Boeing’s Management Crisis | Part Two: Catastrophe on the 737 MAX
In October 2019, text messages surfaced showing that in 2016 Boeing’s chief 737 technical pilot, Mark Forkner, said he could not avert egregious trimming by an automated system that was “running rampant” during test sessions in a 737 MAX flight simulator.
The term “trim” refers to adjusting control surfaces so that the aircraft maintains the altitude set by the pilot without any further control input. Think of this concept as analogous to cruise control in a car. Activating cruise control will hold the car’s speed constant once the driver removes their foot from the gas pedal, which usually feels more comfortable. Similarly, a Boeing pilot might need to exert pressure on the control yoke to fly at a level pitch, which after a while can start to feel uncomfortable. Setting the pitch trim will maintain that altitude even if the pilot releases their pressure on the control column.
Pilots in the 737 can adjust this trim through both power-assisted and manual techniques in ways that are analogous to another system in a car: power steering. When a car’s power steering system fails, it might still be possible to turn the steering wheel, but doing so would require tremendous human force.
Similarly, because the 737 is not yet a completely electronic “fly by wire” aircraft like the more modern Airbus models, to adjust pitch the 737 still relies on pulling steel cables that swivel the horizontal stabilizer. Pilots normally adjust the stabilizer through switches that control powerful electric motors that extend and retract the cables. Boeing also connects those cables to wheels mounted on the cockpit’s center pedestal that pilots may be able to crank by hand if the electric, power-assisted trim system is malfunctioning or inactive.
How Can a Plane’s Trim Malfunction?
One type of trim malfunction is known as a “runaway” stabilizer trim. In this situation, the electric motor providing the power assist for the stabilizer trim system can stick in the “on” position. In that case, the motor can continue to swivel the rear stabilizer well beyond the levels specified by the pilots, forcing the nose excessively up or down.
Technically, it is still not yet known with absolute certainty what sort of trimming Forkner was describing. It is also uncertain if he was describing an implementation of the MAX’s flight dynamics in the simulator cab that differed from behavior onboard an actual, flying MAX.
But in response to a legal request for documents, Forkner asserted the Fifth Amendment’s protection against self-incrimination. Moreover, the specific automated system that appears to be the leading cause of both crashes is closely related to the stabilizer trim system. That’s because the automated system could be turned off by two electric trim cutout switches behind the throttles that provide power assistance to the stabilizer trim system as well.
Boeing’s Maneuvering Characteristics Augmentation System (MCAS)
That automated system is the MAX’s new Maneuvering Characteristics Augmentation System, or MCAS. Boeing introduced MCAS, a flight control software subsystem, to simulate the handling characteristics of an older 737 by the MAX so that the FAA would maintain the single, original type certification. In other words, MCAS attempted to compensate for the MAX’s less stable aerodynamics in unusual circumstances so that it would handle like any other 737. The MAX was the first 737 model on which MCAS appeared.
At one point, Forkner says:
“Oh, shocker alert! MCAS is now active down to M point 2. It’s running rampant in the sim on me.. . .So basically, I lied to the regulators unknowingly.”
One analysis of these texts by an airline pilot suggests that MCAS, which was previously intended to function at high speeds, had now been extended to also function at Mach 0.2, a speed much lower than FAA regulators had been told. Previously (and possibly because of the high-speed functioning restriction) the same Forkner requested and received permission from the FAA so Boeing could delete all mentions of MCAS from the flight crew operations manual for the MAX, thus hiding the system from pilots. Before the Lion Air crash, pilots had no knowledge of the MCAS software. Consequently, there was no way to train for an MCAS malfunction because one can’t train for a malfunction in a system not known to exist in the first place. Pilots received no indication through instrumentation that MCAS existed or had activated, either.
An MCAS fault has since been shown to be the primary failure—though not the only failure—that doomed both the Lion Air and Ethiopian Airlines flights. What Forkner appears to be talking about was that MCAS was “running rampant” at much lower speeds, but not because of sensor failures which created both disasters.
The MCAS version operating on both the Lion Air and Ethiopian Airlines aircraft would activate when it detected a high angle of attack—the first sign of a potential stall—at a slow airspeed not far above the minimum. But to decide if a stall might be starting, MCAS first depended on readings from one of two airflow sensors mounted on the outside of the aircraft. Key among these are “angle of attack” sensor vanes on the aircraft’s exterior below the side windshields. One sensor is mounted on the captain’s side, and another is mounted on the first officer’s side.
These delicate sensor vanes, which have “DO NOT TOUCH” painted below them, are vulnerable to frequent malfunctions from common incidents, like bird strikes and bumps with jetways. Approximately 200 of these sensor hardware failures have been reported recently.
Despite the multiple sensors, the MCAS software only relied on input from a single AOA sensor. That implementation violated a central tenet of aeronautical safety engineering: system redundancy, once explained to me by an American Airlines captain as “backups, upon backups, upon more backups.”
In other words, corrupted data resulting from a failure in a single AOA sensor would nevertheless be interpreted by MCAS—with catastrophic results. This is known as a “single point of failure.”
How Has Congress Handled Boeing’s Failure?
Members of Congress, investigators, and commentators are unanimous in expressing bewilderment that Boeing would have conditioned the operation of the MCAS software on a single sensor without a redundant backup in case of a sensor failure.
During Congressional hearings, Representative Peter DeFazio cited an email written by a Boeing engineer in 2015, two years before the FAA cleared the MAX to fly. The engineer raised concerns that MCAS could activate unnecessarily based on a single sensor reading: “Are we vulnerable to a single AOA sensor failure with the MCAS implementation, or is there some checking that occurs?”
In other words, the failure of a single sensor would jeopardize the integrity of the entire MCAS system. That just so happens to be exactly what happened during both disasters.
What’s more disturbing is that the software code that acts on the sensor data was written not by Boeing engineers, but by inexperienced contractors in India working for only about $9.00 an hour. These contractors—none of whom with experience as pilots—failed to build in any capability for the pilots to override potential system errors through the software. The only way to override it was by switching off the stabilizer trim system that also powered MCAS.
The Conditions of the MCAS Failure
For MCAS to engage, three conditions needed to be present: the AOA sensor detected an abnormally high angle of attack, the flaps were retracted, and the autopilot was disengaged. When the software interpreting the AOA sensor’s data suspected a potential stall—even a “false stall” due to a sensor anomaly—the “stick shaker” stall warning alert would result in a rattling control column. The Lion Air and Ethiopian pilots both received this alert.
MCAS would then activate. It would aggressively shove the plane’s nose down, kicking off the “sudden, violent and terrifying event” described at the beginning of this series.
Boeing assumed that pilots would immediately recognize this condition as a runaway stabilizer trim condition and follow the corrective steps in a memorized checklist. What Boeing somehow failed to recognize was that an MCAS activation resulting from a sensor fault presented nothing like a runaway stabilizer. Because of the sensor fault, MCAS lit up the instrument panel like a pinball machine, with all kinds of contradictory and confusing indications and flashing warning lights. Some of the warnings were audible, like the “Don’t Sink” automated voice, and a deafening clacker which indicated that airspeed was too high.
Boeing also failed to recognize that pilots could be faced with multiple failures at once. Besides the sensor failure, evidence is mounting that autothrottle systems on both the Lion Air and Ethiopian aircraft may have failed at the same time as MCAS because excessive speed appears to be prominent features of both disasters. James Fallows, the editor of the Atlantic, found reports of MAX autothrottle failures in pilot complaints filed with NASA’s database.
Although Boeing designed the 737 for airports in Europe and Asia, the company appears to have also assumed that the MAX would only be flown by pilots with the kind of training and experience mandated in the United States since 2013. To fly a commercial passenger or cargo jetliner in this country, a first officer needs to hold an Airline Transport Pilot (ATP) certificate, requiring at least 1,500 hours of total flying time as a pilot. But in Europe, a first officer only needs about 250 hours, of which about 80 percent can be in flight simulators. The first officer in the Ethiopian crash only had flown for roughly 200 hours. Complete details of the pilots’ training and experience in both MAX disasters remain unclear.
Suppose that a pilot interrupted the first dive, possibly by pulling up on the control column to raise the elevators in the tail. Unless the pilots either extended the flaps or switched off two electric trim cutout switches behind the throttles, MCAS was programmed to engage repeatedly. Each time, MCAS would shove the nose down with increasing force, reactivating every nine seconds with only a five-second pause between cycles. If pilots did not interrupt the dive and regain control of the airplane, MCAS would persist until the airplane was diving in the full nose-down position.
Before the MAX, the U.S. Department of Defense required Boeing to remove the repetitive cycling on a version of MCAS deployed in a military jet. That was another troubled Boeing product: the KC-135 tanker used to refuel fighter jets during flight. With that plane, Boeing sought to alert pilots at the first signs of a stall triggered by jet fuel flowing into tanks at the rear of the tanker that could weigh down the plane’s tail and push the nose up. It defies logic as to why Boeing, through their contractors, deliberately programmed this repetition into the non-military MCAS version on the 737 MAX. This gripping New York Times animation, simulating how the Lion Air pilots battled to maintain control of their doomed aircraft, portrays the crisis situation Boeing executives knew (or should have known) could result.
Although not conclusive, Forkner’s text messages strongly suggest that Boeing’s management should have been aware as early as 2016 that the MCAS software could seize control of a MAX and force uncommanded maneuvers. The Lion Air crew fought these repetitive cycles 21 times; each time the plane dived and then pitched back up until, on the 22nd dive, the plane finally impacted the Java Sea.
The young Ethiopian pilots presumably had a better understanding of this situation. They most likely knew about an alarming emergency airworthiness directive issued by the FAA and published worldwide on the Internet in November 2018. That bulletin read in part:
This emergency AD was prompted by analysis performed by the manufacturer showing that if an erroneously high single angle of attack (AOA) sensor input is received by the flight control system, there is a potential for repeated nose-down trim commands of the horizontal stabilizer. This condition, if not addressed, could cause the flight crew to have difficulty controlling the airplane, and lead to excessive nose-down altitude, significant altitude loss, and possible impact with terrain.
Reading through the report of the Ethiopian flight is difficult because the pilots of the Ethiopian flight came very close to averting this disaster. That’s because those pilots correctly identified how to disable MCAS. Consistent with the FAA’s emergency directive and a Boeing manual update, they shut off the two electric stabilizer trim cutout switches, which shut down MCAS and stopped the repeated roller-coaster-like diving every 14 seconds.
But several minutes had elapsed and by that time, in the calamity of the crisis, the crew failed to do something critically important. They didn’t reduce the takeoff thrust, which they set before departure at 94 percent of the engines’ maximum thrust. With the plane still accelerating, shutting off the cutout switches also shut off the power assist that would swivel the stabilizer, freezing it with the nose still pitched downward and the plane diving.
But when the crew tried to manually crank the wheels connected by cables to the stabilizer, they didn’t have the physical strength required. By not reducing thrust, the pilots let the plane accelerate into the redline zone almost 50 miles per hour beyond the airplane’s maximum safe airspeed. Airflow forces on control surfaces don’t vary according to a linear relationship with speed, but an exponential one. The extreme opposing force of that airflow on the stabilizer was too strong for the pilots to manually move the stuck cable cranks connected to that control surface.
Violating Boeing’s checklist, the frantic pilots then turned the cutout switches back on to restart the stabilizer trim power assist. But that action also reactivated MCAS, which shoved the nose down one last time. About thirty seconds later, the plane impacted the ground at roughly 600 MPH.
Part three of this series covers eminent business professors’ perspectives on the Boeing tragedy.