MIT's Cathy Wu on Why Reinforcement Learning Keeps Breaking — and the Workaround That Cut Training Costs 30x

Reinforcement learning has a reputation for brittleness that rarely survives contact with a conference slide deck. A new MIT profile of Cathy Wu, an associate professor in the Department of Civil and Environmental Engineering and the Institute for Data, Systems, and Society, is unusually candid about it: after an early RL-for-traffic result that MIT says went viral in 2018, Wu spent roughly two years at MIT watching follow-up attempts fail. What came out of that stretch is a practical finding about which problems to train on — one with implications well beyond traffic.

The failure that produced the method

According to MIT, Wu's 2018 work during her PhD at UC Berkeley applied RL to automatically analyze how autonomous vehicles might affect traffic flow across a range of different networks. She then did a postdoc at Microsoft on RL theory and joined the MIT faculty, drawn by CEE's sustainability focus and IDSS's emphasis on pushing data science into other disciplines.

Then the method stopped cooperating. "That was stressful," Wu says in the profile. "It was unclear whether the problem was me (the advisor), my students, the traffic domain, or RL itself."

That ambiguity will be familiar to anyone who has tried to productionize RL. When a run fails, the failure is rarely legible: the reward shaping, the environment, the seed, the algorithm choice, and the problem formulation are all plausible suspects, and the debugging loop is expensive. Wu's group eventually turned the question into a research result. In 2022, MIT says, she and her students identified that RL algorithms are so sensitive that an algorithm which works on one problem may not work on even a closely related one.

That is a diagnosis rather than a fix, but it reframes the engineering problem. If sensitivity is a property of the algorithms rather than a sign of sloppy tuning, then the response is not more tuning — it is selecting the problems you train on.

Train on the 10 percent that works

The payoff, which Wu says she is proudest of "because it was like the light at the end of a long tunnel of negative results," arrived in 2023. MIT describes the finding this way: across a group of related problems, RL may fail to train well on 90 percent of them but train quite well on the remaining 10 percent. Train on the subset that both solves and generalizes well, and the resulting models collectively perform well across the broader set — including problems that direct training would not have cracked.

Wu's team then built an algorithm to decide which problems to train on. MIT says it improved training efficiency by up to 30 times: a workload that would normally have required 100 training models may only require three.

A 30x reduction in model count is the kind of number that changes what is economically feasible, not just what is academically interesting. If designing a transportation system means modeling and analyzing dozens or hundreds of variants — Wu's framing of why evidence-driven design is out of reach with today's tools — then the binding constraint is compute and researcher time per variant, not any single model's peak performance. Cutting the number of models needed attacks the constraint directly.

The profile does not specify the benchmark suite, the baseline method, the domains the 30x figure was measured on, or whether the gain holds as problem families get more heterogeneous. Those caveats matter: "up to" figures in RL often reflect a best case on a particular family of tasks. Readers should treat it as a reported research result, not a validated engineering spec.

Wu says a good chunk of her group now works on contextual RL — the setting where RL tries to solve a space of related problems rather than one instance. That is a meaningful reframing. Most applied optimization work in industry is exactly this: not one routing problem, but a new variant every week as constraints, fleets, or demand patterns shift.

Eco-driving as the policy test case

The applied result MIT highlights is eco-driving: intelligently controlling vehicle speeds to cut excessive stopping and starting. Wu's work shows such measures could reduce vehicle emissions by between 11 and 22 percent, according to MIT, and she describes it as "a demonstration that RL can be used to inform transportation policy on problems of practical importance."

An 11-to-22 percent range is wide, which is itself informative — the benefit presumably depends heavily on network, traffic density, and penetration rate. The profile does not say whether these figures come from simulation or field deployment, what share of vehicles would need to participate, what baseline the reduction is measured against, or how the control would be implemented in practice (infrastructure signals, in-vehicle systems, or policy mandate). For a product team, those are the first questions, and none are answered here.

What the result does establish is a workflow: use RL to make the analysis of many system variants cheap enough that a policy question becomes empirically answerable. Wu frames this as the point of the work. "I am a big fan of evidence-based policy," she says, adding that she believes some questions people argue about endlessly "can be analyzed systematically using data and have objective answers."

Why this generalizes past traffic

MIT says the algorithms Wu's team has produced are intended to streamline the development of solvers for hard optimization problems generally — logistics, supply chains, manufacturing, and resource allocation among them. Wu calls her approach use-inspired basic research: start from a consequential practical problem, find where existing methods fall short, and let the problem shape the research direction.

That positioning is worth noting at a moment when most RL attention is pointed at post-training for language models. The combinatorial-optimization thread is older, less fashionable, and arguably closer to deployment in operations-heavy industries, where the alternative is hand-built solvers that take years per variant — the bottleneck Wu says she observed among transportation researchers during her PhD.

Wu has received a 2023 National Science Foundation CAREER award and the Ole Madsen Mentoring Award in 2025, per MIT. Her advice to students facing large problems is deliberately unglamorous: "Be patient. Start small. Societal impact is a lifelong endeavor, not something to be accomplished in a few years."

What to watch

• Whether the problem-selection algorithm behind the reported up-to-30x efficiency gain is released as usable tooling, and how it performs on optimization families outside transportation. MIT does not say whether code is available.
• Replication of the 11–22 percent eco-driving emissions figure outside the original setting, including what penetration rate of controlled vehicles is required and whether any field trial is planned. Not disclosed in the profile.
• Whether contextual RL — solving a space of related problems — gets adopted by logistics and supply chain teams who currently rebuild solvers per variant, which is the clearest commercial test of the idea.
• How the 2022 sensitivity finding influences evaluation practice. If RL performance can swing between closely related problems, single-benchmark results in applied RL papers deserve more skepticism than they typically get.
• Whether regulators and transportation agencies actually use RL-generated evidence in rulemaking, which is the claim Wu is making for the work and the hardest one to verify.