Your Navigation App Is Making Traffic Unmanageable

Illustration: Chris Philpot

Miguel Street is a winding, narrow route through the Glen Park neighborhood of San Francisco. Until a few years ago, only those living along the road traveled it, and they understood its challenges well. Now it's packed with cars that use it as a shortcut from congested Mission Street to heavily traveled Market Street. Residents must struggle to get to their homes, and accidents are a daily occurrence.

The problem began when smartphone apps like Waze, Apple Maps, and Google Maps came into widespread use, offering drivers real-time routing around traffic tie-ups. An estimated 1 billion drivers use such apps in the United States alone.

Today, traffic jams are popping up unexpectedly in previously quiet neighborhoods around the country and the world. Along Adams Street, in the Boston neighborhood of Dorchester, residents complain of speeding vehicles at rush hour, many with drivers who stare down at their phones to determine their next maneuver. London shortcuts, once a secret of black-cab drivers, are now overrun with app users. Israel was one of the first to feel the pain because Waze was founded there; it quickly caused such havoc that a resident of the Herzliya Bet neighborhood sued the company.

The problem is getting worse. City planners around the world have predicted traffic on the basis of residential density, anticipating that a certain amount of real-time changes will be necessary in particular circumstances. To handle those changes, they have installed tools like stoplights and metering lights, embedded loop sensors, variable message signs, radio transmissions, and dial-in messaging systems. For particularly tricky situations-an obstruction, event, or emergency-city managers sometimes dispatch a human being to direct traffic.

But now online navigation apps are in charge, and they're causing more problems than they solve. The apps are typically optimized to keep an individual driver's travel time as short as possible; they don't care whether the residential streets can absorb the traffic or whether motorists who show up in unexpected places may compromise safety. Figuring out just what these apps are doing and how to make them better coordinate with more traditional traffic-management systems is a big part of my research at the University of California, Berkeley, where I am director of the Smart Cities Research Center.

Here's how the apps evolved. Typically, the base road maps used by the apps represent roads as five functional classes, from multilane freeways down to small residential streets. Each class is designed to accommodate a different number of vehicles moving through per hour at speeds that are adjusted for local conditions. The navigation systems-originally available as dedicated gadgets or built into car dashboards and now in most smartphones-have long used this information in their routing algorithms to calculate likely travel time and to select the best route.

Initially, the navigation apps used these maps to search through all the possible routes to a destination. Although that worked well when users were sitting in their driveways, getting ready to set out on a trip, those searches were too computationally intensive to be useful for drivers already on the road. So software developers created algorithms that identify just a few routes, estimate the travel times of each, and select the best one. This approach might miss the fastest route, but it generally worked pretty well. Users could tune these algorithms to prefer certain types of roads over others-for example, to prefer highways or to avoid them.

The digital mapping industry is a small one. Navteq (now Here Technologies) and TomTom, two of the earliest digital-map makers, got started about 30 years ago. They focused mainly on building the data sets, typically releasing updated maps quarterly. In between these releases, the maps and the routes suggested by the navigation apps didn't change.

When navigation capabilities moved to apps on smartphones, the navigation system providers began collecting travel speeds and locations from all the users who were willing to let the app share their information. Originally, the system providers used these GPS traces as historical data in algorithms designed to estimate realistic speeds on the roads at different times of day. They integrated these estimates with the maps, identifying red, yellow, and green routes-where red meant likely congestion and green meant unrestricted flow.

As the historical records of these GPS traces grew and the coverage and bandwidth of the cellular networks improved, developers started providing traffic information to users in nearly real time. Estimates were quite accurate for the more popular apps, which had the most drivers in a particular region.

And then, around 2013, Here Technologies, TomTom, Waze, and Google went beyond just flagging traffic jams ahead. They began offering real-time rerouting suggestions, considering current traffic on top of the characteristics of the road network. That gave their users opportunities to get around traffic slowdowns, and that's how the chaos began.

What The Traffic Apps Don't Know Illustration: Chris Philpot A sporting event at a nearby stadium [A] causes a traffic backup on the highway that bypasses the center of this imaginary urban area. That's a problem for our hypothetical driver trying to get home from work, so she turns to a navigation app for help. The shortest-and, according to the app, the fastest-alternate route [blue line] winds through a residential neighborhood with blind turns, a steep hill [B], and a drawbridge [C], which can create unexpected delays for those unfamiliar with its scheduled openings. The red route cuts through the city center [D] and in front of an elementary school [E]; the app doesn't know school just let out for the day. Fortunately, our driver knows the area, so she selects the purple route, even though the app indicates that it isn't the fastest option. Drivers unfamiliar with the area and looking for a shortcut to the stadium could find themselves in chaotic-even hazardous-situations.

On its face, real-time rerouting isn't a problem. Cities do it all the time by changing the signal, phase, and timing of traffic lights or flashing detour alerts on signs. The real problem is that the traffic management apps are not working with existing urban infrastructures to move the most traffic in the most efficient way.

First, the apps don't account for the peculiarities of a given neighborhood. Remember the five classes of roads along with their estimated free-flow speeds I mentioned? That's virtually all the apps know about the roads themselves. For example, Baxter Street in Los Angeles-also a scene of increased accidents due to app-induced shortcutting-is an extremely steep road that follows what originally was a network of goat paths. But to the apps, this road looks like any other residential road with a low speed limit. They assume it has parking on both sides and room for two-way traffic in between. It doesn't take into account that it has a 32 percent grade and that when you're at the top you can't see the road ahead or oncoming cars. This blind spot has caused drivers to stop unexpectedly, causing accidents on this once-quiet neighborhood street.

The algorithms also may not consider other characteristics of the path they choose. For example, does it include roads on which there are a lot of pedestrians? Does it pass by an elementary school? Does it include intersections that are difficult to cross, such as a small street crossing a major thoroughfare with no signal light assistance?

I recently experienced what such cluelessness can cause. I was in congested traffic on a multilane road when an app offered to get me out of the traffic by sending me into a residential neighborhood. It routed me right past an elementary school at 8:15 a.m. There were crossing guards, minivans double parked, kids jumping out of cars, and drivers facing the bright morning sun having a hard time seeing in the glare. I only added to the chaos.

On top of all these problems, these rerouting apps are all out for themselves. They take a selfish view in which each vehicle is competing for the fastest route to its destination. This can lead to the router creating new traffic congestion in unexpected places.

Consider cars crossing a thoroughfare without the benefit of a signal light. Perhaps the car on the smaller road has a stop sign. Likely, it was designed as a two-way stop because traffic on the larger road was typically light enough that the wait to cross was comfortably short. Add cars to that larger road, however, and breaks in the traffic become few, causing the line of cars waiting at the stop sign to flow onto neighboring streets. If you're in the car on the larger road, you may be zipping along to your destination. But if you're on the smaller road, you may have to wait a very long time to cross. And if the apps direct more and more cars to these neighborhood roads, as may happen when a nearby highway is experiencing abnormal delays, the backups build and the likelihood of accidents increases.

To compound the "selfish routing" problem, each navigation application provider-Google, Apple, Waze (now owned by Google)-operates independently. Each provider receives data streamed to its servers only from the devices of its users, which means that the penetration of its app colors the system's understanding of reality. If the app's penetration is low, the system may fall back on historical traffic speeds for the area instead of getting a good representation of existing congestion. So we have multiple players working independently with imperfect information and expecting that the entire road network is available to absorb their users in real time.

Meanwhile, city transportation engineers are busy managing traffic with the tools they have at their disposal, like those on-ramp metering lights, messaging signs, and radio broadcasts suggesting real-time routing adjustments that I mentioned previously. Their goal is to control the congestion, maintain a safe and effective travel network, and react appropriately to such things as accidents, sporting events, and, in emergency situations, evacuations.

The city engineers are also working in isolation, with incomplete information, because they have no idea what the apps are going to do at any moment. The city now loses its understanding of the amount of traffic demanding access to its roads. That's a safety issue in the short term and a planning issue in the long term: It blinds the city to information it could use to develop better traffic-mitigation strategies-for example, urging businesses to consider different work shifts or fleet operators to consider different routes.

Photo: Richard Vogel/AP Goats Only? The steep and narrow Baxter Street in Los Angeles, originally a network of goat paths, looked like any other residential road to the traffic apps, which rerouted drivers along it and caused chaos.

So you may have recently benefitted from one of these shortcuts, but it's doubtful that you're winning the long game. To do that takes thinking about the system as a whole and perhaps even considering aggregate fuel consumption and emissions. Only then can we use these rerouting algorithms for the benefit of all citizens and our environment.

In the meantime, neighborhoods and citizens are fighting back against the strangers using their streets as throughways. In the early days of the problem, around 2014, residents would try to fool the applications into believing there were accidents tying up traffic in their neighborhood by logging fake incidents into the app. Then some neighborhoods convinced their towns to install speed bumps, slowing down the traffic and giving a route a longer base travel time.

A town in New Jersey, Leonia, simply closed many of its streets to through traffic during commute hours, levying heavy fines for nonresident drivers. Neighboring towns followed suit. And all faced the unintended consequence of their local businesses now losing customers who couldn't get through the town at those hours.

The city of Los Angeles recently responded to the issues on Baxter Street by recasting the street as one-way: downhill only. It's still not ideal; it means longer trips for residents coming and going from their homes, but it reduced the chaos.

Last year, an unfortunate situation in Los Angeles during the 2017 wildfires clearly demonstrated the lack of congruence among the rerouting apps and traditional traffic management: The apps directed drivers onto streets that were being closed by the city, right into the heart of the fire. This is not the fault of the algorithms; it is simply extremely difficult to maintain an up-to-date understanding of the roads during fast-moving events. But it does illustrate why city officials need a way to connect with or even override these apps. Luckily, the city had a police officer in the area, who was able to physically turn traffic away onto a safer route.

These are mere stopgap measures; they serve to reduce, not improve, overall mobility. What we really want is a socially optimum state in which the average travel time is minimized everywhere. Traffic engineers call this state system optimum equilibrium, one of the two Wardrop principles of equilibrium. How do we merge the app-following crowds with an engineered flow of traffic that at least moves toward a socially optimized system, using the control mechanisms we have on hand? We can begin by pooling everyone's view of the real-time state of the road network. But getting everybody in the data pool won't be easy. It is a David and Goliath story-some players like Google and Apple have massive back-office digital infrastructures to run these operations, while many cities have minimal funding for advanced technology development. Without the ability to invest in new technology, cities can't catch up with these big technology providers and instead fall back on regulation. For example, Portland, Ore., Seattle, and many other cities have lowered the speed limits on residential streets to 20 miles per hour.

There are better ways. We must convince the app makers that if they share information with one another and with city governments, the rerouting algorithms could consider a far bigger picture, including information from the physical infrastructure, such as the timing schedule for traffic lights and meters and vehicle counts from static sensors, including cameras and inductive loops. This data sharing would make their apps better while simultaneously giving city traffic planners a helping hand.

As a first step, we should form public-private partnerships among the navigation app providers, city traffic engineering organizations, and even transportation companies like Uber and Lyft. Sharing all this information would help us figure out how to best reduce congestion and manage our mobility.

We have a number of other hurdles to overcome before all the apps and infrastructure tools can work together well enough to optimize traffic flow for everyone.

The real challenge with traffic control is the enormous scale of the problem. Using the flood of data from app users along with the data from city sensors will require a new layer of data analytics that takes the key information and combines it, anonymizes it, and puts it in a form that can be more easily digested by government-operated traffic management systems.

We also need to develop simulation software that can use all this data to model the dynamics of our mobility on an urban scale. Developing this software is a key topic of current research sponsored by the U.S. Department of Energy's Energy Efficient Mobility Systems program and involving Here Technologies and three national laboratories: Lawrence Berkeley, Argonne, and Pacific Northwest. I am involved with this research program through the Berkeley lab, where I am a guest scientist in the Sustainable Transportation Initiative. To date, a team supported by this program, led by me and staffed by researchers from the three laboratories, has developed simulations for a number of large cities that can run in just minutes on DOE supercomputers. In the past, such simulations took days or weeks. I expect that new approaches to manage congestion that account for the many complexities of the problem will emerge from these simulations.

In one of our projects, we took 22 million origin-and-destination pairs-or trip legs, as defined by the San Francisco County Transportation Authority-and created a simulation for the San Francisco Bay Area that defines the shortest travel time route for each leg as well as the congestion patterns on each route for a full day. We added an algorithm that reroutes vehicles when the simulation anticipates significant congestion. We discovered that approximately 40,000 vehicles are typically rerouted per hour at the peak congestion times in the morning and 120,000 vehicles are rerouted per hour in the evening congestion period; an incident on a highway, of course, would cause these numbers to jump.

This simulation demonstrates how much traffic planners can do to rebalance traffic flow, and it provides numbers that, right now, are not directly available. The next question is how much of the road network you want to use, trading off highway congestion for some additional traffic on neighborhood roads.

Our next step will be to modify our algorithm to consider neighborhood constraints. We know, for example, that we don't want to reroute traffic into school zones during drop-off and pickup times, and that we should modify navigation algorithms appropriately.

We hope to soon put these tools in the hands of government transportation agencies.

That's what we're trying to do with technology to address the problem. But there are nontechnical hurdles as well. For example, location data can contain personal information that cannot be shared indiscriminately. And current business models may make for-profit companies reluctant to give away data that has value.

Solving both the technical and nontechnical issues will require research and public-private partnerships before we can assemble this cooperative ecosystem. But as we learn more about what drives the dynamics of our roads, we will be able to develop effective routing and traffic controls that take into account neighborhood concerns, the business objectives of fleet owners, and people's health and convenience.

I am confident that most people, when well informed, would be open to a little inconvenience in the furtherance of the common good. Wouldn't you be willing to drive a few extra minutes to spare a neighborhood and improve the environment?

This article appears in the October 2019 print issue as "When Apps Rule the Road."

About the Authors

Jane Macfarlane is director of the Smart Cities Research Center at the University of California Berkeley's Institute of Transportation Studies, where she works on data analytics for emerging transportation issues.