Waymo advances AI research with our multifunctional Waymax simulator

For over a decade, Waymo has been addressing complex AI and ML challenges in autonomous driving, robotics and computer vision. We have contributed to the research community through our Waymo Open Dataset initiative and through our academic publications in top conferences. Our next chapter in this collaboration with the community is Waymax, which can unlock new research horizons in the simulation and planning agent domains.

Simulation is a critical tool for testing and evaluating autonomous driving system performance, which is why we invest heavily in realistic simulation environments (such as our Simulation City) to evaluate the performance of the Waymo Driver.

To be an effective evaluation tool, a simulator requires huge volumes of carefully curated data that are representative of the real world. Not only does it need to look real, but it's critical that other road agents — such as human drivers, pedestrians, and cyclists — behave realistically in a simulated environment, matching the experiences our vehicles would encounter in the real world. This is why sim agents and their connections to planning research have become increasingly important.

Waymo’s own simulation tools benefit from tens of millions of real-world autonomous miles collected by the Waymo Driver. We’re thrilled to be able to share our unique capabilities with the academic community through the first-of-its-kind, lightweight simulator.

In developing the Waymax simulator, we focused on building a more powerful and more easily accessible tool for researchers than what exists today. Compared to other simulators, Waymax offers a range of unique features and capabilities previously unavailable. Waymax is:

• Lightweight. It enables researchers to conduct experiments efficiently and iterate quickly.
• Informed by troves of real-world data. To model realistic driving behaviors, Waymax can leverage the 570+ hours of real world driving examples from multiple cities that are present in the Waymo Open Motion Dataset.
• Hardware-accelerated. Waymax is written entirely in JAX, making use of in-graph compilation to deliver blazing-fast simulations.
• Multi-agent, so researchers can analyze and model the complex behaviors among multiple road users and unlock new insights into multi-agent systems.

Researchers can access Waymax software on GitHub and use it straight out of the box. It already incorporates the minimal representation of objects and bounding boxes adopted by the Waymo Open Motion Dataset, which eliminates the need for users to run their own perception system from raw sensor data, and comes bundled with metrics and pre-built sim agents, streamlining the research process.

As with all our engagements with the research community, we are dedicated to continuously enhancing and evolving the Waymax platform, and we look forward to seeing what amazing results it will enable.