Being able to predict the behavior of other agents on the road is absolutely crucial for achieving level 4-5 autonomy. Other agents not only have their goals and methods of realizing them, but they also hold beliefs about each other’s internal states and adjust their actions based on their best predictions of the actions of other agents.

 

Inverted AI has developed a predictive model called “Imagining the Road Ahead” (or ITRA for short) achieves world-class performance on the Interpret Challenge; specifically less than 20cm average displacement error when predicting 3 seconds into the future given 1 second of past trajectories. See our ITRA paper for more details.

 

ITRA should be thought of like Open AI's GPT language model which given some history of recent words predicts the next words to come. ITRA conditions on a map (as shown above) and some amount of historical position data of all agents (darker coloring of agents in the animations above) then jointly predicts the future for all agents well into the future, one step at a time (here 6 seconds in the animations). The green lines in the figures show diverse and realistic predicted futures for various agents. Watching carefully the bottom right animation highlights the elusive but invaluable reactivity where the first car exiting the roundabout slows slightly to avoid collision with the overtaking car.

 

We are currently developing improved variants of ITRA that make fewer infractions and are directable, allowing for both easy and intuitive generation of custom scenarios and natural related variants. Inverted AI is uniquely well suited to developing such models.

 

Apart from the obvious scientific benefit of advancing AI research and competing to better predict evolving road situations, the main point of our world-class predictive models is that we use them to drive reactive, directable NPCs in AV simulation. Doing this closes the sim-to-real gap between synthetic AV training and evaluation environments and the real world. Clearly training in a simulator with ITRA NPCs decreases the amount of real-world interaction that has to be experienced. What is more, validating in an environment with ITRA NPCs directly leads to improved safety as greater coverage of real-world behavior is ensured.