Maybe you have a large amount of labeled data (we do too). And maybe you can extract from this an empirical distribution of initial conditions or even full replay trajectories. But what if you could multiply that data infinitely, in particular by learning a model that can generate realistic initial conditions, multiplying your scenario coverage literally right from the start. Inverted AI researchers have invented a model that can do just that.

Let us explain what we mean via a short quiz. In the examples below some of the vehicle arrangements are labelled training data. Some are samples from our INITIALIZE cloud API. Can you tell which are which? (the answer appears at the bottom of this post)

Note that vehicles that overlap onto the black are in the training data too so that doesn't help. Red lines indicate vehicle fronts.

This model is a "permutation invariant normalizing flow built on equivariant continuous dynamics" and is the subject of a paper just released on arXiv and under review at NeurIPS 2022.

An intuitive sense of how it works can be gathered from watching its internal flow do its job, starting from randomly placed vehicles and "flowing" them to final conditional locations. Here the condition is the underlying map image.

Because this model is a normalizing flow it can be used for outlier detection, a fact that we exploit in our own data labelling pipeline, and you might too. For instance thresholding the conditional likelihood of a configuration under this model is a good way to detect mislabeled images.

LP stands for "log probability." There is a large, detectable difference in conditional log probability for test samples that have been modified to have, for instance, as shown in blue, even a single vehicle mislabeled in terms of orientation.

The use case of such a model for autonomous simulation should be obvious, but, if for some reason it is not, here are some samples from Initialize used to generate initial conditions for a scenario in CARLA.

These are all completely synthetic and, for those familiar with the CARLA autopilot, are traffic configurations that would simply not occur under the traffic manager.

Integrating INITIALIZE into your simulator via our cloud API is straightforward. Contact us for support and details.

The underlying research and model are also useful for object detection. This was demonstrated in the paper on the CLEVR dataset where the flow was trained to conditionally detect objects. Here the flow transforms boxes in space so that they region propose bounding boxes for the objects in the scene.

We promised the answer key for the quiz above. Here it is.

Data means training data. All of the others are synthetic. The last column are samples that have more vehicles on that map than ever appeared in the training data!

Interested in the technical details? Check out the arXiv paper just released.