Example Colab

Colab example for running simulation with ScenarioNet: Open In Colab

Colab example for reading established ScenarioNet dataset: Open In Colab

Waymo Example

In this example, we will show you how to convert a small batch of Waymo scenarios into the internal Scenario Description. After that, the scenarios will be loaded to MetaDrive simulator for closed-loop simulation. First of all, please install MetaDrive and ScenarioNet following these steps Installation.

1. Setup Requirements

For any dataset, the first step after installing ScenarioNet is to install the corresponding official toolkit as we need to use it to parse the original data and convert to our internal scenario description. For Waymo data, we already have the parser in ScenarioNet so just install the TensorFlow and Protobuf via:

pip install tensorflow==2.11.0
conda install protobuf==3.20

Note

You may fail to install protobuf if using pip install protobuf==3.20. If so, install via conda install protobuf=3.20.

For other datasets like nuPlan and nuScenes, you need to setup nuplan-devkit and nuscenes-devkit respectively. Guidance on how to setup these datasets and connect them with ScenarioNet can be found at Datasets.

2. Prepare Data

Access the Waymo motion data at Google Cloud. Download one tfrecord scenario file from waymo_open_dataset_motion_v_1_2_0/uncompressed/scenario/training_20s. In this tutorial, we only use the first file training_20s.tfrecord-00000-of-01000. Just click the download button ⬇️ on the right side to download it. And place the downloaded tfrecord file to a folder. Let’s call it exp_waymo and the structure is like this:

exp_waymo
├──training_20s.tfrecord-00000-of-01000

Note

For building database from all scenarios, install gsutil and use this command: gsutil -m cp -r "gs://waymo_open_dataset_motion_v_1_2_0/uncompressed/scenario/training_20s" . Likewise, place all downloaded tfrecord files to the same folder.

3. Build Mini Database

Run the following command to extract scenarios in exp_waymo to exp_converted:

python -m scenarionet.convert_waymo -d /path/to/exp_converted/ --raw_data_path /path/to/exp_waymo --num_files=1

Note

When running python -m, make sure the directory you are at doesn’t contain a folder called scenarionet. Otherwise, the running may fail. For more details about the command, use python -m scenarionet.convert_waymo -h

Now all extracted scenarios will be placed in exp_converted directory. If we list the directory with ll command, the structure will be like:

exp_converted
├──exp_converted_0
├──exp_converted_1
├──exp_converted_2
├──exp_converted_3
├──exp_converted_4
├──exp_converted_5
├──exp_converted_6
├──exp_converted_7
├──dataset_mapping.pkl
├──dataset_summary.pkl

This is because we use 8 workers to extract the scenarios, and thus the converted scenarios will be stored in 8 subfolders. If we go check exp_converted_0, we will see the structure is like:

├──sd_waymo_v1.2_2085c5cffcd4727b.pkl
├──sd_waymo_v1.2_27997d88023ff2a2.pkl
├──sd_waymo_v1.2_3ece8d267ce5847c.pkl
├──sd_waymo_v1.2_53e9adfdac0eb822.pkl
├──sd_waymo_v1.2_8e40ffb80dd2f541.pkl
├──sd_waymo_v1.2_df72c5dc77a73ed6.pkl
├──sd_waymo_v1.2_f1f6068fabe77dc8.pkl
├──dataset_mapping.pkl
├──dataset_summary.pkl

Therefore, the subfolder produced by each worker is actually where the converted scenarios are placed. To aggregate the scenarios produced by all workers, the exp_converted/dataset_mapping.pkl stores the mapping from scenario_id to the path of the target scenario file relative to exp_converted. As a result, we can get all scenarios produced by 8 workers by loading the database exp_converted.

4. Database Operations

Several basic operations are available and allow us to split, merge, move, and check the databases. First of all, let’s check how many scenarios are included in this database built from training_20s.tfrecord-00000-of-01000:

python -m scenarionet.num -d /path/to/exp_converted/

It will show that there are totally 61 scenarios. For machine learning applications, we usually want to split training/test sets. To this end, we can use the following command to build the training set:

python -m scenarionet.split --from /path/to/exp_converted/ --to /path/to/exp_train --num_scenarios 40

Again, use the following commands to build the test set:

python -m scenarionet.split --from /path/toexp_converted/ --to /path/to/exp_test --num_scenarios 21 --start_index 40

We add the start_index argument to select the last 21 scenarios as the test set. To ensure that no overlap exists, we can run this command:

python -m scenarionet.check_overlap --d_1 /path/to/exp_train/ --d_2 /path/to/exp_test/

It will report No overlapping in two database!. Now, let’s suppose that the /exp_train/ and /exp_test/ are two databases built from different source and we want to merge them into a larger one. This can be achieved by:

python -m scenarionet.merge --from /path/to/exp_train/ /path/to/exp_test -d /path/to/exp_merged

Let’s check if the merged database is the same as the original one:

python -m scenarionet.check_overlap --d_1 /path/to/exp_merged/ --d_2 /path/to/exp_converted

It will show there are 61 overlapped scenarios. Congratulations! Now you are already familiar with some common operations. More operations and details is available at Operations.

5. Simulation

The database can be loaded to MetaDrive simulator for scenario replay or closed-loop simulation. First of all, let’s replay scenarios in the exp_converted database:

python -m scenarionet.sim -d /path/to/exp_converted --render 2D

By adding --render 3D flag, we can use 3D renderer:

python -m scenarionet.sim -d /path/to/exp_converted --render 3D

Note

--render advanced enables the advanced deferred rendering pipeline, but an advanced GPU better than RTX 2060 is required.