yao-shao / waymo_kitti_adapter Goto Github PK

View Code? Open in Web Editor NEW

99.0 4.0 29.0 14.07 MB

A tool converting Waymo dataset format to Kitti dataset format.

License: MIT License

Python 100.00%

self-driving-car kitti-dataset waymo waymo-open-dataset autonomous-vehicles kitti-data

waymo_kitti_adapter's Introduction

Waymo_Kitti_Adapter

This is a tool converting Waymo open dataset format to Kitti dataset format.

Author: Yao Shao

Contact: [email protected]

Instruction

Follow the instructons in QuickStart.md, clone the waymo open dataset repo, build and test it.
Clone this repo to your computer, then copy the files in protocol buffer folder and paste them into waymo open dataset folder.
Copy adapter.py to waymo-od folder. Open adapter.py and change the configurations at the top so that it suits to your own computer's path.
The folder tree may look like this, the downloaded waymo dataset should be in the folder named waymo_dataset, and the generated kitti dataset should be in the folder kitti_dataset/. Feel free to change them to your preferred path by rewriting the configurations in adapter.py.

.
├── adapter.py
├── waymo_open_dataset
│   ├── label_pb2.py
│   ├── label.proto
│   └── ...
├── waymo_dataset
│   └── frames
├── kitti_dataset
│   ├── calib
│   ├── image_0
│   ├── image_1
│   ├── image_2
│   ├── image_3
│   ├── image_4
│   ├── lidar
│   └── label
├── configure.sh
├── CONTRIBUTING.md
├── docs
├── LICENSE
├── QuickStart.md
├── README.md
├── tf
├── third_party
├── tutorial
└── WORKSPACE

Run adapter.py.

Data specification

Cameras

Waymo dataset contains five cameras:

FRONT = 0;
FRONT_LEFT = 1;
FRONT_RIGHT = 2;
SIDE_LEFT = 3;
SIDE_RIGHT = 4;

all the names below with post-fix 0-4 is corresponding to these five cameras.

Label

label_0 to label_4 contains label data for each camera and label_all fonder contain all the labels.

All in vehicle frame.

For each frame, here is the data specification:

#Values    Name      Description
----------------------------------------------------------------------------
   1    type         Describes the type of object: 'Car', 'Van', 'Truck',
                     'Pedestrian', 'Person_sitting', 'Cyclist', 'Tram',
                     'Misc' or 'DontCare'
   1    truncated    Float from 0 (non-truncated) to 1 (truncated), where
                     truncated refers to the object leaving image boundaries
   1    occluded     Integer (0,1,2,3) indicating occlusion state:
                     0 = fully visible, 1 = partly occluded
                     2 = largely occluded, 3 = unknown
   1    alpha        Observation angle of object, ranging [-pi..pi]
   4    bbox         2D bounding box of object in the image (0-based index):
                     contains left, top, right, bottom pixel coordinates
   3    dimensions   3D object dimensions: height, width, length (in meters)
   3    location     3D object location x,y,z in camera coordinates (in meters)
   1    rotation_y   Rotation ry around Y-axis in camera coordinates [-pi..pi]
   1    camera_num	the camera number which the object belongs to, only exist 
                     in label_all

Calib

P0-P4 : intrinsic matrix for each camera
R0_rect : rectify matrix
Tr_velo_to_cam_0 - Tr_velo_to_cam_4 : transformation matrix from vehicle frame to camera frame

Image

image_0 - image_4 : images for each

Lidar

Point cloud in vehicle frame.

x y z intensity

For more details, see readme.txt by KITTI.

References

waymo_kitti_adapter's People

Contributors

Stargazers

Watchers

waymo_kitti_adapter's Issues

how to tranform 3d points from one frame to another

hi,
Yao-Shao,
thanks for your awesome work. may i ask a question about the transformation? how can man tranform 3d points from one frame to another(or is it even possible to do that?)
i know there is a 4x4 row major transform matrix. but it is 4x4. when we have points whose shape is (n, 3), how can we do the transform operation?
thanks for your reply in advance.

filter location

Why only use location_sf, filter out location_phx and location_other?

2D bounding box is inaccurate

it should not use ‘frame.projected_lidar_labels’ for 2D bounding box

Waymo dataset is a tar file. I don't see any frames

Hello. I downloaded waymo dataset tar file. Inside the tar file, there are no images, just tfrecord files. What should I do now?

[libprotobuf ERROR google/protobuf/wire_format_lite.cc:581] String field 'waymo.open_dataset.Context.name' contains invalid UTF-8 data when parsing a protocol buffer. Use the 'bytes' type if you intend to send raw bytes.

Hi, I was running adapter.py file and I encountered this error [libprotobuf ERROR google/protobuf/wire_format_lite.cc:581] String field 'waymo.open_dataset.Context.name' contains invalid UTF-8 data when parsing a protocol buffer. Use the 'bytes' type if you intend to send raw bytes., Can you please help me resolve this issue?

bazel test waymo_open_dataset/metrics/ops/... fails!

I followed your instructions and I get an error:

WARNING: /home/sarim/.cache/bazel/_bazel_sarim/3bfa1001fe6f3e1721e8eb15a171abf8/external/local_config_tf/BUILD:3823:1: target 'libtensorflow_framework.so' is both a rule and a file; please choose another name for the rule
INFO: Analyzed 4 targets (0 packages loaded, 0 targets configured).
INFO: Found 3 targets and 1 test target...
FAIL: //waymo_open_dataset/metrics/ops:detection_metrics_ops_test (see /home/sarim/.cache/bazel/_bazel_sarim/3bfa1001fe6f3e1721e8eb15a171abf8/execroot/__main__/bazel-out/k8-opt/testlogs/waymo_open_dataset/metrics/ops/detection_metrics_ops_test/test.log)
INFO: Elapsed time: 2.220s, Critical Path: 2.07s
INFO: 2 processes: 2 linux-sandbox.
INFO: Build completed, 1 test FAILED, 2 total actions
//waymo_open_dataset/metrics/ops:detection_metrics_ops_test              FAILED in 2.0s
  /home/sarim/.cache/bazel/_bazel_sarim/3bfa1001fe6f3e1721e8eb15a171abf8/execroot/__main__/bazel-out/k8-opt/testlogs/waymo_open_dataset/metrics/ops/detection_metrics_ops_test/test.log

INFO: Build completed, 1 test FAILED, 2 total actions

Please tell me how to fix this, thanks!

Unable to plot 3D bounding box

Hello, I am using the following code to plot 3D bounding boxes in image plane:

    box_3d = []
    center = label_info['bbox3d_loc']
    dims = label_info['bbox3d_dim']
    rot_y = label_info['rot_y']

    for i in [1, -1]:
        for j in [1, -1]:
            for k in [0, 1]:
                point = np.copy(center)
                point[1] = center[1] + i * dims[2] / 2 * np.cos(-rot_y + np.pi / 2) + (j * i) * dims[1] / 2 * np.cos(
                        -rot_y)
                point[0] = center[0] + i * dims[2] / 2 * np.sin(-rot_y + np.pi / 2) + (j * i) * dims[1] / 2 * np.sin(
                        -rot_y)
                point[2] = center[2] - k * dims[0]

                point = np.append(point, 1)
                point = np.dot(label_info['cam_to_img'], point)
                point = point[:2] / point[2]
                point = point.astype(np.int16)
                box_3d.append(point)
    point = np.copy(center)
    point[0] = center[0]
    point[2] = center[2]
    point[1] = center[1]
    point = np.append(point, 1)
    point = np.dot(label_info['cam_to_img'], point)
    point = point[:2] / point[2]
    point = point.astype(np.int16)
    box_3d.append(point)
    for i in range(4):
        point_1_ = box_3d[2 * i]
        point_2_ = box_3d[2 * i + 1]
        cv2.line(img, (point_1_[0], point_1_[1]), (point_2_[0], point_2_[1]), colors['pink'], 1)

    for i in range(8):
        point_1_ = box_3d[i]
        point_2_ = box_3d[(i + 2) % 8]
        cv2.line(img, (point_1_[0], point_1_[1]), (point_2_[0], point_2_[1]), colors['pink'], 1)
    cv2.circle(img, (box_3d[-1][0], box_3d[-1][1]), 10, colors['red'], -1)

However, the bounding boxes are displaced up and slightly to the left or right. Please let me know what I can do here. I am using the P0 intrinsic matrix (label_info['cam_to_img'])

do not use this repo ?

Hey all,

please do not use this repo. conversion is entirely wrong. not just in ground truth, but also in lots of cases.

Wrong lidar data transformation?

Hi @Yao-Shao ! Thank you for your very helpful tool! I have a problem with converted lidar data. Apparently, the data from some lidars wasn't been converted into vehicle coordinate system. 3D visualization shows it: