weikaitan / toronto-3d Goto Github PK

A Large-scale Mobile LiDAR Dataset for Semantic Segmentation of Urban Roadways

point-clouds dataset deep-learning lidar semanticsegmentation 3d point-cloud-segmentation point-cloud-dataset

toronto-3d's Introduction

Toronto-3D: A Large-scale Mobile LiDAR Dataset for Semantic Segmentation of Urban Roadways

[Paper] [Download] [Results] [Codes]

Toronto-3D is a large-scale urban outdoor point cloud dataset acquired by an MLS system in Toronto, Canada for semantic segmentation. This dataset covers approximately 1 km of road and consists of about 78.3 million points. Here is an overview of the dataset and the tiles. The approximate location of the dataset is at (43.726, -79.417).

Point clouds has 10 attributes and classified in 8 labelled object classes. There is a data preparation tip to handle UTM coordinates to avoid problems. There are also some known issues.

Details on the dataset can be found at CVPRW2020. Revisions on the labels will lead to different results from the published paper, and updated results will be updated here.

If you have questions, or any suggestions to help us improve the dataset, please contact Weikai Tan.

Semantic segmentation results (%)

More results to be added

Default: point coordinates only

Method	OA	mIoU	Road	Road mrk.	Natural	Bldg	Util. line	Pole	Car	Fence
PointNet++	84.88	41.81	89.27	0.00	69.0	54.1	43.7	23.3	52.0	3.0
PointNet++ MSG	92.56	59.47	92.90	0.00	86.13	82.15	60.96	62.81	76.41	14.43
PointNet++ *	91.66	58.01	92.71	7.68	84.30	81.83	67.44	63.30	60.92	5.92
DGCNN	94.24	61.79	93.88	0.00	91.25	80.39	62.40	62.32	88.26	15.81
KPFCNN	95.39	69.11	94.62	0.06	96.07	91.51	87.68	81.56	85.66	15.72
MS-PCNN	90.03	65.89	93.84	3.83	93.46	82.59	67.80	71.95	91.12	22.50
TGNet	94.08	61.34	93.54	0.00	90.83	81.57	65.26	62.98	88.73	7.85
MS-TGNet	95.71	70.50	94.41	17.19	95.72	88.83	76.01	73.97	94.24	23.64
RandLA-Net Hu, et al., 2021	92.95	77.71	94.61	42.62	96.89	93.01	86.51	78.07	92.85	37.12
Rim et al., 2021	72.55	66.87	92.74	14.75	88.66	93.52	81.03	67.71	39.65	56.90
MappingConvSeg Yan, et al., 2021	93.17	77.57	95.02	39.27	96.77	93.32	86.37	79.11	89.81	40.89
Use RGB
RandLA-Net Hu, et al., 2021 (RGB)	94.37	81.77	96.69	64.21	96.92	94.24	88.06	77.84	93.37	42.86
Rim et al., 2021 (RGB)	83.60	71.03	92.84	27.43	89.90	95.27	85.59	74.50	44.41	58.30
MappingConvSeg Yan, et al., 2021	94.72	82.89	97.15	67.87	97.55	93.75	86.88	82.12	93.72	44.11
Others
Han et al., 2021 (Intensity + Normal)	93.60	70.80	92.20	53.80	92.80	86.00	72.20	72.50	75.70	21.20

* use same radii and k as TGNet

Codes for training your own network

Code for RandLA-Net
Code for KPFCNN updated thanks to @Yarroudh

Point cloud attributes

XYZ
RGB
Intensity
GPS time
Scan angle rank

Classes

Road (label 1)
Road marking (label 2)
Natural (label 3)
Building (label 4)
Utility line (label 5)
Pole (label 6)
Car (label 7)
Fence (label 8)
unclassified (label 0)

Data preparation tip

The XY coordinates are stored in UTM format. The Y coordinate may exceed decimal digits in float type commonly used in point cloud processing algorithms. Directly read and process the coordinates could result in loss of detail and wrong geometric features.

I set a UTM_OFFSET = [627285, 4841948, 0] to subtract from the raw coordinates. You may use any other numbers to reduce number of digits.

Example of potential issues during grid_subsampling operation used in KPConv and RandLA-Net: both subsampled to grid size 6cm

without offset	with offset

Known issues

Point RGB assignments on taller vehicles.

Point RGB artifact assignments on moving vehicles.

Point acquisition on moving vehicles.

Download

Dataset can be downloaded at OneDrive or 百度网盘(提取码：aewp). Check Changelog for changes.

Toronto-3D belongs to Mobile Sensing and Geodata Science Lab, University of Waterloo. Toronto-3D is distributed under the CC BY-NC 4.0 License

Citation

Please consider citing our work:

@inproceedings{tan2020toronto3d,
    title={{Toronto-3D}: A large-scale mobile lidar dataset for semantic segmentation of urban roadways},
    author={Tan, Weikai and Qin, Nannan and Ma, Lingfei and Li, Ying and Du, Jing and Cai, Guorong and Yang, Ke and Li, Jonathan},
    booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops},
    pages={202--203},
    year={2020}
}

Acknowledgements

Teledyne Optech is acknowledged for providing mobile LiDAR point cloud data collected by Maverick. Thanks Jing Du and Dr. Guorong Cai from Jimei University for point cloud labelling.

Thanks Intel ISL for including our dataset in the Open3D-ML 3D Machine Learning module.

Changelog

[2023-02-07] Added code for RandLA-Net
[2020-04-23] Uploaded newest version. Fixed some labelling errors. Major revision on cars.
[2020-03-22] Uploaded newest version.

toronto-3d's People

Contributors

Stargazers

Watchers

toronto-3d's Issues

about for car labels

Hi @WeikaiTan, thanks for your nice work. Could you please share the car labels data you used in the study with me? Thanks you so much.

RandLA-Net weights

Hello @WeikaiTan,
Is it possible to share with as the pre-trained model on RandLA-Net? I'm trying to train the model on data but I get weak mIoU (28%) using your code. I kept the same parameters.

About Visualization

Hello, how do I visualize segmentation results?

Found 'nan' value in the label of the data

Hi @WeikaiTan, thanks for your efforts in this area!
However, I just found there is a series of nan value in the label of the data, can you please check the label value of the dataset?

I just use np.unique(labels) and found the following results:

Many thanks!

Example DataLoader

Hi @WeikaiTan

Thank you for this great dataset! I wonder if there are any examples for loading and using the dataset?

Best,
Yongcheng

Labeling tool for Toronto3D

Hi, thanks for sharing the great work. I appreciate it. Can you please tell me which labeling tool you have used for preparing the segmentation annotations for this dataset? Thank you.

Could you provide a dataloader.py for the Toronto3D dataset?

Hi @WeikaiTan

Could you provide a dataloader.py for the Toronto3D dataset? I want to see the specific data format to suit for the semantic segmentation baseline. I'm testing my algorithm on Toronto3D dataset, but can't load the data successfully. Or, could you please provide the code of RandLA-Net on Toronto3D as you shown in the table?

In addition, have you ever run the Open3D-ML framework for Toronto3D dataset? I have run RandLANet on Toronto3D dataset, but I only get half the mIoU with TF version as reported. The pytorch version always show can't pickle bugs due to the torch.multiprocessing module.

Thanks a lot in advance.

Xiaobing Han

About the evaluation metrics

Hi,

Thanks for your nice work. However, the evaluation metrics of OA may be confusing. OA should be the ratio of the number of correctly predicted points to the total number of points, right?

classification error

Hi, thank you for providing this excellent dataset. I noticed that there are several errors in the labeling. I have attached an example in which the floor above the cars is classified in the same floor class. Does this affect the results or not very much?

Making a standardized dataset for torch points3d

I would like to congratulate you for assembling this dataset. Like other semantic3d datasets that are available for analysis like S3DIS, Scannet etc.. in torch opints3d can we make this one similar to that? Please let mek now what you think. for your reference here is a link https://torch-points3d.readthedocs.io/en/latest/src/api/datasets.html

LICENSE

Could you please specify the license of the dataset? Thanks

About the Class Label

Hi @WeikaiTan ,thanks for your nice work. I found that the legend of class labels in Figure 1 exits some problems in your papar. The Pole class is missing and that results the Car and Fence get a wrong color-to-label.

求预训练模型

作者您好，请问能否提供一下toronto-3d数据集训练randlanet的预训练模型吗我的邮箱[email protected]

您好，请问是否能提供KPFconv和PointNet++的预训练模型

您好，请问目前公开的准确率的模型是否能提供预训练模型，我自己测试的KPFconv以及pointnet++并不能达到上述准确度，目前测试的准确度和链接论文所获取的结果相似。链接

Bug in L003.ply

Hi
I found some bug in L003.ply when I use plyfile to read L003.ply. much intensity of some points are NaN。 Codes are follows:

plydata = PlyData.read(filename)
pc = plydata['vertex'].data
pc_array = np.array([i.tolist() for i in pc ])

about evalutation tools

Hi, @WeikaiTan ,

Thanks for releasing such a promising dataset. Besides the dataset, could you also release the evalutation tools for segmentation task? It's essential for building a complete benchmark .

THX!

Question about the Rand-LA net performance on your dataset

Hi, I noticed that you reported the RandLA-Net performance on the Toronto-3D dataset, could you please release the pre-trained model and evaluation code for RandLA-Net?