Hi.

I have a question regarding the input features of the network. You are using x, y, z, distance and intensity. Aren't the first four breaking the CNN advantage of translation invariance and could this maybe result in some memorizing of the locations of objects? Did you perhaps test it with intensity and distance only?
Also fusing with rgb would definitely be interesting.

i want to train a new network wothout the pretrained model , what should i do ?

I followed https://github.com/Durant35/SqueezeSeg/blob/master/src/nodes/segment_node.py by @Durant35. But the code seems something wrong, show as below.

the first one is visualized by opencv from your npy file, the second one is created by the code from durant, they are the same frame from kitti. but they looks something different. And with such preprocess, the training result is very bad. Could you open source the projection code? @BichenWuUCB

I am running SqueezeSeg using TF 1.5.1 using default training command on AWS EC2 p3.2xlarge and I got this error after a while from training (demo works fine thought):

2018-02-19 19:10:34.304462: step 150, loss = 2.75 (127.6 images/sec; 0.251 sec/batch)
2018-02-19 19:10:36.817802: step 160, loss = 1.65 (127.7 images/sec; 0.251 sec/batch)
2018-02-19 19:11:38.855599: W tensorflow/core/kernels/queue_base.cc:285] _0_fifo_queue: Skipping cancelled dequeue attempt with queue not closed
Timed out waiting for notification
Traceback (most recent call last):
  File "./src/train.py", line 223, in <module>
    tf.app.run()
  File "/home/ubuntu/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 124, in run
    _sys.exit(main(argv))
  File "./src/train.py", line 219, in main
    train()
  File "./src/train.py", line 212, in train
    coord.join(enq_threads)
  File "/home/ubuntu/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/training/coordinator.py", line 387, in join
    six.reraise(*self._exc_info_to_raise)
  File "/home/ubuntu/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/tensorflow/python/training/coordinator.py", line 295, in stop_on_exception
    yield
  File "./src/train.py", line 98, in enqueue
    weight_per_batch = imdb.read_batch()
  File "/home/ubuntu/SqueezeSeg/src/imdb/imdb.py", line 87, in read_batch
    record = np.load(self._lidar_2d_path_at(idx)).astype(np.float32, copy=False)
  File "/home/ubuntu/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/numpy/lib/npyio.py", line 421, in load
    pickle_kwargs=pickle_kwargs)
  File "/home/ubuntu/anaconda3/envs/tensorflow_p27/lib/python2.7/site-packages/numpy/lib/format.py", line 688, in read_array
    array.shape = shape
ValueError: cannot reshape array of size 187116 into shape (64,512,6)

Firstly, thank you very much for sharing your fantastic work.
My concern is about the pixel-wise features of the 2D projection: what does it happen when more than one point falls in a pixel? The obvious thing is computing the average, but if I'm not wrong, you don't address this issue in your paper. Thanks in advance.

Hi Sir,
I successfully run the demo.py file but while running train.py I am getting this error, could anyone will help here. Help is appreciated.
Attaching.txt file here for the error
train_terminal.txt

Also, I m getting this error while running eval.py
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcublas.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcudnn.so.5 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcufft.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:135] successfully opened CUDA library libcurand.so.8.0 locally
No checkpoint file found
Wait 60s for new checkpoints to be saved ...
No checkpoint file found
Wait 60s for new checkpoints to be saved ...
No checkpoint file found
Wait 60s for new checkpoints to be saved ...

Hi! I got some problem when I was downloading your data. It seems to be some problem with your link. Could your please share your data again? BTW, the 3D raw data is also needed. THX!

hello, all my friends,
in the code:
norm_lidar = (lidar - self._mc.INPUT_MEAN) / self._mc.INPUT_STD

x, y, z, intensity, distance

mc.INPUT_MEAN = np.array([[[10.88, 0.23, -1.04, 0.21, 12.12]]])
mc.INPUT_STD = np.array([[[11.47, 6.91, 0.86, 0.16, 12.32]]])

And I don't understand how the values(INPUT_MEAN and INPUT_STD) are from?

I opened this issue because I think that there is a typo in the repository title:
Implementation of SqueezeSeg, convolutional neural networks for LiDAR point clout segmentation

Hi, when I follow the code and do the evaluation, the result quite different with the result which shown in the paper, especially in pedestrian and cyclist. And I do not change any parameters. Is this reasonable?

I am using the model to seg more than 4 classes , I chane the NUM_CLASS. Would you please show me how to set the CLS_LOSS_WEIGHT、CLS_COLOR_MAP、BILATERAL_THETA_A、 BILATERAL_THETA_R in model config.

First, thanks for the great work! I pull this repo and successfully ran it with the data you provided.

Now, I'd like to hook up this one directly with Lidar to run a real-time test. However, it seems you already did some pre-process the Lidar data and when you feed you have:

• lidar_mask
• lidar_input

those two placeholders

I wonder how can I prepare those? Say I got a PointCloud2 from LiDAR. Any references?

Is there any way to run this project without Nvidia GPU (CPU only)?

Hello, I am trying to install the required modules to run the training.sh and I get an error when running

pip install -r requirements.txt

ERROR: Could not find a version that satisfies the requirement tensorflow-gpu==1.0.0 (from -r requirements.txt (line 5)) (from versions: none)
ERROR: No matching distribution found for tensorflow-gpu==1.0.0 (from -r requirements.txt (line 5))

Does anybody have the same issue ?
Bonus question : is it possible to use tensorflow instead of tensorflow-gpu ?

Thank you very much for your help !

@BichenWuUCB Hi thanks for the wonderful , i had few queries

1. can you please share the pre trained model ?
2. how long will it take to train on the kitti dataset
   thanks in advance

hello,
I am trying to project KITTI velodyne data to 2d sphere and having some questions.
I tried to read your .npy data file like
data = np.fromfile("2011_09_26_0001_0000000000.npy", dtype=np.float32)
and reshape it like
data = data.reshape((-1,5))
when I call data
it turns out like this
[ 2.2366853e+08, 1.2387478e-40, 3.4833497e-15, 4.4898648e+21, 1.5767864e-19],
[ 2.1442303e-07, 2.2228396e-15, 7.6830766e+31, 1.7177136e+19, 6.7425655e+22],
[ 1.5767864e-19, 1.8727951e+31, 2.2228527e-15, 2.7904159e+29, 1.5767847e-19]

Does each of them contain data like [x, y, z, sqrt(x^2+y^2+z^2), intensity] ?
I found x,y,z and sqrt(x^2+y^2+z^2) don't match.

Can someone share the training and validation data that contains LiDAR point-cloud projected to a 2D spherical surface. I cannot download it. Thank you very much.

Sorry, when i check the code. I don't know how to generate the ground truth from the raw data. Can u direct show me in which file?

While trying to run train.sh I encountered this error please do help me out in any way. Thanks!
Traceback (most recent call last):
File "/home/jashshah999/Downloads/SqueezeSeg/src/train.py", line 220, in
tf.app.run()
File "/home/jashshah999/venv/local/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 44, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "/home/jashshah999/Downloads/SqueezeSeg/src/train.py", line 216, in main
train()
File "/home/jashshah999/Downloads/SqueezeSeg/src/train.py", line 64, in train
imdb = kitti(FLAGS.image_set, FLAGS.data_path, mc)
File "/home/jashshah999/Downloads/SqueezeSeg/src/imdb/kitti.py", line 20, in init
self._image_idx = self._load_image_set_idx()
File "/home/jashshah999/Downloads/SqueezeSeg/src/imdb/kitti.py", line 33, in _load_image_set_idx
'File does not exist: {}'.format(image_set_file)
AssertionError: File does not exist: ImageSet/.txt

Also before this, I had to convert all the paths to absolute so anything that had ./ (SOMETHING) had to be changed to /home/jashshah999/Downloads/SqueezeSeg/(SOMETHING)

Please do help me out!
Thanks.

Hi,
I have a problem with using a dataset that is published by the author of SqueezeSeg.
In a paper, 8,057 are used for training and 2,791 are used for validation. However, the given dataset is not divided. The total 10,848 frames just are given. I wonder how to divide the total dataset into the training and validation set. Please help me !!

Thanks,

I am getting an error when trying to run demo.py. It says that there is no module named kitti_squeezeSeg_config. How do I obtain this module?
The exact error is as shown below:

Traceback (most recent call last):
File "demo.py", line 20, in
from config import *
File "/home/jashshah999/TensorFlow/venv/SqueezeSeg/src/config/init.py", line 1, in
from kitti_squeezeSeg_config import kitti_squeezeSeg_config
ImportError: No module named 'kitti_squeezeSeg_config'

Any help is appreciated
Thanks

Hi,
Thanks for your work. I want to try project the predicted 2D labels into the raw LiDar Point Cloud, such as your the first demonstration of SqueezeSeg. Is there any ways?

I'm experimenting some issues importing the pretrained model because I'm running the code with joblib 0.16 and the pickles were exported with 0.10, any idea?

Thanks for sharing your work!
I noticed that the data you provide is lidar_2d, which is projected from 3D lidar data. I am wondering whether other projection method would be better? Could anybody provide the 3D lidar data? Thanks!

Hi! I notice that the BATCH_SIZE is hard-coded to 1 and you will change that later. Have you fix this problem? If yes, would you update the code? If no, what should I do to fix that? Thanks!!!

It didn't end as I have rest for 7 days

When I run python demo.py I encounter this error:

.
.
.
  File "demo.py", line 54, in detect
    saver = tf.train.Saver(model.model_params)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1051, in __init__
    self.build()
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1081, in build
    restore_sequentially=self._restore_sequentially)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 675, in build
    restore_sequentially, reshape)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 402, in _AddRestoreOps
    tensors = self.restore_op(filename_tensor, saveable, preferred_shard)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 242, in restore_op
    [spec.tensor.dtype])[0])
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_io_ops.py", line 668, in restore_v2
    dtypes=dtypes, name=name)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 763, in apply_op
    op_def=op_def)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2395, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1264, in __init__
    self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): Unsuccessful TensorSliceReader constructor: Failed to get matching files on ./data/SqueezeSeg/model.ckpt-23000: Not found: ./data/SqueezeSeg
         [[Node: save/RestoreV2_5 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save/Const_0, save/RestoreV2_5/tensor_names, save/RestoreV2_5/shape_and_slices)]]

NOTE: I install tensorflow==1.0.0 instead of tensorflow-gpu==1.0.0 because I haven't NVIDIA, is this correct?

Hi,

I want to compare your method with another one. Could you share the instance segmentation evaluation code for equal comparison? Thank you

Is there any way to visualize Velodyne point cloud and this project detection/tracking marker in the Rviz?

if i use the typical freeze_graph.py method, it cannot produce the placeholders. I wonder if the fifoqueue causes this question. could you share the frozen script

Hi :),

I wanted to train the model. I followed all the steps described in the readme.md.
The demo script ran perfectly but for the training I always confront the same issue( it ran just for the first step and it stops after) .
./scripts/train.sh -gpu 0 -image_set train -log_dir ./log/
Shape of the pretrained parameter of conv1 does not match, use randomly initialized parameter
Cannot find conv1_skip in the pretrained model. Use randomly initialized parameters
Cannot find fire_deconv10/squeeze1x1 in the pretrained model. Use randomly initialized parameters
Cannot find fire_deconv10/expand1x1 in the pretrained model. Use randomly initialized parameters
Cannot find fire_deconv10/expand1x1 in the pretrained model. Use randomly initialized parameters
Cannot find fire_deconv10/expand3x3 in the pretrained model. Use randomly initialized parameters
Cannot find fire_deconv11/squeeze1x1 in the pretrained model. Use randomly initialized parameters
Cannot find fire_deconv11/expand1x1 in the pretrained model. Use randomly initialized parameters
Cannot find fire_deconv11/expand3x3 in the pretrained model. Use randomly initialized parameters
Cannot find fire_deconv12/squeeze1x1 in the pretrained model. Use randomly initialized parameters
Cannot find fire_deconv12/expand1x1 in the pretrained model. Use randomly initialized parameters
Cannot find fire_deconv12/expand3x3 in the pretrained model. Use randomly initialized parameters
Cannot find fire_deconv13/squeeze1x1 in the pretrained model. Use randomly initialized parameters
Cannot find fire_deconv13/expand1x1 in the pretrained model. Use randomly initialized parameters
Cannot find fire_deconv13/expand3x3 in the pretrained model. Use randomly initialized parameters
Cannot find conv14_prob in the pretrained model. Use randomly initialized parameters
WARNING:tensorflow:From /home/rahul/$SQSG_ROOT/src/nn_skeleton.py:736: calling softmax (from tensorflow.python.ops.nn_ops) with dim is deprecated and will be removed in a future version.
Instructions for updating:
dim is deprecated, use axis instead
Model statistics saved to ./log///train/model_metrics.txt.
WARNING:tensorflow:From ./src/train.py:109: all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02.
Instructions for updating:
Please use tf.global_variables instead.
WARNING:tensorflow:From /home/rahul/anaconda2/lib/python2.7/site-packages/tensorflow/python/util/tf_should_use.py:189: initialize_all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02.
Instructions for updating:
Use tf.global_variables_initializer instead.
2019-01-07 11:09:21.153978: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-01-07 11:09:34.158950: step 0, loss = 2.98 (2.6 images/sec; 12.406 sec/batch) terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc
./scripts/train.sh: line 73: 9311 Aborted (core dumped) python ./src/train.py --dataset=KITTI --pretrained_model_path=./data/SqueezeNet/squeezenet_v1.1.pkl --data_path=./data/ --image_set=$IMAGE_SET --train_dir="$logdir/train" --net=$NET --max_steps=$STEPS --summary_step=100 --checkpoint_step=1000 --gpu=$GPUID

Does anyone have any idea how to solve this issue ?

Thank you :)

The sentence is constantly like this:
Wait 60s for new checkpoints to be saved ...
Wait 60s for new checkpoints to be saved ...
Wait 60s for new checkpoints to be saved ...

I am a newbie with Ubuntu but I tried searching Google for this, I'm confused on what I'm supposed to do after cloning the repository here.

How am I supposed to rename the root directory as $SQSG_ROOT? What does that even mean?

I'm researching online too obviously but could somebody link some documentation or something to explain this? Thanks!

Hi,Thank you very much for sharing the code! Can I predict the point cloud data generated by single-line radar or 16-line radar? I am very curious about this issue.

Hi
since the code you shared train the model only on the front view of 90, how to train the net on Lidar data which is the full view of 360? Can you give me some advice ? thanks .

In your paper it is pointed that for an input tensor of HxWxC, the squeeze layer of the fire module has C/4 filters, and the expand layers have both C/2 filters, but in the code you are setting:
e1x1 = e3x3 = 4s1x1

With this sizes, the fire module does not preserve the equality between the input size and the output size.

Thx!

How would BILATERAL_THETA_A, BILATERAL_THETA_R and ANG_THETA_A vary for datasets with more/different labels?

python src/demo.py
I am encountering the following error while running demo.py. due to same name. Can you tell the way out?

File "src/demo.py", line 20, in <module> from config import * File "./SqueezeSeg/src/config/__init__.py", line 1, in <module> from config.kitti_squeezeSeg_config import kitti_squeezeSeg_config File "./SqueezeSeg/src/config/kitti_squeezeSeg_config.py", line 7, in <module> from config import base_model_config ImportError: cannot import name 'base_model_config'

Hi, thanks for sharing your code! Are you planning to upload also the spherical projection part (described in your paper), in order to convert data from point cloud to projection images?

Hello,

Thanks for sharing such a nice code in advance :-)

I trained SqueezeNet model using lidar_2d data and evaluated it and saved pred_....npy files like demo.py

but I don't know how to visualize it. I just got a lot of .npy files on my computer.

In your YouTube Video, there is visualizing tool which can visualize KITTI dataset and your .npy dataset at the same time. but I don't think https://github.com/MarkMuth/QtKittiVisualizer will work for me.

will you plan to share this visualization tool in the future?
Thank you

I see the numpy array has 6 features, can you please confirm if its X,Y,Z, Vertical Angle, Azimuth Angle? and its in this order? We have lidar data of our own and like to run this model on it.

i got the solution
give rwx permission to SqueezeSeg-master folder

chmod -R 777 SqueezeSeg-master

Originally posted by @praveenkurabarkumar in #33 (comment)

Is there a way to get a visualization of the segmented data without Rviz? Something similar to the first GIF on the Readme:

I'm curious about during the data processing, why is 512 used to set W. It makes sense to set H=64 since it is 64 beams LiDAR, but how is 512 obtained for W? I recall 0.09° is the horizontal resolution and you picked 90° as the front view. Is 512 empirical for making projected images look like the real camera images?
Thanks!

I want to test this net on my own lidar data. Could you give me the source code which is used to convert to input_data?

Hello again,
Which criterion have you followed in order to balance the dataset? You have manually set the CLASS_LOSS_WEIGH, i'm guessing how. I'm thinking in taking the ratios of number of points per class among all labeled points, but this weights will sum up to 1. For example, if there are 100 labeled points and 33 belong to 'car', the weight for 'car' is 0.33.
Thank you very much!

Hi and Thank you for your work.
I am now testing it with GTX 1070(8GB) and having a problem of out of memory while training( with tensorflow-gpu 1.0 ). Should I modify the config files? Would you guys please give me some advices? Thanks and looking forward to your reply.