Database Class about kpconv-pytorch HOT 14 CLOSED

I just learned PyTorch while coding this repo, so I just followed tutorials on PyTorch website (here or here) and everytime I had a problem, I found a solution on stackoverflow or similar forums.

About the length, it is useless, the code never calls it, so you don't have to worry about it.

I would advise not to use iterable dataset as they are not very versatile. Prefer map style datasets and try to take inspiration from the dataset I already coded like S3DIS.

from kpconv-pytorch.

Hi @HuguesTHOMAS Thank you for your reply. I have one problem using the S3DIS as example, in it you have mentioned the data is small enough to be loaded into the memory. For me it is not so is it still feasible to use the same structure? Thanks.

from kpconv-pytorch.

If you have a lot of small files (like lidar scans or RGBD images), you can use SemanticKitti dataset as an example. Otherwise if you have large scenes it is going to be much more difficult for you because I did not code anything for that case.

I kinda thought of this problem and here is my advise on it. If you have a lot of very large scenes, you need to break down your dataset in small tiles (each tile in a separate file) so that you can load the points fast. Then when you need a new input cloud (lets imagine a sphere like I do in S3DIS), you need to compute which tiles your sphere intersect, load them, and then get the points in the sphere. This will not be simple to implement but it should be good to keep a fast training.

from kpconv-pytorch.

Okay, Mine is Lidar point cloud which have in total of 0.5 B points but it is tiled into smaller squares. I think I need to understand your S3DIS code more. Anyways, do you find using spherical neighbors highly impacting accuracy? I thought of making small square area with input points just more than ( let's say ) 4096 points and then reduce the size to 4096 to make it equal for all iterations. I read that (from the paper) you have implemented something pretty good to consider the variation in the input points. I am yet to find that snip of your code. Or is it such that I don't have to create a stable number of input points. The network would adapt to it?

Thank you so much for giving your time to reply my questions

from kpconv-pytorch.

Or is it such that I don't have to create a stable number of input points. The network would adapt to it?

Yes it is like that.

First the network does not need a constant number of point as input like Pointnet. The input number of points is variable. It is possible because we use "stacked" batch instead of regular ones. We don't have a batch with shape [B, N, ...] but we stack all clouds like this: [N1 + N2 + N3, ...], therefore N1, N2, and N3 do not have to be the same.

Second, to have a relatively stable memory consumption, the network automatically adapts the batch size depending on the size of the input clouds. Let's say you ask for batch_num = 8. The network is not going to always use 8 input clouds in each batch. Instead it will add clouds to the batch until it reaches a certain limit N_max. Sometimes the batch will contain 5 big clouds, and some other times it will contain 10 smaller clouds but in average it will be around 8. You can see this in the calibration function of S3DIS dataset:

from kpconv-pytorch.

@Arjun-NA Hello, has your problem been solved? I found the same dataset, but didn't know how to train the model?

from kpconv-pytorch.

@xdw-123 Hi , I used tensorflow version for now, in which NPM3d dataset code is easily convertible to this dataset. If I have any progress with pytorch I will update it soon here.

from kpconv-pytorch.

@xdw-123 Hi , I used tensorflow version for now, in which NPM3d dataset code is easily convertible to this dataset. If I have any progress with pytorch I will update it soon here.

Thank you for your reply. Do you mean that dataset DALES can be converted to this required dataset format through NPM3d dataset code?

from kpconv-pytorch.

@xdw-123 You can download the DALES in ply format which is available with the mail they send you

This location can be then set in the database file. no need of any other conversions.

from kpconv-pytorch.

@xdw-123 You can download the DALES in ply format which is available with the mail they send you

This location can be then set in the database file. no need of any other conversions.

Thank you for your reply. I'll try it right away.There are three formats for the same dataset in the pictures you send. Do you want only one?Is it necessary to have all three?

from kpconv-pytorch.

@Arjun-NA Hello, I tried running Python 3.5 training_NPM3D.py.Many problems have arisen.But most of them are read and write errors in ply files. Can you upload your modifications?

from kpconv-pytorch.

@xdw-123 sorry for the late reply.
I missed out that the ply file that they provide is in ascii format and you would need in binary format. I will upload the conversion file in here. I have added few lines in readme for you to use.
Please let me know if there's any errors with it.

from kpconv-pytorch.

Hello @Arjun-NA, @xdw-123 , I've managed to train RandLA-Net on DALES with this, I'm currently investigating what do I need to change in this dataset to fit KPConv. I'm assuming you could use it too for the DALES dataset.

Best Regards,

from kpconv-pytorch.

Hi @eyalol , I am not sure about the dataset RandLA-Net, please check with Dr. Hughes for more information. Thanks

from kpconv-pytorch.