cguangyan-bit / pointgpt Goto Github PK

Here are the different PointGPT-S models:

PointGPT-S Models

I thought all the classification models were trasfered to the ScanObjectNN dataset then evaluated against {OBJ_BG, OBJ_ONLY, PB_T50_RS, ModelNet40-1k, ModelNet40-8k}. So why is there five different links for classification models? Shouldn't there be only one link because they were all transfer learned on ScanObjectNN?

Please let me know where I have misunderstood. Thanks.

Hello, when can the code be released? I really want to learn about it

Hello! I am fascinated by your great idea and have been experimenting with your code, but I found that there might be some problems with your function of multicard finetuning:
if "--launcher" is set to none and set two or more GPUs like CUDA_VISIBLE_DEVICES=0,1, NaN problems will occur in the first epoch"NaN or Inf found in input tensor"
if "--launcher" is set to "pytorch", errors about environmental variables like "RANK" not defined or "WORLD_SIZE" not define will be raised. In the corresponding block, I found a "TO DO"

Have you met the problem when doing the experiment yourselves? Please tell me how it shall be solved, and how tour DDP can be used? Thanks!

Hi,

Thanks for the great work.
I pretrained the network on ShapeNet, but only got a loss of around 30%, is this normal? I got a loss of 2% when I pretrained with pointMAE.
Also, I tried to finetune on ModelNet, but got an error as followed:
Traceback (most recent call last):
File "main.py", line 95, in
main()
File "main.py", line 89, in main
finetune(args, config, train_writer, val_writer)
File "/home/pointGPT/PointGPT/tools/runner_finetune.py", line 176, in run_net
losses.update([torch.ones_like(loss1).item()])
ValueError: only one element tensors can be converted to Python scalars

I'm just wondering what i'm missing here. Thanks a lot!

Thanks for sharing the paper and code. It's a great work.

I notice the tasks PointGPT conducted are relatively easy because they focus on the 3D objects. Have you tried some scene-level 3D tasks, such as object detection and sematic segmentation? I wonder the performances of PointGPT on these tasks.

Thanks

Hi,

Which command should be used to start the pretraining on multiple GPUs? I guessed from looking at the code, that the parameter --launcher pytorch needs to be set. Is there something else to set like environment variables etc?

Best,
Hugues

Thanks for the great work.
I have two questions about the post-pretraining.

1. I notice the current codebase does not suppport the LabeledHybrid dataset. Could you please give more description on this dataset and how do you do you this dataset for the post-pretraining (e.g., the loss design)?

2. Without the post-pretraining, how much performance will drop for the PointGPT-B and PointGPT-L?

• How long did it take to do the auto-regressive SSL pretraining and on what hardware?
• How long did it take to do the post-pretraining and on what hardware?

Thank you.

Hello, I'm new in the field of analysis of Point Cloud.

And I wonder how I should organize the structure of dataset UHD and LHD after downloading the zip file you have provided.

When I only unzip HybridDatasets and put it in ./data/HybridDatasets and then start to pretrain ,it show the error that"FileNotFoundError: UnlabeledHybrid: [Errno 2] No such file or directory: 'data/HybridDatasets/pretrain/train.txt'".

Wish for response!

Hi @CGuangyan-BIT ,

Thanks for your great work! The performance of PointGPT is really impressive. May I ask when will the PointGPT-B/L model be released?

Hi,
When I load the trained model from your link. The accuracy is not consistent with the accuracy you report. For example, in Small Point GPT, you report 94% acc. However, I got 93.27% with the processed test data. Can you explain this problem?
thanks

Hello,

Thank you so much for your work on PointGPT and for making the pretrained models availble! I was doing some experiments with your work, and I noticed that the download links for the pretrained and post-pretrained PointGPT-B models are the same (both are linked to the pretrained.pth file), so I just wanted to let you know about that :)

Hi Thanks for released the code
I wonder know how to fine tune ScanObjectNN

I haved Pre-train on ShapeNet
and used CUDA_VISIBLE_DEVICES= python main.py --config cfgs/finetune_scan_hardest.yaml
--finetune_model --exp_name <output_file_name> --ckpts <path/to/pre-trained/model>

But different class number ShapeNet is 40 and ScanObjectNN is 15
and num_group are different Pre-train is 64, but fine tune ScanObjectNN is 128

Thanks alot~

error is below

size mismatch for blocks.cls_head_finetune.8.weight: copying a param with shape torch.Size([40, 256]) from checkpoint, the shape in current model is torch.Size([15, 256]).
size mismatch for blocks.cls_head_finetune.8.bias: copying a param with shape torch.Size([40]) from checkpoint, the shape in current model is torch.Size([15])

Hi, where can we get the visualization of the result?

Hi, thanks for the great work!
I tried to download the ShapeNetPart Datasets from https://shapenet.cs.stanford.edu/media/shapenetcore_partanno_segmentation_benchmark_v0_normal.zip
But I failed to download it.

Hi!
Please tell me how to classify using a model that has been fine-tuned.
Please tell me what to change.
Thank you.

in this file PointGPT-1.2\extensions\emd\cuda\emd_kernel.cu
I get a wrong "error: command 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin\nvcc.exe' failed with exit code 1" when I make

 cd ./extensions/emd
python setup.py install --user

but,I found a way through a random query
for example,I changed code

#ifndef _EMD_KERNEL
#define _EMD_KERNEL

#include <cmath>
#include <vector>

#include <ATen/ATen.h>
#include <ATen/cuda/CUDAApplyUtils.cuh>  // at::cuda::getApplyGrid
// #include <THC/THC.h>

#define CHECK_CUDA(x) TORCH_CHECK(x.type().is_cuda(), #x " must be a CUDA tensor")
#define CHECK_CONTIGUOUS(x) TORCH_CHECK(x.is_contiguous(), #x " must be contiguous")
#define CHECK_INPUT(x) CHECK_CUDA(x); CHECK_CONTIGUOUS(x)
#ifndef CHECK_EQ
#define CHECK_EQ(a, b) assert((a) == (b))
#endif

Because I'm still a rookie in code and deep learning,so I can't know whether I accurately do it or not(and my English is not very good,but I am really interested in your code,I would appreciate it if you could give me some advice,thank you very much )

In the paper you describe the relative direction prompt in equation 5 as $RDP_i = \textrm{PE}((\mathbf{C}_{i+1}^s - \mathbf{C}_i^s) / \parallel \mathbf{C} _{i+1}^s - \mathbf{C} _i^s \parallel_2), \quad i \in \{ 1, \dots, n' \}$.

However, in the code I cannot find such scaling factor and it looks like you actually just do $RDP_i = \textrm{PE}(\mathbf{C} _{i+1}^s - \mathbf{C} _i^s)$.

The relevant lines of code are https://github.com/CGuangyan-BIT/PointGPT/blob/a8bdf85ad469615b3e462a187474d36929a6bbfb/models/PointGPT.py#L380C1-L386C57 and https://github.com/CGuangyan-BIT/PointGPT/blob/a8bdf85ad469615b3e462a187474d36929a6bbfb/models/PointGPT.py#L642C1-L647C1

Does this mean you actually use a relative position prompt instead of a relative direction prompt? Is there a mistake in the paper, is there a mistake in the code, or am I missing something?