Can I export the pruned model? about acdc HOT 5 CLOSED

Hi detectRecog, thank you :) During training, the best sparse and dense checkpoints, as well as the last checkpoint, are saved automatically, it will be wherever you set your --experiments_root_path to, along with all other artifacts of the run, something like my_experiment_root_path/run_type/run_name/timestamp/best_sparse_checkpoint.ckpt or my_experiment_root_path/run_type/run_name/timestamp/last_checkpoint.ckpt. Please feel free to reopen if this doesn't seem to be happening for you.

from acdc.

@ohaijen Hi Jen, thank you for your rapid response! I find the location of the 'best_sparse_checkpoint.ckpt' file and I can observe the notably high sparsity achieved by ACDC. However, I don't know how to export the pruned network (such as using torch.jit.trace) with removing zero-weight channels and even conv blocks to accelerate the inference speed on the edge device. Do you have any ideas?

from acdc.

hi @detectRecog, if you are looking for real speedups at an edge device, you might prefer a pruning approach that gives structured pruning (ie, removes entire filters). Since AC/DC is generally unstructured, you actually keep most filters (95% sparsity on ResNet50 removes something like 20% of filters completely). Would you like me to point you to some structured pruning resources?

from acdc.

hi @detectRecog, if you are looking for real speedups at an edge device, you might prefer a pruning approach that gives structured pruning (ie, removes entire filters). Since AC/DC is generally unstructured, you actually keep most filters (95% sparsity on ResNet50 removes something like 20% of filters completely). Would you like me to point you to some structured pruning resources?

Yes, you're so kind. Thank you so much. I also notice that deepsparse starts to offer libraries for running sparse models on CPU. It is a good beginning for deploying sparse models. I believe sparse models are very promising on edge inference.

from acdc.

hi @detectRecog, if you are looking for real speedups at an edge device, you might prefer a pruning approach that gives structured pruning (ie, removes entire filters). Since AC/DC is generally unstructured, you actually keep most filters (95% sparsity on ResNet50 removes something like 20% of filters completely). Would you like me to point you to some structured pruning resources?

could you recommend some structured pruning methods?

from acdc.