Giter Club home page Giter Club logo

dipoorlet's Issues

如何解决"CUDA out of memory"

我的硬件是单张RTX 3050,使用指令
python -m torch.distributed.launch --use_env -m dipoorlet -I dipoorlet_work_dir/ -N 1000 -D trt -M models/mobilev2_model.onnx -A mse -O dipoorlet_brecq/ --brecq
执行模型量化,产生了CUDA out of memory的运行报错。我检查了所有可以使用的命令行参数,没有发现可以调整数据加载批次的命令,请问有什么手段可以消除这个报错吗?

校准数据

你好,请问校准数据是什么格式,就是0.bin的格式,需要用图片转换到特定格式吗?

Run adaround failed with a toy onnx model

when i try to quant a model with adaround. But below error occurs:

onnxruntime.capi.onnxruntime_pybind11_state.InvalidGraph: [ONNXRuntimeError] : 10 : INVALID_GRAPH : This is an invalid model. Type Error: Type 'tensor(uint8)' of input parameter (input) of operator (QuantizeLinear) in node (QuantizeLinear0) is invalid.

image

精度损失大

你好,使用默认minmax校准量化转换后的模型,目标检测mAP降低10个点,这是为什么

针对rv1126芯片量化mobilenet-0.25 ,得到的quant_model.onnx,使用rknntoolkit-1.7.1加载失败

按照example里关于rv平台的量化示例,对mobilenet模型进行量化,能正常的到量化后的onnx模型,但是用rknn-toolkit转换失败
企业微信截图_1688605910143
看报错信息,应该是官方不支持gemm量化后的算子,我查看了官方rknntoolkit仓库里关于加载量化模型的示例,发现瑞芯微官方提供的shufflenet模型最后的gemm前后确实也没有加quant/dequant op
https://github.com/rockchip-linux/rknn-toolkit/tree/master/examples/common_function_demos/load_quantized_model/onnx
企业微信截图_1688606204587

quant_model.zip
mobilenet.zip

dipoorlet 支持动态输入吗

在试用dipoorlet PTQ量化 torch 导出的onnx模型时报错: ValueError: cannot reshape array of size 172800 into shape (0,0,3,180,320)。

  1. torch.onnx.export() 导出时指定了dynamic_axes, 具体如下:
torch.onnx.export(
        model, # torch model
        dummy_input, # random dummy input
        onnx_path, # save path of onnx format model
        export_params=True, # export all params
        verbose=True, # enable debug message
        training=torch.onnx.TrainingMode.EVAL, # export the model in inference mode
        input_names=input_names, # names to assign to input nodes of computation graph
        output_names=output_names, # names to assign to output nodes of computation graph
        opset_version=16, # version of opset
        # dynamic axes setting for dynamic input/output shapes
        dynamic_axes={
            "LR_bins":{0: "batch_size", 1:"temporal_dim"},
            "HR":{0: "batch_size", 1:"temporal_dim"}
        }
  1. 使用dipoorlet量化时具体报错如下:
root@autodl-container-032d11993c-d711a821:~/autodl-tmp/Dipoorlet_Examples# sh verification_trial.sh
[2023-11-07 14:59:14 dipoorlet](__main__.py 118): INFO Do tensor calibration...
Minmax update: 0it [00:00, ?it/s]
Traceback (most recent call last):
  File "/root/miniconda3/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/root/miniconda3/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/root/autodl-tmp/Dipoorlet/dipoorlet/__main__.py", line 119, in <module>
    act_clip_val, weight_clip_val = tensor_calibration(onnx_graph, args)
  File "/root/autodl-tmp/Dipoorlet/dipoorlet/tensor_cali/tensor_cali_base.py", line 6, in tensor_calibration
    act_clip_val = tensor_cali_dispatcher(args.act_quant, onnx_graph, args)
  File "/root/autodl-tmp/Dipoorlet/dipoorlet/utils.py", line 297, in wrapper
    return dispatch(args[0])(*(args[1:]), **kw)
  File "/root/autodl-tmp/Dipoorlet/dipoorlet/tensor_cali/basic_algorithm.py", line 18, in find_clip_val_minmax
    stats_min_max = forward_get_minmax(onnx_graph, args)
  File "/root/autodl-tmp/Dipoorlet/dipoorlet/forward_net.py", line 215, in forward_get_minmax
    ort_inputs[name] = data[name][:].reshape(onnx_graph.get_tensor_shape(name))
ValueError: cannot reshape array of size 172800 into shape (0,0,3,180,320)

关于MSE准则迭代求解最优scale的疑问

你好,在函数forward_net_octav中有如下mse准则下迭代求解最优scale的代码:

abs_x = np.abs(ort_inputs[i])
s_n = abs_x.sum() / abs_x[abs_x > 0].size
for _ in range(20):
    s_n_plus_1 = abs_x[abs_x > s_n].sum() / \
               (1 / (4 ** 8) / 3 / unsigned * abs_x[abs_x <= s_n].size + abs_x[abs_x > s_n].size)
    if np.abs(s_n_plus_1 - s_n) < 1e-6:
        break
    s_n = s_n_plus_1

想请问下这里

 s_n_plus_1 = abs_x[abs_x > s_n].sum() / \
               (1 / (4 ** 8) / 3 / unsigned * abs_x[abs_x <= s_n].size + abs_x[abs_x > s_n].size)

迭代更新scale公式的物理含义是什么呢?是如何推导得到的呢?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.