How to add empty input to a node about optimizer HOT 8 CLOSED

@wjj19950828 Great, I will debug it sooner

from optimizer.

@wjj19950828

auto* empty_node = g->create(kUndefined, 1);
empty_node->output()->setUniqueName(""); // empty_node->output() is a `Value` that represents a empty tensor 

empty_node->insertBefore(node); // insert empty_node to node lists that should be topo sort

node->addInput(empty_node->output())

Are you able to use the above codes to verifying again ?

from optimizer.

@daquexian Do you have any good suggestions～

from optimizer.

@wjj19950828 you can try it following

Tensor empty_tensor;
empty_tensor.sizes().push_back(0); // make the tensor dims to [0]
empty_tensor.elem_type() =  ONNX_NAMESPACE::TensorProto_DataType_FLOAT ;  // you should modify it  as needed
Value *empty_value = graph.addInitializerAndCreateValue(empty_tensor);  // create a value object

node->addInput(empty_value)  // add input to node in order

from optimizer.

@HSQ79815 thank you for your reply～

my code as follow

Tensor mask_index, past;
 
mask_index.sizes().push_back(0); // Add mask_index empty node
mask_index.elem_type() = TensorProto_DataType_INT32;
past.sizes().push_back(0); // Add mask_index empty node
past.elem_type() = TensorProto_DataType_FLOAT;
Value *attention_inputs_mask_index = graph.addInitializerAndCreateValue(mask_index);
Value *attention_inputs_past = graph.addInitializerAndCreateValue(past);

attention_node->addInput(attention_inputs_mask_index);
attention_node->addInput(attention_inputs_past);

The generated Attention Node is as shown below

But when I run forward with ORT, the following error will be reported

The specific reason is because the code here causes

It means that past tensor does not consider it to be nullptr in ORT

But in the following Attention Node ORT can run

Do you know how I should modify,thanks a lot~

from optimizer.

@wjj19950828 are you able to share your model ? I try to reproduce

from optimizer.

@HSQ79815 The reproduction script and the onnx model are at the following link

Attention.onnx is the onnx model of a single Attention node generated by a python script, which can be run by onnx_run.py

ner_model_test_0706.onnxis the onnx model I generated by adding the attention fuse pass, the empty node is generated by the above code, but running ORT will report the above error

链接: https://pan.baidu.com/s/1bN-1VgJ9I8CxpHfDQxrQhg?pwd=mwhh 提取码: mwhh

感谢～

from optimizer.

@HSQ79815 thanks a lot~

The above code solved my problem

from optimizer.