Comments (1)
Hi, thanks for raising the issue.
First of all, the fusion rules in fusion_rules.json
indicates whether two ops will be fused in the corresponding device. When there comes a network for latency prediction, nn-meter will do: 1. parse the network to get the network structure, and convert the network architecture to nn-meter ir graph; 2. detect all kernels based on fusion_rules.json
of the predictor, for example, if "bn_relu" and "conv_bn" both obey the fusion ("BF_bn_relu": {"obey": true}, "BF_conv_bn": {"obey": true}), then all "conv + bn + relu" in the network will be fused to a kernel "conv-bn-relu"; 3. go through the network after kernel detection, and predict kernel-wise latency based on the kernel configs.
In your mentioned fusion rules, "CBC" and "RBC" are legacy test cases, so that there is no need to test these two cases when building your own predictor. Furthermore, nn-Meter building tools also removed test cases of fusing three ops such as "conv_bn_relu" and "dwconv_bn_relu", since the kernel "conv_bn_relu" will be detected by "conv_bn" and "bn_relu".
As for your second question, the detected kernel is based on fusion rules but not equal to fusion rules. As I mentioned before, a detected kernel of "conv_bn_relu" should based on two fusion rules, "conv_bn" and "bn_relu". In the released CPU predictor "cortexA76cpu_tflite21", we have tested all fusion rules and summarized all detected kernels in this page.
The third question is about the default predictorbuild_config.yaml
. Here we only listed some common kernel for prediction. However, the kernel relies on both the inference framework and the hardware, as both of them impact the fusion rules. We suggest detect fusion rules for your own backend, and then determine all kernels by fusion rule detection results.
The last question about whether to merge some kernels into one predictor, my suggestion is to take a look at the profiling latency. According to our experience, the latency of concat/reshape/add/relu are relatively small when compared with conv/dwconv/fc op (which we call it a dominant op). If you found two kernels with the same dominant op has similar latency results, you could consider to merge the two kernels in one predictor.
from nn-meter.
Related Issues (20)
- Why removed nodes are not removed from inbound
- can nn-meter be used for pytorch/tensorflow on the server device? HOT 2
- Comparing string to list
- Can you explain why stride[::3] has to be [1, 1]?
- Do we need self._fetch_connections(kernels)?
- Why used [kernel_extent_w, kernel_extent_h] as k_size?
- Why set output_shape to 0?
- Used get_h to calculate outw...Is this a typo HOT 1
- self.tmp_dir not found HOT 1
- How did you measure FLOPS of different CPU and GPU in the baseline comparison experiment? HOT 1
- Using NNI with nn-Meter HOT 4
- how to predict model generated from keras model.save()? HOT 2
- Can you provide the specific time and accuracy of each epoch of tflite_cpu training the conv-bn-relu kernel? HOT 3
- Question on Padding Calculation for Building PyTorch Networks HOT 3
- KeyError in graph_tool
- Error in Prediction of certain Model Families HOT 2
- in the next two rounds, the accuracy rate drops HOT 3
- How can I get the models in datasets.zip?
- NotImplementedError
- How to use PyTorch or TensorFlow to check the predicted value HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nn-meter.