Comments (8)
Hi @Ramsonjehu We plan to release the quantize predictor after submitting our research paper, in about October or later. 😃
from nn-meter.
Hi, Ramson,
Thanks for your interest! nn-Meter 2.0 provides the building tools for users to build latency predictors for custom devices. You can download and try the latest main branch. Currently, we support tflite/openvino backends. Maybe you can try to connect the tflite micro framework via the nn-Meter backend interface doc. Note that you need to implement the followings: (i) set up the device and the connection; (ii) profiling a model/kernel and get the latency results on your edge device
from nn-meter.
Hi @Lynazhang ,
Can you elaborate a little bit on the following?
Note that you need to implement the followings: (i) set up the device and the connection; (ii) profiling a model/kernel and get the latency results on your edge device
from nn-meter.
Hi Ramson,
nn-Meter building tool provides a whole pipeline for users to build customized latency predictors. The first step, set up the device and the connection, is to build a backend class for TFLite micro, here is a guidance: https://github.com/microsoft/nn-Meter/blob/main/docs/builder/prepare_backend.md#-build-customized-backend-.
The second step, profiling a model/kernel and get the latency results on your edge device, is to sample several kernel model and profiling the kernel on device to get their latency, and use the profiled results to train a latency predictor. Here is a guidance: https://github.com/microsoft/nn-Meter/blob/main/docs/builder/build_kernel_latency_predictor.md. Note that we only support devices which could profile kernel models with tensors of different shape as inputs.
In addition, there are also some tasks to do, such as creating workspace and detecting fusion rules. Here is a overview of the whole pipeline: https://github.com/microsoft/nn-Meter/blob/main/docs/builder/overview.md. If you have any question in using nn-Meter, please feel free to contact us.
from nn-meter.
Hi @JiahangXu ,
I have implemented the backend class and detected fusion rules. Currently building kernel latency predictors. I would like to know if any updates on reference code for dataset generation as mentioned in the issue #53 .
from nn-meter.
Hi Ramson,
I'm sorry that we are still working on it. Actually, the dataset generation code contains two features, the code to generate tensorflow models, and a converter to generate tf2 models (keras h5 file) to nn-meter ir. The first part is almost ready by now, and the second part is preparing for PR and still need to test its stability. We plan to complete arrange the full dataset generation code this week.
from nn-meter.
Hi @JiahangXu ,
Thanks for the update. I'm also curious about quantize predictor, when is it planned to release?
from nn-meter.
Close this issue given no updates in a long time. Please reach out if there is anything we can assist with.
from nn-meter.
Related Issues (20)
- Why removed nodes are not removed from inbound
- can nn-meter be used for pytorch/tensorflow on the server device? HOT 2
- Comparing string to list
- Can you explain why stride[::3] has to be [1, 1]?
- Do we need self._fetch_connections(kernels)?
- Why used [kernel_extent_w, kernel_extent_h] as k_size?
- Why set output_shape to 0?
- Used get_h to calculate outw...Is this a typo HOT 1
- self.tmp_dir not found HOT 1
- How did you measure FLOPS of different CPU and GPU in the baseline comparison experiment? HOT 1
- Using NNI with nn-Meter HOT 4
- how to predict model generated from keras model.save()? HOT 2
- Can you provide the specific time and accuracy of each epoch of tflite_cpu training the conv-bn-relu kernel? HOT 3
- Question on Padding Calculation for Building PyTorch Networks HOT 3
- KeyError in graph_tool
- Error in Prediction of certain Model Families HOT 2
- in the next two rounds, the accuracy rate drops HOT 3
- How can I get the models in datasets.zip?
- NotImplementedError
- How to use PyTorch or TensorFlow to check the predicted value HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nn-meter.