Comments (2)
This code was not used yet, just exploring an idea.
I think of node grouping and transformations as a two step process:
-
Framework-specific code that maps all layer names to a standard naming convention. This code should be as small as possible and only handles the differences between frameworks. For example, in converts "Linear", "fully connected", "fc", ...etc. to "linear", and converts Convolution, Conv, Conv2d, ...etc. to "conv".
-
Framework-agnostic code that merges and transforms layers for better visualization. This works on the result from step 1. For example, this merges a "conv/bn/relu" into one layer for nicer optimization.
Ideally, the grouping rules in 2 should be user-customizable using a simple syntax. For example, if someone doesn't like to merge linear and RELU they can easily remove the "linear/relu" rule. Similarly, if someone uses Conv followed by sigmoid a lot and want to merge them they should be able to add a "conv/sigmoid" rule easily.
The current syntax is simply "a/b/c" which catches layers that follow each other. I was experimenting with adding more rules, such as sibling layers and so on.
Then, from that I was thinking that if we build a nice parser for these rules then we might be able to use it for the mapping in 1 as well. For example, to map the TF graph to the standard graph we use in step 2 we need to do things like merging Conv and weights and biases nodes into one, or to remove all "assign" nodes. These were the two additional lines I added, but I haven't gotten the chance to continue down that path.
from hiddenlayer.
Ok, good to know.
I was able to get descent graphs for a variety of models from the tf-slim model zoo using three sets of rules. These rules are applied sequentially to simplify the graph:
Rules are passed to DirectedGraph()
's constructor:
You could use our framework-specific rules. Or, you could use your own rules. Or, a modified copy of our rules -- up to the caller.
As a general rule, I greatly support the dichotomy between framework-specific code and framework-agnostic code. In practice, when the dichotomy is very small, I tend to weigh that against perhaps unecessarily complicating the source code.
Here's what that means in our case. The rules currently passed in are the following:
TF CODE:
PYTORCH CODE:
The set of entries is sooooo small, re-factoring the code (beyond passing the rules as parameters) might feel somewhat artificial. However, if we discover very unique rules that do compromise (i.e., completely break) our current implementation, then yes, we absolutely should revisit the matter.
from hiddenlayer.
Related Issues (20)
- AttributeError: module 'torch.onnx' has no attribute '_optimize_trace' HOT 7
- May I get the very first and last tensor shape?
- module 'torch.jit' has no attribute '_get_trace_graph' HOT 3
- Pytorch Model Branch Duplication
- AttributeError: module 'torch.jit' has no attribute '_get_trace_graph' HOT 3
- ONNX symbolic expected a constant value of the rand argument
- tensor shapes not displayed in rendered graph HOT 5
- can it show variable name at every node?
- demo pytorch_graph.ipynb not working
- How to display branch nodes / parallel blocks HOT 2
- Not displaying graph as expected in Jupyter Notebook (either during training or after training) HOT 1
- How exactly should look the second parametr?
- build_graph() does not return HOT 2
- AttributeError: 'Digraph' object has no attribute '_repr_svg_' HOT 2
- How to display dimensions and change font to Times HOT 1
- TypeError for Pytorch Model HOT 4
- support yolov5 HOT 2
- Error when trying to print the Model
- Error while trying to run the example
- How to plot bert model? (Transfomer models) HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hiddenlayer.