Comments (13)
I write a simple one
import keras
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation
from keras.optimizers import SGD
import keras.engine.saving
import keras.models
import math
import numpy as np
import stopwatch
import onnxmltools
data_size = 100000
x_train = np.array(np.random.rand(data_size,39),dtype='float32')
y_train = np.zeros(shape=(data_size,2),dtype='float32')
model = Sequential()
model.add(Dense(units=22,input_shape=(39,), activation='tanh'))
model.add(Dense(units=22, activation='tanh'))
model.add(Dense(units=2, activation='tanh'))
sgd = SGD(lr=0.01, momentum=0.9)
model.compile(loss='categorical_crossentropy',optimizer=sgd,)
sw = stopwatch.Stopwatch()
sws=[]
for i in range(2):
sw = stopwatch.Stopwatch()
sw.reset();
sw.stop();
sws.append(sw)
for i in range(100):
sws[0].start()
model.train_on_batch(x_train, y_train)
sws[0].stop()
sws[1].start()
y_predict=model.predict(x_train,batch_size=len(x_train))
sws[1].stop()
print("{0} {1:.3f} {2:.3f} ".format(i,sws[0].duration/(i+1),sws[1].duration/(i+1)))
onnx_model = onnxmltools.convert_keras(model, target_opset=7)
onnxmltools.save_model(onnx_model,"model.onnx")
Hope helps
from onnx-go.
Hello,
TL;DR: no you are not, onnx-go is not (yet) optimized for performances.
Longer answer:
onnx-go is not optimized for performances (yet). In your case, it may be because of the way the Gorgonia
backend itself is implemented. Actually, launching a backed.Run()
initiates a lot of stuff under the hood (a gorgonia.TapeMachine
is created each time for example). It is something that we are aware of, and hiding all of this within the Run()
method should allow performance tuning without breaking the API (for example. there could be a reset method to recycle some elements, or create a worker pool of machines to execute the code - this is just ideas.)
Anyway, this is pure speculation and to enhance we need to know where the bottleneck is and to measure the performances. Your example could be a perfect starting point to start to analyze and to do some measurements.
Maybe we can turn your example into a _test.go
and run some bench and analysis.
Do you mind sharing your complete example with us?
On top of that, we can also try to run your tests in concurrency via some goroutines to see how it behaves. The concurrency is also a goal as @blackrez and I would like to be able to run onnx-go inside web service.
from onnx-go.
@owulveryck , it's ok to share the model and data.It's only some study.Give me a email address or something?
from onnx-go.
Is it huge? can you copy/paste the python code here?
from onnx-go.
A basic performance analysis has started in issue #68
from onnx-go.
I made a very simple test on my machine without actually checking the results; Gorgonia is winning against Tensorflow (if my test is ok):
Python
import keras
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation
from keras.optimizers import SGD
import keras.engine.saving
import keras.models
import math
import numpy as np
import time
model = Sequential()
import onnxmltools
data_size = 100000
x_train = np.array(np.random.rand(data_size,39),dtype='float32')
y_train = np.zeros(shape=(data_size,2),dtype='float32')
model.add(Dense(units=22,input_shape=(39,), activation='tanh'))
model.add(Dense(units=22, activation='tanh'))
model.add(Dense(units=2, activation='tanh'))
sgd = SGD(lr=0.01, momentum=0.9)
model.compile(loss='categorical_crossentropy',optimizer=sgd,)
model.train_on_batch(x_train, y_train)
total = 0
for i in range(100):
x_test = np.array(np.random.rand(data_size,39),dtype='float32')
start = time.time()
y_predict=model.predict(x_test)
end = time.time()
total += (end - start)
print(total / (i+1))
onnx_model = onnxmltools.convert_keras(model, target_opset=7)
onnxmltools.save_model(onnx_model,"model.onnx")
The prediction is on average 870ms
Go
This code should run something similar:
datasize := 100000
backend := gorgonnx.NewGraph()
model := onnx.NewModel(backend)
b, err := ioutil.ReadFile(os.Args[1])
if err != nil {
log.Fatal(err)
}
err = model.UnmarshalBinary(b)
if err != nil {
log.Fatal(err)
}
var d time.Duration
for i := 0; i < 100; i++ {
input := tensor.New(tensor.WithShape(datasize, 39), tensor.Of(tensor.Float32), tensor.WithBacking(tensor.Random(tensor.Float32, datasize*39)))
model.SetInput(0, input)
t := time.Now()
err = backend.Run()
if err != nil {
log.Fatal(err)
}
d += time.Since(t)
fmt.Println(time.Duration(float64(d) / float64(i+1)))
}
Gives on average 320ms (and around 200ms with a patch that i will commit to the tensor package soon).
from onnx-go.
try this keras predict ,it's faster
y_predict=model.predict(x_train,batch_size=len(x_train))
from onnx-go.
A patch has been commited to the tensor.Tensor
package; it really changes the performances and memory consumption.
Can you update your go workspace and give a new try?
from onnx-go.
onnx-go improved.my ratio is 0.07s(keras) vs 0.3s
from onnx-go.
@bitnick10 , would you be kind enough to provide the python code and the Go code you are actually executing to get your results? This would allow us to run the exact same test as you do.
Maybe a gist.github.com can do the job.
Thanks
from onnx-go.
I copied keras of mine and goalng of yours.I changed this line that your keras code without it y_predict=model.predict(x_train,batch_size=len(x_train))
which can make keras fast a lot if you add batch_size=len(x_train)
and keras settings like this
{
"floatx": "float32",
"epsilon": 1e-07,
"backend": "tensorflow",
"image_data_format": "channels_last"
}
from onnx-go.
You're right, setting the batch_size
improve the performances of kheras and make it more efficient.
One thing I've noticed is that the tensor.Random(tensor.Float32, datasize*39)
of the Go file generates numbers that are not between 0 and 1; this makes the computation of the Tanh
expensive.
I've tried with a fix small value, and I am falling down to a 100ms in average for Go while I have around 50ms for the Python version.
I made profiling, and the next move will be to enhance the broadcasting (see issue #68):
Or maybe we can try to generate a model that does not "compact the tensor" and use broadcasting for testing. But I don't know if it's possible.
from onnx-go.
This PR from the tensor package gives good results;
I am closing this issue. A new issue can be raised to work on performances again.
from onnx-go.
Related Issues (20)
- Will this project be maintained further and are contributions still welcomed?
- Implement operator `LinearRegressor` for backend `gorgonia`
- "Asymmetric padding" error
- panic: negative dimension size does not make sense
- ../../go/src/gorgonia.org/tensor/dense_compat.go:442:23: undefined: array.Interface HOT 2
- Updated depens
- poor performance (run model)
- run() function calls newMachine() everytime HOT 1
- Question: unsqueeze: axes in not an []int64 HOT 3
- Support for empty tensors
- Tape machine does not reset properly for some models HOT 2
- Implement operator `PReLU` for backend `Gorgonia`
- Implement operator `Cast` for backend `Gorgonia`
- panic: index out of range HOT 4
- the Error during UnmarshalBinary(b)
- Is Conv1D supported?
- Implement operator `Gather` for backend `YYY`
- Any plan to support Graph attribute type?
- Time taken to initialize model
- shame on the developers HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from onnx-go.