jessicayung / blog-code-snippets Goto Github PK
View Code? Open in Web Editor NEWCode snippets used in tutorials on my blog.
Code snippets used in tutorials on my blog.
Comment about stateful LSTM is incorrect because model.hidden
is not passed in LSTM. It only stores the hidden state. Implemented LSTM is stateless.
For example:
Stateful LSTM uses hidden states:
rnn = nn.LSTM(10, 20, 2)
input = torch.randn(5, 3, 10)
h0 = torch.randn(2, 3, 20) c0 = torch.randn(2, 3, 20)
output, (hn, cn) = rnn(input, (h0, c0))
Stateless LSTM doesn't use hidden states:
rnn = nn.LSTM(10, 20, 2)
input = torch.randn(5, 3, 10)
output, (hn, cn) = rnn(input)
Great code, thanks.
Minor issue probably but I'm getting the following warning:
//anaconda3/envs/torch/lib/python3.7/site-packages/torch/nn/_reduction.py:43: UserWarning: size_average and reduce args will be deprecated, please use reduction='sum' instead.
warnings.warn(warning.format(ret))
torch.__version__ == '1.2.0'
If I figure out the solution I'll post here.
Hi Jessica,
I am wonder that if the noise of AR(5) generated data is set to 1.0 (according to your blog). LSTM is very hard (impossible) to learn this generated data.
Could you explain how to debug LSTM with generated AR data (noise=1)? Thanks!
Test data is being generated but not used.
Using model(X_test)
yields an error
If you set per_element = False
(line 19) the NN will fail because the batch_size=num_train
assumption will lead to a dimensionality error with self.lstm call on line 77.
One way to fix this would be to set batch_size = num_train / input_size
and pass in batch_size=batch_size
... but it's also possible to do an arbitrary batch size, trick would again be to get the .view() on line 77 to satisfy the self.lstm dimensionality.
You also will need to transform the loss function on line 104 so that y_train dimensions match y_pred.
You should first transpose both matrices. The best way to track the problem is using a time series with the values 1, 2, 3, etc. and you will find the error.
So, you should use:
X_train = X_train.t().view(input_size, -1, 1)
X_test = X_test.t().view(input_size, -1, 1)
instead of:
X_train = X_train.view([input_size, -1, 1])
X_test = X_test.view([input_size, -1, 1])
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.