Comments (3)
from maml.
That's strange. I haven't seen anything like that before. Have you tuned the update_lr? (alpha in the paper) It's possible that it might be much too large or much too small.
Can you visualize the motor current for multiple tasks?
Are you using a different sample for the inner and outer objectives? (You should be.)
from maml.
Yes I played around a lot with the learning rates. I found out that it is very sensible to those learning rates, and doesn't learn at all, if its wrong. Currently i have meta_lr 0.0001 and update_lr 0.00001 while having num_updates 3. I recognized, if I make num_updates > 1, I have to make update_lr rather small that it learns anything - is this probably because the updates regarding every single task get to different, when going many steps into one task?!
I think for some reason, now the equilibrium is no problem anymore, don't know why. It takes me about 100 iterations of finetuning with a higher learning rate to come to a good accuracy.
What number of inner gradient steps do you normally use for good results?
The motor currents for the multiple tasks are not so different. Just having their spikes etc. at different locations.
What exactly does the console output of the preloss and postloss mean. Loss before updating the inner gradients and afterwards?!
Are you using a different sample for the inner and outer objectives? (You should be.)
I guess the code below is creating different samples for the inner and outer objective?! a single tasks, b meta training?!
inputa = tf.slice(image_tensor, [0,0,0], [-1,num_classes*FLAGS.update_batch_size, -1])
inputb = tf.slice(image_tensor, [0,num_classes*FLAGS.update_batch_size, 0], [-1,-1,-1])
labela = tf.slice(label_tensor, [0,0,0], [-1,num_classes*FLAGS.update_batch_size, -1])
labelb = tf.slice(label_tensor, [0,num_classes*FLAGS.update_batch_size, 0], [-1,-1,-1])
from maml.
Related Issues (20)
- ModuleNotFoundError: No module named 'tensorflow.contrib' HOT 1
- Pretrained weights for omniglot
- Some question about cross-entropy loss xent
- How to open the results in LOGs
- Memory efficient MAML
- wrong comparison in maml.py
- What weights you are using during testing
- Prelosses and Postlosses HOT 2
- First-order approximation HOT 11
- Sinusoid results
- include how to cite paper on read me
- Definition of tasks in the Meta Phase
- Why 20000 tasks are set, but only one batch task is used HOT 1
- something about Computational Graph
- Testing always creates same results across 600 test points
- Why was a scheduler/annealing rate not used?
- itr is referenced before assignment in main.py, line 160
- model parameters of meta-training
- Interpretation of the results HOT 1
- Create tests file to highlight usage
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from maml.