Official PyTorch Implementation of "Learning to Learn with Generative Models of Neural Network Checkpoints"
Home Page: https://www.wpeebles.com/Gpt
License: BSD 2-Clause "Simplified" License
Hi, after setting my API key, installing the requirements and executing the following prompt
python main.py --config-path configs/test --config-name cifar10_loss.yaml num_gpus=4
I get the following error:
...
Wrote config to: test_results/cifar10_loss/config.yaml
wandb: Currently logged in as: fabioferreira. Use `wandb login --relogin` to force relogin
wandb: WARNING `config_exclude_keys` is deprecated. Use `config=wandb.helper.parse_config(config_object, exclude=('key',))` instead.
wandb: ERROR Error while calling W&B API: entity learn2learn not found during upsertBucket (<Response [404]>)
Problem at: /ferreira-ltft/G.pt/Gpt/utils.py 61 setup_env
wandb: ERROR It appears that you do not have permission to access the requested resource. Please reach out to the project owner to grant you access. If you have the correct permissions, verify that there are no issues with your networking setup.(Error 404: Not Found)
Error executing job with overrides: ['num_gpus=4']
Traceback (most recent call last):
File "main.py", line 349, in main
torch.multiprocessing.start_processes(
File "/home/ferreira/.miniconda/envs/ltft/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 198, in start_processes
while not context.join():
File "/home/ferreira/.miniconda/envs/ltft/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 160, in join
raise ProcessRaisedException(msg, error_index, failed_process.pid)
torch.multiprocessing.spawn.ProcessRaisedException:
-- Process 0 terminated with the following error:
Traceback (most recent call last):
File "/home/ferreira/.miniconda/envs/ltft/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 69, in _wrap
fn(i, *args)
File "/ferreira-ltft/G.pt/main.py", line 335, in single_proc_train
train(cfg)
File "/ferreira-ltft/G.pt/main.py", line 127, in train
seed = setup_env(cfg)
File "/ferreira-ltft/G.pt/Gpt/utils.py", line 61, in setup_env
wandb.init(
File "/home/ferreira/.miniconda/envs/ltft/lib/python3.8/site-packages/wandb/sdk/wandb_init.py", line 1144, in init
run = wi.init()
File "/home/ferreira/.miniconda/envs/ltft/lib/python3.8/site-packages/wandb/sdk/wandb_init.py", line 773, in init
raise error
wandb.errors.UsageError: It appears that you do not have permission to access the requested resource. Please reach out to the project owner to grant you access. If you have the correct permissions, verify that there are no issues with your networking setup.(Error 404: Not Found)
Let's say I generate millions of checkpoints of a model using Adam optimizer and it reaches a minimum loss of e.g. 0.8. Can G.pt generate weights for loss lower than 0.8?
hello, this is one of the best projects I have ever seen.
I have some questions about the generation of the pretraining data. Let's say MNIST, for each run, we select 200 checkpoints to save. In the paper I can see "and we randomly save a subset of checkpoints from each training run".
So my question is: does "random" means the random between training steps? And I am also curious about the ratio of saving(200/(25*steps_one_epoch)). Do you try to save more or less ratio of data, and does the saving ratio influence the performance?
Can you please guide me on how to draw Figure 8 in the original paper? : G.pt learns a multimodal distribution over local error minima.
First of all, one of the most impressive research projects I've ever come across. Thank you for being so cool!
Second, I'm very curious if it's possible to train a general version of G.pt, so maybe take all the models on huggingface and dump them in + as many other datasets as possible.
Would G.pt as it is now be useful for optimizing, say, a language model or some domain it wasn't trained on?
Dear researcher,
Thanks for sharing this great work. A question about the code is that the mixed precision training (i.e., amp) is disabled. Does the amp training lead to training instability or poor performance in your experiments?
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.