cybertronai / imagenet18 Goto Github PK
View Code? Open in Web Editor NEWTrain ImageNet in 18 minutes on AWS
License: The Unlicense
Train ImageNet in 18 minutes on AWS
License: The Unlicense
In train_imagenet_nv.py, 'phase' always equals None, since set_epoch(epoch) will delete cur_phase when it is not None.
line 143: dm.set_epoch(epoch)
line 318: self.phases.remove(phase)
line 155: phase = dm.get_phase(epoch)
Hi
I am new to AWS. Which resources increase request need to be filled for making this work?
Thanks for your good codes. If I want to use it on my own GPU machines. Can I do this? How should I config the IP and other configuration?
Thanks.
Hello and thanks for your great work.
I tried to rerun your code on 2xP3.16xlarge. Everything seems fine up to creating machines. They run but ncluster cannot ssh to both machines:
1.imagenet: Exception connecting to "IP" via ssh (could be a timeout): [Errno None] Unable to connect to port 22 on "IP".
Is there any specific details that I needed to take care of?
I checked the created VMs and both allow SSH on port 22.
It is worth mentioning that I copied the key from the following path in the machine I am running your code /home/ubuntu/.ncluster/ncluster-ubuntu-831839378763-us-east-2.pem and could not ssh to both machines too. connection refused!
Running the main script command:
python train.py
I get:
Exception are [AssertionError('No images match pytorch-efa01 in us-west-2, check name and image permissions')]
Traceback (most recent call last):
File "train.py", line 453, in <module>
main()
File "train.py", line 305, in main
job = ncluster.make_job(name=args.name,
File "/usr/local/lib/python3.8/site-packages/ncluster/aws_backend.py", line 1294, in make_job
raise exceptions[0]
File "/usr/local/lib/python3.8/site-packages/ncluster/aws_backend.py", line 1268, in make_task_fn
tasks[i] = make_task(f"{i}.{name}", run_name=run_name,
File "/usr/local/lib/python3.8/site-packages/ncluster/aws_backend.py", line 1070, in make_task
image = u.lookup_image(image_name)
File "/usr/local/lib/python3.8/site-packages/ncluster/aws_util.py", line 397, in lookup_image
assert len(images) > 0, f"No images match {str(wildcard)} in {get_region()}, check name and image permissions"
AssertionError: No images match pytorch-efa01 in us-west-2, check name and image permissions
> /usr/local/lib/python3.8/site-packages/ncluster/aws_util.py(397)lookup_image()
-> assert len(images) > 0, f"No images match {str(wildcard)} in {get_region()}, check name and image permissions"
I have set up the snapshots & created volumes from them using the scripts provided.
fix --spot flag
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.