Comments (9)
sean-smith
commented on September 28, 2024
1
@rsignell-usgs Taking a look
from no-tears-cluster.
sean-smith
commented on September 28, 2024
@rsignell-usgs Can you run a few commands for me? The line that's breaking is https://github.com/aws-samples/no-tears-cluster/blob/release/scripts/bootstrap.sh#L31 which I assume has to due with how Cloud9 manages aws credentials (my guess would be that Cloud9 changed something about their credential management in multi-user environment).
To start can you run:
sudo pip-3.6 --disable-pip-version-check --no-cache-dir install aws-parallelcluster==2.8.0 --upgrade
Did you specify a pcluster version in the cloudformation stack?
from no-tears-cluster.
sean-smith
commented on September 28, 2024
Ok I was able to reproduce, disregard my previous comment. I'll figure out what's breaking
from no-tears-cluster.
sean-smith
commented on September 28, 2024
Ok when I run:
$ sudo pip-3.6 --disable-pip-version-check --no-cache-dir install aws-parallelcluster==2.8.0 --upgrade
...
$ pcluster list --color
Traceback (most recent call last):
File "/usr/bin/pcluster", line 33, in <module>
sys.exit(load_entry_point('aws-parallelcluster==2.8.1', 'console_scripts', 'pcluster')())
File "/usr/bin/pcluster", line 22, in importlib_load_entry_point
for entry_point in distribution(dist_name).entry_points
File "/usr/local/lib/python3.6/site-packages/importlib_metadata/__init__.py", line 558, in distribution
return Distribution.from_name(distribution_name)
File "/usr/local/lib/python3.6/site-packages/importlib_metadata/__init__.py", line 215, in from_name
raise PackageNotFoundError(name)
importlib_metadata.PackageNotFoundError: No package metadata was found for aws-parallelcluster
Looks like it's an issue with pip in the virtual env.
https://stackoverflow.com/questions/54552367/pip-cannot-find-metadata-file-environmenterror
from no-tears-cluster.
sean-smith
commented on September 28, 2024
Hmm so I got it working with pip3 instead of pip-3.6
from no-tears-cluster.
rsignell-usgs
commented on September 28, 2024
@sean-smith, I'm guessing you didn't mean that I just need to install pcluster using pip3, right?
For me, that produced the same "no package metadata" error:
__| __|_ )
_| ( / Amazon Linux AMI
___|\___|___|
https://aws.amazon.com/amazon-linux-ami/2018.03-release-notes/
No packages needed for security; 4 packages available
Run "sudo yum update" to apply all updates.
AWS-HPC-Quickstart-ResearcherB0ACBDEC-1058NFFMNDDPU:~/environment $ pcluster
Traceback (most recent call last):
File "/usr/bin/pcluster", line 33, in <module>
sys.exit(load_entry_point('aws-parallelcluster==2.8.0', 'console_scripts', 'pcluster')())
File "/usr/bin/pcluster", line 22, in importlib_load_entry_point
for entry_point in distribution(dist_name).entry_points
File "/usr/local/lib/python3.6/site-packages/importlib_metadata/__init__.py", line 558, in distribution
return Distribution.from_name(distribution_name)
File "/usr/local/lib/python3.6/site-packages/importlib_metadata/__sudo pip-3.6 --disable-pip-version-check --no-cache-dir install aws-parallelcluster==2.8.0 --upgrade
raise PackageNotFoundError(name)
importlib_metadata.PackageNotFoundError: No package metadata was found for aws-parallelcluster
AWS-HPC-Quickstart-ResearcherB0ACBDEC-1058NFFMNDDPU:~/environment $ sudo pip3 --disable-pip-version-check --no-cache-dir install aws-parallelcluster==2.8.0 --upgrade
Collecting aws-parallelcluster==2.8.0
Downloading https://files.pythonhosted.org/packages/fb/06/46f3d1717920d7a77db753c6e0fa055fb4a484fab7c8d06a9340576871bb/aws-parallelcluster-2.8.0.tar.gz (110kB)
100% |████████████████████████████████| 112kB 4.2MB/s
/usr/local/lib/python3.6/site-packages/_distutils_hack/__init__.py:19: UserWarning: Distutils was imported before Setuptools, but importing Setuptools also replaces the `distutils` module in `sys.modules`. This may lead to undesirable behaviors or errors. To avoid these issues, avoid using distutils directly, ensure that setuptools is installed in the traditional way (e.g. not an editable install), and/or make sure that setuptools is always imported before distutils.
"Distutils was imported before Setuptools, but importing Setuptools "
/usr/local/lib/python3.6/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")
Requirement already up-to-date: setuptools in /usr/local/lib/python3.6/site-packages (from aws-parallelcluster==2.8.0)
Collecting boto3>=1.14.3 (from aws-parallelcluster==2.8.0)
Downloading https://files.pythonhosted.org/packages/26/48/40dbd811d4131fc3076b0ee0b4b6f86352fdfc2d5fe824213521d5164223/boto3-1.14.53.tar.gz (97kB)
100% |████████████████████████████████| 102kB 15.7MB/s
Collecting future<=0.18.2,>=0.16.0 (from aws-parallelcluster==2.8.0)
Downloading https://files.pythonhosted.org/packages/45/0b/38b06fd9b92dc2b68d58b75f900e97884c45bedd2ff83203d933cf5851c9/future-0.18.2.tar.gz (829kB)
100% |████████████████████████████████| 829kB 20.9MB/s
Collecting tabulate<=0.8.3,>=0.8.2 (from aws-parallelcluster==2.8.0)
Downloading https://files.pythonhosted.org/packages/c2/fd/202954b3f0eb896c53b7b6f07390851b1fd2ca84aa95880d7ae4f434c4ac/tabulate-0.8.3.tar.gz (46kB)
100% |████████████████████████████████| 51kB 28.0MB/s
Requirement already up-to-date: ipaddress>=1.0.22 in /usr/local/lib/python3.6/site-packages (from aws-parallelcluster==2.8.0)
Requirement already up-to-date: PyYAML>=5.1.2 in /usr/local/lib64/python3.6/site-packages (from aws-parallelcluster==2.8.0)
Requirement already up-to-date: botocore<1.18.0,>=1.17.53 in /usr/local/lib/python3.6/site-packages (from boto3>=1.14.3->aws-parallelcluster==2.8.0)
Requirement already up-to-date: jmespath<1.0.0,>=0.7.1 in /usr/local/lib/python3.6/site-packages (from boto3>=1.14.3->aws-parallelcluster==2.8.0)
Requirement already up-to-date: s3transfer<0.4.0,>=0.3.0 in /usr/local/lib/python3.6/site-packages (from boto3>=1.14.3->aws-parallelcluster==2.8.0)
Requirement already up-to-date: python-dateutil<3.0.0,>=2.1 in /usr/local/lib/python3.6/site-packages (from botocore<1.18.0,>=1.17.53->boto3>=1.14.3->aws-parallelcluster==2.8.0)
Requirement already up-to-date: docutils<0.16,>=0.10 in /usr/local/lib/python3.6/site-packages (from botocore<1.18.0,>=1.17.53->boto3>=1.14.3->aws-parallelcluster==2.8.0)
Requirement already up-to-date: urllib3<1.26,>=1.20; python_version != "3.4" in /usr/local/lib/python3.6/site-packages (from botocore<1.18.0,>=1.17.53->boto3>=1.14.3->aws-parallelcluster==2.8.0)
Requirement already up-to-date: six>=1.5 in /usr/local/lib/python3.6/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.18.0,>=1.17.53->boto3>=1.14.3->aws-parallelcluster==2.8.0)
Installing collected packages: boto3, future, tabulate, aws-parallelcluster
Running setup.py install for boto3 ... done
Running setup.py install for future ... done
Running setup.py install for tabulate ... done
Running setup.py install for aws-parallelcluster ... done
Successfully installed aws-parallelcluster-2.8.0 boto3-1.14.53 future-0.18.2 tabulate-0.8.3
AWS-HPC-Quickstart-ResearcherB0ACBDEC-1058NFFMNDDPU:~/environment $ pcluster list --color
Traceback (most recent call last):
File "/usr/bin/pcluster", line 33, in <module>
sys.exit(load_entry_point('aws-parallelcluster==2.8.0', 'console_scripts', 'pcluster')())
File "/usr/bin/pcluster", line 22, in importlib_load_entry_point
for entry_point in distribution(dist_name).entry_points
File "/usr/local/lib/python3.6/site-packages/importlib_metadata/__init__.py", line 558, in distribution
return Distribution.from_name(distribution_name)
File "/usr/local/lib/python3.6/site-packages/importlib_metadata/__init__.py", line 215, in from_name
raise PackageNotFoundError(name)
importlib_metadata.PackageNotFoundError: No package metadata was found for aws-parallelcluster
from no-tears-cluster.
rsignell-usgs
commented on September 28, 2024
I was able to install pcluster into my Cloud 9 workspace using conda (installing aws-parallelcluster=2.8.0 from the conda-forge channel), but when I list, I do not see the new stack. Only an old stack "habenero2" I created using pcluster 2.6.1 back in May:
from no-tears-cluster.
rsignell-usgs
commented on September 28, 2024
Ah, okay, I did pcluster create hpc-cluster and (after waiting 5 minutes) I see now:
(aws) AWS-HPC-Quickstart-ResearcherB0ACBDEC-1058NFFMNDDPU:~/environment $ pcluster list --color
hpc-cluster CREATE_COMPLETE 2.8.0
habanero2 CREATE_COMPLETE 2.6.1
I'm guessing the No-Tears cloud formation script quit before it created the cluster?
from no-tears-cluster.
sean-smith
commented on September 28, 2024
yup - I'm testing the changes, looks like the pip command we use to install isn't working as desired.
from no-tears-cluster.
Related Issues (15)
- Make VPC and Subnet Optional
- Post_install breaks on compute nodes in Pcluster 2.9.x
- expired credentials HOT 2
- Timeout comment incorrect
- Cloud9 instance: no space left on device HOT 2
- No-Ingress Cluster Access
- failed build HOT 2
- Cloud9 Not Bootstrapping HOT 1
- nodejs 14 HOT 1
- post_install fails on [email protected] HOT 1
- SRA toolkit now named 'sratoolkit'
- Gitter? HOT 1
- Make user creation optional HOT 2
- No-Tears-Cluster CLI command HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from no-tears-cluster.