masroorhasan / docker-pyspark Goto Github PK

HDFS integration options

• Mounted volumes
• Separate container with shared volumes
• Baked in

update-alternatives: using /usr/bin/g++ to provide /usr/bin/c++ (c++) in auto mode
update-alternatives: warning: skip creation of /usr/share/man/man1/c++.1.gz because associated file /usr/share/man/man1/g++.1.gz (of link group c++) doesn't exist
Setting up dirmngr (2.2.4-1ubuntu1.2) ...
Setting up virtualenv (15.1.0+ds-1.1) ...
Setting up python-secretstorage (2.3.1-2) ...
Setting up python-keyring (10.6.0-1) ...
Setting up build-essential (12.4ubuntu1) ...
Setting up gpg-wks-client (2.2.4-1ubuntu1.2) ...
Setting up gnupg (2.2.4-1ubuntu1.2) ...
Processing triggers for libc-bin (2.27-3ubuntu1) ...
Removing intermediate container c189d87de154
---> c6f5f8ab4ae9
Step 5/7 : RUN pip install --upgrade pip && pip install numpy && pip install pyspark
---> Running in 7328315d23d5
Collecting pip
Downloading https://files.pythonhosted.org/packages/5c/e0/be401c003291b56efc55aeba6a80ab790d3d4cece2778288d65323009420/pip-19.1.1-py2.py3-none-any.whl (1.4MB)
Installing collected packages: pip
Found existing installation: pip 9.0.1
Not uninstalling pip at /usr/lib/python2.7/dist-packages, outside environment /usr
Successfully installed pip-19.1.1
Traceback (most recent call last):
File "/usr/bin/pip", line 9, in
from pip import main
ImportError: cannot import name main
The command '/bin/sh -c pip install --upgrade pip && pip install numpy && pip install pyspark' returned a non-zero code: 1