Comments (4)
I have a slightly different error when I use pydatalab in a clean docker image derived from debian:jesse
root@1c7027b9b3bd:/# python
Python 2.7.9 (default, Mar 1 2015, 12:57:24)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import datalab.bigquery as bq
>>> bq.Query('SELECT 1')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/datalab/bigquery/_query.py", line 82, in __init__
context = datalab.context.Context.default()
File "/usr/local/lib/python2.7/dist-packages/datalab/context/_context.py", line 92, in default
project = _project.Projects.get_default_id(credentials)
File "/usr/local/lib/python2.7/dist-packages/datalab/context/_project.py", line 109, in get_default_id
projects, _ = Projects(credentials)._retrieve_projects(None, 2)
File "/usr/local/lib/python2.7/dist-packages/datalab/context/_project.py", line 80, in _retrieve_projects
raise e
oauth2client.client.HttpAccessTokenRefreshError: invalid_scope: Empty or missing scope not allowed.
My error is specific to a missing scope. I was able to resolve this issue and use pydatalab in my own docker image, but I had to modify the pydatalab source code.
In context/_utils.py, I changed line 53 from
return oauth2client.client.GoogleCredentials.get_application_default()
to
credentials = oauth2client.client.GoogleCredentials.get_application_default()
credentials = credentials.create_scoped(['https://www.googleapis.com/auth/cloud-platform'])
return credentials
Here is my DockerFile
FROM debian:jessie
RUN echo "deb-src http://ftp.us.debian.org/debian testing main" >> /etc/apt/sources.list && \
apt-get update -y && \
apt-get install --no-install-recommends -y -q \
curl wget unzip git vim build-essential ca-certificates pkg-config \
libatlas-base-dev liblapack-dev gfortran \
libpng-dev libfreetype6-dev libxft-dev \
rsync \
libxml2-dev \
python2.7 python-dev python-setuptools python-zmq openssh-client && \
easy_install pip && \
mkdir -p /tools
RUN mkdir -p /tools/node && \
wget -nv https://nodejs.org/dist/v4.3.2/node-v4.3.2-linux-x64.tar.gz -O node.tar.gz && \
tar xzf node.tar.gz -C /tools/node --strip-components=1 && \
rm node.tar.gz
ENV PATH $PATH:/tools/node/bin:/tools/google-cloud-sdk/bin
ENV PYTHONPATH /env/python
RUN mkdir -p /tools/node && \
wget -nv https://nodejs.org/dist/v4.3.2/node-v4.3.2-linux-x64.tar.gz -O node.tar.gz && \
tar xzf node.tar.gz -C /tools/node --strip-components=1 && \
rm node.tar.gz
RUN /tools/node/bin/npm install \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
[email protected] && \
cd / && \
/tools/node/bin/npm install -g forever && \
/tools/node/bin/npm install -g typescript
RUN wget -nv https://dl.google.com/dl/cloudsdk/release/google-cloud-sdk.zip && \
unzip -qq google-cloud-sdk.zip -d tools && \
rm google-cloud-sdk.zip && \
tools/google-cloud-sdk/install.sh --usage-reporting=false \
--path-update=false --bash-completion=false \
--disable-installation-options && \
tools/google-cloud-sdk/bin/gcloud -q components update \
gcloud core bq gsutil compute preview alpha beta && \
touch /tools/google-cloud-sdk/lib/third_party/google.py
RUN pip install https://github.com/googledatalab/pydatalab/archive/master.zip
ADD datalab /usr/local/lib/python2.7/dist-packages/datalab
ADD hello.js /hello.js
ADD key.json /key.json
CMD forever hello.js
I was able to successfully run the sample code provided.
import datalab.bigquery as bq
table_name = "my_project:my_dataset.my_table"
bq.Table(table_name).exists()
This isn't a complete solution, but potentially it is a starting point to help find the root cause.
from pydatalab.
I've hit this too, and I think @parthea is right about adding a call to create_scoped
.
I've sent out #55 to fix it. We will probably want to expand on that fix later to support additional scopes, but at least using the cloud-platform scope should improve the situation.
from pydatalab.
@ojarjur Can this issue be closed? I couldn't produce the exception reported:
RequestException: HTTP request failed: Invalid Credentials
Although, I did run into a different exception related to missing scope which was resolved by #55.
from pydatalab.
@parthea Yes, we believe the root cause is fixed in the latest releases, and enough time has passed without another report, that I think we can close this.
@tamsett Thanks so much for bringing this to our attention, and If you still see this issue with the latest Datalab release, then please let us know.
from pydatalab.
Related Issues (20)
- pydatalab vs google-cloud-biquery HOT 1
- Please pin dependency version ranges, not specific dependency versions HOT 5
- pydatalab is breaking pandas-gbq HOT 3
- google.datalab.utils.Iterator.__iter__ stuck in "infinite" loop HOT 3
- Possible issue with the time displayed when using %chart magic HOT 1
- Issue installing datalab jupyter extension HOT 8
- PythonOperator / Airflow Contrib Operators HOT 1
- _recurse_subqueries not compose the main query properly if subquery has dependency queries HOT 1
- Remove use of a deprecated python package, google-cloud HOT 2
- The unsigned dtypes mapped to String
- allow compression in the write_to function
- ImportError: cannot import name 'Aligner' HOT 2
- datalab ml python3 compatibility HOT 5
- Incompatibility with Python 3.7 HOT 7
- Add error handling to object.read_stream for reading corrupted text files from GCS
- Does not build with pip-compile because of dependency resolution with six HOT 1
- Can't assing batch priority to job with Query.execute()
- Object Upload fails on non Latin-1 characters HOT 1
- OAuth credentials not retained in GCP Cloud DataLab
- Datalab BigQuery examples fail due to library not specifying job location HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pydatalab.