Comments (11)
That's a very interesting idea
Some sample packages can be found here
https://github.com/datasets/
Some code can be found here
https://github.com/okfn/dptools/blob/master/bin/load-postgresql.py
https://github.com/okfn/dptools/blob/master/bin/load-sqlite.py
https://github.com/okfn/jsontableschema-py
https://github.com/okfn/jts-sql
https://github.com/trickvi/datapackage
see also
trickvi/datapackage#22
from pandas-datareader.
Not a lot of luck with trickvi/datapackage
In [1]: import datapackage
In [2]: datapkg = datapackage.DataPackage('http://data.okfn.org/data/cpi/')
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-2-21b6afb5084e> in <module>()
----> 1 datapkg = datapackage.DataPackage('http://data.okfn.org/data/cpi/')
//anaconda/lib/python3.4/site-packages/datapackage/datapackage.py in __init__(self, *args, **kwargs)
100 elif len(args) == 1:
101 self.base = args[0]
--> 102 descriptor = self.get_descriptor()
103 super(DataPackage, self).__init__(**descriptor)
104 else:
//anaconda/lib/python3.4/site-packages/datapackage/datapackage.py in get_descriptor(self)
588 # Load the descriptor json contents
589 str_descriptor = descriptor.read()
--> 590 json_descriptor = json.loads(str_descriptor)
591
592 # Return the descriptor json contents (as the dict json.load returns
//anaconda/lib/python3.4/json/__init__.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
310 if not isinstance(s, str):
311 raise TypeError('the JSON object must be str, not {!r}'.format(
--> 312 s.__class__.__name__))
313 if s.startswith(u'\ufeff'):
314 raise ValueError("Unexpected UTF-8 BOM (decode using utf-8-sig)")
TypeError: the JSON object must be str, not 'bytes'
from pandas-datareader.
Sorry to hear that @femtotrader - can you post this issue in https://discuss.okfn.org/c/open-knowledge-labs/data-packages ? someone will sort it for you ASAP
from pandas-datareader.
I'm trying to signup with GitHub and get
Sorry, there was an error authorizing your account. Perhaps you did not approve authorization?
from pandas-datareader.
from pandas-datareader.
odo http://odo.readthedocs.org/ might be considered trickvi/datapackage#45
from pandas-datareader.
@femtotrader I will have a look at this on my and and try to be of as much assistance as I can (both issues). Thanks for reporting. I will keep you updated here in this issue as well.
from pandas-datareader.
trickvi/datapackage is now fixed for Python 3.
It's possible to do
>> import datapackage
>>
>> # Note trailing slash is important for data.okfn.org
>> datapkg = datapackage.DataPackage('http://data.okfn.org/data/cpi/')
>>
>> print(datapkg.title)
Annual Consumer Price Index (CPI)
>> print datapkg.description
Annual Consumer Price Index (CPI) for most countries in the world. Reference year is 2005.
>> # Weird example just to show how to work with data rows
>> print(sum([row['CPI'] for row in datapkg.data]))
405442.60078415077
so converting to a DataFrame can be done using
df = pd.DataFrame(list(datapkg.data))
but that's probably quite long when processing very long table.
Maybe a method should be add to convert to DataFrame when Pandas is installed
it's now possible to do
>> df['CPI'].sum()
405442.60078415077
from pandas-datareader.
A new Python DataPackage client is available at https://github.com/datapackages/datapackage-py
from pandas-datareader.
New Python Data Package client: https://github.com/frictionlessdata/datapackage-py
There is also a plugin for reading Tabular Data Package resources into Pandas data frames: https://github.com/frictionlessdata/jsontableschema-pandas-py
from pandas-datareader.
Seems unnecessary given other packages.
from pandas-datareader.
Related Issues (20)
- requests.exceptions.SSLError: HTTPSConnectionPool(host='stats.oecd.org', port=443): Max retries exceeded with url: /SDMX-JSON/data/PRICES_CPI/all/all (Caused by SSLError(SSLError(1, '[SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:997)')))
- Line 234 of tiingo.py needs to be updated according to new pandas update HOT 2
- 'DataFrame' object has no attribute 'append' HOT 4
- AttributeError: module 'configparser' has no attribute 'SafeConfigParser' HOT 14
- get_nasdaq_symbols() is failing today. API endpoint problem? HOT 6
- get_sector_performance_av returning Meta Data HOT 1
- FutureWarning: The argument 'date_parser' is deprecated and will be removed in a future version. Please use 'date_format' instead, or read your data in as 'object' dtype and then call 'to_datetime' HOT 2
- ImportError: cannot import name 'reduce' from 'pandas_datareader.compat' when using pandas_datareader.wb HOT 1
- TypeError when indexing response from DataReader with 'yahoo' as data source HOT 1
- Future release(s) of pandas-datareader HOT 2
- AttributeError: module 'configparser' has no attribute 'SafeConfigParser' HOT 2
- Actualize docs after setup modernization
- web.DataReader + "fred": Failed Downloads
- TypeError HOT 1
- TypeError: string indices must be integers, not 'str' HOT 1
- STDERR is used instead of an exception HOT 1
- PyPi Long Description Content Type is incorrect
- MOEX: AttributeError: 'DataFrame' object has no attribute 'append' HOT 1
- ModuleNotFoundError: No module named 'distutils' after importing pandas-datareader HOT 1
- OECDReader: Unable to read URL
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pandas-datareader.