library-data-platform / ldlite Goto Github PK
View Code? Open in Web Editor NEWLDLite is a client-side reporting tool for Okapi-based services
License: Apache License 2.0
LDLite is a client-side reporting tool for Okapi-based services
License: Apache License 2.0
Python 3.10.6
In querying /item-storage/items, I am receiving this error output while using LDLite to get the data from my institutions Folio tenent (see attachment). I am using the sample script from the ldlite git hub to get the data and create the tables: https://github.com/library-data-platform/ldlite/blob/main/examples/folio_demo.py.
I haven't been able to get the item records or anything else after that point in the script. I have tried breaking up the queries and table creation parts into multiple cell in my Jupyter notebook and have managed to get some of the tables created but it stalls out on items (see attachment).
I was trying to run a copy of a notebook that worked fine previously (I think back in September 2022). The current version of the notebook is on Google Colaboratory, and it installs the latest LDLite (currently 0.0.35) with !pip install --upgrade ldlite
.
It all starts okay. Connecting to FOLIO Snapshot worked fine, and I successfully completed several queries and selections. But when I try to set up a db cursor so I can follow the example for executing a SQL query (cur = db.cursor()
), I get the following error:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
[<ipython-input-63-9a71221bf8ff>](https://localhost:8080/#) in <module>
----> 1 cur = db.cursor()
AttributeError: 'NoneType' object has no attribute 'cursor'
If instead I change the database connection to db = ld.experimental_connect_db_sqlite(filename='ldlite-sqlite2.db')
, I get an error when I try to query the LDLite instance. That is, this command
_ = ld.query(table='users', path='/users', query='cql.allRecords=1 sortby id')
yields
ldlite: querying: /users
reading
scanning
transforming
---------------------------------------------------------------------------
OperationalError Traceback (most recent call last)
[/usr/local/lib/python3.8/dist-packages/ldlite/_jsonx.py](https://localhost:8080/#) in _transform_json(db, dbtype, table, total, quiet, max_depth)
416 table_j = table + '__t' if i == 0 else table + '__t' + str(i + 1)
--> 417 _transform_data(dbtype, '', cur2, [(1, table_j)], jdict, newattrs, 1, row_ids, max_depth, {})
418 if not quiet:
7 frames
OperationalError: no such column: False
During handling of the above exception, another exception occurred:
RuntimeError Traceback (most recent call last)
[/usr/local/lib/python3.8/dist-packages/ldlite/_jsonx.py](https://localhost:8080/#) in _transform_json(db, dbtype, table, total, quiet, max_depth)
422 pbar.close()
423 except (RuntimeError, psycopg2.Error, sqlite3.OperationalError, duckdb.CatalogException) as e:
--> 424 raise RuntimeError('running JSON transform: ' + str(e))
425 finally:
426 cur.close()
RuntimeError: running JSON transform: no such column: False
In trying to build an automated harvesting script for our production tenant, I've repeatedly encountered an issue where a query (particularly to the inventory-storage endpoints) will hang during the "reading" phase and require a keyboard interrupt to exit. Perhaps we could add a default timeout or allow specifying a requests timeout (and number of retries) as an argument to the query method?
Reporting for a colleague at Five Colleges:
When upgrading LDLite, he was unable to do transformations. He fixed this with a downgrade to .31 and python 3.83.
"I'm running a windows system, with python 3.88. LDlite is running in a virtual environment. I did the standard pip -m .... to update.
didn't work so i created a new venv, also didn't work but that one was python 3.10. Tried it in pipev too. Unfortunately I've since reinstalled with .31 in a python 3.83 venv, which is now working fine, i can't replicate it"
RuntimeError Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_42136/1165802898.py in
----> 1 ld.query(table='p_users', path='/users', query='cql.allRecords=1 sortby id')
c:\Users\aneslin\Documents\python\ldplite\ldp\lib\site-packages\ldlite_init_.py in query(self, table, path, query, json_depth, limit, transform)
399 newattrs = {}
400 if json_depth > 0:
--> 401 jsontables, jsonattrs = _transform_json(self.db, self.dbtype, table, count, self._quiet, json_depth)
402 newtables += jsontables
403 newattrs = jsonattrs
c:\Users\aneslin\Documents\python\ldplite\ldp\lib\site-packages\ldlite_jsonx.py in _transform_json(db, dbtype, table, total, quiet, max_depth)
393 cur2 = db.cursor()
394 while True:
--> 395 row = cur.fetchone()
396 if row is None:
397 break
RuntimeError: Invalid Input Error: Attempting to execute an unsuccessful or closed pending query result
Error:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.