materials-data-facility / forge Goto Github PK
View Code? Open in Web Editor NEWForge is the Python package to access data in the Materials Data Facility
Home Page: https://www.materialsdatafacility.org/
License: Apache License 2.0
forge's People
Contributors
Stargazers
Watchers
forge's Issues
Add Helper to Simplify Search by source_id
We currently have a helper for source_name but not for source_id. Having a helper for source_id seems important since that's what we pass back from the connect client.
Either we should have a separate helper or provide a flag (maybe even a default) that strips the _vx.y from a standard source_id to get the source_name.
Add README.md to test directory
Add a README file to the test directory detailing test coverage / limitations and how to use the tests
Do we need both Forge and ConnectClient
Separate libraries for publishing and retrieving data seems too complicated.
Syncing Query with Globus SDK
It seems like the Query object in Forge could be a valuable addition to the Globus SDK. They already have a query helper class that is much less sophisticated. Is merging our changes with Globus SDK something worth considering?
Globus error when downloading data to path with a space
If the user has a space in their filepath, they get the following error. It's uncertain at this time if this comes from Globus or MDF Forge -- investigate to make sure it's not an MDF forge issue
Authentication issue for first time user
I was going through the Programmatic Interface steps 1-3 in the MDF homepage https://materialsdatafacility.org/
from mdf_forge.forge import Forge
mdf = Forge()
As a first time user for me, the MDF_Forge authentication failed when i tried to authenticate via orcid id. But the second time the authentication worked. May be this issue has to be documented somewhere.
In the step three, the mdf method call has changed.
Old ----------> New
mdf.get_globus -----------> mdf.globus_download
mdf.get_https(r) ------------> mdf.http_download(r[0])
Search Helper to Find Dataset by DOI
WHoleTale has a flow where they have a DOI reference in a notebook, and want to query MDF to get the data location associated with the DOI. It would be helpful to have a helper to do this. @jgaff
Forge API calls not working on CI with anonymous=True
Using Forge(anonymous=True) sometimes results in the following type of error, especially when running on CI resources (like github actions):
test_env/lib/python3.8/site-packages/fair_research_login/local_server.py:205: in wait_for_code
resp = self._auth_code_queue.get(block=True, timeout=self.timeout)
/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/queue.py:178: in get
raise Empty
E _queue.Empty
During handling of the above exception, another exception occurred:
matminer/data_retrieval/tests/test_retrieve_MDF.py:9: in <module>
class MDFDataRetrievalTest(unittest.TestCase):
matminer/data_retrieval/tests/test_retrieve_MDF.py:13: in MDFDataRetrievalTest
mdf_dr = MDFDataRetrieval(anonymous=True)
matminer/data_retrieval/retrieve_MDF.py:41: in __init__
self.forge = Forge(anonymous=anonymous, **kwargs)
test_env/lib/python3.8/site-packages/mdf_forge/forge.py:107: in __init__
super().__init__(index=index, search_client=search_client,
test_env/lib/python3.8/site-packages/mdf_toolbox/globus_search/sub_helpers.py:21: in __init__
super().__init__(*args, **kwargs)
test_env/lib/python3.8/site-packages/mdf_toolbox/globus_search/search_helper.py:163: in __init__
self.__search_client = mdf_toolbox.login(
test_env/lib/python3.8/site-packages/mdf_toolbox/auth.py:183: in login
native_client.login(requested_scopes=scopes,
test_env/lib/python3.8/site-packages/fair_research_login/client.py:165: in login
auth_code = self.get_code(requested_scopes, refresh_tokens,
test_env/lib/python3.8/site-packages/fair_research_login/client.py:215: in get_code
auth_code = ch.authenticate(url=auth_url)
test_env/lib/python3.8/site-packages/fair_research_login/code_handler.py:104: in authenticate
return self.get_code()
test_env/lib/python3.8/site-packages/fair_research_login/local_server.py:133: in get_code
return self.server.wait_for_code()
test_env/lib/python3.8/site-packages/fair_research_login/local_server.py:210: in wait_for_code
raise LocalServerError()
E fair_research_login.exc.LocalServerError
------------------------------- Captured stdout --------------------------------
Error: Unable to create client for 'search' service.
Anonymous access may not be allowed.
Starting login with Globus Auth, press ^C to cancel.
Is there a way to prevent these problems when anonymous=True? Or is there a better way to provide API key headlessly (i.e., thru env variable)?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.