Comments (8)
Thanks for the report.
Functions decorated with flows.step
are not meant to return AiiDA datatypes directly. The function typically runs on an external computer, which does not necessarily have access to the Aiida database, and so it is not possible to instantiate nodes directly there (additionally the execution environment may not even have Aiida installed).
I would imagine that this behavior is due to aiida.orm.Int
not playing well with (cloud)pickle.
from aiida-dynamic-workflows.
Can confirm:
from aiida-dynamic-workflows.
@louisponet IMO this may be an AiiDA issue (OTOH I don't think any guarantees are made about cloudpickling AiiDA dataypes)
from aiida-dynamic-workflows.
But this raises a good point that we should be able to control more carefully how datatypes returned from steps are encoded.
Currently if you
@flows.step(returns=["v1"])
def test(nk: int) -> int:
return nk
z = flows.engine.apply(test, nk=4)
t = aiida.engine.run(z.on(cluster_env))
then the Data node produced will be of type PyRemoteData
, which is not particularly informative.
Indeed this feature (encoding types more richly) is something that we have been discussing recently.
Over the coming weeks we will add issues/projects here as we migrate to GitHub; I'll be sure to ping you on any relevant issues.
from aiida-dynamic-workflows.
Right, understood. I wonder if we should try to come up with a sensible conversion between bare python objects to aiida nodes. That would be useful on many fronts, but also needs to be well specified so that it does not become messy.
I have done some explorational work with dict2node functions w.r.t going from json to aiida nodes.
Thanks! I'm looking forward to working more with this :D.
from aiida-dynamic-workflows.
I wonder if we should try to come up with a sensible conversion between bare python objects to aiida nodes.
Sounds like a good idea; additionally different datatypes may have a less opaque serialization format than pickle, and that could be pluggable too.
I think the main design decision is how signal to AiiDA what type was returned (recall that the Python function actually runs in a cluster environment with no access to AiiDA directly). We could use the function's return type annotation for this, however we would also need to define what should happen if the function decides not to conform to its type signature.
from aiida-dynamic-workflows.
Ah right, I see that's indeed not very straightforward. The solution you'd be looking for would not change anything to the current syntax etc right?
One could imagine things like annotations that signal what desired AiiDA node a return value would be converted to etc, but that would maybe not be the most elegant solution.
from aiida-dynamic-workflows.
I have opened aiidateam/aiida-core#5285 to track the reported bug upstream.
I'll leave this issue open here until that one is resolved (and further discussion about feature requests should happen in a separate issue).
from aiida-dynamic-workflows.
Related Issues (7)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aiida-dynamic-workflows.