Comments (5)
Thanks for finding this! The problem is that the @synchronization
decorator doesn't unindent the source of the function its attached to before trying to compile it into an AST. I wasn't able to find any built in functions to unindent the source lines, so the two manual ways I can think of are:
- Regex match indentation from the first line and remove it from all subsequent lines
- Call string.lstrip() on the lines up to a line matching a function definition but no further
Both seem kind of derpy, but I'll pick one and implement it some time today. If anyone has some input or alternative I'm happy to hear it.
from deco.
I've committed b48e575 to address this issue and fix the bug with indentation handling.
However, I also ran across another issue with the example. Because the synchronized
class doesn't provide a __get__
method, it won't function correctly when used on class methods. To address this in the immediate future I added a __get__
method that throws an exception explaining what's happening. Perhaps in the future we can support this functionality but I'm not really sure if deco would become that much more useful for how much more complicated it would need to be.
@dataoverflow, if you had a specific use case in mind for this and could provide it I would be really interested to see how you would make use of this functionality.
from deco.
@alex-sherman my use case is a class which follows interfaces from sklearn
(fit/predict etc) so it can be used along with existing sklearn
implementation. My fit()
method in my class has a loop which would benefit from concurrent execution. It'd be great if I could annotate methods of my class with deco
. Refactoring this code into a module purely so I can use deco
seem not to be a great option in my opinion.
from deco.
Thanks for the reply! I don't have much familiarity with sklearn
so it's cool to see someone using deco
with it. I tried just briefly to get the decorators to work with class methods, but ran into some complications. I unfortunately don't have enough free time at the moment to work through this, but will definitely consider it at a later date. If you're interested here are the main complications:
Descriptor API and implementing get
Both concurrent and synchronized decorators need to implement __get__(self, obj, type = None)
methods in order to be used as class methods. That problem lies in being able to return basically a copy of the wrapper object that has a modified __call__
attribute which will contain the appropriate self
in it's *args
.
More complicated AST parsing
This is really the hardest problem to solve. The AST parsing has remained really simple so far because deco can expect concurrent functions to be in the global namespace. Once that assumption is gone, detecting which function calls are actually concurrent functions becomes more complicated.
TL;DR:
Working with class methods is maybe possible, but definitely hard. I hope to revisit this at a later date, but hopefully you'll still find deco
useful in the mean time.
Thanks again for mentioning this!
from deco.
Alex, thank you for looking into this, even if at a later date! I found deco
very useful already. I am using it in a different part of my project and it works great.
from deco.
Related Issues (20)
- Can deco decorates nested functions? HOT 1
- conc_test.py was eror in my python 3 HOT 1
- Processor limit? HOT 3
- Can I specify the number of processes to use manually? HOT 1
- Decorators from deco cannot be used on class methods HOT 3
- KeyError with peterbe.com easy example on WinPython 3.6.3.0-64 HOT 4
- Bug with deco function call HOT 2
- wiki problems HOT 1
- Issues With Hanging Processes / Restart HOT 2
- Is nbody.py a Deco example? HOT 1
- Specify fixed number of CPUs? HOT 1
- User problem: where to look for when passing and receiving complex data structures? HOT 2
- Is it a good idea to pass read-only variables in a global like manner to the @concurrent function? HOT 3
- "posonlyargs" missing from arguments HOT 3
- Code slows down without sleep HOT 7
- get() without synchronized returns a tuple? HOT 2
- Multiple for loops HOT 4
- TypeError: cannot pickle '_nrt_python._MemInfo' object when passing a numba Dict to the @concurrent function HOT 2
- error_callback possible for deco?
- Setting and handling timeout HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deco.