Example in readme lacking .wait about deco HOT 11 CLOSED

The .wait() is added automatically by the @synchronized decorator. When using @synchronized it should usually never be necessary to worry about explicit synchronization.

from deco.

Why do I have to do it here then?

from deco.

.wait() is called only when it's necessary, so whenever an argument or the result of a concurrent function is read or the @synchronized function returns. For timing things like you're doing, you're using .wait() totally correctly though. I should probably clarify this point in documentation somewhere.

As an alternative, you could probably time calls outside the @synchronized function. This guarantees that .wait() has been called and removes the need for it to be called explicitly.

from deco.

Oh. I think I get it. The @synchronized function, upon completion, does the equivalent of doing .wait() inside it.

from deco.

No, that's not it either. I'm sorry I'm so slow to get this but why doesn't this work without the .wait():

from deco import concurrent, synchronized

def slow():
    time.sleep(5)

@concurrent
def download(url, data):
    slow()
    data[url] = url

@synchronized
def run(data):
    for url in list('123456'):
        download(url, data)
#    download.wait()

somemute = {}
t0=time.time()
run(somemute)
t1=time.time()
pprint(somemute)
print "TOOK", t1-t0

The output of this is:

{}
TOOK 0.0250489711761

Why doesn't the @synchronized decorator automatically do the .wait()?

from deco.

Also, why 3 processors? Why not 4 or 11 or 47? Or why the number of cpus?

from deco.

That's actually a good catch, I'll need to look into this further but the AST parsing in v0.2 did not handle this case correctly. For whatever reason it now works in v0.3 and the latest version in github. I'll need to add an example/test that includes this, thanks for pointing it out!

The hard coded number of processors also got removed in the latest version.

Thanks for finding this!

from deco.

Thanks for the support. I'm so impressed by this that I decided to blog about it. This way it helps me understand it better. And remember it when I forget everything I've learned in 3 days.
https://www.peterbe.com/plog/deco

from deco.

Wow that's awesome! Do you mind if I link back to that in the readme somewhere? You've got a great set of examples that might help other people figure out how to use deco.

By the way, you can actually limit the number of process, @concurrent passes it's arguments to multiprocessing.Pool.

@concurrent(processes = 5)
def really_slow_and_intensive_thing():
    ...

Pool has some other interesting arguments you can look at here.

from deco.

Yes, please link to it.
I'll make an update about the ability to do processes=5.

from deco.

Now I have a good follow-up too:
https://www.peterbe.com/plog/time-to-do-concurrent-cpu-bound-work

from deco.