Comments (11)
@mhindery I don't think you need a closure. You can just reference the function directly:
>>> import tenacity
>>> @tenacity.retry
... def x():
... print(x.retry.statistics)
...
>>> x()
{'start_time': 743610.875068273, 'attempt_number': 1, 'idle_for': 0}
from tenacity.
Is it possible to access the number of retries which have occurred?
If you want the overall retry count for all tasks running an f
function, you should use f.retry.statistics["attempt_number"] - active_task_count
, where this non-local active_task_count
should be increased only at the first attempt of each task (which might be quite hard to do).
If you want the retry count for a single task, I'm afraid it's not possible with the retry.statistics
, unless you're not calling the same function more than once, since its context is lexical, and the retry count in this case is part of a dynamic context. If you have several tasks running the same decorated function, the statistics is mixed.
This example shows this behavior and includes a solution for the second problem using contextvars
:
import asyncio, contextvars, datetime, random, tenacity
attempt = contextvars.ContextVar("attempt")
@tenacity.retry
async def f(n):
attempt.set(attempt.get(0) + 1)
values = [n, datetime.datetime.utcnow(), attempt.get(),
f.retry.statistics["attempt_number"]]
print(" ".join(map(str, values)))
await asyncio.sleep(random.random())
raise tenacity.TryAgain
async def call():
for n in range(5):
asyncio.ensure_future(f(n))
asyncio.run(call())
The result:
0 2019-08-08 20:39:30.991814 1 1
1 2019-08-08 20:39:30.992007 1 1
2 2019-08-08 20:39:30.992135 1 1
3 2019-08-08 20:39:30.992247 1 1
4 2019-08-08 20:39:30.992353 1 1
4 2019-08-08 20:39:30.993940 2 6
1 2019-08-08 20:39:30.994081 2 6
0 2019-08-08 20:39:30.994182 2 6
3 2019-08-08 20:39:30.994273 2 6
2 2019-08-08 20:39:30.994358 2 6
1 2019-08-08 20:39:31.074111 3 7
2 2019-08-08 20:39:31.128905 3 8
3 2019-08-08 20:39:31.206795 3 9
2 2019-08-08 20:39:31.284957 4 10
0 2019-08-08 20:39:31.289798 3 11
4 2019-08-08 20:39:31.516829 3 12
1 2019-08-08 20:39:31.652761 4 13
0 2019-08-08 20:39:31.713638 4 14
1 2019-08-08 20:39:31.772541 5 15
3 2019-08-08 20:39:31.841446 4 16
1 2019-08-08 20:39:32.051496 6 17
4 2019-08-08 20:39:32.062492 4 18
2 2019-08-08 20:39:32.174466 5 19
3 2019-08-08 20:39:32.236457 5 20
1 2019-08-08 20:39:32.303372 7 21
0 2019-08-08 20:39:32.323257 5 22
4 2019-08-08 20:39:32.607363 5 23
4 2019-08-08 20:39:32.632185 6 24
3 2019-08-08 20:39:32.810179 6 25
1 2019-08-08 20:39:32.930142 8 26
2 2019-08-08 20:39:33.020101 6 27
3 2019-08-08 20:39:33.045923 7 28
1 2019-08-08 20:39:33.065823 9 29
4 2019-08-08 20:39:33.120670 7 30
2 2019-08-08 20:39:33.143518 7 31
0 2019-08-08 20:39:33.270527 6 32
0 2019-08-08 20:39:33.440524 7 33
4 2019-08-08 20:39:33.497371 8 34
0 2019-08-08 20:39:33.543119 8 35
0 2019-08-08 20:39:33.692038 9 36
3 2019-08-08 20:39:33.778935 8 37
0 2019-08-08 20:39:33.793688 10 38
0 2019-08-08 20:39:33.925558 11 39
2 2019-08-08 20:39:33.956431 8 40
1 2019-08-08 20:39:34.039304 10 41
4 2019-08-08 20:39:34.475522 9 42
1 2019-08-08 20:39:34.536371 11 43
0 2019-08-08 20:39:34.696280 12 44
2 2019-08-08 20:39:34.750217 9 45
3 2019-08-08 20:39:34.754077 9 46
1 2019-08-08 20:39:34.921033 12 47
4 2019-08-08 20:39:35.233154 10 48
0 2019-08-08 20:39:35.268114 13 49
2 2019-08-08 20:39:35.517181 10 50
3 2019-08-08 20:39:35.529995 10 51
1 2019-08-08 20:39:35.534860 13 52
0 2019-08-08 20:39:35.671832 14 53
4 2019-08-08 20:39:35.922999 11 54
1 2019-08-08 20:39:36.212128 14 55
4 2019-08-08 20:39:36.289996 12 56
3 2019-08-08 20:39:36.305002 11 57
2 2019-08-08 20:39:36.358790 11 58
0 2019-08-08 20:39:36.472697 15 59
1 2019-08-08 20:39:36.489496 15 60
1 2019-08-08 20:39:36.558043 16 61
3 2019-08-08 20:39:36.678750 12 62
4 2019-08-08 20:39:36.728588 13 63
2 2019-08-08 20:39:37.223850 12 64
2 2019-08-08 20:39:37.240736 13 65
0 2019-08-08 20:39:37.251575 16 66
4 2019-08-08 20:39:37.400507 14 67
2 2019-08-08 20:39:37.428611 14 69
1 2019-08-08 20:39:37.428857 17 69
4 2019-08-08 20:39:37.526645 15 70
3 2019-08-08 20:39:37.557577 13 71
1 2019-08-08 20:39:37.683582 18 72
1 2019-08-08 20:39:37.815516 19 73
0 2019-08-08 20:39:37.939429 17 74
2 2019-08-08 20:39:37.962317 15 75
2 2019-08-08 20:39:38.092201 16 76
4 2019-08-08 20:39:38.141038 16 77
If you sort these lines, you'll see the attempt
context variable is the attempt number for that dynamic context. The number of retries is just attempt.get() - 1
.
For the first approach (the retry count summing retries from all tasks), you'll need to increase the global active_task_count
when attempt.get() == 1
.
from tenacity.
Each tenacity.Retrying
has a statistics
attribute that you can use to access the number of retries.
I'll let this bug open to make sure its documened somewhere.
from tenacity.
Something related which might be helpful for people looking at this; We wanted to have access to the statistics during the execution, and used a closure for that. Skeleton code for it (which for example uses a statistic to put into a header of a request) looks like this:
def doRequest(url):
# Closure structure to access retry statistics
@retry(reraise=True)
def doRequest_closure(url):
try:
headers.update({"AttemptNumber": str(doRequest_closure.retry.statistics["attempt_number"])})
except:
pass
return r = requests.post(url, headers=headers)
return doRequest_closure(url)
from tenacity.
Feels like wrapping it with another decorator might be a pretty way of doing it.
e.g.
@tenacity.retry
@tenacity.pass_statistics
def foo(statistics, value):
print(f'Attempt #{statistics.attempt_number}: {value}'
from tenacity.
All of the above one's are correct in their own cases :) I think a decorator could be useful, as this functionality of accessing statistics during execution looks like something not that uncommon to me.
@jd this indeed works if you have a function, but not in the case we used it in (a static method on a class). :
class Utils(object):
# works
@staticmethod
def doRequest():
# Closure structure to access retry statistics
@tenacity.retry(reraise=True)
def doRequest_closure():
print(doRequest_closure.retry.statistics)
return doRequest_closure()
# doesn't work
@staticmethod
@tenacity.retry(reraise=True, stop=tenacity.stop_after_attempt(1))
def doRequest2():
print(doRequest2.retry.statistics)
The doesn't work case raises this error:
<ipython-input-8-a41ae46bbd83> in doRequest2()
13 @tenacity.retry(reraise=True, stop=tenacity.stop_after_attempt(1))
14 def doRequest2():
---> 15 print(doRequest2.retry.statistics)
NameError: global name 'doRequest2' is not defined
from tenacity.
You should define it as a classmethod or a method, not a staticmethod. It calls itself inside a class, so it's not static.
from tenacity.
I don't mind putting in a small amount of time to support a @tenacity.pass_statistics
for what it's worth.
from tenacity.
@naphta that's not needed, see the discussion.
from tenacity.
@mhindery I don't think you need a closure. You can just reference the function directly:
>>> import tenacity >>> @tenacity.retry ... def x(): ... print(x.retry.statistics) ... >>> x() {'start_time': 743610.875068273, 'attempt_number': 1, 'idle_for': 0}
@jd but in this case the function x needs to access to attributes defined outside of their own scope. But what happens in the case that you cant modify the original function and add just the retry decorator? I think in that cases could be nice to have or another decorator, or maybe there is a way to use a custom callback for tihs?
from tenacity.
With respect to global vs local retry stats, retry.statistics
is local like you'd expect. Perhaps not for async
functions, but I suspect most are not using async
.
from tenacity import retry, stop_after_attempt
@retry(stop=stop_after_attempt(3))
def foo():
attempt = foo.retry.statistics['attempt_number']
print(f"Attempt {attempt}...")
if attempt < 3:
raise Exception("Failed")
foo()
foo()
prints
Attempt 1...
Attempt 2...
Attempt 3...
Attempt 1...
Attempt 2...
Attempt 3...
from tenacity.
Related Issues (20)
- Any help needed for a new release? HOT 2
- attempt_number should be set to 0 HOT 1
- `tests/test_tornado.py::TestTornado - AttributeError: 'TestTornado' object has no attribute 'runTest'` w/ pytest-8.2.0 HOT 4
- Question: How to handle dynamically changing retry (think api throttling) HOT 1
- Readme formatting fell apart HOT 1
- Can't we define a retry method in the instance method?
- Disable when developing? HOT 2
- HelloGitHub Badge
- Add `__aexit__`and `__aenter__` to AttemptManager
- ImportError tenacity.asyncio HOT 15
- Bug introduced in tenacity 8.4.0: ModuleNotFoundError: No module named 'tenacity.asyncio' HOT 1
- 8.4.0 release is broken due to missing files HOT 11
- Coalesce multiple ORs into a single object
- Access all occurred exceptions in `RetryError` HOT 1
- Retry in recursive context is broken since 8.3.0 HOT 3
- 8.4.2 breaks ability to compose regular functions with `retry_base` HOT 3
- 8.4.2 breaks suggested way to overwrite tenacity behaviors on unit tests HOT 7
- 8.5.0 includes breaking changes on the statistics object HOT 6
- `wait_random_exponential` does not respect `min` HOT 1
- 9.0.0: Unintended or missing behavior in retry_with
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tenacity.