Comments (2)
I am copy-pasting my response from your other issue for anyone else who reads this issue:
Hey @sushaanttb,
Have a look at #21 and the discussion within. You will see some more info there about what is going on with different retry settings, and how MAX_RETRY_SECONDS effects the outputs. That PR has not been merged and it doesn't look like it will, so if you want the behaviour you suggest, I would suggest you use my fork which fixes this issue and uses a more suitable value of 60 seconds for MAX_RETRY_SECONDS.
from azure-openai-benchmark.
Hey @michaeltremeer
Thanks for the comment!
I did had a look on the existing issues, including Issue#20 but as it was marked closed and then as your PR was also "still" not merged in the main branch, I was a bit confused on why it was still not merged in the main repo and as it was also becoming a bit confusing. I accordingly re-raised some of these issues again.
Btw, your PR got merged yesterday after I raised those issues, so congratulations and great work!.
Lastly, I like how the documentation is now more clear and also saw your results with different MAX_RETRY_SECONDS,, in which 60s was found to be giving better results.
But the point is it's still hardcoded, & thus should we not still give the option to make it configurable? We can keep the default to 60s.If not, we should def. add those benchmark stats results for reference in the README for anyone trying to understand why 60s was chosen.
from azure-openai-benchmark.
Related Issues (20)
- Any plan to adding visualization result in this package? HOT 3
- Benchmark tool Output to CSV HOT 2
- Utilization Statistics - avg, P95
- Typo when setting "presence penalty" parameter
- RequestBuilder is prepared with hardcoded value of "gpt-4-0613"
- Renaming "frequence_penalty" function argument to "frequency_penalty"
- Considering "retry-after" header as well when retrying throttled requests.
- Documenting reference for header values used.
- RETRY_AFTER_MS_HEADER not getting properly "honored" HOT 1
- Decoupling Throttling retries logic & Throttling backoff logic. HOT 4
- Fixing the while loop logic of retry mechanism for 429 requests HOT 1
- Documenting about MAX_RETRY_SECONDS HOT 2
- Documenting about the two types of retry logics for 429 errors. HOT 2
- Count of Completed Requests & Total Requests is same
- Properly reflecting the count of "requests"
- Purpose of stats call condition in while loop?
- Difference in the e2e_avg latency observed in the benchmarking tool and Azure portal
- Issue with azure benchmarking tool on passing input tokens greater than 30k
- Issue while running benchmarking tool in setting the rpm
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from azure-openai-benchmark.