Comments (4)
Looks great, a few questions to understand the approach better:
* Are real world app performance tests called benchmark tests?
Benchmarking is typically used in the industry to test performance of libraries, frameworks, etc. Just following that convention here.
* Under setting up benchmark data, `INSERT AND UPDATE` are going to happen regardless, because both of these are a huge part. Why is syncing again not ideal? The dataset will run all over again after syncing and that is a problem, why?
Syncing data from Sandbox will take a lot of time (~30m), so ideally we don't want to do it over and over again for every test. Backing up and restoring the data between test runs will be faster.
* If we backup and restore the dataset with no sync, are we sure we'll never receive a regression with syncing after data update because with the current approach this use case will not be included
In the sense? The regression is something we track not over a single test run, but over multiple test runs. We would want to see degraded performance of a query over a couple of days along with a significant difference in query performance before we label it as a regression. I am not sure if you were referring to something else.
from simple-android.
The proposal is looking good @vinaysshenoy . Having a base class for benchmark tests seems the easiest option, compared to having a lint rule for it.
from simple-android.
Looks great, a few questions to understand the approach better:
- Are real world app performance tests called benchmark tests?
- Under setting up benchmark data,
INSERT AND UPDATE
are going to happen regardless, because both of these are a huge part. Why is syncing again not ideal? The dataset will run all over again after syncing and that is a problem, why? - If we backup and restore the dataset with no sync, are we sure we'll never receive a regression with syncing after data update because with the current approach this use case will not be included
from simple-android.
In the sense? The regression is something we track not over a single test run, but over multiple test runs. We would want to see degraded performance of a query over a couple of days along with a significant difference in query performance before we label it as a regression. I am not sure if you were referring to something else.
The second answer clarifies it, thanks.
from simple-android.
Related Issues (20)
- A better way to add feature flags HOT 6
- `ProgressMaterialButton` HOT 16
- Migrating to Java Optional
- Use Dagger `Qualifier` instead of `Named` HOT 5
- Instruction to setup up JDK 1.8 installation on Windows Machines (Readme) HOT 2
- Suggestion[Update Git clone command) Readme HOT 4
- Improve the navigation framework HOT 4
- Proposal for automatic performance reporting of SQL queries HOT 5
- Proposal for Fragment Results API HOT 12
- Dependency Dashboard
- Reset PIN HOT 1
- Action Required: Fix Renovate Configuration
- Reporting a vulnerability HOT 2
- Hide "Record every patient…" text on home when a snackbar is visible HOT 1
- Switch to using App Bundle format for APK to reduce install file size HOT 3
- Gzip request bodies of network calls in the app HOT 4
- If the process restarts while a registration is happening, the app crashes HOT 1
- "Every patient" span of the message on the patients tab is missing on Lollipop devices HOT 1
- Build a custom view to enable fast entry of blood pressures HOT 12
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from simple-android.