Comments (3)
- add a new flag to always use the system's environment
So long as it's a flag and not the default, sure, that would definitely see some use. Security isn't a major concern for the average user, I think it's fine to have an option to circumvent the environment removal.
- use the -D flag that disables denoise to also disable the restricting of the environment
Dunno. Bit unintuitive... And I'm not too keen on mixing denoise with the environment in such a way, when really you may want to disable one and not the other.
make ReBench errors more helpful, currently the environment is not printed but only the working directory and the command
+1 for printing the environment (at least with the debug flag)
explicitly point out that the PATH might be missing or incomplete if a command fails with an error that indicates a missing binary
Definitely that one, although it would only work for missing binaries and not so easily for more obscure errors that can be caused by the unexpected lack of an env. Still, I like it.
Alternatively, would there be any value in running experiments by default with no environment EXCEPT a default PATH? I.e. /bin:/usr/bin:/usr/sbin:/usr/local/bin
. All directories where you need to have extra permissions to create files in. It's not as secure as there being no PATH like it currently is, but if we're assuming there's already harmful binaries on the machine that were put in those directories, you've already got a big problem.
Maybe my thinking is backwards, but I would think it's an acceptable compromise. Hell, running sudo bash
has a default PATH in its env, it would be odd otherwise.
Definitely needs to explicitly mention "Running with default PATH "the default path"" if you go for that.
from rebench.
I am not too concerned about security really. We throw security out of the window with denoise in many ways already.
So, it's mostly about reproducibility.
And yes, a default PATH that's the same everywhere seems sensible.
The one question then is how to treat the env: {PATH: ''}
setting, would think we still need to replace the PATH. Not sure I am prepared to implement env: {PATH: '/foo/:$PATH'}
and variants though...
from rebench.
Oh wow, I always assumed it was security reasons since that's usually why people tiptoe around the value of PATH. Makes sense that it's reproducibility in this context.
Not sure I follow the setting bit, but shouldn't Rebench assume that any provided env variable with no value ought not to be set at all? If env: {PATH: ''}
is provided, then shouldn't its behaviour be to unset PATH if it was in the env, or do nothing otherwise?
Not sure I am prepared to implement
env: {PATH: '/foo/:$PATH'}
and variants though...
I'm really not sure that feature is needed. Kinda goes against your idea of reproducibility since it opens you up to many a weird bug ("oh look I thought my binary was in /foo/
but turns out it wasn't, so instead of rebench reporting that I messed up, it's been using the wrong version of my binary this whole time instead of failing")
The only benefit I see is that we assume there's a default PATH, then you can extend the default PATH slightly more easily by typing /foo/:$PATH
instead of /foo/:/bin:/usr/bin:/usr/sbin:/usr/local/bin
(which isn't crazy value)
from rebench.
Related Issues (20)
- [T.4.3] Improve Documentation and Tutorials
- Parser for perf output fails on some input HOT 4
- Include CSV header in data files HOT 1
- ReBench environment variable inconsistency HOT 6
- Fast abort of benchmarks when same binary is missing HOT 1
- Parse git branch info when running in developer checkout
- `ReBenchLog` gauge adapter doesn't work on case-sensitive filesystems HOT 7
- Could we use hwnoise to assess whether performance fluctuations are coming from hardware noise?
- Can we drop Python 2 support? HOT 1
- Add option to explicitly disable all use of denois HOT 4
- There seem to be 'None' string values for extraArgs that are send to ReBenchDB HOT 1
- Make invocation number accessible in command line. HOT 2
- Add support for single-value metrics
- Add support to capture custom version details
- CSV Headers should be added only exactly once, to avoid being confused with data
- Reenable testing on PyPy on GitHub Actions once the PyYAML issue is resolved HOT 1
- ReBench commandline exit code should not be non-zero when there's nothing to do
- The ~ in a path should be replaced by the home directory HOT 1
- signal.SIGKILL is only available on Unix-like systems HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rebench.