google / blkcgroup Goto Github PK
View Code? Open in Web Editor NEWAutomatically exported from code.google.com/p/blkcgroup
License: Apache License 2.0
Automatically exported from code.google.com/p/blkcgroup
License: Apache License 2.0
The primary motivation for this test is to verify that the control group based block layer isolation functions properly. This test creates and configures control groups and launches worker tasks in them, then evaluates how effective the isolation is. Concepts ======== Control Groups provide a mechanism for aggregating/partitioning sets of tasks, and all their future children, into hierarchical groups with specialized behavior. A cgroup hierarchy arranges cgroups in a tree so that resources can be partitioned at multiple levels. Split hierarchies allow for separate cgroup hierarchies per-resource for more flexible isolation. See Documentation/cgroups/cgroups.txt in the kernel source for more details. Requirements ============ 0. Kernel options. This test requires a Linux kernel 2.6.26 or later. The following Kconfig options must be set: CONFIG_CGROUPS=y CONFIG_RESOURCE_COUNTERS=y CONFIG_DEBUG_BLK_CGROUP=y CONFIG_BLK_CGROUP=y CONFIG_IOSCHED_CFQ=y CONFIG_DEFAULT_CFQ=y CONFIG_CFQ_GROUP_IOSCHED=y 1. Blkio controller. The blkio controller must be set up properly. This can be simply done: $ mount -t cgroup -o cpuset none /dev/cgroup/cpuset $ mount -t cgroup -o io none /dev/cgroup/io More thorough documentation is at Documentation/cgroups/blkio-controller.txt in the kernel tree. 2. A relatively recent version of dd. This test uses "conv=fdatasync" and "oflag=direct" arguments for dd, which are available in recent versions of coretools. Building the tests ================== We have to make some binaries that the test uses. $ make # build a binary to service random read workers Running the tests ================= $ ./regression_test.py # Runs a suite of coverage tests or $ ./mixed_cgroup_test.py # Runs a short smoke test. Test data is output to stdout. The very end of output will have a summary line like: "-----ran 34 experiments, 33 passed, 1 failed" Searching for "FAILED" in the test output will tell you which experiment failed (and by what margin). Passed experiments will indicate the achieved DTF margin. If you get errors about state left behind from the previous test, use the -c flag: $ ./regression_test.py -c which deletes state left behind from interrupted tests. Specifically, it deletes all containers prefixed with blkcgroupt. Adding new tests/writing new tests ================================== Creating tests is pretty straightforward; if an invalid experiment is specified in the test, the harness will exit with an error. For example, current kernels do not support cgroup weights that would give less than 10% of disk time, so we can't run a test where workers would get 5% of the time. The central part of the test is a list of experiments, strings that specify a list of worker jobs, their corresponding weights of total cgroup weight, and the permitted "error": the difference we tolerate between requested weights and what is observed. It's generally a good idea to work off of examples in the existing tests. The current supported workers are: rdseq: read sequential rdseq.dir: read sequential (direct) rdrand: read random wrseq: write sequential (buffered) wrseq.sync: write synchronous (buffered) wrseq.dir: write sequential (direct) io_load_read: small reads, bouncing threads io_load_write: small writes, bouncing threads A full description of the experiment grammar is in blkcgroup_test_lib.py. In addition to the list of experiments, there are some arguments passed to the test harness: workvol: the volume where test files will be created. seq_read_mb: Size in megabytes, of input files used for rdseq workers. Scales the overall time duration of all tests. For instance, if a container has N identical workers, each uses a separate file of size 1/N. For other worker types, their input or output file size is automatically scaled to a related size that takes roughly the same elapsed time as does rdseq reading seq_read_mb. TODO ==== We plan on adding new features to this test soon, including - more worker types - more tests
commit 08bc8153da306e35ebb5705c34a715366659ef3d
Author: Rakesh Iyer <[email protected]>
Date: Mon Jan 9 16:10:59 2012 -0800
Calculate & log total throughput for experiments.
Calculate and log the aggregate throughput for all the experiments
run. The throughput is calculated by looking at the value of
io.io_service_bytes for the device at the root cgroup and dividing
this by the number of seconds that the experiment was run. For now
we simply log the value calculated at the end of the experiment.
Note that the value of the throughput has no bearing on whether the
test passes or fails.
The end result is that the log file looks similar to this:
...
15:40:06 INFO All workers have now completed or been killed by fastest worker.
15:40:06 INFO Experiment completed in 23.4 seconds
15:40:06 INFO Aggregate Throughput = 71.453175 MB/s
15:40:06 INFO experiment 4 achieved DTFs: 586, 204, 208
15:40:06 INFO experiment 4 PASSED: max observed error is 14, allowed is 35
Original issue reported on code.google.com by [email protected]
on 10 Jan 2012 at 12:56
Patched the blkcgroup code so that it builds using blaze
After the review, I'll merge this branch into:
/trunk (I think)
Original issue reported on code.google.com by [email protected]
on 26 Sep 2011 at 11:38
Attachments:
Branch name: enable-tracing
Purpose of code changes on this branch:
- Fix up comments from previous review.
- Allow for traces to be collected during test runs.
When reviewing my code changes, please focus on:
http://code.google.com/p/blkcgroup/source/detail?r=76cfa9f590
http://code.google.com/p/blkcgroup/source/detail?r=08402f4031
After the review, I'll merge this branch into:
/master
Original issue reported on code.google.com by [email protected]
on 17 Jan 2012 at 6:18
Branch name: memcg_fix
Purpose of code changes on this branch: The patch set enables the blkcgroup
test code to run on memcg as well as fakenuma
After the review, I'll merge this branch into:
/master
Original issue reported on code.google.com by [email protected]
on 2 Feb 2012 at 6:42
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.