Comments (3)
Hi @hertz1987,
If your data is properly "marked" as metadata, then IO class 'metadata' should be the best classifier for this purpose. If you want to add a restriction, that only metadata from the beginning of the core is cached, then you can combine it with LBA classifier 'metadata&lba:le:'.
Additionally, if you skip 'Eviction priority' setting for an IO class, then the data belonging to that IO class does not get evicted from cache.
So defining an IO class as ',metadata&lba:le:,,1' (notice no value for Eviction priority between the last 2 commas) should be the way to go for classifying metadata at the beginning of the core. Just remember to make sure it doesn't collide with the default metadata configuration, which is 'metadata&done', which means that data recognized as metadata is classified there, and no further rules are processed (that's the result of '&done'). You should either give your classifier lower ID than the default metadata and add '&done' to your classifier, or remove '&done' from the default metadata classifier and give you classifier higher ID, or remove the default classifier at all.
Classification just by LBA should work as well, but it will cache all data from that range, regardless if it is indeed metadata or not.
from ocf.
Hi @hertz1987,
If your data is properly "marked" as metadata, then IO class 'metadata' should be the best classifier for this purpose. If you want to add a restriction, that only metadata from the beginning of the core is cached, then you can combine it with LBA classifier 'metadata&lba:le:'.
Additionally, if you skip 'Eviction priority' setting for an IO class, then the data belonging to that IO class does not get evicted from cache.
So defining an IO class as ',metadata&lba:le:,,1' (notice no value for Eviction priority between the last 2 commas) should be the way to go for classifying metadata at the beginning of the core. Just remember to make sure it doesn't collide with the default metadata configuration, which is 'metadata&done', which means that data recognized as metadata is classified there, and no further rules are processed (that's the result of '&done'). You should either give your classifier lower ID than the default metadata and add '&done' to your classifier, or remove '&done' from the default metadata classifier and give you classifier higher ID, or remove the default classifier at all.
Classification just by LBA should work as well, but it will cache all data from that range, regardless if it is indeed metadata or not.
Thank you for your last reply, I tried using "lba" classifier but it didn't seem to have worked, and it seemed the statistics for each IO classification didn't reflect the actual data:
my IO classification configuration file is as follows:
IO class id,IO class name,Eviction priority,Allocation
0,unclassified,22,1
2,lba:lt:0&done,,1
3,lba:le:20971520&done,,1
4,lba:gt:20971520&done,22,1
In rule #2 I specified a rule which will never be satisfied(lba<0)
In rule #3 I specified a rule that all I/Os accessing the first 10GB of the core device should be permanently cached.
In rule $4 I specified a rule that all I/Os accessing data at offset>10GB should be cached but with a relatively low priority.
So in my expectation, if I write some data of 512 bytes to the core device at offset=0 ( dd if=/dev/zero of=/dev/intelcas3-3 bs=512 count=1 ), there should be write full misses in "casadm -P -i 3 -j 3 -d 3" output, Instead:
- The statistics output from IO class ID=3 is not showing any writes:
casadm -P -i 3 -j 3 -d 3
WARNING: Intel(R) CAS is running on a non-validated OS!
IO class ID 3
IO class name lba:le:20971520&done
Eviction priority Pinned
Selective allocation Yes
╔══════════════════╤═══════╤═════╤═════════════╗
║ Usage statistics │ Count │ % │ Units ║
╠══════════════════╪═══════╪═════╪═════════════╣
║ Occupancy │ 0 │ 0.0 │ 4KiB blocks ║
║ Free │ 0 │ 0.0 │ 4KiB blocks ║
║ Clean │ 0 │ 0.0 │ 4KiB blocks ║
║ Dirty │ 0 │ 0.0 │ 4KiB blocks ║
╚══════════════════╧═══════╧═════╧═════════════╝
╔══════════════════════╤═══════╤═════╤══════════╗
║ Request statistics │ Count │ % │ Units ║
╠══════════════════════╪═══════╪═════╪══════════╣
║ Read hits │ 0 │ 0.0 │ Requests ║
║ Read partial misses │ 0 │ 0.0 │ Requests ║
║ Read full misses │ 0 │ 0.0 │ Requests ║
║ Read total │ 0 │ 0.0 │ Requests ║
╟──────────────────────┼───────┼─────┼──────────╢
║ Write hits │ 0 │ 0.0 │ Requests ║
║ Write partial misses │ 0 │ 0.0 │ Requests ║
║ Write full misses │ 0 │ 0.0 │ Requests ║
║ Write total │ 0 │ 0.0 │ Requests ║
╟──────────────────────┼───────┼─────┼──────────╢
║ Pass-Through reads │ 0 │ 0.0 │ Requests ║
║ Pass-Through writes │ 0 │ 0.0 │ Requests ║
║ Serviced requests │ 0 │ 0.0 │ Requests ║
╟──────────────────────┼───────┼─────┼──────────╢
║ Total requests │ 0 │ 0.0 │ Requests ║
╚══════════════════════╧═══════╧═════╧══════════╝
╔══════════════════╤═══════╤═════╤═════════════╗
║ Block statistics │ Count │ % │ Units ║
╠══════════════════╪═══════╪═════╪═════════════╣
║ Blocks reads │ 0 │ 0.0 │ 4KiB blocks ║
╟──────────────────┼───────┼─────┼─────────────╢
║ Blocks writes │ 0 │ 0.0 │ 4KiB blocks ║
╚══════════════════╧═══════╧═════╧═════════════╝
- The 512 byte write was reflected in statistics output from IO class ID=0(unclassified):
casadm -P -i 3 -j 3 -d 0
WARNING: Intel(R) CAS is running on a non-validated OS!
IO class ID 0
IO class name Unclassified
Eviction priority 22
Selective allocation Yes
╔══════════════════╤════════╤═══════╤═════════════╗
║ Usage statistics │ Count │ % │ Units ║
╠══════════════════╪════════╪═══════╪═════════════╣
║ Occupancy │ 259 │ 0.1 │ 4KiB blocks ║
║ Free │ 477149 │ 100.0 │ 4KiB blocks ║
║ Clean │ 258 │ 99.6 │ 4KiB blocks ║
║ Dirty │ 1 │ 0.4 │ 4KiB blocks ║
╚══════════════════╧════════╧═══════╧═════════════╝
╔══════════════════════╤═══════╤═══════╤══════════╗
║ Request statistics │ Count │ % │ Units ║
╠══════════════════════╪═══════╪═══════╪══════════╣
║ Read hits │ 47 │ 97.9 │ Requests ║
║ Read partial misses │ 0 │ 0.0 │ Requests ║
║ Read full misses │ 0 │ 0.0 │ Requests ║
║ Read total │ 47 │ 97.9 │ Requests ║
╟──────────────────────┼───────┼───────┼──────────╢
║ Write hits │ 1 │ 2.1 │ Requests ║
║ Write partial misses │ 0 │ 0.0 │ Requests ║
║ Write full misses │ 0 │ 0.0 │ Requests ║
║ Write total │ 1 │ 2.1 │ Requests ║
╟──────────────────────┼───────┼───────┼──────────╢
║ Pass-Through reads │ 0 │ 0.0 │ Requests ║
║ Pass-Through writes │ 0 │ 0.0 │ Requests ║
║ Serviced requests │ 48 │ 100.0 │ Requests ║
╟──────────────────────┼───────┼───────┼──────────╢
║ Total requests │ 48 │ 100.0 │ Requests ║
╚══════════════════════╧═══════╧═══════╧══════════╝
╔══════════════════╤═══════╤═══════╤═════════════╗
║ Block statistics │ Count │ % │ Units ║
╠══════════════════╪═══════╪═══════╪═════════════╣
║ Blocks reads │ 263 │ 100.0 │ 4KiB blocks ║
╟──────────────────┼───────┼───────┼─────────────╢
║ Blocks writes │ 1 │ 100.0 │ 4KiB blocks ║
╚══════════════════╧═══════╧═══════╧═════════════╝
At this point I am confused about why the 512 bytes write was classified as "unclassified" by rule #0 instead of by rule #2(lba<=20971520), was it because "lba" classifier is only supported on Open CAS but not on Intel CAS? or maybe there are syntax errors in my configuration file?
Any clarification or help would be greatly appreciated, thank you!
from ocf.
After running some tests on OpenCAS I found out that this problem can only be reproduced on IntelCAS and not on OpenCAS, so transitioning from IntelCAS to OpenCAS should solve my problem.
@Deixx Thank you so much for your help!
from ocf.
Related Issues (20)
- [ocf env] The completion order is not correct. HOT 1
- Crash during failover standby initialization: BUG_ON() in metadata_raw.c:276
- [Enhancement] Need for larger cache-line size
- [Feature requirement] Support extra read on core to warm cache HOT 9
- ocf_io_set_data() does not accept offset != 0
- Make it possible to configure promotion policy separately for reads and writes HOT 1
- Memory Leak for seq cutoff initialization HOT 3
- Sporadically pyocf Queue will raise RuntimeException in test HOT 1
- Cache exported object returns error from submit_flush() if no data set in io
- Provide API to query metadata hit
- Add support for bigger cache line sizes
- Add inteligent cache prefetch
- BUG() in cleaning_policy_acp_init_common() during activate (acp.c:288)
- inaccuracies in sequential cutoff promotion logic
- Cannot copy large file into VM when using OCF in SPDK
- OCF hangs after a very specific I/O pattern
- OCF crashes on big flush requests
- OCF doesn't properly identify sequential IO and splits cache requests HOT 1
- If eviction failed, clean data from any partition
- Is open cas ocf continuously maintained and updated? Or has open cas ocf become an abandoned project? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ocf.