Comments (3)
I'm describing the length of the hex bucket string, which is equivalent to be the depth of directories. The mistake in the docs is that there are 16^length
dirs at the bottom level, so that many "buckets".
Yes: the main thing is to convey that length == 3 means buckets like a0f
, which is what you need to know when configuring tantar instances.
The fact that it is a depth of directory tree is just to explain what it does internally so a deployer can grok what it does and how to chose a length (because tantar validation has to list a whole bucket and hold it in memory so one is balancing memory consumption vs fs dir listing ops). I guess technically validation also holds (length-1) lists of intervening dirs (16 dirs each), but that's close enough to a constant that I didn't bother explaining it. Practically, length of 3-4 is good for most use cases, 1-2 for testing, ... I don't think I'd ever use 5.
I will fix the mistake.
from storage-inventory.
attempt to clarify in PR #493
from storage-inventory.
just to add: the implementation could have been written to make a flatter dir structure and it would still work:
i) the btree like structure currently implemented
ii) flat
bucketLength=1: 0, 1, ...f (as it is now)
bucketLength=2: 00, 01, ...ff (flat, 256 buckets, like git does in .git/objects
)
bucketLength=3: 000, 001, ...fff (flat, 4096 buckets, like swift adapter does)
iii) flatter and fancier but complicated for no good reason:
bucketLength=3: 00, 01, ..., ff in one level, then 0, 1, ... f in the second, then files
All of these have the same logical "hex" buckets from the outside so tantar operators don't need to know which one is implemented.
from storage-inventory.
Related Issues (20)
- minoc: make service work if renamed at deploy time
- raven: make service work if renamed at deploy time HOT 1
- luskan: self-lookup is an anti-pattern HOT 1
- raven: add support for files API params
- tantar: dryrun mode doesn't produce any output HOT 1
- raven: does not work without grant providers HOT 2
- model change: StorageSite
- critwall: use Protocol.securityMethod to decide how to authenticate download calls
- cadc-storage-adapter-swift should support a pool of clients HOT 1
- raven: should not hand out anon URLs for resouyrces that are not accessible anonymously
- baldur: should verify the namespace syntax when reading baldur.properties
- baldur: permission rules should match raven style HOT 1
- cadc-storage-adapter-fs: remove hard-coded assumption of md5 file checksum
- cadc-storage-adapter-swift: remove hard-coded assumption of md5 file checksum
- cadc-storage-adapter: add stale transaction cleanup API HOT 1
- logControl permissions hard-coded: luskan, raven, minoc HOT 1
- NPE in swift adapter when reverting a file segment
- `minoc` metadata updates are not propagated to all storage locations HOT 1
- minoc: check If-Modified-Since request header
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from storage-inventory.