cadc-storage-adapter-fs clarification about storage-inventory HOT 3 CLOSED

I'm describing the length of the hex bucket string, which is equivalent to be the depth of directories. The mistake in the docs is that there are 16^length dirs at the bottom level, so that many "buckets".

Yes: the main thing is to convey that length == 3 means buckets like a0f, which is what you need to know when configuring tantar instances.

The fact that it is a depth of directory tree is just to explain what it does internally so a deployer can grok what it does and how to chose a length (because tantar validation has to list a whole bucket and hold it in memory so one is balancing memory consumption vs fs dir listing ops). I guess technically validation also holds (length-1) lists of intervening dirs (16 dirs each), but that's close enough to a constant that I didn't bother explaining it. Practically, length of 3-4 is good for most use cases, 1-2 for testing, ... I don't think I'd ever use 5.

I will fix the mistake.

from storage-inventory.

attempt to clarify in PR #493

from storage-inventory.

just to add: the implementation could have been written to make a flatter dir structure and it would still work:

i) the btree like structure currently implemented

ii) flat
bucketLength=1: 0, 1, ...f (as it is now)
bucketLength=2: 00, 01, ...ff (flat, 256 buckets, like git does in .git/objects)
bucketLength=3: 000, 001, ...fff (flat, 4096 buckets, like swift adapter does)

iii) flatter and fancier but complicated for no good reason:
bucketLength=3: 00, 01, ..., ff in one level, then 0, 1, ... f in the second, then files

All of these have the same logical "hex" buckets from the outside so tantar operators don't need to know which one is implemented.

from storage-inventory.