avalanche-io / c4 Goto Github PK

Given some new flag, provide a (de)duplication report that shows totals for amount of data that is duplicated.

And general code cleanup.

Add support for command style arguments (i.e. c4 cp or c4 id).

C4id: C4id1+C4id2

id to pair of ids db for merkle trees.

Currently, implementing io.Writer is great, but it should also implement hash.Hash to allow for more general use.

A simple database that maps keys to IDs, and IDs to IDs.

Build command-line tool that generates C4 IDs.

REST, websockets

Add command group to the c4 cli for identification related commands.

Watch local filesystem path for updates.

Should it go to here?:
https://www.etcentric.org/wp-content/uploads/2015/09/C4-ID-ETC-Whitepaper.pdf

Support functions to simplify testing.

Initially configuration mocking, but later we will likely need mocks for remote remote connections and the like.

Soon we will be making a change to the base58 character set used represent C4 IDs.

Old character set:

"123456789abcdefghijkmnopqrstuvwxyzABCDEFGHJKLMNPQRSTUVWXYZ"

New character set:

"123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz"

This must be done to insure that sorting of C4 IDs produces the same order as sorting the raw bytes.

Note that this is the same as the bitcoin base58 character set.

There are package dependancies such as "github.com/cheekybits/is", which while being very handy add unnecessary complexity. So we should avoid them when possible. In particular when the external package is an Avalanche-io package, which makes the separation a little silly. I'm still learning the best ways to work with go projects.

It does make sense not to keep the C4 interface clean and minimal, so if we integrate packages into C4 directly we should be careful to avoid exporting those APIs.

First a cp clone that IDs the copied file on the fly, then add multiple destinations.

Ideally this will work just as fast as regular cp in the single target case, and faster then cp in the muilti-target case if the targets are on different IO busses / devices.

c4 cp should act as a drop in replacement for standard cp, so it must match feature for feature except for id reporting and a new flag to support multiple targets.

It would be nice to have comparative performance tests as part of this as well.

The C4 spec requires JSON for services, so we should default when not matching output for cp compatibility etc.

It takes 2 parts to package / represent a file system tree.

1. A Merkle Tree that describes all files.
2. A description file dispassions and attributes.

c4 filesystem scans get recorded to the local K/V database directly if c4d is not configured or not reachable.

Setup docker container for base c4 environment.

Consolidating dev branch in prep for merge to master.

Persistent interface for LAN clients.

In prep for the upcoming persistent data needs, (env, credentials, db storage, etc), we need a way to set configuration.

REST, and websockets.

A category of functionality that is handy but inappropriate for the standard API.

For the command line tool c4 this means adding a new mode util.

Utilities should be things like:

• Alternate representations of c4 ids.
• Convert from old character set ids.

List other utilities here as they come up.

A mechanism to find and represent the differences between two directory trees in a way that can be followed as instructions to conform one into the other.

I got into the weeds on advanced algorithms so I'm walking that back a bit to simply get the basic features out (fun stuff though). Then we'll come back and improve the performance with better computer science.