Giter Club home page Giter Club logo

earthstar-cli's Introduction

Earthstar CLI

🚨🚨🚨 The Earthstar CLI is not under active development! Please check out user-scripts instead, which offers pretty much all of the same functionality as the CLI, but with the benefit that you can extend functionality with your own scripts, and tweak existing ones.


A complete command-line suite for everything Earthstar. Manage identities, back up shares on your computer, and sync with replica servers over the internet.

  • earthstar identities - Generate and manage Earthstar identities.
  • earthstar shares - Create, sync, and interact with shares.
  • earthstar servers - Add and remove replica servers you want to sync with.
  • earthstar upgrade - Upgrade the CLI to the latest version.

This tool also provides a special filesystem synchronisation feature, allowing you to use your filesystem as a way to view and modify share data. More on that below!

Installation

In time we will have pre-compiled binaries of this tool. Until then, you'll need to have Deno installed (instructions here).

If you had the previous Node version of the CLI installed, make sure to uninstall it using npm uninstall -g earthstar-cli.

Once Deno is installed, run the following command in your terminal:

deno install --allow-read --allow-write --allow-net --allow-run --allow-env --no-check --unstable -f --location https://earthstar-project.org -n earthstar https://deno.land/x/earthstar_cli/src/main.ts

There are a few permission flags there. Here is what they are for:

  • --allow-read - Reading Earthstar databases on the filesystem, filesystem sync.
  • --allow-write - Writing Earthstar databases to the filesystem, filesystem sync.
  • --allow-net - Synchronising with remote servers, checking for latest CLI version.
  • --allow-run - Running the upgrade script.
  • --allow-env - Gets your HOME path when suggesting where to store your shares.

You will see a message once installation is complete.

If you ever want to uninstall, run deno uninstall earthstar.

earthstar identities

Create new Earthstar identities or save existing ones, as well as set a current identity which is used by default for commands which write data to shares.

  • list - List all saved identities.
  • switch - Switch the identity used by default for setting data.
  • generate <shortname> - Generate a new identity with a shortname.
  • add <address> <secret> - Manually add an existing identity with a address and secret.
  • remove - Forget an identity.
  • info - Display an identity's public address and/or secret.

Extra flags can be found for each of these commands by using the --help flag.

earthstar shares

Manage, create, modify, and sync your Earthstar shares. Take it one step further, and view and modify their contents through your filesystem!

Your shares are saved locally as Sqlite databases in a directory of your choice, making them fully functional even when you're offline.

  • generate <name> - Generate a new share address using a shortname you provide.
  • add <shareAddress> - Add an existing share using its public address.
  • list - List all known shares.
  • remove - Forget a known share.
  • set - Write data to a path on a share.
  • paths - List all document paths on a share.
  • latest - Display the latest document at a given path.
  • contents - Get the latest content at a given path.
  • sync - Sync your shares with known replica servers OR a local share database.
  • sync-files - Bidirectionally synchronise a share's documents with a directory on your filesystem.
  • set-share-dir - Set the filesystem directory where your share data is stored.

Extra flags can be found for each of these commands by using the --help flag.

earthstar servers

Manage a list of replica servers you know and trust, and which will be synced with when you run earthstar shares sync.

Even if you have many shares, replica servers will only have knowledge of those they knew of beforehand. Your shares will never be leaked to replica servers that don't know about them already.

  • list - List known replica servers
  • add <url> - Add a replica server by its URL.
  • remove - Forget a replica server.

Extra flags can be found for each of these commands by using the --help flag.

Filesystem synchronisation

earthstar shares sync-files synchronises a share with a directory, representing a share's documents as files on your computer. If you modify those files, those modifications can be synchronised back to the share, and synced with other peers!

This makes it possible for you to interact with share documents with the tools you already know and use.

  • Synchronise a knowledgebase of markdown files across different machines and users (e.g. Obsidian)
  • Create websites with your friends (and serve it from one of your replica servers!)
  • Start a sneakernet of funny images you pass by USB key, synced from one computer to the next.

There are some caveats:

  • Earthstar's current maximum size for documents is 4 megabytes. If you try to sync a document larger than this, sync will throw a warning. This limit will be addressed in the near future.
  • Binary files will be encoded to base64, increasing their size by ~30%. This will also be addressed in the near future.
  • Writing to certain 'owned' paths may not be permitted, throwing an error when you try to sync. If you don't care about losing the local data, you can use the --overwriteFilesAtOwnedPaths flag.

earthstar-cli's People

Contributors

cinnamon-bun avatar dependabot[bot] avatar sgwilym avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

Forkers

kustomzone

earthstar-cli's Issues

Installation fails (on Mac OS)

$ deno install --allow-read --allow-write --allow-net --allow-run --allow-env --no-check --unstable -f --location https://earthstar-project.org -n earthstar https://deno.land/x/earthstar_cli/src/main.ts

Download https://deno.land/x/crayon/types.ts
Warning Implicitly using latest version (3.3.2) for https://deno.land/x/crayon/types.ts
Download https://deno.land/x/[email protected]/types.ts
error: Module not found "https://deno.land/x/crayon/types.ts".
    at https://deno.land/x/[email protected]/index.ts:8:24
deno 1.23.3 (release, x86_64-apple-darwin)
v8 10.4.132.8
typescript 4.7.4

Earthstar "shares" command not working on Windows 10

Tested running the Earthstar cli on a Windows Powershell and command prompt.
It all installs well, but there seems to be a functional issue with the "shares" command where it never returns the control back to the user.

Environment:

Windows 10 Pro Version 21H1 (OS Build 19043.1645)
PowerShell 7.2.2
Copyright (c) Microsoft Corporation.

https://aka.ms/powershell
Type 'help' to get help.

PS C:\Users\johan> earthstar --version
v8.3.3
PS C:\Users\johan> deno --version
deno 1.21.0 (release, x86_64-pc-windows-msvc)
v8 10.0.139.17
typescript 4.6.2

Creating an identity and setting up a share worked in the C:\Users\johan\Shares folder. The sharename.sqlite and sharename.sqlite-journal files are created.

However the command which are related to "shares" never seem to finish or return back the prompt to the user.

The command earthstar shares list does not end; unclear why. Tried using elevated command prompt without success.

The command earthstar identities list works fine.

The command earthstar upgrade works fine.

The command earthstar servers list works fine.

Tried adding the "deno.exe" executable to the "allowed apps" list, but it does not seem to make a difference.

Sync with a directory of files

Do a bulk import/export between earthstar documents and a directory of files.

This might require a lot of configuration, maybe it's better as a separate script.

How do we make earthstar paths safe to use as filenames?

Do we allow subdirectories, or put everything in one flat directory?

Can we use timestamps for two-way sync, or do we have to ask the user what to do when the data doesn't match?

How do we tell the difference between deleted files and empty files? Earthstar doesn't have deletion yet, only empty values (See earthstar-project/earthstar#20 Deletion)

Do we represent directories themselves in Earthstar, or ignore them?

trouble installing on MacOS - node-gyp and xcode tools

Problem

If running...

npm install --global earthstar-cli

Gives the error...

> node-gyp rebuild --release

No receipt for 'com.apple.pkg.CLTools_Executables' found at '/'.

No receipt for 'com.apple.pkg.DeveloperToolsCLILeo' found at '/'.

No receipt for 'com.apple.pkg.DeveloperToolsCLI' found at '/'.

gyp: No Xcode or CLT version detected!```

The problem is related to setup of the xcode command line tools. They might already be set up, but node is picky about them. (node-gyp is the build tool that compiles native C code such as SQLite, which is used inside Earthstar.)

Solution

At least on MacOS Catalina (15), the solution is

sudo rm -rf /Library/Developer/CommandLineTools
xcode-select --install

Sources:

Not compatible with Deno 1.26.0

Versions:

  • Deno 1.26.0
  • Earthstar CLI v8.3.5

The Deno.setRaw() error happens when running a command like "earthstar shares latest":

$ earthstar shares latest
 ? Choose a share › error: Uncaught (in promise) TypeError: Deno.setRaw is not a function
      (Deno.setRaw as setRaw)(
                             ^
    at Input.#readChar (https://deno.land/x/[email protected]/prompt/_generic_prompt.ts:285:30)
    at Input.#readKey (https://deno.land/x/[email protected]/prompt/_generic_prompt.ts:273:50)
    at Input.read (https://deno.land/x/[email protected]/prompt/_generic_prompt.ts:169:52)
    at Input.#execute (https://deno.land/x/[email protected]/prompt/_generic_prompt.ts:111:21)
    at async Input.prompt (https://deno.land/x/[email protected]/prompt/_generic_prompt.ts:90:14)
    at async Command.fn (https://deno.land/x/[email protected]/src/share.ts:490:36)
    at async Command.execute (https://deno.land/x/[email protected]/command/command.ts:1026:7)
    at async Command.parse (https://deno.land/x/[email protected]/command/command.ts:940:16)
    at async https://deno.land/x/[email protected]/src/main.ts:23:1

Seems to be related to this denoland/deno#15796

Work-around

Downgrade Deno to version 1.25.4 for now.

Better feedback for providing a too-long shortname

What's the problem you want solved?

earthstar identities generate foobar
ValidationError: name must be between 4 and 4 characters long, but is 6

My first choice was > 4 and I imagine most people will do that too...

Error is a somewhat confusing read also 😅

Is there a solution you'd like to recommend?

Length ~ 16 chars to avoid this?

Trying to set an invalid document does not give useful feedback

testing inside denoland/deno:alpine-1.25.4 (https://hub.docker.com/r/denoland/deno/)

earthstar identities generate foob --current
Added @foob to stored identities.
/ # earthstar shares generate foob
Generated share address.
 ? No directory for storing shares has been set yet. Where you would you like it? (/root/Shares) › /root/Shares
Added +foob.kvebxpmd1k0a
'schemaVersion  share~/Shares # earthstar shares set
 ? Choose a share › +foob.kvebxpmd1k0a
 ? Choose a path › foo.txt
 ? Enter document content › barbar
Could not set the document.
invalid_document

Import/export a single earthstar document to a file

Add some commands to make it easier to import/export single documents to files, either as JSON (including the document metadata) or just as the document content.

// new commands
earthstar document
earthstar content [--binary]

// new flags to existing command
earthstar set --fromFile <filename> [--binary]

Usage

// print out a whole document as JSON
earthstar document <dbFilename> <path>

// prints out just document.content
earthstar content <dbFilename> <path>

// redirect it to a file, to export the data out of earthstar
earthstar content <dbFilename> <path> > myfile

// read data from a file to use as the document's content
earthstar set <dbFilename> <authorFile> <path> --fromFile <filename>

// new option --binary means treat the file as binary instead of utf-8
// which means it needs to be encoded/decoded to base64 for storage in Earthstar
earthstar content <dbFilename> <path> --binary
earthstar set <dbFilename> <authorFile> <path> --fromFile <filename> --binary

Or maybe instead of --fromFile, it could try to read from stdin so you could do earthstar set db.sqlite author.json /path < myfile

Solve filesystem confusion

The CLI interacts with the filesystem in two ways:

  1. It persists share data to disk using ReplicaDriverFs. The contents of this directory are meant to be opaque to users.
  2. It syncs a share to a directory using the earthstar share sync-fs command, writing changes from the disk to the replica and vice versa. The contents of this directory are intended to be interacted with by users.

I saw this cause some confusion with testers before the CLI was launched. Which is the one for Earthstar's use, and which one is for users to interact with?

The only idea I have to counter this is to hide the directories used by ReplicaDriverFs entirely, and ferret them into some application support folder someplace.

CLI installation not working

What's the problem you want solved?

When trying to follow instructions from https://earthstar-project.org/get-started/install-cli, there is an error and the cli does not install.

command:
deno install --allow-read --allow-write --allow-net --allow-run --allow-env --no-check --unstable -f --location https://earthstar-project.org -n earthstar https://deno.land/x/earthstar_cli/src/main.ts

error:
error: Module not found "https://deno.land/x/crayon/types.ts". at https://deno.land/x/[email protected]/index.ts:8:24

❯ deno --version
deno 1.23.1 (release, aarch64-apple-darwin)
v8 10.4.132.8
typescript 4.7.2

Is there a solution you'd like to recommend?

🤷

Publishing image for the CLI from earthstar-project namespace?

If I run:

docker build -t earthstar-project/earthstar:latest .

With the following Dockerfile:

FROM denoland/deno:alpine-1.25.4

RUN deno install \
  --allow-read \
  --allow-write \
  --allow-net \
  --allow-run \
  --allow-env \
  --no-check \
  --unstable \
  -f \
  --location https://earthstar-project.org \
  -n earthstar \
  https://deno.land/x/earthstar_cli/src/main.ts

Then I'm basically good to go to have an image which I can use to package for Co-op Cloud.

The thing to figure out in this issue is how to publish these images as the CLI changes.

It would be best if you would consider publishing yourselves here from this repo via some CI thingy? Or manually:

docker build -t earthstar-project/earthstar:latest .
docker push earthstar-project/earthstar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.