charmbracelet / charm Goto Github PK
View Code? Open in Web Editor NEWThe Charm Tool and Library 🌟
License: MIT License
The Charm Tool and Library 🌟
License: MIT License
when we run charm backup-keys
also provide a seed phrase
been trying to self-host charm following the tls instructions but it's pretty confusing. including some more instructions on how to do it on a fresh vps would be useful, as well as explaining what tls brings over and against the basic ssh connection.
I was looking for a way to store my secrets recently, and a common occurrence among the suggestions and solutions was to encrypt each secret using another key and store the encrypted keys in a secure place where only you have access to it (automated solutions would handle the encrypting and storing for you). but the problem with those is, the secrets are not synced across all my devices. and if I loose the access to my device or the "master" key, I loose all my secrets.
Today this idea popped in my head: If charm can securely store and sync my markdown notes across my devices, why can it not store my secrets? So the immediate solution would be to create a markdown file with all the secrets in it but that's not really a solution at all imo.
So I propose the following feature request:
charm secrets # list all secrets' names (similar to `charm keys`)
charm secrets add SECRET_NAME < cat secret_file # add a secret (similar to `glow stash`)
charm secrets get SECRET_NAME # get the value of the secret
charm secrets remove SECRET_NAME # get the value of the secret
this would allow things like
some_tool --passphrase $(charm secrets get SECRET_NAME)
command name and format can change
add tests for linking multiple clients to the same charm account
Currently only unencrypted SSH keys are supported, we should also provide support for password encrypted keys.
The new /.well-known/openid-configuration
endpoint recently introduced by #59 is no longer returning a valid jwks_uri
:
[rubiojr@nano tavern] curl localhost:35354/.well-known/openid-configuration
{"issuer":"","authorization_endpoint":"","token_endpoint":"","jwks_uri":"http://:35354/v1/public/jwks","userinfo_endpoint":"","id_token_signing_alg_values_supported":null}
I think this is the result of #76 removing localhost
as the default in the server configuration struct.
Sorry to be off topic here, but I was enticed to come on this thread from its title, as I'm too, unsure how to get started with charm.
In fact, it's a bit worse, I wonder what I could use charm for. I tried to understand the possible usages from its website but was left wondering.
Maybe there is some documentation or a simple wiki page ? Or better, a video showing us what good things we can do with it ?
Originally posted by @stephaneeybert in #93 (comment)
Hi,
I try to backup my keys but after the import, I lose my files.
Steps to reproduce
export CHARM_HOST=localhost
charm fs cp delta.md charm:/
charm fs tree charm:/
charm backup-keys
rm -fr ~/.local/share/charm
charm fs tree charm:/
charm import-keys charm-keys-backup.tar
charm fs tree charm:/
A simple tar -czf
of the keys directory work fine.
I use ubuntu.
Steps to reproduce
test
directory or file: charm fs cp -r test charm:/test
charm fs rm charm:/test
Expected result
The test
directory is removed from the charm server.
Observed result
The test
directory is not removed from the server and the client doesn't exit with an error.
It seems the rm
command does not handle charm
URIs so charm:
is interpreted as a path element and encrypted as part of the path to delete. The server is using os.RemoveAll, which returns no error if the path doesn't exist, so the client will get a 200 response from the server, making the user think the operation completed successfully.
Apologies if I've missed something obvious here. I was looking to test charm KV and ran the following commands:
$ charm keygen
✔ Generated keys
$ charm id
We were’t able to authenticate via SSH, which means
there’s likely a problem with your key.
You can generate SSH keys by running charm keygen . You
can also set the environment variable CHARM_SSH_KEY_PATH
to point to a specific private key, or use -i specifify a
location.
I expected charm id
to report an identity associated with the keys I just created.
I tried this in a basic Docker container and got the same result:
$ docker run -it ubuntu:latest /bin/bash
...
# apt update && apt install -y wget
...
# wget https://github.com/charmbracelet/charm/releases/download/v0.10.3/charm_0.10.3_linux_amd64.deb
...
# dpkg -i charm_0.10.3_linux_amd64.deb
Selecting previously unselected package charm.
(Reading database ... 4470 files and directories currently installed.)
Preparing to unpack charm_0.10.3_linux_amd64.deb ...
Unpacking charm (0.10.3) ...
Setting up charm (0.10.3) ...
# charm keygen
✔ Generated keys
# charm id
We were’t able to authenticate via SSH, which means
there’s likely a problem with your key.
You can generate SSH keys by running charm keygen . You
can also set the environment variable CHARM_SSH_KEY_PATH
to point to a specific private key, or use -i specifify a
location.
I also tried running a local charm
server, but got the same result (output from the same Docker container):
# charm serve &
[1] 2609
2022/03/08 21:34:47 Opening SQLite db: data/db/charm_sqlite.db
2022/03/08 21:34:47 http://localhost:35354
2022/03/08 21:34:47 Starting SSH server on :35353
2022/03/08 21:34:47 Starting HTTP server on: :35354
2022/03/08 21:34:47 Starting HTTP health server on: :35356
# export CHARM_HOST=localhost
# charm id
We were’t able to authenticate via SSH, which means
there’s likely a problem with your key.
You can generate SSH keys by running charm keygen . You
can also set the environment variable CHARM_SSH_KEY_PATH
to point to a specific private key, or use -i specifify a
location.
Is there something that I'm missing?
support expiration time (lifetime) of the key
add subcommand like charm where
to show where your charm-related keys, DBs, and other info is stored. Currently, the only way to find this path is in the Backup option of the TUI.
When setting a username, pre-populate the input field in the "set username" section with the current username, if one is set.
I was expecting the database file to reside inside the directory specified with --data-dir
, based on:
rubiojr@nano charm] ./charm serve -h
Start the SSH and HTTP servers needed to power a SQLite-
backed Charm Cloud.
Usage:
charm serve [flags]
Aliases:
serve, server
Flags:
--data-dir string Directory to store SQLite db, SSH keys and file data
It doesn't seem to be the case currently:
[rubiojr@nano charm] mkdir /tmp/charmtest
[rubiojr@nano charm] cd /tmp/charmtest/
[rubiojr@nano charmtest] charm serve --data-dir foo
2021/12/09 14:13:50 Opening SQLite db: ./data/db
2021/12/09 14:13:52 Starting SSH server on :35353
2021/12/09 14:13:52 HTTP server listening on: :35354
^C
[rubiojr@nano charmtest] find .
.
./foo
./foo/.ssh
./foo/.ssh/charm_server_rsa.pub
./foo/.ssh/charm_server_rsa
./data
./data/stats
./data/files
./data/db
./data/db/charm_sqlite.db
[rubiojr@nano charmtest]
Looks like that happens in
Line 25 in 4c30cde
DataDir
value is set in Line 33 in 4c30cde
I'm using CHARM_SERVER_DATA_DIR=charmdata charm serve
as a workaround that does the trick.
EDIT: tested with charm
pulled from main today
was curious what I could do with charm and was looking around. there is mention of seamless user auth. and I see jwt stuff in the godoc. but how to use it is alittle unclear. maybe add more docs to readme or add links to read more about how to use charm in this way.
type successMsg bool
should be a struct{}
type since successMsg
is literally just a success message. We're using other messages for errors.
https://github.com/charmbracelet/charm/blob/master/ui/linkgen/linkgen.go#L34
https://github.com/charmbracelet/charm/blob/master/ui/link/link.go#L41
It seems it's not currently possible to setup a custom location for the charm client keys (https://github.com/charmbracelet/charm/blob/main/client/env.go#L11), other than changing the "host" directory for the keys.
Would it be acceptable for DataPath
to be part of the Charm client Config
and default to the current way we calculate DataPath
to avoid breaking changes?
Happy to PR this.
I know ya'll are working on the documentation but I was just trying to use the CLI as a proof of concept.
I was able to link two devices nicely, I can see them both with charm keys
and the "Current Key" switches in a sensible way between the two machines.
kv
commands work fine for a single machine, but I can't figure out how to share across linked accounts. charm kv sync
doesn't appear to do anything for me. Am I misunderstanding the usage? Is this only the purview of skate
? I saw this (charmbracelet/skate#31) and I thought that I could just use charm kv
.
Thanks for the cool tools!
It would be nice if the crypto piece of charm could expand to signing data with my identity and verifying signed data.
Im specifically thinking about use cases like:
https://github.com/dominictarr/scuttlebutt
https://www.w3.org/TR/activitypub/
https://saltpack.org/
https://book.keybase.io/docs/cli#signing
where data is replicated and verified using signatures.
Requirements:
I recently tried the program Skate v0.1.2 and I think I observed a big bug with kv. A very big file (000001.vlog or 000003.vlog) is generated at the path ~/.local/share/charm/<charm server>/kv/charm.sh.skate.default/
. It is 2 GiB long it is mostly filled with null bytes.
This is very problematic as this prevent me to run Skate on my Raspberry Pi which have not enough RAM to load this file.
I observed this bug by using Skate release v0.1.2 on my local machine and on the self-hosted Charm server I tried the commit f4634fe and the release v0.9.1.
Listing out the keys using the the db.Keys() function gives me wrong results.
If I run a small program that creates 3 kv pairs and then use db.Keys() to list them out I get a duplicate key value.
I ran my example program to generate the keys, then used the charm cli tool to list my keys out, then run the program again which shows the difference between .Keys() and .View() functions:
➜ charm-test charm kv list @charm-kv-keys-test-db
key1 value1
key2 value2
key3 value3
➜ charm-test go run main.go
Charm kv .Keys()
0: key3
1: key2
2: key3
Charm kv .View() to list keys and values
key1 - value1
key2 - value2
key3 - value3
package main
import (
"github.com/charmbracelet/charm/kv"
"github.com/dgraph-io/badger/v3"
"fmt"
"log"
)
func main(){
// Open a database (or create one if it doesn’t exist)
db, err := kv.OpenWithDefaults("charm-kv-keys-test-db")
if err != nil {
log.Fatal(err)
}
defer db.Close()
// Save some data
if err := db.Set([]byte("key1"), []byte("value1")); err != nil {
log.Fatal(err)
}
if err := db.Set([]byte("key2"), []byte("value2")); err != nil {
log.Fatal(err)
}
if err := db.Set([]byte("key3"), []byte("value3")); err != nil {
log.Fatal(err)
}
fmt.Println("Charm kv .Keys()")
if keys,err := db.Keys(); err != nil{
log.Fatal(err)
}else{
for i,k := range keys{
fmt.Printf("%d: %s\n", i, string(k))
}
}
fmt.Println("Charm kv .View() to list keys and values")
// Go full-blown Badger and use transactions to list values and keys
db.View(func(txn *badger.Txn) error {
opts := badger.DefaultIteratorOptions
opts.PrefetchSize = 10
it := txn.NewIterator(opts)
defer it.Close() //nolint:errcheck
for it.Rewind(); it.Valid(); it.Next() {
item := it.Item()
k := item.Key()
err := item.Value(func(v []byte) error {
fmt.Printf("%s - %s\n", k, v)
return nil
})
if err != nil {
panic(err)
}
}
return nil
})
}
Disclaimer: I only started looking at Go last week, so could be that I'm doing something dumb here!
However, when running a debug session to try and figure it out, I did find that I could get it to work if I changed the following line in the kv.go file from
ks = append(ks, it.Item().Key())
to
ks = append(ks, it.Item().KeyCopy(nil))
I recently tried to self-host charm using the docker-compose file from this repository.
I am running charm behind Nginx with SSL certificates and I am able to use it on a single client. I am able to see files generated through charm fs cp ...
and also skate
or charm kv
generates the corresponding databases.
So I added a second client through charm link
I get a success and can see the key in charm keys
.
However, when I try to run charm fs ls /
on the second client, I receive an error.
Error: open /: Couldn't validate the authenticity of the ciphertext and associated data.
Trying to link a third client gives the same result/error.
Also I can see in the logs that the client tries to authenticate, which returns a 200 success. And when trying to use charm fs
I see the server returning 200 as well in the logs. So it seems the client is somewhat broken or having trouble decrypting the contents?
charm | 2022/01/24 14:22:07 <- GET /v1/fs//
charm | 2022/01/24 14:23:40 -> 200 OK 317B 1.384314ms
Using the same command on the first client returns my folder list. Any idea what I'm missing?
Hi developers,
Dependency github.com/jacobsa/crypto
is no longer maintained. The latest commit was 3 years ago, and the author isn't responding to new issues and PRs. It fails to build in riscv64
environment now. Do you have any plan to replace this dependency with a well-maintained alternative? Also, I can maintain a fork of it to make it build in riscv64
and accept new changes, so that Charm can use it.
Right now the charm backup-keys
command adds the kv
directory to the tarball it creates. KV DBs should be omitted so the backup is strictly just keys.
When running the Docker container, it creates files with root as the owner.
jeff@library:~/charm-r/data$ ls -al
total 28
drwxr-xr-x 5 root root 4096 Feb 27 18:08 .
drwxr-xr-x 3 jeff jeff 4096 Feb 27 18:08 ..
drwx------ 2 root root 4096 Feb 27 18:08 db
drwx------ 2 root root 4096 Feb 27 18:08 files
drwx------ 2 root root 4096 Feb 27 18:08 .ssh
-rw-r--r-- 1 root root 8192 Feb 27 18:08 stats
Ideally, there should be a way to set a uid/gid for the application, like how the linuxserver.io folks do their containers that allow environment variables to be set (PUID/PGID). See https://hub.docker.com/r/linuxserver/radarr and https://github.com/linuxserver/docker-baseimage-ubuntu for examples. The base image is where all the smarts are for PUID/PGID.
I was able to create my own docker image using linuxserver/docker-baseimage-ubuntu and adapting the raddar docker config. I'm not sure if that is the approach that should be taken, but the linuxserver folks have already done the hard work.
"charm with custom docker container"
jeff@library:~/charm/data$ ls -al
total 28
drwxr-xr-x 5 jeff jeff 4096 Feb 27 17:47 .
drwxr-xr-x 3 jeff jeff 4096 Feb 27 18:19 ..
drwx------ 2 jeff jeff 4096 Feb 27 17:47 db
drwx------ 3 jeff jeff 4096 Feb 27 17:47 files
drwx------ 2 jeff jeff 4096 Feb 27 17:46 .ssh
-rw-r--r-- 1 jeff jeff 8192 Feb 27 17:47 stats
On a freshly initialized charm server, if you try to copy the first file to charm:/
it breaks the file server:
[rubiojr@nano charm] charm fs cp hello.txt charm:/
[rubiojr@nano charm] charm fs cp hello.txt charm:/bar.txt
Error: server error: 500 Internal Server Error
Usage:
charm fs cp [charm:]PATH [charm:]PATH [flags]
Flags:
-h, --help help for cp
-r, --recursive copy directories recursively
server error: 500 Internal Server Error
Relevant logs from the server:
2021/12/18 12:47:06 Opening SQLite db: ./data/db
2021/12/18 12:47:06 Starting SSH server on :35353
2021/12/18 12:47:06 HTTP server listening on: :35354
2021/12/18 12:47:39 ssh api-auth
2021/12/18 12:47:39 JWT for user d0e256c3-e84a-4170-b4b8-98545e66b7cb
2021/12/18 12:47:39 <- POST /v1/encrypt-key [::1]
2021/12/18 12:47:39 Adding encrypted key 7ae84ea9-f714-4e3d-ae1b-6e7fb7c1eefc <nil> for user d0e256c3-e84a-4170-b4b8-98545e66b7cb
2021/12/18 12:47:39 -> 200 OK 0B 14.556183ms
2021/12/18 12:47:39 <- POST /v1/fs//?mode=420 [::1]
2021/12/18 12:47:39 -> 200 OK 0B 455.725µs
2021/12/18 12:48:05 ssh api-auth
2021/12/18 12:48:05 JWT for user d0e256c3-e84a-4170-b4b8-98545e66b7cb
2021/12/18 12:48:05 <- POST /v1/fs//6522161c3dfb9b87027e670c7d95dc8115ad8259a2d269?mode=420 [::1]
2021/12/18 12:48:05 cannot post file: open data/files/d0e256c3-e84a-4170-b4b8-98545e66b7cb/6522161c3dfb9b87027e670c7d95dc8115ad8259a2d269: not a directory
2021/12/18 12:48:05 -> 500 Internal Server Error 29B 399.826µs
It seems the server creates the user directory as a regular file instead of a directory and every other FS operation expects that root to be a directory?
Somewhat related, if you try to copy a file to the root of an already initialized server, you'll get a 500:
[rubiojr@nano charm] charm fs cp hello.txt charm:/hello.txt
[rubiojr@nano charm] charm fs cp hello.txt charm:/
Error: server error: 500 Internal Server Error
Usage:
charm fs cp [charm:]PATH [charm:]PATH [flags]
Flags:
-h, --help help for cp
-r, --recursive copy directories recursively
server error: 500 Internal Server Error
[rubiojr@nano charm] ./charm serve
2021/12/18 12:50:40 Opening SQLite db: ./data/db
2021/12/18 12:50:40 Starting SSH server on :35353
2021/12/18 12:50:40 HTTP server listening on: :35354
2021/12/18 12:50:50 ssh api-auth
2021/12/18 12:50:50 Creating user for key 02a9f5a4adccc36187270b76b75d743b5708a417
2021/12/18 12:50:50 JWT for user ba69b17a-6db9-4791-be4a-12bf5401cfec
2021/12/18 12:50:50 <- POST /v1/encrypt-key [::1]
2021/12/18 12:50:50 Adding encrypted key e10b870b-488d-4bb0-8888-eb0d6e7b209a <nil> for user ba69b17a-6db9-4791-be4a-12bf5401cfec
2021/12/18 12:50:50 -> 200 OK 0B 14.337438ms
2021/12/18 12:50:50 <- POST /v1/fs//e9bd3e5aa7f5f458de54de6f2779d130d4d41d7da30b79e780?mode=420 [::1]
2021/12/18 12:50:50 -> 200 OK 0B 506.6µs
2021/12/18 12:50:55 ssh api-auth
2021/12/18 12:50:55 JWT for user ba69b17a-6db9-4791-be4a-12bf5401cfec
2021/12/18 12:50:55 <- POST /v1/fs//?mode=420 [::1]
2021/12/18 12:50:55 cannot post file: open data/files/ba69b17a-6db9-4791-be4a-12bf5401cfec: is a directory
2021/12/18 12:50:55 -> 500 Internal Server Error 29B 389.371µs
Just wanted to let you know that after installing the RPM on Fedora 35, when I enter charm link
I'm greeted with something that looks like it exits early. It shows the linking progress indicator in a frozen state and just skips to showing my prompt again.
~ on ☁️ [email protected]
❯ charm link
You can link the SSH keys on another machine to your Charm
account so both machines have access to your stuff. Keys
can be unlinked at any time.
⣽ Generating link...
~ on ☁️ [email protected]
❯ charm fs
~ on ☁️ [email protected]
❯ charm link
You can link the SSH keys on another machine to your Charm
account so both machines have access to your stuff. Keys
can be unlinked at any time.
⣽ Generating link...
~ on ☁️ [email protected]
❯ charm link
Also, when I run charm
after installing the regular old Linux binary, I get the following error which is because my firewall has restrictions on outgoing ports:
❯ charm
Charm
Uh oh, there’s been an error: dial tcp 18.214.45.34:35353: connect: connection
~/Downloads/charm_0.9.2_linux_x86_64 on ☁️ [email protected] took 3s
It might make sense to use NGINX to proxy 443 (or 80) to 35353 for your production server since this might be a common issue with people behind restrictive enterprise firewalls
I have a Caddy server proxying a few other servers and doing TLS termination for them, so I'd like to do that with the Charm server also, and setup a Charm server using the docker container, behind a Caddy container doing the TLS termination.
not that this helps a lot but the issues looks less boring :)
The setup is straight forward but I'm finding some resistance: if I don't specify CHARM_SERVER_HTTP_SCHEME=https
server side (as documented in the README), the client tries to communicate using plain HTTP with the server eventually (presumably after being told to use http by the server), so I need to set CHARM_SERVER_HTTP_SCHEME=https
. However, if I do that, the server crashes when starting up because it obviously requires key material to do the TLS termination and I'm not setting CHARM_SERVER_TLS_KEY_FILE
and CHARM_SERVER_TLS_CERT_FILE
:
charm | 2021/12/09 16:02:51 HTTPS server listening on: :35354
charm | 2021/12/09 16:02:51 Server crashed: open : no such file or directory
It'd be nice if an external proxy (like Caddy in my case) could do the TLS termination while the server lets the client know it should still use https to communicate (I think this is related to the URL schema being used to send requests.
I'm currently using a small patch and seems to be working as expected, with Charm server serving plain text and Caddy doing the TLS termination, but keeping the client happy and the transport layer secure:
diff --git a/server/http.go b/server/http.go
index 642996f..7fed612 100644
--- a/server/http.go
+++ b/server/http.go
@@ -88,7 +88,7 @@ func (s *HTTPServer) Start() {
}()
log.Printf("%s server listening on: %s", strings.ToUpper(s.cfg.HTTPScheme), listenAddr)
- if useTls {
+ if useTls && !s.cfg.TLSDisableTermination {
log.Fatalf("Server crashed: %s", server.ListenAndServeTLS(s.cfg.TLSCertFile, s.cfg.TLSKeyFile))
} else {
log.Fatalf("Server crashed: %s", server.ListenAndServe())
diff --git a/server/server.go b/server/server.go
index 0ea177c..038e840 100644
--- a/server/server.go
+++ b/server/server.go
@@ -17,21 +17,22 @@ import (
// Config is the configuration for the Charm server.
type Config struct {
- Host string `env:"CHARM_SERVER_HOST" default:"localhost"`
- SSHPort int `env:"CHARM_SERVER_SSH_PORT" default:"35353"`
- HTTPPort int `env:"CHARM_SERVER_HTTP_PORT" default:"35354"`
- HTTPScheme string `env:"CHARM_SERVER_HTTP_SCHEME" default:"http"`
- StatsPort int `env:"CHARM_SERVER_STATS_PORT" default:"35355"`
- HealthPort string `env:"CHARM_SERVER_HEALTH_PORT" default:"35356"`
- DataDir string `env:"CHARM_SERVER_DATA_DIR" default:"./data"`
- TLSKeyFile string `env:"CHARM_SERVER_TLS_KEY_FILE" default:""`
- TLSCertFile string `env:"CHARM_SERVER_TLS_CERT_FILE" default:""`
- TLSConfig *tls.Config
- PublicKey []byte
- PrivateKey []byte
- DB db.DB
- FileStore storage.FileStore
- Stats stats.Stats
+ Host string `env:"CHARM_SERVER_HOST" default:"localhost"`
+ SSHPort int `env:"CHARM_SERVER_SSH_PORT" default:"35353"`
+ HTTPPort int `env:"CHARM_SERVER_HTTP_PORT" default:"35354"`
+ HTTPScheme string `env:"CHARM_SERVER_HTTP_SCHEME" default:"http"`
+ StatsPort int `env:"CHARM_SERVER_STATS_PORT" default:"35355"`
+ HealthPort string `env:"CHARM_SERVER_HEALTH_PORT" default:"35356"`
+ DataDir string `env:"CHARM_SERVER_DATA_DIR" default:"./data"`
+ TLSKeyFile string `env:"CHARM_SERVER_TLS_KEY_FILE" default:""`
+ TLSCertFile string `env:"CHARM_SERVER_TLS_CERT_FILE" default:""`
+ TLSConfig *tls.Config
+ TLSDisableTermination bool `env:"CHARM_SERVER_TLS_DISABLE_TERMINATION"`
+ PublicKey []byte
+ PrivateKey []byte
+ DB db.DB
+ FileStore storage.FileStore
+ Stats stats.Stats
}
// Server contains the SSH and HTTP servers required to host the Charm Cloud.
Diff here. I'm currently unaware of how many kittens per day this patch could eat.
With that patch, I can start the server with the following env variables and have Caddy in front, serving and doing the TLS termination:
CHARM_SERVER_TLS_DISABLE_TERMINATION=1 CHARM_SERVER_HTTP_SCHEME=https CHARM_SERVER_HOST=my.awesome.charm ./charm serve
Did I miss something or having external TLS termination isn't currently possible?
❤️
I've installed charm to test the Zsh completion. The instructions say to put the generated output from charm completion zsh
to a file _charm
in $fpath
. The generated output defines all the necessary completion functions, but does not call _charm
at the end.
This means that in the first tab completion, only the function definitions are loaded, so you need to press TAB a second time to get completions.
I've seen you use https://github.com/spf13/cobra for the completion generation. Something like this would be needed to fix it, so maybe you just need to update Cobra: https://github.com/spf13/cobra/blob/52e6099aead5c377be3c46f34bf0c2d3e0c128b6/zsh_completions.go#L251-L254
Thanks!
Custom SSH keys are supported as of v0.11.0 with a env var. We should also support the -i KEYFILE
flag on all commands.
Hi, I tried to spin up a self hosted charm using docker, the docker image seems to store invalid created_at
dates for encrypt_key
. When using charm serve
instead, there is no problem.
On a first glance regarding the error message:
The docker image seems to set encrypt_key.created_at
to 2021-12-08 12:15:47.525768 +0100 +0100
while charm serve
stores 2021-12-08 12:09:31.894222 +0100 CET
. I guess that produces the error message Error fetching encrypt keys: sql: Scan error on column index 2, name "created_at": unsupported Scan, storing driver.Value type string into type *time.Time
.
brew install charmbracelet/tap/charm
)brew install charmbracelet/tap/skate
)skate set hello world
-> Encryption key mismatchskate set hello world
-> failed crypt checkcharm serve
skate list
-> failed crypt check$ rm -rf ./data "~/Library/Application Support/charm/"
$ cat docker-compose.yml
---
version: "3.1"
services:
soft-serve:
image: charmcli/charm:v0.9.0
container_name: charm
volumes:
- ./data:/data
ports:
- 35353:35353
- 35354:35354
- 35355:35355
- 35356:35356
restart: unless-stopped
$ docker-compose up
Creating network "charm_default" with the default driver
Creating charm ... done
Attaching to charm
charm | 2021/12/08 11:15:41 Opening SQLite db: /data/db
charm | 2021/12/08 11:15:42 Starting SSH server on :35353
charm | 2021/12/08 11:15:42 HTTP server listening on: :35354
charm | 2021/12/08 11:15:47 ssh api-auth
charm | 2021/12/08 11:15:47 Creating user for key 37f0505b79ab53d19e595489d7911c28f348ce6a
charm | 2021/12/08 11:15:47 JWT for user b3ef3220-3beb-401d-8b48-e0fb9a0f30e4
charm | 2021/12/08 11:15:47 <- POST /v1/encrypt-key 172.21.0.1
charm | 2021/12/08 11:15:47 Adding encrypted key 539f7c93-a0fa-4d65-8279-f414e3fbf699 2021-12-08 12:15:47.525768 +0100 +0100 for user b3ef3220-3beb-401d-8b48-e0fb9a0f30e4
charm | 2021/12/08 11:15:47 -> 200 OK 0B 33.6366ms
charm | 2021/12/08 11:15:49 ssh api-auth
charm | 2021/12/08 11:15:49 JWT for user b3ef3220-3beb-401d-8b48-e0fb9a0f30e4
charm | 2021/12/08 11:15:49 Error fetching encrypt keys: sql: Scan error on column index 2, name "created_at": unsupported Scan, storing driver.Value type string into type *time.Time
^CGracefully stopping... (press Ctrl+C again to force)
Killing charm ... done
ERROR: 2
$ charm serve
2021/12/08 12:16:04 Opening SQLite db: ./data/db
2021/12/08 12:16:04 Starting SSH server on :35353
2021/12/08 12:16:04 HTTP server listening on: :35354
2021/12/08 12:16:09 ssh api-auth
2021/12/08 12:16:09 JWT for user b3ef3220-3beb-401d-8b48-e0fb9a0f30e4
2021/12/08 12:16:09 Error fetching encrypt keys: sql: Scan error on column index 2, name "created_at": unsupported Scan, storing driver.Value type string into type *time.Time
Here the commands for the client
$ export CHARM_HOST=127.0.0.1
# during docker-compose up
$ skate set hello world
Error: Encryption key mismatch
Usage:
set KEY[@DB] VALUE [flags]
Flags:
-h, --help help for set
Encryption key mismatch
$ skate set hello world
Error: failed crypt check
Usage:
set KEY[@DB] VALUE [flags]
Flags:
-h, --help help for set
failed crypt check
# during charm serve
$ skate list
Error: failed crypt check
Usage:
list [@DB] [flags]
Flags:
-d, --delimiter string delimiter to separate keys and values (default "\t")
-h, --help help for list
-k, --keys-only only print keys and don't fetch values from the db
-r, --reverse list in reverse lexicographic order
-v, --values-only only print values
failed crypt check
charm serve
charm serve
skate set hello world
-> worksskate list
-> works$ charm --version
charm version v0.9.0 (e554127)
$ rm -rf ./data "~/Library/Application Support/charm/"
$ charm serve
2021/12/08 12:09:17 Opening SQLite db: ./data/db
2021/12/08 12:09:19 Starting SSH server on :35353
2021/12/08 12:09:19 HTTP server listening on: :35354
2021/12/08 12:09:31 ssh api-auth
2021/12/08 12:09:31 Creating user for key 37f0505b79ab53d19e595489d7911c28f348ce6a
2021/12/08 12:09:31 JWT for user 72dad0be-b338-4f45-918f-519c1cae3806
2021/12/08 12:09:31 <- POST /v1/encrypt-key 127.0.0.1
2021/12/08 12:09:31 Adding encrypted key 42f2ce6f-89c5-4c1f-baa4-5cee035cd3d7 2021-12-08 12:09:31.894222 +0100 CET for user 72dad0be-b338-4f45-918f-519c1cae3806
2021/12/08 12:09:31 -> 200 OK 0B 2.310272ms
2021/12/08 12:09:31 <- GET /v1/seq/b1f17f58c3aed7b14e097fbc55b3dc410d9f9c92f3cd974544e358cb0b9e26d82db7808901b0 127.0.0.1
2021/12/08 12:09:31 -> 200 OK 10B 2.989027ms
2021/12/08 12:09:31 <- GET /v1/fs/b1f17f58c3aed7b14e097fbc55b3dc410d9f9c92f3cd974544e358cb0b9e26d82db7808901b0 127.0.0.1
2021/12/08 12:09:31 -> 404 Not Found 29B 568.249µs
2021/12/08 12:09:31 <- POST /v1/seq/b1f17f58c3aed7b14e097fbc55b3dc410d9f9c92f3cd974544e358cb0b9e26d82db7808901b0 127.0.0.1
2021/12/08 12:09:31 -> 200 OK 10B 1.580633ms
2021/12/08 12:09:31 <- POST /v1/fs/b1f17f58c3aed7b14e097fbc55b3dc410d9f9c92f3cd974544e358cb0b9e26d82db7808901b0/825034e5c6b9fc697cb0214f873d34baa8?mode=432 127.0.0.1
2021/12/08 12:09:31 -> 200 OK 0B 1.983326ms
^C
$ cat docker-compose.yml
---
version: "3.1"
services:
soft-serve:
image: charmcli/charm:v0.9.0
container_name: charm
volumes:
- ./data:/data
ports:
- 35353:35353
- 35354:35354
- 35355:35355
- 35356:35356
restart: unless-stopped
$ docker-compose up
Starting charm ... done
Attaching to charm
charm | 2021/12/08 11:09:46 Opening SQLite db: /data/db
charm | 2021/12/08 11:09:46 Starting SSH server on :35353
charm | 2021/12/08 11:09:46 HTTP server listening on: :35354
charm | 2021/12/08 11:09:50 ssh api-auth
charm | 2021/12/08 11:09:50 JWT for user 72dad0be-b338-4f45-918f-519c1cae3806
charm | 2021/12/08 11:09:50 <- GET /v1/fs/b1f17f58c3aed7b14e097fbc55b3dc410d9f9c92f3cd974544e358cb0b9e26d82db7808901b0 172.19.0.1
charm | 2021/12/08 11:09:50 -> 200 OK 308B 9.3245ms
And here are the commands of the client:
$ export CHARM_HOST=127.0.0.1
# while charm serve
$ skate set hello world
# while docker-compose up
$ skate list
hello world
Any skate set
with this user will succeed with docker-compose from now on.
"charm kv" is not working right on an arm64 system (Raspberry Pi, Debian Buster).
> charm kv list
foo bar
~ ......................................................................................................................
> charm kv set foo baz
Error: while opening memtables error: while opening fid: 3 error: while updating skiplist error: mremap size mismatch: r
Usage:
charm kv set KEY[@DB] VALUE [flags]
Flags:
-h, --help help for set
Error: while opening memtables error: while opening fid: 5 error: while updating skiplist error: mremap size mismatch: requested: 72 got: 134217728
Not really sure what's going on, but it has a strong smell of being this:
https://discuss.dgraph.io/t/error-mremap-size-mismatch-on-arm64/15333
for which a similar issue has this patch:
Show current host in the UI -> show the URL to the host or its nickname so it's clear where their data is hosted
(mentioned in the Slack)
Please see charmbracelet/homebrew-tap#1
Encryption example states you can do:
charm encrypt < myfile.txt > encryptedfile.txt
but the command is actually charm crypt encrypt < myfile.txt > encryptedfile.txt
Also when you just run charm crypt
nothing seems to happen which can be confusing to new users. Should have some kind of feedback (maybe pop open help menu since it requires commands or flags to do something)
Setting a style width in the App
style would solve this
Line 387 in fb3611b
Expected non-truncated output:
Uh oh, there’s been an error: dial tcp: address cloud.dev.charm.sh:32984712983741982037410831923987414379843870923:35353: too many colons in address
> ssh git.charm.sh
z4h: failed to upload bootstrap script
Is that a problem with your server or with my zsh?
P.S. I guess it is with my zsh, using z4h. I tried with sh and it worked. Anyway, shouldn't it work with zsh?
Cool tool!
I was wondering why RSA is the default algorithm instead of Ed255 (which you already implemented)?
Also, if you supported P256, there could be a nice integration with TPM 2.0 (e.g. with go-tpm), to have keys that are actually bound to the machine they're generated on vs. lying around in ~/.local/share/charm
.
As I can't run both OS at the same time, I can't complete the verification process.
I've tried backing up and loading the backup on the other OS, but that didn't work either.
➜ dev go get github.com/charmbracelet/charm/cmd/charm
# github.com/charmbracelet/charm/ui/username
../go/src/github.com/charmbracelet/charm/ui/username/username.go:130:20: too many arguments in call to textinput.Blink
have (textinput.Model)
want ()
../go/src/github.com/charmbracelet/charm/ui/username/username.go:130:20: cannot use textinput.Blink(m.input) (type tea.Msg) as type tea.Cmd in return argument: need type assertion
../go/src/github.com/charmbracelet/charm/ui/username/username.go:194:20: undefined: textinput.Update
../go/src/github.com/charmbracelet/charm/ui/username/username.go:234:18: undefined: textinput.Update
../go/src/github.com/charmbracelet/charm/ui/username/username.go:243:7: undefined: textinput.View
➜ dev go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/stephen/.cache/go-build"
GOENV="/home/stephen/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/stephen/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/stephen/go-1.14"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/stephen/go-1.14/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build142608877=/tmp/go-build -gno-record-gcc-switches"
I get why - but it might be nice to at least have an option to choose and use a pre-existing encrypted ssh key, or to encrypt the key and save the password for it in the keychain (I'm on macOS).
This affects binary output in particular. See: charmbracelet/skate#16
When importing keys, your keys in your data dir may be overriden if you accept it, it does not, though, remove the old keys from your account.
Not sure if this is the expected behavior or if we should change it so it unlinks the key as well
Update docs to cover data storage max for our servers, how to change storage limits on self-hosted server, how and when we encrypt
Right now, we don't try to operate on the path specified in the charm import-keys
argument until absolutely necessary. In particular, after potentially prompting the user about overwriting. We should stat the path in the argument first so we can exit before putting throwing up a prompt in front of the user.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.