axiomhq / cli Goto Github PK

View Code? Open in Web Editor NEW

42.0 42.0 11.0 7.72 MB

The power of Axiom on the command line.

Home Page: https://axiom.co

License: MIT License

Makefile 2.78% Go 97.00% Dockerfile 0.22%

axiom cli command-line command-line-tool go golang

cli's People

Contributors

Stargazers

Watchers

Forkers

fossabot kamstrup bahlo ted-gould thecraftman jeremyw-dobeu adeboyedn s-scheele-simonsen excloudx6

cli's Issues

First install & authentication can result in an `API Error 404: Not Found`

What happened

I just installed the cli and followed the basic initial steps to get going:

install cli with ❯ go install github.com/axiomhq/cli/cmd/axiom@latest
auth with ❯ axiom auth login
successful browser auth w/ my github account

The moment the oauth callback returned to localhost the CLI returned gave me a 404. Screenshot shows full context.

OS/versions

~ via 🐹 v1.22.2 via  v20.11.1 on ☁️  (eu-west-2) took 3s 
❯ uname -a
Linux b7 6.8.8-300.fc40.x86_64 #1 SMP PREEMPT_DYNAMIC Sat Apr 27 17:53:31 UTC 2024 x86_64 GNU/Linux

~ via 🐹 v1.22.2 via  v20.11.1 on ☁️  (eu-west-2) 
❯ getenforce
Enforcing

~ via 🐹 v1.22.2 via  v20.11.1 on ☁️  (eu-west-2) 
❯ google-chrome --version
Google Chrome 126.0.6452.3 dev

Ingest Error: 504 API Timeout

Twice in the last 3 months has ingest stopped due to an error on timeout and a failure to reconnect. The fix has been to just keep deployments going at smaller intervals, but for some of our longer uptime services, the timeouts stopping all ingest has been confusing.

This is the error:

Error: could not ingest "stdin" into dataset "NAME": API error 504: Gateway Timeout

The deployments are done inside of a Docker container with this initiating the CLI:

ENTRYPOINT ./APP | axiom ingest DATASET

I'm not sure what would be causing the API timeout multiple times like this, but perhaps there could be a --continue-on-error flag that would keep trying to reconnect through all errors and just keep logging them to the console? The error was only logged once so its unknown how many times it took to give up trying. At least with a continual log we could see the number of attempts.

Error: Chosen deployment is not an Axiom Cloud deployment

Likely caused by having cloud.axiom.co as the base URL without a leading http scheme.

Personal token always invalid

echo "4bde8051-1428-47da-bcb5-f6a30d5ec4ae" | axiom auth login --url="http://localhost:8080" --alias="local" --token-stdin --token-type personal -f
Error: invalid authentication credentials

Access tokens work without issues.

Add package to npm

This would allow easier cli installing on windows.

axiom help environment is a bit ugly

This could be prettier. Perhaps wrap/trim for one, at ~70 chars, like axiom help does

$ ~/go/bin/axiom help environment
AXIOM_DEPLOYMENT: The deployment to use. Overwrittes the choice loaded from the configuration file.

AXIOM_ORG_ID: The organization id of the organization the access token is valid for.

AXIOM_PAGER, PAGER (in order of precedence): A terminal paging program to send standard output to, e.g. "less".

AXIOM_TOKEN: Token The access token to use. Overwrittes the choice loaded from the configuration file.

AXIOM_URL: The deployment url to use. Overwrittes the choice loaded from the configuration file.

VISUAL, EDITOR (in order of precedence): The editor to use for authoring text.

NO_COLOR: Set to any value to avoid printing ANSI escape sequences for color output.

CLICOLOR: Set to "0" to disable printing ANSI colors in output.

CLICOLOR_FORCE: Set to a value other than "0" to keep ANSI colors in output even when the output is piped.

Note how it doesn't fit in a code block ^ :D

`axiom auth select` auto selects the already configured deployment

Token type is not documented on auth login

Typo in the help

$ ~/go/bin/axiom help environment
AXIOM_DEPLOYMENT: The deployment to use. Overwrittes the choice loaded from the configuration file.

s/Overwrittes/Overwrites/

If only one dataset is available, don't present a select options

Your CLT does not support macOS 11.

MacOS 11.0.1
Error: Your CLT does not support macOS 11.
It is either outdated or was modified.
Please update your CLT or delete it if no updates are available.
Update them from Software Update in System Preferences or run:
softwareupdate --all --install --force

If that doesn't show you an update run:
sudo rm -rf /Library/Developer/CommandLineTools
sudo xcode-select --install

Alternatively, manually download them from:
https://developer.apple.com/download/more/.

Error: An exception occurred within a child process:
SystemExit: exit

axiom dataset list panic (with one dataset called `test`)

$ axiom dataset list

Showing 1 dataset:

panic: runtime error: index out of range [-1]

goroutine 1 [running]:
github.com/axiomhq/cli/cmd/axiom/dataset.runList.func2(0x6f02398, 0xc0002a2b70, 0xffffffffffffffff)
        github.com/axiomhq/cli/cmd/axiom/dataset/dataset_list.go:91 +0x177
github.com/axiomhq/cli/pkg/iofmt.FormatToTable(0xc000098000, 0x1, 0xc0000d3c90, 0x0, 0xc0000d3c68, 0x0, 0x0)
        github.com/axiomhq/cli/pkg/iofmt/table.go:52 +0x29f
github.com/axiomhq/cli/cmd/axiom/dataset.runList(0x3eb2758, 0xc000094000, 0xc0000a6348, 0x0, 0x0)
        github.com/axiomhq/cli/cmd/axiom/dataset/dataset_list.go:98 +0x2aa
github.com/axiomhq/cli/cmd/axiom/dataset.newListCmd.func1(0xc0000dfb80, 0x4145360, 0x0, 0x0, 0x0, 0x0)
        github.com/axiomhq/cli/cmd/axiom/dataset/dataset_list.go:43 +0x47
github.com/spf13/cobra.(*Command).execute(0xc0000dfb80, 0x4145360, 0x0, 0x0, 0xc0000dfb80, 0x4145360)
        github.com/spf13/[email protected]/command.go:852 +0x472
github.com/spf13/cobra.(*Command).ExecuteC(0xc0000de000, 0xc0000de000, 0x3eb2758, 0xc000094000)
        github.com/spf13/[email protected]/command.go:960 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
        github.com/spf13/[email protected]/command.go:897
github.com/spf13/cobra.(*Command).ExecuteContext(...)
        github.com/spf13/[email protected]/command.go:890
main.main()
        github.com/axiomhq/cli/cmd/axiom/main.go:96 +0x351

Error: please provide options to select from

$ axiom -I -U=http://localhost:8081 -T=xaat-1234 ingest -f=test.json
Error: please provide options to select from

Likely caused by the function that fetches the datasets to select from (no argument given above) as it cannot list datasets with an API token (xaat- prefix).

Ingest with shell expanded list of files throws "Error: accepts at most 1 arg(s), received x"

The example of the ingest command says:

# Ingest the contents of all files inside /var/logs/nginx with
# extension ".log" into a dataset named "nginx-logs":
$ axiom ingest nginx-logs -f /var/logs/nginx/*.log

But it actually throws Error: accepts at most 1 arg(s), received x, with x being the amount of actual files the shell expanded to.

Export command

axiom config export <login> to export AXIOM_URL, AXIOM_TOKEN and AXIOM_ORG_ID to the curent session.

Remove -c option

It's not clear that it's ever a good idea to switch compression off, and the compression is onyl affecting the internal protocol we talk to Axiom with; so it's more or less a debugging option...

Auth login examples incomplete

axiom auth login --help prints this example:

# Provide parameters on the command-line:
$ echo $AXIOM_ACCESS_TOKEN | axiom auth login --url="https://axiom.eu-west-1.aws.com" --alias="axiom-eu-west-1" --token-stdin

Running it returns an error: Error: --token-type required when --token-stdin is set

Store token in OS keyring

Use something like https://github.com/zalando/go-keyring to access the OS keyring on all major operating systems.

Axiom CLI never sleeps

Have the axiom tool setup to ingest data and it is constantly using CPU. Even when processing no data.

Upgrade token

Follow-up from #34. Would be great to be able to run something like axiom auth update-token to replace an ingest token with a personal access token.

Should also be suggested in the help introduce in #34.

Examples erroneously suggest nginx log ingestion possible

Green user, trying axiom cli for the first time. Goal was to ingest nginx logs.

Googled "axiom nginx logs" which led to a first-party blog post from March 2021. It suggests nginx logs can be directly ingested into axiom cloud.

# Ingest the contents of all files inside /var/logs/nginx
# with extension ".log" into a dataset named "nginx-logs":
$ axiom ingest nginx-logs -f /var/logs/nginx/*.log

This does not work.

axiom ingest mastodon -f /var/log/nginx/access.log
Error: could not detect "/var/log/nginx/access.log" content type: cannot determine content type

We should probably:

Update the internal documentation in root.go

And separately:

Remove or update the blog post
Identify any other docs/blog posts or tutorials that are outdated/incorrect.

panic on "axiom auth status"

Leaving this here for reference. Synchronisation bug, will fix ASAP.

fatal error: concurrent map writes

goroutine 39 [running]:
github.com/axiomhq/cli/internal/cmd/auth.runStatus.func1.1()
        github.com/axiomhq/cli/internal/cmd/auth/auth_status.go:86 +0x38
github.com/axiomhq/cli/internal/cmd/auth.runStatus.func1()
        github.com/axiomhq/cli/internal/cmd/auth/auth_status.go:92 +0x154
golang.org/x/sync/errgroup.(*Group).Go.func1()
        golang.org/x/[email protected]/errgroup/errgroup.go:75 +0x58
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1
        golang.org/x/[email protected]/errgroup/errgroup.go:72 +0x98

goroutine 1 [semacquire]:
sync.runtime_Semacquire(0x140002df9b8?)
        runtime/sema.go:62 +0x2c
sync.(*WaitGroup).Wait(0x140000c7008)
        sync/waitgroup.go:116 +0x74
golang.org/x/sync/errgroup.(*Group).Wait(0x140000c7000)
        golang.org/x/[email protected]/errgroup/errgroup.go:53 +0x2c
github.com/axiomhq/cli/internal/cmd/auth.runStatus({0x10473f1d8?, 0x14000139bc0}, 0x140000cc3c0)
        github.com/axiomhq/cli/internal/cmd/auth/auth_status.go:121 +0x290
github.com/axiomhq/cli/internal/cmd/auth.newStatusCmd.func1(0x140003ac000?, {0x104b1f5a0?, 0x0?, 0x0?})
        github.com/axiomhq/cli/internal/cmd/auth/auth_status.go:53 +0x2c
github.com/spf13/cobra.(*Command).execute(0x14000393500, {0x104b1f5a0, 0x0, 0x0})
        github.com/spf13/[email protected]/command.go:983 +0x82c
github.com/spf13/cobra.(*Command).ExecuteC(0x140000e2300)
        github.com/spf13/[email protected]/command.go:1115 +0x344
github.com/spf13/cobra.(*Command).ExecuteContextC(...)
        github.com/spf13/[email protected]/command.go:1048
main.main()
        github.com/axiomhq/cli/cmd/axiom/main.go:94 +0x474

goroutine 5 [syscall]:
os/signal.signal_recv()
        runtime/sigqueue.go:149 +0x2c
os/signal.loop()
        os/signal/signal_unix.go:23 +0x1c
created by os/signal.Notify.func1.1 in goroutine 1
        os/signal/signal.go:151 +0x28

goroutine 33 [select]:
os/signal.NotifyContext.func1()
        os/signal/signal.go:288 +0x68
created by os/signal.NotifyContext in goroutine 1
        os/signal/signal.go:287 +0x15c

goroutine 34 [sleep]:
time.Sleep(0x8f0d180)
        runtime/time.go:195 +0x10c
github.com/briandowns/spinner.(*Spinner).Start.func1()
        github.com/briandowns/[email protected]/spinner.go:375 +0x40
created by github.com/briandowns/spinner.(*Spinner).Start in goroutine 1
        github.com/briandowns/[email protected]/spinner.go:334 +0x10c

goroutine 35 [select]:
net/http.(*Transport).getConn(0x1400040c000, 0x14000422080, {{}, 0x0, {0x1400042a000, 0x5}, {0x14000428040, 0x1c}, 0x0})
        net/http/transport.go:1393 +0x448
net/http.(*Transport).roundTrip(0x1400040c000, 0x1400042c000)
        net/http/transport.go:591 +0x658
net/http.(*Transport).RoundTrip(0x1046f3a60?, 0x140004121e0?)
        net/http/roundtrip.go:17 +0x1c
github.com/klauspost/compress/gzhttp.(*gzRoundtripper).RoundTrip(0x14000412000, 0x1400042c000)
        github.com/klauspost/[email protected]/gzhttp/transport.go:81 +0x190
net/http.send(0x1400042c000, {0x10473b9e8, 0x14000412000}, {0x1043802f0?, 0x8?, 0x0?})
        net/http/client.go:260 +0x4e0
net/http.(*Client).send(0x14000412030, 0x1400042c000, {0x140003b0698?, 0x104108028?, 0x0?})
        net/http/client.go:181 +0x9c
net/http.(*Client).do(0x14000412030, 0x1400042c000)
        net/http/client.go:724 +0x6f4
net/http.(*Client).Do(...)
        net/http/client.go:590
github.com/axiomhq/axiom-go/axiom.(*Client).Do(0x14000410150, 0x1400042c000, {0x10466cbe0, 0x14000422000})
        github.com/axiomhq/[email protected]/axiom/client.go:286 +0xfc
github.com/axiomhq/axiom-go/axiom.(*Client).Call(0x140000c7040?, {0x10473f018?, 0x140004120f0?}, {0x10454d1f2?, 0x140003b0c18?}, {0x1045517e5?, 0x14000410150?}, {0x0?, 0x0?}, {0x10466cbe0, ...})
        github.com/axiomhq/[email protected]/axiom/client.go:181 +0x58
github.com/axiomhq/axiom-go/axiom.(*UsersService).Current(0x140004180a8, {0x10473f1d8?, 0x14000139bc0?})
        github.com/axiomhq/[email protected]/axiom/users.go:84 +0xfc
github.com/axiomhq/axiom-go/axiom.(*Client).ValidateCredentials(0x14000410150, {0x10473f1d8, 0x14000139bc0})
        github.com/axiomhq/[email protected]/axiom/client.go:165 +0x98
github.com/axiomhq/cli/internal/client.New({0x10473f1d8, 0x14000139bc0}, {0x140000ac080?, 0x20?}, {0x140000c4000, 0x29}, {0x140000ac0c0, 0x12}, 0x0)
        github.com/axiomhq/cli/internal/client/client.go:61 +0x574
github.com/axiomhq/cli/internal/cmd/auth.runStatus.func1()
        github.com/axiomhq/cli/internal/cmd/auth/auth_status.go:89 +0xc0
golang.org/x/sync/errgroup.(*Group).Go.func1()
        golang.org/x/[email protected]/errgroup/errgroup.go:75 +0x58
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1
        golang.org/x/[email protected]/errgroup/errgroup.go:72 +0x98

goroutine 36 [running]:
        goroutine running on other thread; stack unavailable
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1
        golang.org/x/[email protected]/errgroup/errgroup.go:72 +0x98

goroutine 37 [select]:
net/http.(*Transport).getConn(0x14000494000, 0x140004ac080, {{}, 0x0, {0x140004b2000, 0x5}, {0x1400049e090, 0x10}, 0x0})
        net/http/transport.go:1393 +0x448
net/http.(*Transport).roundTrip(0x14000494000, 0x140004b4000)
        net/http/transport.go:591 +0x658
net/http.(*Transport).RoundTrip(0x1046f3a60?, 0x1400049a1e0?)
        net/http/roundtrip.go:17 +0x1c
github.com/klauspost/compress/gzhttp.(*gzRoundtripper).RoundTrip(0x1400049a000, 0x140004b4000)
        github.com/klauspost/[email protected]/gzhttp/transport.go:81 +0x190
net/http.send(0x140004b4000, {0x10473b9e8, 0x1400049a000}, {0x1043802f0?, 0x8?, 0x0?})
        net/http/client.go:260 +0x4e0
net/http.(*Client).send(0x1400049a030, 0x140004b4000, {0x14000490698?, 0x104108028?, 0x0?})
        net/http/client.go:181 +0x9c
net/http.(*Client).do(0x1400049a030, 0x140004b4000)
        net/http/client.go:724 +0x6f4
net/http.(*Client).Do(...)
        net/http/client.go:590
github.com/axiomhq/axiom-go/axiom.(*Client).Do(0x14000498150, 0x140004b4000, {0x10466cbe0, 0x140004ac000})
        github.com/axiomhq/[email protected]/axiom/client.go:286 +0xfc
github.com/axiomhq/axiom-go/axiom.(*Client).Call(0x140000c7040?, {0x10473f018?, 0x1400049a0f0?}, {0x10454d1f2?, 0x14000490c18?}, {0x1045517e5?, 0x14000498150?}, {0x0?, 0x0?}, {0x10466cbe0, ...})
        github.com/axiomhq/[email protected]/axiom/client.go:181 +0x58
github.com/axiomhq/axiom-go/axiom.(*UsersService).Current(0x140004a00a8, {0x10473f1d8?, 0x14000139bc0?})
        github.com/axiomhq/[email protected]/axiom/users.go:84 +0xfc
github.com/axiomhq/axiom-go/axiom.(*Client).ValidateCredentials(0x14000498150, {0x10473f1d8, 0x14000139bc0})
        github.com/axiomhq/[email protected]/axiom/client.go:165 +0x98
github.com/axiomhq/cli/internal/client.New({0x10473f1d8, 0x14000139bc0}, {0x140000ac160?, 0x14?}, {0x140000c4080, 0x29}, {0x1400009c3e0, 0xd}, 0x0)
        github.com/axiomhq/cli/internal/client/client.go:61 +0x574
github.com/axiomhq/cli/internal/cmd/auth.runStatus.func1()
        github.com/axiomhq/cli/internal/cmd/auth/auth_status.go:89 +0xc0
golang.org/x/sync/errgroup.(*Group).Go.func1()
        golang.org/x/[email protected]/errgroup/errgroup.go:75 +0x58
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1
        golang.org/x/[email protected]/errgroup/errgroup.go:72 +0x98

goroutine 38 [select]:
net/http.(*Transport).getConn(0x1400050c000, 0x14000522080, {{}, 0x0, {0x14000528000, 0x5}, {0x1400053a020, 0x20}, 0x0})
        net/http/transport.go:1393 +0x448
net/http.(*Transport).roundTrip(0x1400050c000, 0x1400052a000)
        net/http/transport.go:591 +0x658
net/http.(*Transport).RoundTrip(0x1046f3a60?, 0x140005121e0?)
        net/http/roundtrip.go:17 +0x1c
github.com/klauspost/compress/gzhttp.(*gzRoundtripper).RoundTrip(0x14000512000, 0x1400052a000)
        github.com/klauspost/[email protected]/gzhttp/transport.go:81 +0x190
net/http.send(0x1400052a000, {0x10473b9e8, 0x14000512000}, {0x1043802f0?, 0x8?, 0x0?})
        net/http/client.go:260 +0x4e0
net/http.(*Client).send(0x14000512030, 0x1400052a000, {0x1400048c698?, 0x104108028?, 0x0?})
        net/http/client.go:181 +0x9c
net/http.(*Client).do(0x14000512030, 0x1400052a000)
        net/http/client.go:724 +0x6f4
net/http.(*Client).Do(...)
        net/http/client.go:590
github.com/axiomhq/axiom-go/axiom.(*Client).Do(0x14000510150, 0x1400052a000, {0x10466cbe0, 0x14000522000})
        github.com/axiomhq/[email protected]/axiom/client.go:286 +0xfc
github.com/axiomhq/axiom-go/axiom.(*Client).Call(0x140000c7040?, {0x10473f018?, 0x140005120f0?}, {0x10454d1f2?, 0x1400048cc18?}, {0x1045517e5?, 0x14000510150?}, {0x0?, 0x0?}, {0x10466cbe0, ...})
        github.com/axiomhq/[email protected]/axiom/client.go:181 +0x58
github.com/axiomhq/axiom-go/axiom.(*UsersService).Current(0x140005180a8, {0x10473f1d8?, 0x14000139bc0?})
        github.com/axiomhq/[email protected]/axiom/users.go:84 +0xfc
github.com/axiomhq/axiom-go/axiom.(*Client).ValidateCredentials(0x14000510150, {0x10473f1d8, 0x14000139bc0})
        github.com/axiomhq/[email protected]/axiom/client.go:165 +0x98
github.com/axiomhq/cli/internal/client.New({0x10473f1d8, 0x14000139bc0}, {0x140000c40c0?, 0x24?}, {0x140000c4100, 0x29}, {0x140000ac200, 0x1e}, 0x0)
        github.com/axiomhq/cli/internal/client/client.go:61 +0x574
github.com/axiomhq/cli/internal/cmd/auth.runStatus.func1()
        github.com/axiomhq/cli/internal/cmd/auth/auth_status.go:89 +0xc0
golang.org/x/sync/errgroup.(*Group).Go.func1()
        golang.org/x/[email protected]/errgroup/errgroup.go:75 +0x58
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1
        golang.org/x/[email protected]/errgroup/errgroup.go:72 +0x98

goroutine 49 [select]:
net.(*Resolver).lookupIPAddr(0x104ae81e0, {0x10473f018?, 0x140004123c0}, {0x10454d2e8, 0x3}, {0x14000428040, 0x18})
        net/lookup.go:332 +0x320
net.(*Resolver).internetAddrList(0x10473f018?, {0x10473f018?, 0x140004123c0?}, {0x10454d2e8, 0x3}, {0x14000428040?, 0x30?})
        net/ipsock.go:288 +0x4cc
net.(*Resolver).resolveAddrList(0x10473f0c0?, {0x10473f018, 0x140004123c0}, {0x10454d996, 0x4}, {0x10454d2e8?, 0x0?}, {0x14000428040, 0x1c}, {0x0?, ...})
        net/dial.go:282 +0x400
net.(*Dialer).DialContext(0x14000410000, {0x10473f018, 0x140004121b0}, {0x10454d2e8, 0x3}, {0x14000428040, 0x1c})
        net/dial.go:488 +0x32c
net/http.(*Transport).dial(0x29?, {0x10473f018?, 0x140004121b0?}, {0x10454d2e8?, 0x140000c4000?}, {0x14000428040?, 0x140000ac0c0?})
        net/http/transport.go:1183 +0xdc
net/http.(*Transport).dialConn(0x1400040c000, {0x10473f018, 0x140004121b0}, {{}, 0x0, {0x1400042a000, 0x5}, {0x14000428040, 0x1c}, 0x0})
        net/http/transport.go:1625 +0x61c
net/http.(*Transport).dialConnFor(0x0?, 0x14000440000)
        net/http/transport.go:1467 +0x7c
created by net/http.(*Transport).queueForDial in goroutine 35
        net/http/transport.go:1436 +0x380

goroutine 65 [select]:
net.(*Resolver).lookupIPAddr(0x104ae81e0, {0x10473f018?, 0x140005123c0}, {0x10454d2e8, 0x3}, {0x1400053a020, 0x1c})
        net/lookup.go:332 +0x320
net.(*Resolver).internetAddrList(0x10473f018?, {0x10473f018?, 0x140005123c0?}, {0x10454d2e8, 0x3}, {0x1400053a020?, 0x30?})
        net/ipsock.go:288 +0x4cc
net.(*Resolver).resolveAddrList(0x10473f0c0?, {0x10473f018, 0x140005123c0}, {0x10454d996, 0x4}, {0x10454d2e8?, 0x0?}, {0x1400053a020, 0x20}, {0x0?, ...})
        net/dial.go:282 +0x400
net.(*Dialer).DialContext(0x14000510000, {0x10473f018, 0x140005121b0}, {0x10454d2e8, 0x3}, {0x1400053a020, 0x20})
        net/dial.go:488 +0x32c
net/http.(*Transport).dial(0x29?, {0x10473f018?, 0x140005121b0?}, {0x10454d2e8?, 0x140000c4100?}, {0x1400053a020?, 0x140000ac200?})
        net/http/transport.go:1183 +0xdc
net/http.(*Transport).dialConn(0x1400050c000, {0x10473f018, 0x140005121b0}, {{}, 0x0, {0x14000528000, 0x5}, {0x1400053a020, 0x20}, 0x0})
        net/http/transport.go:1625 +0x61c
net/http.(*Transport).dialConnFor(0x0?, 0x1400053e000)
        net/http/transport.go:1467 +0x7c
created by net/http.(*Transport).queueForDial in goroutine 38
        net/http/transport.go:1436 +0x380

goroutine 81 [select]:
net.(*Resolver).lookupIPAddr(0x104ae81e0, {0x10473f018?, 0x1400049a3c0}, {0x10454d2e8, 0x3}, {0x1400049e090, 0xc})
        net/lookup.go:332 +0x320
net.(*Resolver).internetAddrList(0x10473f018?, {0x10473f018?, 0x1400049a3c0?}, {0x10454d2e8, 0x3}, {0x1400049e090?, 0x30?})
        net/ipsock.go:288 +0x4cc
net.(*Resolver).resolveAddrList(0x10473f0c0?, {0x10473f018, 0x1400049a3c0}, {0x10454d996, 0x4}, {0x10454d2e8?, 0x0?}, {0x1400049e090, 0x10}, {0x0?, ...})
        net/dial.go:282 +0x400
net.(*Dialer).DialContext(0x14000498000, {0x10473f018, 0x1400049a1b0}, {0x10454d2e8, 0x3}, {0x1400049e090, 0x10})
        net/dial.go:488 +0x32c
net/http.(*Transport).dial(0x29?, {0x10473f018?, 0x1400049a1b0?}, {0x10454d2e8?, 0x140000c4080?}, {0x1400049e090?, 0x1400009c3e0?})
        net/http/transport.go:1183 +0xdc
net/http.(*Transport).dialConn(0x14000494000, {0x10473f018, 0x1400049a1b0}, {{}, 0x0, {0x140004b2000, 0x5}, {0x1400049e090, 0x10}, 0x0})
        net/http/transport.go:1625 +0x61c
net/http.(*Transport).dialConnFor(0x0?, 0x140004be000)
        net/http/transport.go:1467 +0x7c
created by net/http.(*Transport).queueForDial in goroutine 37
        net/http/transport.go:1436 +0x380

goroutine 82 [chan receive]:
net.(*Resolver).goLookupIPCNAMEOrder.func4({_, _}, _)
        net/dnsclient_unix.go:659 +0x64
net.(*Resolver).goLookupIPCNAMEOrder(_, {_, _}, {_, _}, {_, _}, _, _)
        net/dnsclient_unix.go:669 +0x9dc
net.(*Resolver).lookupIP(0x104ae81e0, {0x10473f050, 0x140004aa230}, {0x10454d2e8, 0x3}, {0x1400049e090, 0xc})
        net/lookup_unix.go:72 +0xe4
net.glob..func1({0x10473f050?, 0x140004aa230?}, 0x1400049a1b0?, {0x10454d2e8?, 0x140004b2000?}, {0x1400049e090?, 0x1400049e090?})
        net/hook.go:23 +0x44
net.(*Resolver).lookupIPAddr.func1()
        net/lookup.go:324 +0x48
internal/singleflight.(*Group).doCall(0x104ae81f0, 0x140004aa280, {0x1400049e0c0, 0x10}, 0x0?)
        internal/singleflight/singleflight.go:93 +0x34
created by internal/singleflight.(*Group).DoChan in goroutine 81
        internal/singleflight/singleflight.go:86 +0x384

goroutine 66 [chan receive]:
net.(*Resolver).goLookupIPCNAMEOrder.func4({_, _}, _)
        net/dnsclient_unix.go:659 +0x64
net.(*Resolver).goLookupIPCNAMEOrder(_, {_, _}, {_, _}, {_, _}, _, _)
        net/dnsclient_unix.go:669 +0x9dc
net.(*Resolver).lookupIP(0x104ae81e0, {0x10473f050, 0x14000520230}, {0x10454d2e8, 0x3}, {0x1400053a020, 0x1c})
        net/lookup_unix.go:72 +0xe4
net.glob..func1({0x10473f050?, 0x14000520230?}, 0x140005121b0?, {0x10454d2e8?, 0x14000528000?}, {0x1400053a020?, 0x1400053a020?})
        net/hook.go:23 +0x44
net.(*Resolver).lookupIPAddr.func1()
        net/lookup.go:324 +0x48
internal/singleflight.(*Group).doCall(0x104ae81f0, 0x14000520280, {0x1400053a040, 0x20}, 0x0?)
        internal/singleflight/singleflight.go:93 +0x34
created by internal/singleflight.(*Group).DoChan in goroutine 65
        internal/singleflight/singleflight.go:86 +0x384

goroutine 50 [chan receive]:
net.(*Resolver).goLookupIPCNAMEOrder.func4({_, _}, _)
        net/dnsclient_unix.go:659 +0x64
net.(*Resolver).goLookupIPCNAMEOrder(_, {_, _}, {_, _}, {_, _}, _, _)
        net/dnsclient_unix.go:669 +0x9dc
net.(*Resolver).lookupIP(0x104ae81e0, {0x10473f050, 0x14000420230}, {0x10454d2e8, 0x3}, {0x14000428040, 0x18})
        net/lookup_unix.go:72 +0xe4
net.glob..func1({0x10473f050?, 0x14000420230?}, 0x140004121b0?, {0x10454d2e8?, 0x1400042a000?}, {0x14000428040?, 0x14000428040?})
        net/hook.go:23 +0x44
net.(*Resolver).lookupIPAddr.func1()
        net/lookup.go:324 +0x48
internal/singleflight.(*Group).doCall(0x104ae81f0, 0x14000420280, {0x14000428060, 0x1c}, 0x0?)
        internal/singleflight/singleflight.go:93 +0x34
created by internal/singleflight.(*Group).DoChan in goroutine 49
        internal/singleflight/singleflight.go:86 +0x384

goroutine 6 [IO wait]:
internal/poll.runtime_pollWait(0x14b9bfc58, 0x72)
        runtime/netpoll.go:343 +0xa0
internal/poll.(*pollDesc).wait(0x14000172580?, 0x1400027a000?, 0x0)
        internal/poll/fd_poll_runtime.go:84 +0x28
internal/poll.(*pollDesc).waitRead(...)
        internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0x14000172580, {0x1400027a000, 0x4d0, 0x4d0})
        internal/poll/fd_unix.go:164 +0x200
net.(*netFD).Read(0x14000172580, {0x1400027a000?, 0x14b8f6698?, 0x104c50a68?})
        net/fd_posix.go:55 +0x28
net.(*conn).Read(0x14000118000, {0x1400027a000?, 0x140001bdf01?, 0x104273644?})
        net/net.go:179 +0x34
net.dnsPacketRoundTrip({_, _}, _, {{{0x61, 0x70, 0x69, 0x2e, 0x73, 0x74, 0x61, ...}, ...}, ...}, ...)
        net/dnsclient_unix.go:108 +0x11c
net.(*Resolver).exchange(_, {_, _}, {_, _}, {{{0x61, 0x70, 0x69, 0x2e, 0x73, ...}, ...}, ...}, ...)
        net/dnsclient_unix.go:187 +0x394
net.(*Resolver).tryOneName(_, {_, _}, _, {_, _}, _)
        net/dnsclient_unix.go:277 +0x370
net.(*Resolver).goLookupIPCNAMEOrder.func3.1(0x1?)
        net/dnsclient_unix.go:653 +0x68
created by net.(*Resolver).goLookupIPCNAMEOrder.func3 in goroutine 66
        net/dnsclient_unix.go:652 +0x15c

goroutine 7 [IO wait]:
internal/poll.runtime_pollWait(0x14b9bfe48, 0x72)
        runtime/netpoll.go:343 +0xa0
internal/poll.(*pollDesc).wait(0x14000426100?, 0x140001c6000?, 0x0)
        internal/poll/fd_poll_runtime.go:84 +0x28
internal/poll.(*pollDesc).waitRead(...)
        internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0x14000426100, {0x140001c6000, 0x4d0, 0x4d0})
        internal/poll/fd_unix.go:164 +0x200
net.(*netFD).Read(0x14000426100, {0x140001c6000?, 0x14bb23db8?, 0x104c50108?})
        net/fd_posix.go:55 +0x28
net.(*conn).Read(0x140000ae000, {0x140001c6000?, 0x140001c1f01?, 0x104273644?})
        net/net.go:179 +0x34
net.dnsPacketRoundTrip({_, _}, _, {{{0x61, 0x70, 0x69, 0x2e, 0x73, 0x74, 0x61, ...}, ...}, ...}, ...)
        net/dnsclient_unix.go:108 +0x11c
net.(*Resolver).exchange(_, {_, _}, {_, _}, {{{0x61, 0x70, 0x69, 0x2e, 0x73, ...}, ...}, ...}, ...)
        net/dnsclient_unix.go:187 +0x394
net.(*Resolver).tryOneName(_, {_, _}, _, {_, _}, _)
        net/dnsclient_unix.go:277 +0x370
net.(*Resolver).goLookupIPCNAMEOrder.func3.1(0x1c?)
        net/dnsclient_unix.go:653 +0x68
created by net.(*Resolver).goLookupIPCNAMEOrder.func3 in goroutine 66
        net/dnsclient_unix.go:652 +0x15c

goroutine 97 [IO wait]:
internal/poll.runtime_pollWait(0x14b9bfb60, 0x72)
        runtime/netpoll.go:343 +0xa0
internal/poll.(*pollDesc).wait(0x14000328000?, 0x1400032c000?, 0x0)
        internal/poll/fd_poll_runtime.go:84 +0x28
internal/poll.(*pollDesc).waitRead(...)
        internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0x14000328000, {0x1400032c000, 0x4d0, 0x4d0})
        internal/poll/fd_unix.go:164 +0x200
net.(*netFD).Read(0x14000328000, {0x1400032c000?, 0x14b8e19a0?, 0x104c50f18?})
        net/fd_posix.go:55 +0x28
net.(*conn).Read(0x140004b8028, {0x1400032c000?, 0x140003cbf01?, 0x104273644?})
        net/net.go:179 +0x34
net.dnsPacketRoundTrip({_, _}, _, {{{0x61, 0x70, 0x69, 0x2e, 0x61, 0x78, 0x69, ...}, ...}, ...}, ...)
        net/dnsclient_unix.go:108 +0x11c
net.(*Resolver).exchange(_, {_, _}, {_, _}, {{{0x61, 0x70, 0x69, 0x2e, 0x61, ...}, ...}, ...}, ...)
        net/dnsclient_unix.go:187 +0x394
net.(*Resolver).tryOneName(_, {_, _}, _, {_, _}, _)
        net/dnsclient_unix.go:277 +0x370
net.(*Resolver).goLookupIPCNAMEOrder.func3.1(0x1?)
        net/dnsclient_unix.go:653 +0x68
created by net.(*Resolver).goLookupIPCNAMEOrder.func3 in goroutine 82
        net/dnsclient_unix.go:652 +0x15c

goroutine 98 [IO wait]:
internal/poll.runtime_pollWait(0x14b9bf878, 0x72)
        runtime/netpoll.go:343 +0xa0
internal/poll.(*pollDesc).wait(0x140004ae080?, 0x140003d6000?, 0x0)
        internal/poll/fd_poll_runtime.go:84 +0x28
internal/poll.(*pollDesc).waitRead(...)
        internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0x140004ae080, {0x140003d6000, 0x4d0, 0x4d0})
        internal/poll/fd_unix.go:164 +0x200
net.(*netFD).Read(0x140004ae080, {0x140003d6000?, 0x104fe9a48?, 0x104c52b38?})
        net/fd_posix.go:55 +0x28
net.(*conn).Read(0x1400043a030, {0x140003d6000?, 0x140003cff01?, 0x104273644?})
        net/net.go:179 +0x34
net.dnsPacketRoundTrip({_, _}, _, {{{0x61, 0x70, 0x69, 0x2e, 0x61, 0x78, 0x69, ...}, ...}, ...}, ...)
        net/dnsclient_unix.go:108 +0x11c
net.(*Resolver).exchange(_, {_, _}, {_, _}, {{{0x61, 0x70, 0x69, 0x2e, 0x61, ...}, ...}, ...}, ...)
        net/dnsclient_unix.go:187 +0x394
net.(*Resolver).tryOneName(_, {_, _}, _, {_, _}, _)
        net/dnsclient_unix.go:277 +0x370
net.(*Resolver).goLookupIPCNAMEOrder.func3.1(0x1c?)
        net/dnsclient_unix.go:653 +0x68
created by net.(*Resolver).goLookupIPCNAMEOrder.func3 in goroutine 82
        net/dnsclient_unix.go:652 +0x15c

goroutine 83 [IO wait]:
internal/poll.runtime_pollWait(0x14b9bfa68, 0x72)
        runtime/netpoll.go:343 +0xa0
internal/poll.(*pollDesc).wait(0x140004da000?, 0x140004e0000?, 0x0)
        internal/poll/fd_poll_runtime.go:84 +0x28
internal/poll.(*pollDesc).waitRead(...)
        internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0x140004da000, {0x140004e0000, 0x4d0, 0x4d0})
        internal/poll/fd_unix.go:164 +0x200
net.(*netFD).Read(0x140004da000, {0x140004e0000?, 0x14b9bda18?, 0x104c51878?})
        net/fd_posix.go:55 +0x28
net.(*conn).Read(0x140004de000, {0x140004e0000?, 0x14000451f01?, 0x104273644?})
        net/net.go:179 +0x34
net.dnsPacketRoundTrip({_, _}, _, {{{0x61, 0x70, 0x69, 0x2e, 0x64, 0x65, 0x76, ...}, ...}, ...}, ...)
        net/dnsclient_unix.go:108 +0x11c
net.(*Resolver).exchange(_, {_, _}, {_, _}, {{{0x61, 0x70, 0x69, 0x2e, 0x64, ...}, ...}, ...}, ...)
        net/dnsclient_unix.go:187 +0x394
net.(*Resolver).tryOneName(_, {_, _}, _, {_, _}, _)
        net/dnsclient_unix.go:277 +0x370
net.(*Resolver).goLookupIPCNAMEOrder.func3.1(0x1?)
        net/dnsclient_unix.go:653 +0x68
created by net.(*Resolver).goLookupIPCNAMEOrder.func3 in goroutine 50
        net/dnsclient_unix.go:652 +0x15c

goroutine 84 [IO wait]:
internal/poll.runtime_pollWait(0x14b9bfd50, 0x72)
        runtime/netpoll.go:343 +0xa0
internal/poll.(*pollDesc).wait(0x14000098080?, 0x14000460000?, 0x0)
        internal/poll/fd_poll_runtime.go:84 +0x28
internal/poll.(*pollDesc).waitRead(...)
        internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0x14000098080, {0x14000460000, 0x4d0, 0x4d0})
        internal/poll/fd_unix.go:164 +0x200
net.(*netFD).Read(0x14000098080, {0x14000460000?, 0x14b920938?, 0x104c513c8?})
        net/fd_posix.go:55 +0x28
net.(*conn).Read(0x1400005e000, {0x14000460000?, 0x14000455f01?, 0x104273644?})
        net/net.go:179 +0x34
net.dnsPacketRoundTrip({_, _}, _, {{{0x61, 0x70, 0x69, 0x2e, 0x64, 0x65, 0x76, ...}, ...}, ...}, ...)
        net/dnsclient_unix.go:108 +0x11c
net.(*Resolver).exchange(_, {_, _}, {_, _}, {{{0x61, 0x70, 0x69, 0x2e, 0x64, ...}, ...}, ...}, ...)
        net/dnsclient_unix.go:187 +0x394
net.(*Resolver).tryOneName(_, {_, _}, _, {_, _}, _)
        net/dnsclient_unix.go:277 +0x370
net.(*Resolver).goLookupIPCNAMEOrder.func3.1(0x1c?)
        net/dnsclient_unix.go:653 +0x68
created by net.(*Resolver).goLookupIPCNAMEOrder.func3 in goroutine 50
        net/dnsclient_unix.go:652 +0x15c

goroutine 36 [running]:
github.com/axiomhq/cli/internal/cmd/auth.runStatus.func1.1()
        github.com/axiomhq/cli/internal/cmd/auth/auth_status.go:86 +0x38
github.com/axiomhq/cli/internal/cmd/auth.runStatus.func1()
        github.com/axiomhq/cli/internal/cmd/auth/auth_status.go:92 +0x154
golang.org/x/sync/errgroup.(*Group).Go.func1()
        golang.org/x/[email protected]/errgroup/errgroup.go:75 +0x58
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1
        golang.org/x/[email protected]/errgroup/errgroup.go:72 +0x98

Inconsistent parameters vs flags for axiom dataset create

axiom dataset create expects the name for the dataset to create as a flag (--name).
axiom dataset delete expects the name for the dataset to delete as a parameter.

Without prior knowledge of the axiom dataset create command I assumed it would take the dataset name as a parameter just like the other dataset commands.

[stream] Basic features

TUI

### Piped

"cannot decode" error for all commands

All commands are failing with error "cannot decode response with unknown content type". Installed with brew install axiom

~ % axiom stream clavis-demo
! Organization ID is set using "AXIOM_ORG_ID"!
! Token is set using "AXIOM_TOKEN"!
! URL is set using "AXIOM_URL"!
Error: cannot decode response with unknown content type

~ % axiom auth status
! Organization ID is set using "AXIOM_ORG_ID"!
! Token is set using "AXIOM_TOKEN"!
! URL is set using "AXIOM_URL"!
➜ cloud
    ✖ cannot decode response with unknown content type

~/.zprofile:

export AXIOM_TOKEN="{redacted}"
export AXIOM_URL="https://app.axiom.co"
export AXIOM_ORG_ID="clavis-vsr2"
export AXIOM_DEPLOYMENT="cloud"

Shell: zsh
OS: macOS 12.7.2
Go version: go1.21.6 darwin/arm64
Brew version: 4.2.5
Axiom CLI version: 0.12.0 (revision: c70256c)

Add option to ingest to disable spinner

Right now cat myfile | pv | axiom ingest mydataset is breaking the pv output.

Implement own man page generator

The one used by cobra is based on go-md2man which can't handle < > characters because it is interpreting them as HTML entities. Man pages aren't hard. Lets implement something like https://github.com/Xe/x/blob/master/internal/manpage/gen.go in our generation tool.

Or: Just get rid of < and > characters in our commands usage lines.

Streaming ingest doesn't work without activity

Reproduce:

echo '{"foo":"bar"}\n` >> logs
tail -f logs | axiom ingest my-dataset --flush-every 1s

Spins forever without ingesting the event.

Adding a line with

cat >> logs <<EOF
{"bar":"baz"}
EOF

ingests the previous event as well.

But what if there are no new events?

Unknown command "login"

% axiom
? This seems to be your first time running this CLI. Do you want to login to Axiom? No
! Skipped setup. Most functionality will be limited.

To login to Axiom, run:
$ axiom login
% axiom login
Error: unknown command "login" for "axiom"

Add support for streams

With some kind of --flush-every

No way to specify a Unix timestamp as the time format

Probably the most ubiquitous timestamp format in existence.

Missing OrganizationID when using Personal Access Token with cloud deployment

I have dug around the https://github.com/axiomhq/axiom-go to figure out why it returned Error: missing organization id when initializing the client in the cli code and I found the following comment:

// When the organization ID is not set, use `AXIOM_ORG_ID`. In case the url
// is the Axiom Cloud url and the access token is a personal token, the
// organization ID is explicitly required and an error is returned, if it is
// not set.

The cli usage documentation doesn't show that explicit requirement and the demo also doesn't show the setting of AXIOM_ORG_ID.

Incorrect error with `axiom dataset list` for bad deployment alias

When using an alias to that is no longer valid the axiom dataset list
command returns an error of the form:

$ AXIOM_DEPLOYMENT=bogus-no-longer-valid-env axiom dataset list
Error: cannot decode response with unknown content type

This can be cross-checked with axiom auth status which will show valid and non-valid auth tokens by alias

Add Brew Update/Upgrade instructions?

via @cdeutsch

Configure with env only doesn't work

Running axiom dataset list without any deployments but AXM_URL and AXM_TOKEN set returns an error:

✖ No deployments configured!

  Setup a deployment by logging into it:
  $ axiom auth login

This should work without any deployments.

Ingestion problems after #25

A regression was introduced with f534579 which causes ingestion to fail:

logs.json:

{"time": "17/May/2015:08:05:32 +0000", "remote_ip": "93.180.71.3", "remote_user": "-", "request": "GET /downloads/product_1 HTTP/1.1", "response": 304, "bytes": 0, "referrer": "-", "agent": "Debian APT-HTTP/1.3 (0.8.16~exp12ubuntu10.21)"}
{"time": "17/May/2015:08:05:32 +0000", "remote_ip": "93.180.71.3", "remote_user": "-", "request": "GET /downloads/product_1 HTTP/1.1", "response": 304, "bytes": 0, "referrer": "-", "agent": "Debian APT-HTTP/1.3 (0.8.16~exp12ubuntu10.21)"}
{"time": "17/May/2015:08:05:32 +0000", "remote_ip": "93.180.71.3", "remote_user": "-", "request": "GET /downloads/product_1 HTTP/1.1", "response": 304, "bytes": 0, "referrer": "-", "agent": "Debian APT-HTTP/1.3 (0.8.16~exp12ubuntu10.21)"}

cat ~/Downloads/logs.json | axiom ingest lukas-logs:

Error: could not ingest "stdin" into dataset "lukas-logs": API error 400: 400 Bad Request: "invalid event at index 17: ReadObject: expect { or , or } or n, but found t, error found in #10 byte of ...|:42 +00{\"time\": \"17/|..., bigger context ...|{\"time\": \"17/May/2015:08:05:42 +00{\"time\": \"17/May/2015:08:05:42 +0000\", \"remote_ip\": \"|..."

Having just a single event inside logs.json fails silently, without ingesting and raising an error.

Add .gitconfig tweak to instructions steering users around go get failure

[url "ssh://[email protected]/"]
  insteadOf = https://github.com/

Proper integration tests

Partly a follow-up from #39.

We need some integration tests that use the cli binary to actually interact with an Axiom deployment.

Trim didn't work

Issues this command

axiom dataset trim waxiom-mr-notes --duration="85440h"

...and got this result

✓ Trimmed dataset waxiom-mr-notes (dropped 0 blocks)

But still seeing data in the Stream view.

Version

Axiom CLI, release 0.1.12 (revision: aae376f)
  build user:       goreleaser
  build date:       2021-03-10T13:40:27Z
  go version:       go1.16

Axiom, release v1.6.0-20210317T1714+eb8a91d1f (cloud)

Read the manual at https://docs.axiom.co/cli`})

Ingest tokens don't work

The cli tries to check /datasets and it can't, so it fails with 403.

Error: unknown plan "hobby"

❯ axiom --version
Axiom CLI version 0.7.0

After issuing axiom auth login I'm getting a following error:

? You need to login to Axiom. Should I open your default browser? Yes
Waiting for authentication...
Error: unknown plan "hobby"

Is the CLI app only supported for basic/enterprise accounts?

Configurable output format

Would be amazing if you could pass for example --json and get the data as JSON so it can be processed further (jq, etc.).

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.