arangodb / foxx-cli Goto Github PK

View Code? Open in Web Editor NEW

44.0 20.0 8.0 735 KB

CLI for managing Foxx services.

License: Apache License 2.0

JavaScript 97.53% EJS 2.47%

foxx-cli's Introduction

foxx-cli

CLI for managing and developing ArangoDB Foxx services.

foxx-cli is a standalone tool for the shell that can be used to talk to ArangoDB servers to manage Foxx services.

To learn more about Foxx, see the official ArangoDB Foxx documentation.

foxx-cli relies on the Foxx HTTP API.

Install
Usage
Special files
Contribute
License

Install

foxx-cli runs on Node.js and can be installed with yarn:

yarn global add foxx-cli

Or with npm:

npm install --global foxx-cli

Note: using yarn you can also run foxx-cli from your project's devDependencies:

yarn add --dev foxx-cli
yarn foxx help

If you're using a recent version of npm you can also use npx:

npx -p foxx-cli foxx help

Usage

After you've installed foxx-cli, you should be able to use the foxx program. You can learn more about the different commands foxx supports by using the --help flag.

foxx --help

You can also use the --help flag with commands to learn more about them, e.g.:

foxx install --help # Help for the "install" command

foxx server --help # Help for the "server" command

foxx server list --help # Subcommands are supported, too

If you have no prior knowledge of Foxx, you can get started by installing ArangoDB locally and then creating a new Foxx service in the current directory using the init command:

foxx init -i # answer the interactive questions

If you want an example, you can also let init create an example service for you:

foxx init -e # create an example service please

You can also just use foxx init to create a minimal service without the example code.

You can inspect the files created by the program and tweak them as necessary. Once you're ready, install the service at a mount path using the install command:

foxx install /hello-foxx # installs the current directory

You should then be able to view the installed service in your browser at the following URL:

http://localhost:8529/_db/_system/hello-foxx

If you continue to work on your Foxx service and want to upgrade the installed version with your local changes use the upgrade command to do so.

foxx upgrade /hello-foxx # upgrades the server with the current directory

Special files

manifest.json

The manifest.json or manifest file contains a service's meta-information. See the Foxx reference documentation.

The directory containing a service's manifest.json file is called the root directory of the service.

foxxignore

If you want to exclude files from the service bundle that will uploaded to ArangoDB you can create a file called .foxxignore in the root directory of your service. Each line should specify one pattern you wish to ignore:

Patterns starting with ! will be treated as an explicit whitelist. Paths matching these patterns will not be ignored even if they would match any of the other patterns.

Example: !index.js will override any pattern matching a file called index.js.
Patterns starting with / will only match paths relative to the service's root directory.

Example: /package.json will not match node_modules/joi/package.json.
Patterns ending with / will match a directory and any files inside of it.

Example: node_modules/ will exclude all node_modules directories and all of their contents.
A single * (glob) will match zero or more characters (even dots) in a file or directory name.

Example: .* will match any files and directories with a name starting with a dot.
A double ** (globstar) will match zero or more levels of nesting.

Example: hello/**/world will match hello/world, hello/foo/world, hello/foo/bar/world, and so on.
Patterns starting with # are considered comments and will be ignored.

For more details on the pattern matching behaviour, see the documentation of the minimatch module (with the dot flag enabled).

If no .foxxignore file is present in the service's root directory the following patterns will be ignored automatically: .git/, .svn/, .hg/, *.swp, .DS_Store.

Should you need to include files that match these patterns for some reason, you can override this list by creating an empty .foxxignore file.

You can also create a .foxxignore file in the current directory using the ignore command:

foxx ignore # creates a file pre-populated with the defaults

foxx ignore --force # creates an empty file

To add individual patterns to the .foxxignore file just pass them as additional arguments:

foxx ignore .git/ .svn/ # you can pass multiple patterns at once

foxx ignore '*.swp' # make sure to escape special characters

foxxrc

If you define servers using the server commands, a .foxxrc file will be created in your $HOME directory, which is typically one of the following paths:

/home/$USER on Linux
/Users/$USER on macOS
C:\Users\$USER on Windows

This file contains sections for each server which may contain server credentials should you decide to save them.

Contribute

Feel free to dive in! Open an issue or submit PRs.

All JavaScript code should conform to standard coding style.

This README conforms to the standard-readme specification.

License

The Apache License, Version 2.0. For more information, see the accompanying LICENSE file.

foxx-cli's People

Contributors

Stargazers

Watchers

Forkers

fossabot jschill simran-b xgenvn m0ppers pluma serajhqi oog-gamro

foxx-cli's Issues

foxx run error

foxx-cli version: 1.1.3
ArangoDB version: 3.3.8

When using:

foxx run /some-service some-script --server http://localhost:8529 --database Test

I am seeing an error:

Unexpected ArangoDB error (Code: 400):
Unexpected end of JSON input

/some-service is mounted in the Test database and has a some-script.

Please let me know if this is my error or yours. Thanks.

TypeError: db.useDatabase is not a function

within foxx v2.1.0

foxx install -D db_bw --server sw_fasty /settings settings
Sorry! An unexpected error occurred. This is likely a bug in foxx-cli.
Please open an issue at https://github.com/arangodb/foxx-cli/issues with a full copy of the following error message and a description of what you were trying to do when this problem occurred.

TypeError: db.useDatabase is not a function
    at module.exports (/Users/olivierbonnaure/.nvm/versions/node/v16.15.0/lib/node_modules/foxx-cli/lib/util/client.js:7:8)
    at Object.handler (/Users/olivierbonnaure/.nvm/versions/node/v16.15.0/lib/node_modules/foxx-cli/lib/cmds/install.js:136:16)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

We apologize for the inconvenience.

TypeError: Cannot read property 'isDirectory' of null

I am trying to install some Foxx microservices into an ArangoDB based Docker image, something like:

# escape=`
FROM arangodb:latest

WORKDIR /

# Enable https support for apt-get
RUN apt-get update `
    && apt-get install -y apt-transport-https ca-certificates

# Enable node.js & yarn repositories
RUN curl -sL https://deb.nodesource.com/setup_8.x | bash -
RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
RUN echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list

# Install yarn
RUN apt-get update `
    && apt-get install -y yarn `
    && rm -rf /var/lib/apt/lists/*

# Install Foxx CLI
RUN yarn global add foxx-cli

# Install Foxx microservices
RUN mkdir foxx
COPY ./foxx/* foxx/
RUN for service in foxx/*; do foxx install $service; done

but I get the following error:

TypeError: Cannot read property 'isDirectory' of null
    at /usr/local/share/.config/yarn/global/node_modules/foxx-cli/dist/util/fs.js:58:21
    at Generator.next (<anonymous>)
    at step (/usr/local/share/.config/yarn/global/node_modules/foxx-cli/dist/util/fs.js:3:191)
    at /usr/local/share/.config/yarn/global/node_modules/foxx-cli/dist/util/fs.js:3:361
    at <anonymous>

Note that this installed version 1.3.0 of the foxx-cli.

Errors using bundle / zip

On Mac OSX, running in terminal:

foxx zip -o test

Sorry! An unexpected error occurred. This is likely a bug in foxx-cli.
Please open an issue at https://github.com/arangodb/foxx-cli/issues with a full copy of the following error message and a description of what you were trying to do when this problem occured.

TypeError: bundle is not a function
at Object.handler (/usr/local/lib/node_modules/foxx-cli/src/cmds/bundle.js:86:11)
at

We apologize for the inconvenience.

Error: self signed certificate

ArangoDB 3.2.8 on Windows 2016
ArangoDB is using a self-signed certificate created with OpenSSL and running on port 8530 with port 8529 disabled.
The ArangoDB server is accessible via a web UI.

I have saved a server with the command:
foxx server set myserver https://{uid}:{password}@myserver.corp.local:8530

I validated that the contents of my .foxxrc properly decoded the command and is storing the parameters properly.

Running the command:
foxx --server myserver list from the command line and the response is:

Sorry! An unexpected error occurred. This is likely a bug in foxx-cli.
Please open an issue at https://github.com/arangodb/foxx-cli/issues with a full copy of the following
error message and a description of what you were trying to do when this problem occured.

Error: self signed certificate
    at TLSSocket.<anonymous> (_tls_wrap.js:1103:38)
    at emitNone (events.js:106:13)
    at TLSSocket.emit (events.js:208:7)
    at TLSSocket._finishInit (_tls_wrap.js:637:8)
    at TLSWrap.ssl.onhandshakedone (_tls_wrap.js:467:38)

We apologize for the inconvenience.

generateCrudRoutes is not defined

Standing in a newly created folder:

> yarn add --dev foxx-cli
> yarn foxx init test

yarn run v1.5.1
warning package.json: No license field
$ /workspace/github/foxx/node_modules/.bin/foxx init test
Sorry! An unexpected error occurred. This is likely a bug in foxx-cli.
Please open an issue at https://github.com/arangodb/foxx-cli/issues with a full copy of the following error message and a description of what you were trying to do when this problem occured.

ReferenceError: ejs:3
    1| 'use strict';
    2|
 >> 3| <% if (generateCrudRoutes && typeof documentCollections !== 'undefined') {
    4|   documentCollections.forEach(function (collection) { -%>
    5| module.context.use('/<%- collection %>', require('./api/<%- collection %>'), '<%- collection %>');
    6| <% })

generateCrudRoutes is not defined
    at eval (eval at compile (/workspace/github/foxx/node_modules/ejs/lib/ejs.js:584:12), <anonymous>:11:8)
    at returnedFn (/workspace/github/foxx/node_modules/ejs/lib/ejs.js:615:17)
    at exports.render (/workspace/github/foxx/node_modules/ejs/lib/ejs.js:414:37)
    at /workspace/github/foxx/node_modules/foxx-cli/dist/generator/index.js:6:12
    at Generator.next (<anonymous>)
    at step (/workspace/github/foxx/node_modules/foxx-cli/dist/generator/index.js:25:191)
    at /workspace/github/foxx/node_modules/foxx-cli/dist/generator/index.js:25:437
    at new Promise (<anonymous>)
    at /workspace/github/foxx/node_modules/foxx-cli/dist/generator/index.js:25:99
    at generateFile (/workspace/github/foxx/node_modules/foxx-cli/dist/generator/index.js:10:17)

We apologize for the inconvenience.
error An unexpected error occurred: "Command failed.
Exit code: 1
Command: sh
Arguments: -c /workspace/github/foxx/node_modules/.bin/foxx init test
Directory: /workspace/github/foxx
Output:
".
info If you think this is a bug, please open a bug report with the information provided in "/workspace/github/foxx/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Handle common system errors

We should probably provide more meaningful messages for some of the following errors:

https://nodejs.org/api/errors.html#errors_common_system_errors

We already handle ECONNREFUSED. We should probably extend that to also handle ECONNRESET (possibly indicating ArangoDB went away during a connection), EPIPE (possibly indicating a server-side problem) and ETIMEDOUT (possibly indicating ArangoDB is under heavy load), at the least.

`upgrade`/`install`/`replace` ignore foxxignore file during bundling

Using foxx bundle and then installing from the bundle should behave the same as letting the bundling happen automatically. But foxx bundle respects the ignore file whereas the other commands don't. That seems to be a bug (maybe a path resolution issue?).

foxx server test

It would be neat if there was a way to test that the server definition is correct (valid credentials etc) by pinging the server. This could also be used to ensure the server version is actually compatible with foxx-cli, i.e. 3.1.10/3.2.alpha2 or later.

ArangoError handling

The CLI commands currently mostly don't do any error handling as far as they're implemented yet at all.

Specifically the following commands and errors should be dealt with (everything else is probably unexpected):

config
- ERROR_SERVICE_NOT_FOUND
deps
- ERROR_SERVICE_NOT_FOUND
download
- ERROR_SERVICE_NOT_FOUND
install
- ERROR_INVALID_MOUNTPOINT
- ERROR_SERVICE_MOUNTPOINT_CONFLICT
- ERROR_SERVICE_SOURCE_NOT_FOUND
- ERROR_SERVICE_SOURCE_ERROR
- ERROR_SERVICE_MANIFEST_NOT_FOUND
- ERROR_MALFORMED_MANIFEST_FILE
- ERROR_INVALID_SERVICE_MANIFEST
- ERROR_SERVICE_OPTIONS_MALFORMED
list?
replace
- ERROR_SERVICE_NOT_FOUND
- ERROR_SERVICE_SOURCE_NOT_FOUND
- ERROR_SERVICE_SOURCE_ERROR
- ERROR_SERVICE_MANIFEST_NOT_FOUND
- ERROR_MALFORMED_MANIFEST_FILE
- ERROR_INVALID_SERVICE_MANIFEST
- ERROR_SERVICE_OPTIONS_MALFORMED
run
- ERROR_SERVICE_NOT_FOUND
- ERROR_SERVICE_NEEDS_CONFIGURATION
- ERROR_SERVICE_UNKNOWN_SCRIPT
- ERROR_MODULE_NOT_FOUND
- ERROR_MODULE_FAILURE
scripts
- ERROR_SERVICE_NOT_FOUND
set-dev
- ERROR_SERVICE_NOT_FOUND
set-prod
- ERROR_SERVICE_NOT_FOUND
show
- ERROR_SERVICE_NOT_FOUND
test
- ERROR_SERVICE_NOT_FOUND
- ERROR_SERVICE_NEEDS_CONFIGURATION
- ERROR_MODULE_NOT_FOUND
- ERROR_MODULE_FAILURE
uninstall?
upgrade
- ERROR_SERVICE_NOT_FOUND
- ERROR_SERVICE_SOURCE_NOT_FOUND
- ERROR_SERVICE_SOURCE_ERROR
- ERROR_SERVICE_MANIFEST_NOT_FOUND
- ERROR_MALFORMED_MANIFEST_FILE
- ERROR_INVALID_SERVICE_MANIFEST
- ERROR_SERVICE_OPTIONS_MALFORMED

Maybe we should also handle 404s that should be encountered in older versions of ArangoDB (i.e. pre-3.1).

Usage not really usage

The Usage part in the README does not really describe any usage. It point to more Help sources.
Adding a few detailed examples would be an enormous help to new users.

query parameter "mount" with value fails to match the required pattern

When i try to
$ foxx install --server dev ./ --database appdb

I am getting this error
I have already setup dev
Unexpected ArangoDB error (Code: 400):
query parameter "mount" with value "./" fails to match the required pattern: /(?:\/[-_0-9a-z]+)+/i

Any help is appreciated.

Yes Sir

Uploading a file with bugs in it as a service is no problem. It does not really matter what the javascript contains, it is always accepted. When I use the same file using the web interface to upload it I get an error. So foxx cli is somehow circumventing the checks.

Command I use
foxx upgrade --server pp --database piekenpijp /incoming alldocs.zip

ejs library subject to CVE-2022-29078 vulnerability

https://nvd.nist.gov/vuln/detail/CVE-2022-29078

Resolved with v3.1.7

hapijs error

Even though i have added node_modules/joi and node_modules/@Hapi to ignore list install command gives me error below:

`Server encountered errors executing a JavaScript file:

@ node_modules@hapi\address\lib\domain.js:14:21
12 | tldSegmentRx: /^a-zA-Z?$/,
13 | domainSegmentRx: /^a-zA-Z0-9?$/,

14 | URL: Url.URL || URL // $lab:coverage:ignore$
| ^
15 | };
16 |
17 |
ArangoError 3103: failed to invoke module
Reason: ReferenceError: URL is not defined
File: C:\Windows\TEMP\arangod_a06176\services-tmp-4780-528358610\node_modules@hapi\address\lib\domain.js

For details check the arangod server logs.`

I am using arangodb community edition 3.6.4 and foxx_cli 2.0.0.

Compatibility with mocha reporters

It would be nice if the test command was compatible with mocha formatters. If we reimplement the mocha API in Foxx, we should provide a "reporter" that just records the raw events that would be sent to the reporter so we can replay them in foxx-cli.

Current directory (target foxx service) is not detected properly under MINGW64 in Windows 10

The commands were run without altering the system state in any way in between:
MINGW64 (Default git terminal on windows):

CMD:

Probably the current working directory is not being detected properly for some reason. Tools like npm, yarn and other similar CLIs work in MINGW64 without issues.

Migrate away from request

Request is deprecated. Got should be simple enough for what we are doing.

https://www.npmjs.com/package/got

Request is not actually used to talk to ArangoDB, it's only used to fetch (input bundle) sources passed as URLs.

Setting Servers

When I command "foxx install /hello-foxx", it shows "Server refused authorization. Either your credentials are invalid or the user has insufficient privileges." Should I set up a server or do anything else?

Add linter to CLI

Users may want to sanity check their packages before uploading it to ArangoDB.

The foxx-cli therefore could start to implement a linting facility that would do certain checks for package inetgrity and sanity.

Could not generate temp file

Hello.
There is a cluster of three servers (v3.6.1).
Everything is configured and everything works.
But when I try to add a fox-app, I get an error.

{
 "code":500,
 "error":true,
 "errorMessage":"could not generate temp file: failed to create directory '/tmp/arangod_NFLJap/uploads': Permission denied",
 "errorNum":4
}

All services are launched like this:

/usr/sbin/arangod --uid arangodb --gid arangodb --pid-file /var/run/arangodb3/agency.pid
/usr/sbin/arangod --uid arangodb --gid arangodb --pid-file /var/run/arangodb3/dbserver.pid
/usr/sbin/arangod --uid arangodb --gid arangodb --pid-file /var/run/arangodb3/coordinator.pid

but a directory is created with such rights:

ls -l /tmp | grep NFLJa
drwx------ 2 root          root               4096 Apr 12 21:29 arangod_NFLJap/

How is this possible?
Maybe I have errors in the settings?

Thank you.

foxx install command doesn't support "unix://" socket connections

It seems that the FOXX-CLI package doesn't support "unix://" socket connections.

While trying to use the FOXX-CLI package within a Docker Container for automatic deployment of services at buildtime I encountered the following problem:

when I try to install / deploy my FOXX-Service with the command foxx install ... the command states that the "unix://" scheme is invalid/unsupported.
The output of foxx install --help confirms that as there are no options or examples listed using the "unix://" scheme.

As mentioned earlier in arangodb/arangodb#4393 there seems to a problem with the presence of the loopback interface at the time when the Docker "entrypoint script" is executed.
In consequence the default TCP/HTTP scheme "http://localhost:8529" isn't working.
The "entrypoint script" uses hereing the "unix://" scheme to solve this issue when connections to the server shall be established.

When I connect into the running container the TCP scheme is available and I can run the FOXX-CLI with no problems.

Not able to install new Foxx service on ArangoDB

Foxx CLI version 2.0.0
ArangoDB version 3.6.3

I am probably doing something wrong since this is the first time trying to install our Foxx service via the Foxx CLI. To generate this error I did the following:

Cloned repository
Changed directory into where our Foxx service resides (myservice)
Performed an "npm install"
Then attempted to load the Foxx service to the database via this command:
foxx install --server https://testing.mydomain.co -u root -P /myservice

I received this error:

HttpError: Payload Too Large
at new HttpError (/home/ec2-user/.nvm/versions/node/v15.0.1/lib/node_modules/foxx-cli/node_modules/arangojs/error.js:146:21)
at Object.resolve (/home/ec2-user/.nvm/versions/node/v15.0.1/lib/node_modules/foxx-cli/node_modules/arangojs/connection.js:373:32)
at callback (/home/ec2-user/.nvm/versions/node/v15.0.1/lib/node_modules/foxx-cli/node_modules/arangojs/connection.js:191:26)
at IncomingMessage. (/home/ec2-user/.nvm/versions/node/v15.0.1/lib/node_modules/foxx-cli/node_modules/arangojs/lib/request.node.js:108:21)
at IncomingMessage.emit (node:events:339:22)
at endReadableNT (node:internal/streams/readable:1289:12)
at processTicksAndRejections (node:internal/process/task_queues:80:21)

We apologize for the inconvenience.

I tried adding node_modules/ to a .foxxignore file but then it fails because the service is looking for npm modules

Foxx upgrade error on Gitlab-runner

I am using gitlab-runner for CI/CD. I get error while trying to upgrade a service on pipeline.

The same command works when I execute it locally - not on gitlab-runner.

Running with gitlab-runner 11.7.0 (8bb608ff)
  on ci Xrx1zKeK
Using Docker executor with image node:11.7.0 ...
Pulling docker image node:11.7.0 ...
Using docker image sha256:7d3b4aa102297eb27045a4abf42a0bf024fba1f9a626a71e567b55641e79b590 for node:11.7.0 ...
Running on runner-Xrx1zKeK-project-3222783-concurrent-0 via ci...
Fetching changes...
Removing .cache/
Removing node_modules/
Checking out source
$ yarn global add foxx-cli
yarn global v1.13.0
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Installed "[email protected]" with binaries:
      - foxx
Done in 10.75s.
$ foxx upgrade /graphql ./packages/user/service/foxx --server http://root:xxxxxxx@myserverip:8529 -D ccio_dev
Sorry! An unexpected error occurred. This is likely a bug in foxx-cli.
Please open an issue at https://github.com/arangodb/foxx-cli/issues with a full copy of the following error message and a description of what you were trying to do when this problem occured.

Error: connect EHOSTUNREACH myserverip:8529

Error with foxx install if the current working directory is not the service source code directory

When calling foxx install /my-mount-point path/to/my/service I get:

(node:11437) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: ENOENT: no such file or directory, open 'main.js'
(node:11437) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
Sorry! An unexpected error occurred. This is likely a bug in foxx-cli.
Please open an issue at https://github.com/arangodb/foxx-cli/issues with a full copy of the following error message and a description of what you were trying to do when this problem occured.

Error: ENOENT: no such file or directory, open 'main.js'

We apologize for the inconvenience.
error Command failed with exit code 1.

But if I run the following command, it works: cd path/to/my/service && foxx install /my-mount-point .

foxx deps --interactive

It would be neat if foxx deps could fetch the current deps and prompt for new values (defaulting to current value) to make setting up dependencies of mounted services easier.

foxx replace --force

It would be neat if foxx replace could be used as an upsert (fall back to install if the mount path does not exist).

Error running foxx with nodist on Windows

Running foxx-cli on a Windows Server 2016 machine with nodist installed.
https://github.com/marcelklehr/nodist

The error that comes up when running just the command foxx is:

C:\Program Files (x86)\Nodist\bin\node_modules\foxx-cli\src\cmds\bundle.js:56
exports.handler = async function handler(argv) {
                        ^^^^^^^^
SyntaxError: Unexpected token function
    at Object.exports.runInThisContext (vm.js:76:16)
    at Module._compile (module.js:513:28)
    at Object.Module._extensions..js (module.js:550:10)
    at Module.load (module.js:458:32)
    at tryModuleLoad (module.js:417:12)
    at Function.Module._load (module.js:409:3)
    at Module.require (module.js:468:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> (C:\Program Files (x86)\Nodist\bin\node_modules\foxx-cli\src\index.js:21:12)
    at Module._compile (module.js:541:32)

This may be an issue with nodist more than foxx-cli but I thought worth logging.

foxx install on Oasis server?

Just got a 14 days trial on Oasis, set users, db and so on, got foxx server set done, but it fails on foxx install /api . --server dev-cloud --database mydb.

It exists with the following:

Connection reset by peer. The server closed the connection unexpectedly.
This indicates connectivity issues or a server problem.
error Command failed with exit code 1.

Foxx CLI can't deploy on Oasis?

foxx config --interactive

It would be neat if foxx config could fetch the current config and prompt for new values (defaulting to current value) to make configuration of mounted services easier.

Provide usage examples

Every command should provide usage examples as appropriate: