hypercore-protocol / cli Goto Github PK
View Code? Open in Web Editor NEWA CLI for peer-to-peer file sharing using the Hypercore Protocol.
License: MIT License
A CLI for peer-to-peer file sharing using the Hypercore Protocol.
License: MIT License
Need tooling to delete hypers from your local cache, see #20
I see that the cli requires node 14+ currently. Its great that it's documented in multiple places, but it's easy to enforce and provide a meaningful error message.
I'll submit a PR.
restarted computer, reinstalled hypercore-cli but i still keep getting "the daemon is not active", any ideas to fix this?
Creating a hyperdrive which contains a soft link returns an error.
mkdir -p a/b
touch a/file.txt
(cd a/b; ln -s ../file.txt)
hyp sync a
# Syncing...
# Comparing...
# Error: Aborting file-tree comparison, a symlink or hardlink loop was detected at /b/file.txt
However it works when using a differently named symlink.
mkdir -p a/b
touch a/file.txt
(cd a/b; ln -s ../file.txt file2.txt)
hyp sync a
# Syncing...
# Synced
For use in shell scripts or to pipe hyp’s output into other commands, it would be practical to have an option to only get the address of the created drive/bee from hyp create
. Example output, currently:
> hyp create drive
Drive Created: hyper://87f05a94b1916674fb00d4cf4da0ed3ae9d9491ebd706afbb01e2a83332f4a4e
Seeding hyperdrive
The line containing the address appears to be output via stderr, not stdout, so currently some juggling like this is required to read only the hyper://…
address:
hyp create drive 2>&1 | grep -oE "hyper://\w+"
An -o
/--only
option (like grep) or -q
/--quiet
option (like many tools) that makes hyp output only the hyper://…
url could be helpful.
hyp sync hyper://e498a28a5755d724143c170cb454340873ebbc7a7e7e938d392d883f566ca86e
Hyperspace daemon started
Mirroring daemon started
Error: Block not available from peers
i ran hyp seed on my server and on my local i ran above command it throws me the error
In our user experience research with the dat cli, we found that --exit
was more immediately intuitive than --live=false
, seems like that crept in again with the hyp
cli!
Hello;
I installed with pnpm install -g @hyperspace/cli
and get that in the output:
WARN deprecated [email protected]: cross-spawn no longer requires a build toolchain, use it instead
WARN deprecated [email protected]: Package no longer supported. Contact Support at https://www.npmjs.com/support for more info.
```
I receive strange errors in 'hyp info' command on hyperdrives that are inaccessible / not seeded well.
Error: Block not available from peers
at Object.decode (/usr/lib/node_modules/@hyperspace/cli/node_modules/@hyperspace/rpc/index.js:11:17)
at RPC._onmessage (/usr/lib/node_modules/@hyperspace/cli/node_modules/arpeecee/index.js:113:33)
at Parser._nextState (/usr/lib/node_modules/@hyperspace/cli/node_modules/arpeecee/parser.js:127:14)
at Parser._readMessage (/usr/lib/node_modules/@hyperspace/cli/node_modules/arpeecee/parser.js:54:19)
at Parser.recv (/usr/lib/node_modules/@hyperspace/cli/node_modules/arpeecee/parser.js:35:44)
at RPC._write (/usr/lib/node_modules/@hyperspace/cli/node_modules/arpeecee/index.js:170:22)
at WritableState.update (/usr/lib/node_modules/@hyperspace/cli/node_modules/streamx/index.js:168:14)
IMHO unavailability of a hyperdrive should be considered and treated as normal situation. I would expect information in 'hyp info' like this:
06e4fa..ff (hyperdrive): 0 connected | [↓0 B/s ↑0 B/s] - Not seeding, drive data inaccessible/incomplete
Seems to pull from the package.json in the cwd instead of the install path. May be windows only.
Running hyp sync
on a hyperdrive will destroy the contents of any drives you have mounted drives own. Also hyp diff
will report that the contents will be removed.
Also see hyperspace-org/hyperdrive-service#7
I would expect the contents of the mounted hyperdrive to be left unchanged, and at worst the drive to be unmounted.
When downloading via npm on Windows 10 and Node 14.5.0, I get the error: Error: spawn UNKNOWN
for a lot of commands that I try.
> hyp sync hyper://b282d5efe484143816362d33b3f9b3ea45ecfb8a6ada97e278fdfdc6a725e22f/
Mirroring daemon started
Error: spawn UNKNOWN
I've attached a minimally-trimmed log of my installation and usage below (click to expand):
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.
Install the latest PowerShell for new features and improvements! https://aka.ms/PSWindows
PS C:\Users\jemma> node
Welcome to Node.js v14.5.0.
Type ".help" for more information.
>
PS C:\Users\jemma> npm install -g @hyperspace/cli
npm WARN deprecated [email protected]: cross-spawn no longer requires a build toolchain, use it instead
les\@hyperspace\cli\bin\hyp.js
> [email protected] install C:\Users\jemma\scoop\apps\nvm\current\nodejs\nodejs\node_modules\@hyperspace\cli\node_modules\fd-lock
> node-gyp-build
> [email protected] install C:\Users\jemma\scoop\apps\nvm\current\nodejs\nodejs\node_modules\@hyperspace\cli\node_modules\sodium-native
> node-gyp-build "node preinstall.js" "node postinstall.js"
> [email protected] install C:\Users\jemma\scoop\apps\nvm\current\nodejs\nodejs\node_modules\@hyperspace\cli\node_modules\utp-native
> node-gyp-build
> [email protected] install C:\Users\jemma\scoop\apps\nvm\current\nodejs\nodejs\node_modules\@hyperspace\cli\node_modules\leveldown
> node-gyp-build
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@~2.1.2 (node_modules\@hyperspace\cli\node_modules\chokidar\node_modules\fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"win32","arch":"x64"})
+ @hyperspace/[email protected]
added 278 packages from 185 contributors in 50.725s
PS C:\Users\jemma> hyp
Usage: hyp <command> [opts...]
General Commands:
hyp info [urls...] - Show information about one (or more) hypers.
hyp seed {urls...} - Download and make hyper data available to the network.
hyp unseed {urls...} - Stop making hyper data available to the network.
hyp create {drive|bee} - Create a new hyperdrive or hyperbee.
Hyperdrive Commands:
hyp drive ls {url} - List the entries of the given hyperdrive URL.
hyp drive mkdir {url} - Create a new directory at the given hyperdrive URL.
hyp drive rmdir {url} - Remove a directory at the given hyperdrive URL.
hyp drive cat {url} - Output the content of the given hyperdrive URL.
hyp drive put {url} [content] - Write a file at the given hyperdrive URL.
hyp drive rm {url} - Remove a file or (if --recursive) a folder at the given hyperdrive URL.
ves. Can optionally "commit" the difference.
hyp drive sync {source_path_or_url} [target_path_or_url] - Continuously sync changes between two folders in your local filesystem or in hyperdrives.
hyp drive http {url} - Host a hyperdrive as using HTTP on the localhost.
Hyperbee Commands:
hyp bee ls {url} - List the entries of the given hyperbee URL.
hyp bee get {url} - Get the value of an entry of the given hyperbee URL.
hyp bee put {url} [value] - Set the value of an entry of the given hyperbee URL.
hyp bee del {url} - Delete an entry of the given hyperbee URL.
Daemon Commands:
hyp daemon status - Check the status of the hyperspace daemon.
hyp daemon stop - Stop the hyperspace and mirroring daemons if active.
Aliases:
hyp sync -> hyp drive sync
hyp diff -> hyp drive diff
hyp ls -> hyp drive ls
hyp cat -> hyp drive cat
hyp put -> hyp drive put
Learn more at https://github.com/hypecore-protocol/cli
PS C:\Users\jemma> hyp daemon status
Hyperspace v1.13.0
Your address: 24.143.103.2:51735 (Hole-punchable)
PS C:\Users\jemma> mkdir ManyDrives
Directory: C:\Users\jemma
Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 12/11/2020 12:06 PM ManyDrives
PS C:\Users\jemma> cd .\ManyDrives\
PS C:\Users\jemma\ManyDrives> hyp
Usage: hyp <command> [opts...]
General Commands:
hyp info [urls...] - Show information about one (or more) hypers.
hyp seed {urls...} - Download and make hyper data available to the network.
hyp unseed {urls...} - Stop making hyper data available to the network.
Hyperdrive Commands:
hyp drive ls {url} - List the entries of the given hyperdrive URL.
hyp drive mkdir {url} - Create a new directory at the given hyperdrive URL.
hyp drive cat {url} - Output the content of the given hyperdrive URL.
hyp drive put {url} [content] - Write a file at the given hyperdrive URL.
hyp drive rm {url} - Remove a file or (if --recursive) a folder at the given hyperdrive URL.
hyp drive diff {source_path_or_url} {target_path_or_url} - Compare two folders in your local filesystem or in hyperdrives. Can optionally "commit" the difference.
hyp drive sync {source_path_or_url} [target_path_or_url] - Continuously sync changes between two folders in your local filesystem or in hyperdrives.
Hyperbee Commands:
hyp bee ls {url} - List the entries of the given hyperbee URL.
hyp bee get {url} - Get the value of an entry of the given hyperbee URL.
hyp bee put {url} [value] - Set the value of an entry of the given hyperbee URL.
Daemon Commands:
hyp daemon status - Check the status of the hyperspace daemon.
hyp daemon stop - Stop the hyperspace and mirroring daemons if active.
hyp sync -> hyp drive sync
hyp ls -> hyp drive ls
hyp cat -> hyp drive cat
hyp put -> hyp drive put
Learn more at https://github.com/hypecore-protocol/cli
PS C:\Users\jemma\ManyDrives> hyp sync hyper://b282d5efe484143816362d33b3f9b3ea45ecfb8a6ada97e278fdfdc6a725e22f/
Mirroring daemon started
Error: spawn UNKNOWN
PS C:\Users\jemma\ManyDrives> hyp sync hyper://b282d5efe484143816362d33b3f9b3ea45ecfb8a6ada97e278fdfdc6a725e22f/ blog
Mirroring daemon started
Error: spawn UNKNOWN
PS C:\Users\jemma\ManyDrives> mkdir blog
Directory: C:\Users\jemma\ManyDrives
Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 12/11/2020 12:07 PM blog
PS C:\Users\jemma\ManyDrives> hyp sync hyper://b282d5efe484143816362d33b3f9b3ea45ecfb8a6ada97e278fdfdc6a725e22f/ blog
Mirroring daemon started
Error: spawn UNKNOWN
PS C:\Users\jemma\ManyDrives> hyp sync hyper://b282d5efe484143816362d33b3f9b3ea45ecfb8a6ada97e278fdfdc6a725e22f blog
Mirroring daemon started
Error: spawn UNKNOWN
PS C:\Users\jemma\ManyDrives> hyp sync hyper://b282d5efe484143816362d33b3f9b3ea45ecfb8a6ada97e278fdfdc6a725e2^W^W^C
PS C:\Users\jemma\ManyDrives> hyp drive sync hyper://b282d5efe484143816362d33b3f9b3ea45ecfb8a6ada97e278fdfdc6a725e22f blog
Mirroring daemon started
Error: spawn UNKNOWN
PS C:\Users\jemma\ManyDrives> hyp drive sync hyper://b282d5efe484143816362d33b3f9b3ea45ecfb8a6ada97e278fdfdc6a725e22f/ blog
Mirroring daemon started
Error: spawn UNKNOWN
PS C:\Users\jemma\ManyDrives> rmdir blog
PS C:\Users\jemma\ManyDrives> hyp drive sync hyper://b282d5efe484143816362d33b3f9b3ea45ecfb8a6ada97e278fdfdc6a725e22f/ blog
Mirroring daemon started
Error: spawn UNKNOWN
PS C:\Users\jemma\ManyDrives> hyp
Usage: hyp <command> [opts...]
General Commands:
hyp info [urls...] - Show information about one (or more) hypers.
hyp seed {urls...} - Download and make hyper data available to the network.
hyp unseed {urls...} - Stop making hyper data available to the network.
Hyperdrive Commands:
hyp drive ls {url} - List the entries of the given hyperdrive URL.
hyp drive mkdir {url} - Create a new directory at the given hyperdrive URL.
hyp drive cat {url} - Output the content of the given hyperdrive URL.
hyp drive put {url} [content] - Write a file at the given hyperdrive URL.
the difference.
s.
Hyperbee Commands:
hyp bee ls {url} - List the entries of the given hyperbee URL.
hyp bee put {url} [value] - Set the value of an entry of the given hyperbee URL.
hyp bee del {url} - Delete an entry of the given hyperbee URL.
Daemon Commands:
hyp daemon stop - Stop the hyperspace and mirroring daemons if active.
hyp diff -> hyp drive diff
hyp ls -> hyp drive ls
hyp put -> hyp drive put
Mirroring daemon started
PS C:\Users\jemma\ManyDrives> hyp seed hyper://b282d5efe484143816362d33b3f9b3ea45ecfb8a6ada97e278fdfdc6a725e22f/
Error: spawn UNKNOWN
PS C:\Users\jemma\ManyDrives> hyp daemon stop
mirroring daemon stopped
hyperspace daemon is running. Attempting to stop...
hyperspace daemon stopped
PS C:\Users\jemma\ManyDrives> hyp daemon
Invalid command: daemon
PS C:\Users\jemma\ManyDrives> hyp daemon status
Daemon not active
PS C:\Users\jemma\ManyDrives> hyp seed hyper://b282d5efe484143816362d33b3f9b3ea45ecfb8a6ada97e278fdfdc6a725e22f/
Hyperspace daemon started
Error: spawn UNKNOWN
PS C:\Users\jemma\ManyDrives> hyp info
Hyperspace daemon started
Error: spawn UNKNOWN
PS C:\Users\jemma\ManyDrives> hyp daemon status
Daemon not active
PS C:\Users\jemma\ManyDrives> hyp sync blog hyper://b282d5efe484143816362d33b3f9b3ea45ecfb8a6ada97e278fdfdc6a725e22f/
Hyperspace daemon started
Error: spawn UNKNOWN
PS C:\Users\jemma\ManyDrives> hyp create drive
Hyperspace daemon started
Error: spawn UNKNOWN
PS C:\Users\jemma\ManyDrives> hyp cat hyper://515bbbc1db2139ef27b6c45dfa418c8be6a1dec16823ea7cb9e61af8d060049e/file.txt
Hyperspace daemon started
Error: spawn UNKNOWN
PS C:\Users\jemma\ManyDrives> node --version
v14.5.0
PS C:\Users\jemma\ManyDrives> hyp cat hyper://515bbbc1db2139ef27b6c45dfa418c8be6a1dec16823ea7cb9e61af8d060049e/file.txt
Hyperspace daemon started
Error: spawn UNKNOWN
PS C:\Users\jemma\ManyDrives> hyp sync blog hyper://b282d5efe484143816362d33b3f9b3ea45ecfb8a6ada97e278fdfdc6a725e22f/
Hyperspace daemon started
Error: spawn UNKNOWN
PS C:\Users\jemma\ManyDrives> hyp daemon status
Daemon not active
PS C:\Users\jemma\ManyDrives> hyp daemon start
Invalid command: daemon
PS C:\Users\jemma\ManyDrives> hyp daemon seed
Invalid command: daemon
PS C:\Users\jemma\ManyDrives> hyp
Usage: hyp <command> [opts...]
General Commands:
hyp info [urls...] - Show information about one (or more) hypers.
hyp seed {urls...} - Download and make hyper data available to the network.
hyp unseed {urls...} - Stop making hyper data available to the network.
hyp create {drive|bee} - Create a new hyperdrive or hyperbee.
Hyperdrive Commands:
hyp drive ls {url} - List the entries of the given hyperdrive URL.
hyp drive mkdir {url} - Create a new directory at the given hyperdrive URL.
hyp drive rmdir {url} - Remove a directory at the given hyperdrive URL.
hyp drive cat {url} - Output the content of the given hyperdrive URL.
hyp drive put {url} [content] - Write a file at the given hyperdrive URL.
hyp drive rm {url} - Remove a file or (if --recursive) a folder at the given hyperdrive URL.
hyp drive diff {source_path_or_url} {target_path_or_url} - Compare two folders in your local filesystem or in hyperdrives. Can optionally "commit" the difference.
hyp drive sync {source_path_or_url} [target_path_or_url] - Continuously sync changes between two folders in your local filesystem or in hyperdrives.
hyp drive http {url} - Host a hyperdrive as using HTTP on the localhost.
Hyperbee Commands:
hyp bee ls {url} - List the entries of the given hyperbee URL.
hyp bee get {url} - Get the value of an entry of the given hyperbee URL.
hyp bee put {url} [value] - Set the value of an entry of the given hyperbee URL.
hyp bee del {url} - Delete an entry of the given hyperbee URL.
Daemon Commands:
hyp daemon status - Check the status of the hyperspace daemon.
hyp daemon stop - Stop the hyperspace and mirroring daemons if active.
Aliases:
hyp sync -> hyp drive sync
hyp diff -> hyp drive diff
hyp ls -> hyp drive ls
hyp cat -> hyp drive cat
hyp put -> hyp drive put
Learn more at https://github.com/hypecore-protocol/cli
Examples from a computer where I synced to (ie source is on the other computer).
hyp drive rm hyper://...
Accessing network...
Error: Cannot delete the root folder
hyp drive rm -r hyper://...
Accessing network...
Error: Cannot delete the root folder
hyp drive rmdir hyper://...
Accessing network...
Error: Cannot delete the root folder
hyp drive rmdir -r hyper://...
Error: URL is required
I want to delete this drive from this computer but it seems that I cannot. What should I do?
I'm curious to know where the drive is located on the filesystem (here macOS)?
This error will dump into the CLI output and make everything ugly:
Error: Block not available from peers
at Object.decode (/Users/paulfrazee/.nvm/versions/node/v14.15.1/lib/node_modules/@hyperspace/cli/node_modules/@hyperspace/rpc/index.js:11:17)
at RPC._onmessage (/Users/paulfrazee/.nvm/versions/node/v14.15.1/lib/node_modules/@hyperspace/cli/node_modules/arpeecee/index.js:113:33)
at Parser._nextState (/Users/paulfrazee/.nvm/versions/node/v14.15.1/lib/node_modules/@hyperspace/cli/node_modules/arpeecee/parser.js:127:14)
at Parser._readMessage (/Users/paulfrazee/.nvm/versions/node/v14.15.1/lib/node_modules/@hyperspace/cli/node_modules/arpeecee/parser.js:54:19)
at Parser.recv (/Users/paulfrazee/.nvm/versions/node/v14.15.1/lib/node_modules/@hyperspace/cli/node_modules/arpeecee/parser.js:35:44)
at RPC._write (/Users/paulfrazee/.nvm/versions/node/v14.15.1/lib/node_modules/@hyperspace/cli/node_modules/arpeecee/index.js:170:22)
at WritableState.update (/Users/paulfrazee/.nvm/versions/node/v14.15.1/lib/node_modules/@hyperspace/cli/node_modules/streamx/index.js:168:14)
at updateWriteNT (/Users/paulfrazee/.nvm/versions/node/v14.15.1/lib/node_modules/@hyperspace/cli/node_modules/streamx/index.js:479:6)
at processTicksAndRejections (internal/process/task_queues.js:79:21) {
code: '',
errno: 0,
details: ''
}
Because of the network, running hyp drive sync
may take a long time or even stall out. We should give some indication about what's happening so that users aren't just confused.
The new dat doctor
For example, to run hyp sync
in a cron job once every hour, it would help if hyp skips checking for confirmation (currently it asks Begin sync? [y/N]
). Several command line tools have a -y
/--yes
or similar option to always assume the answer is yes. Could hyp get this too?
Running hyp seed hyper://<bee address>
results in a message that says Seeding <bee address>
, but running hyp info
after that still results in the bee "Not seeding and not online" and others are unable to seed or access the bee.
Alternatively, if I run a Node server locally that serves a hyperbee (built using this tutorial from Paul Frazee) others are able to seed/access that same bee. After running a hyp info
here while the server is running, the CLI says "Not seeding but online (announcing)". But again, they can actually seed and access the bee.
Since the bee is able to be accessed when the Node server is running, but not when trying to seed it manually from the CLI, I'm assuming there's an issue with the CLI and seeding bees. I am new to Hypercore in general, so perhaps it's user error, but I thought I'd surface this in case it's not.
It would be great to redirect hyp text output to stdout, so that it would be redirected into file, filtered through pipe with grep, etc.
C:\Users\abusn>hyp create bee
Bee Created: hyper://1a516a871719c7e9162368be8e945a47d60df42929f6027aff62493942f36e84
events.js:292
throw er; // Unhandled 'error' event
^
Error: key is required
at Object.encodingLength (C:\Users\abusn\AppData\Roaming\npm\node_modules\@hyperspace\cli\node_modules\hyperspace-mirroring-service\rpc\messages.js:52:34)
at Parser.send (C:\Users\abusn\AppData\Roaming\npm\node_modules\@hyperspace\cli\node_modules\arpeecee\parser.js:138:24)
at RPC._push (C:\Users\abusn\AppData\Roaming\npm\node_modules\@hyperspace\cli\node_modules\arpeecee\index.js:159:25) at RPC._request (C:\Users\abusn\AppData\Roaming\npm\node_modules\@hyperspace\cli\node_modules\arpeecee\index.js:203:10)
at Method.request (C:\Users\abusn\AppData\Roaming\npm\node_modules\@hyperspace\cli\node_modules\arpeecee\index.js:247:29)
at HRPCServiceMirror.mirror (C:\Users\abusn\AppData\Roaming\npm\node_modules\@hyperspace\cli\node_modules\hyperspace-mirroring-service\rpc\index.js:63:25)
at HyperspaceMirroringClient.mirror (C:\Users\abusn\AppData\Roaming\npm\node_modules\@hyperspace\cli\node_modules\hyperspace-mirroring-service\client.js:57:32)
at command (file:///C:/Users/abusn/AppData/Roaming/npm/node_modules/@hyperspace/cli/lib/commands/create.js:25:32)
at processTicksAndRejections (internal/process/task_queues.js:93:5)
at async Object.obj.command (file:///C:/Users/abusn/AppData/Roaming/npm/node_modules/@hyperspace/cli/bin/hyp.js:101:7)
Emitted 'error' event on HRPCSession instance at:
at RPC.<anonymous> (C:\Users\abusn\AppData\Roaming\npm\node_modules\@hyperspace\cli\node_modules\hyperspace-mirroring-service\rpc\index.js:117:101)
at RPC.emit (events.js:327:22)
at WritableState.afterDestroy (C:\Users\abusn\AppData\Roaming\npm\node_modules\@hyperspace\cli\node_modules\streamx\index.js:439:19)
at RPC._destroy (C:\Users\abusn\AppData\Roaming\npm\node_modules\@hyperspace\cli\node_modules\arpeecee\index.js:190:5)
at WritableState.updateNonPrimary (C:\Users\abusn\AppData\Roaming\npm\node_modules\@hyperspace\cli\node_modules\streamx\index.js:187:16)
at WritableState.update (C:\Users\abusn\AppData\Roaming\npm\node_modules\@hyperspace\cli\node_modules\streamx\index.js:172:70)
at updateWriteNT (C:\Users\abusn\AppData\Roaming\npm\node_modules\@hyperspace\cli\node_modules\streamx\index.js:479:6)
at processTicksAndRejections (internal/process/task_queues.js:79:21)
Need tools to transfer or backup your keys
Move all the live-output status output to the info
command
When I start hypercore with hyperspace --storage $STORAGE --host $HOST --port $PORT
, then run:
❯ ps -aux | rg hyp
jrestivo 1067902 0.1 0.0 635796 54572 ? SLsl 11:08 0:00 /nix/store/5kkjf5wvfvclqj76k9f69c95gw5mfbhc-nodejs-14.17.4/bin/node /nix/store/if04chi2qyhxr5mvifmq9wfhswydj3ms-hyperspace/bin/hyperspace --storage ~/.hyperspace --host hyperspace --port 0
jrestivo 1069029 0.0 0.0 9644 6232 pts/11 S+ 11:11 0:00 rg hyp
flakes [$!+?]
❯ hyp info
The daemon is not active. Please run:
hyp daemon start
❯ hyp daemon status
Hyperspace v3.19.0
Hyperspace API v1.15.1
Your address: (Not Hole-punchable)
This is preventing me from using any of the hyp
commands. Any advice? Both hyp info
and hyp daemon start
connect to the daemon, as it prints out:
Remote client opened
Remote client closed
This will give the networking status of hypers not managed by the mirror service but active in the daemon
My expectation is hyperspace's socket env variable defines where the socket is created. However when the full path is defined, they command freezes when trying to start the daemon.
HYPERSPACE_SOCKET="/tmp/mysock" hyp daemon start
# Hyperspace daemon started
# [freezes]
I believe this is either a bug or an incorrect error message, it depends on whether the client is supposed to be usable for sharing hypercores which are neither hyperdrives nor hyperbees.
When I try to seed a hypercore, I get the following error:
hyp seed hyper://70e...6712/
node:events:342
throw er; // Unhandled 'error' event
^
Error: key is required
at Object.encodingLength (/.nvm/versions/node/v16.1.0/lib/node_modules/@hyperspace/cli/node_modules/hyperspace-mirroring-service/rpc/messages.js:52:34)
at Parser.send (/.nvm/versions/node/v16.1.0/lib/node_modules/@hyperspace/cli/node_modules/arpeecee/parser.js:138:24)
at RPC._push (/.nvm/versions/node/v16.1.0/lib/node_modules/@hyperspace/cli/node_modules/arpeecee/index.js:159:25)
at RPC._request (/.nvm/versions/node/v16.1.0/lib/node_modules/@hyperspace/cli/node_modules/arpeecee/index.js:203:10)
at Method.request (/.nvm/versions/node/v16.1.0/lib/node_modules/@hyperspace/cli/node_modules/arpeecee/index.js:247:29)
at HRPCServiceMirror.mirror (/.nvm/versions/node/v16.1.0/lib/node_modules/@hyperspace/cli/node_modules/hyperspace-mirroring-service/rpc/index.js:63:25)
at HyperspaceMirroringClient.mirror (/.nvm/versions/node/v16.1.0/lib/node_modules/@hyperspace/cli/node_modules/hyperspace-mirroring-service/client.js:57:32)
at command (file:///.nvm/versions/node/v16.1.0/lib/node_modules/@hyperspace/cli/lib/commands/seed.js:32:29)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async Object.obj.command (file:///.nvm/versions/node/v16.1.0/lib/node_modules/@hyperspace/cli/bin/hyp.js:108:7)
Emitted 'error' event on HRPCSession instance at:
at RPC.<anonymous> (/.nvm/versions/node/v16.1.0/lib/node_modules/@hyperspace/cli/node_modules/hyperspace-mirroring-service/rpc/index.js:117:101)
at RPC.emit (node:events:377:35)
at WritableState.afterDestroy (/.nvm/versions/node/v16.1.0/lib/node_modules/@hyperspace/cli/node_modules/streamx/index.js:439:19)
at RPC._destroy (/.nvm/versions/node/v16.1.0/lib/node_modules/@hyperspace/cli/node_modules/arpeecee/index.js:190:5)
at WritableState.updateNonPrimary (/.nvm/versions/node/v16.1.0/lib/node_modules/@hyperspace/cli/node_modules/streamx/index.js:187:16)
at WritableState.update (/.nvm/versions/node/v16.1.0/lib/node_modules/@hyperspace/cli/node_modules/streamx/index.js:172:70)
at updateWriteNT (/.nvm/versions/node/v16.1.0/lib/node_modules/@hyperspace/cli/node_modules/streamx/index.js:479:6)
at processTicksAndRejections (node:internal/process/task_queues:82:21)
When I create a hyperdrive with hyp I can seed it without any issues.
Version info:
hyp --version 1.5.1
Running hyperspace/3.19.0 linux-x64 node-v16.1.0
Server is already running with the following status
API Version : 1.15.1
Holepunchable : true
It might be nice to have a structure for adding plugins to the core hyp
CLI tool.
This could have a couple of benefits:
With a couple of corresponding cons:
Questions to address:
Takes a hyper key, tells you if the key's being seeded and other metadata
The hyperspace daemon doesn't seem to match what daemon status is saying
Tried running daemon stop
and hyperspace doesn't stop. Both client and daemon were latest
It'd be really handy if we could alias hyper URLs to some registry internal to hk
to reduce typing. Something like...
hk alias bob hyper://1234..af
hk drive ls bob
hk drive cat bob/index.html
When I try to seed a directory with 99GB of data spread across 2.6 million files, I get a memory allocation error after 2 minutes:
peermaps@vmi595217:~/data/peermaps/edb$ time hyp drive sync . -y
Creating new hyperdrive...
Source: .
Target: hyper://a2f90b0157f7a9e810410e5480f40e345621002757fd1afeda31980c361dd9bc//
Syncing...
Comparing...
<--- Last few GCs --->
06[28308:0x60bc730] 130401 ms: Mark-sweep (reduce) 4096.2 (4102.0) -> 4095.5 (4103.3) MB, 4123.0 / 14.2 ms (+ 0.0 ms in 16 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 4148 ms) (average mu = 0.061, current mu = 0.006[28308:0x60bc730] 133968 ms: Mark-sweep (reduce) 4096.5 (4102.3) -> 4095.9 (4103.5) MB, 3350.9 / 15.6 ms (+ 175.9 ms in 16 steps since start of marking, biggest step 108.7 ms, walltime since start of marking 3567 ms) (average mu = 0.036, current mu = 0
<--- JS stacktrace --->
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
1: 0xa222f0 node::Abort() [hyp]
2: 0x96411f node::FatalError(char const*, char const*) [hyp]
3: 0xb97f1e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [hyp]
4: 0xb98297 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [hyp]
5: 0xd52fd5 [hyp]
6: 0xd53b5f [hyp]
7: 0xd61beb v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [hyp]
8: 0xd657ac v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [hyp]
9: 0xd33e5b v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [hyp]
10: 0x107c5af v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [hyp]
11: 0x1423279 [hyp]
Aborted
real 2m18.102s
user 4m57.573s
sys 1m21.883s
Watching htop
the whole time, the highest I saw the memory usage go was up to 10% on this machine with 60GB of memory.
peermaps@vmi595217:~/data/peermaps/edb$ node -v
v14.17.0
peermaps@vmi595217:~/data/peermaps/edb$ hyp --version
2.0.0
To reproduce, the data is also seeded to /ipfs/QmVCYUK51Miz4jEjJxCq3bA6dfq5FXD6s2EYp6LjHQhGmh
Support for ignoring files and directories has been lost during the move from dat to hyper.
While there is support retained in Beaker browser, though that no longer uses .datignore
it does retain a memory of what was ignore in a previous sync. Maybe this just needs a cli command to allow interaction with the ignore list for a hyperdrive, and for that list to be respected when running sync/diff.
Also see hyperspace-org/hyperdrive-service#7, and hypercore-protocol/hyperdrive-daemon#53
I have installed, removed and reinstalled hyp and still getting same error. When installing, I receive several warnings about deprecated components. However it finishes install. Once I enter hyp daemon start or any other command beginning with hyp, I get the following. OS is KDE Neon 5.20.
/usr/local/lib/node_modules/@hyperspace/cli/bin/hyp.js:6
import subcommand from 'subcommand'
^^^^^^^^^^
SyntaxError: Unexpected identifier
at Module._compile (internal/modules/cjs/loader.js:723:23)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
at Module.load (internal/modules/cjs/loader.js:653:32)
at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
at Function.Module._load (internal/modules/cjs/loader.js:585:3)
at Function.Module.runMain (internal/modules/cjs/loader.js:831:12)
at startup (internal/bootstrap/node.js:283:19)
at bootstrapNodeJSCore (internal/bootstrap/node.js:623:3)```
Here are the warnings I get while installing.
```$ sudo npm install -g @hyperspace/cli
npm WARN deprecated [email protected]: Debug versions >=3.2.0 <3.2.7 || >=4 <4.3.1 have a low-severity ReDos regression when used in a Node.js environment. It is recommended you upgrade to 3.2.7 or 4.3.1. (https://github.com/visionmedia/debug/issues/797)
npm WARN deprecated [email protected]: "Please update to latest v2.3 or v2.2"
npm WARN deprecated [email protected]: cross-spawn no longer requires a build toolchain, use it instead
/usr/local/bin/hyp -> /usr/local/lib/node_modules/@hyperspace/cli/bin/hyp.js
> [email protected] install /usr/local/lib/node_modules/@hyperspace/cli/node_modules/fd-lock
> node-gyp-build```
Any assistance would be greatly appreciated!
ipfs uses 8080, so this is maybe not the best default port to use since the audience for ipfs and hyperdrive is very overlapping. 8080 is also a common port for local development.
hi here is my backend.js
const { Client } = require('hyperspace')
const HyperBee = require('hyperbee')
const { kvPairs : dictionaryPairs } = require('websters-english-dictionary')
start()
async function start() {
const { corestore, replicate } = new Client()
const store = corestore()
const core = store.get({ name: 'dictionary' })
const db = new HyperBee(core, { keyEncoding: 'utf-8', valueEncoding: 'utf-8' })
const batch = db.batch()
for (const { key, value } of dictionaryPairs()) {
await batch.put(key, value)
}
await batch.flush()
//print bee key
console.log(' bee ', db.feed.key.toString('hex'))
// Now announce on hyp
await replicate(core)
}
when i am running this code its giving me a console a key on console :
ba118da4eb768dde70d6c1e016ff5b26f3e97ef91ff097c9f21d7c4f8c4699e1
I am seeding the above key as : hyp seed ba118da4eb768dde70d6c1e016ff5b26f3e97ef91ff097c9f21d7c4f8c4699e1
now on second server i am doing :: hyp sync ba118da4eb768dde70d6c1e016ff5b26f3e97ef91ff097c9f21d7c4f8c4699e1 ------ above line giving me error:
root@tower6:~# hyp sync ba118da4eb768dde70d6c1e016ff5b26f3e97ef91ff097c9f21d7c4f8c4699e1
Error: The hyper:// was expected to be a hyperdrive, got hyperbee
please help me how to seed and sync bee
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.