Tracking issue for distributing as static binary to:

• GitHub releases
• homebrew
• apt
• etc.

Shouldn't deploy after showing the version.

I can't seem to deploy to my ipfs-cluster instance. I'm not sure if it's related to the cluster using https, or if it's just broken in general.

The error that it spits out is: SyntaxError: Unexpected token C in JSON at position 0

I got this error after configuring the .env file.
Am I making it wrong, or is it a bug?

ℹ 📡 Beaming new hash to DNS provider Cloudflare…
(node:12729) [DEP0066] DeprecationWarning: OutgoingMessage.prototype._headers is deprecated
✖ 💔 Updating Cloudflare DNS didn't work.

HTTPError: Response code 400 (Bad Request)

We can speed up the first view of the /ipfs/<cid> preview url by using the ipfs https gateway on the service where the site is pinned, and ease the burden on the ipfs.io http gateway a little.

Service specific gateway available at

• https://ipfs.infura.io/
• https://gateway.pinata.cloud

Having to manually forward a port just so that a disposable IPFS node can be reached by Pinata is a lot of trouble.

Maybe there is an easy-to-use npm package to do some basic UPnP port forwarding without being as comprehensive as perhaps libp2p wants to do it so we don't have to block on that.

Keeping an eye on this anyway:
libp2p/js-libp2p#104

@agentofuser I was wondering why did you choose prettier to lint the code. Just wondering since most of the time I've seen standard.js being used which seems much more cleaner (less commas, more spaces, etc).

Tracking issue for pinning service support. Going to prioritize those that have some kind of free-forever tier. And among those, the ones that require no signup or have the least friction to signup.

• Infura.io
• Pinata.cloud
• Temporal.cloud
• Eternum.io

 ✖ 💔  Updating Cloudflare DNS didn't work.
32   ┌──────────────────────────────────────────────────────────────────────────┐
33   │                                                                          │
34   │ HTTPError: Response code 400 (Bad Request)                               │
35   │                                                                          │
36   └──────────────────────────────────────────────────────────────────────────┘

I really wish this included why.
The 403 error has an errors key in the json response that contains error messages that say why it didn't work.
I bet this 400 Bad Request response also had error messages.
This cli should output those messages.

Generalizing from #9 to include:

• IPNS
• ENS
• etc.

Right now ipfs-deploy only builds on node 10.15.3 whereas the stable version of node is 12.1.0

I'd appreciate it if I didn't also need nvm to install this and could just use stable node

NOTE: Racket sites output to the project root, and not in a production root subfolder of the project root. In other words, project root and production document root are one and the same.

Some details of the experience:
https://js.ipfs.io/ipns/QmZJBQBXX98AuTcoR1HBGdbe5Gph74ZBWSgNemBcqPNv1W/Zero-Config-CLI-to-Deploy-Static-Websites-to-IPFS.html

So once I redeploy on cloudflare, the content doesn't update. What gives!?

As I was making #73, I had the idea of restructuring the directories in the following way:

• bin/ - as it is right now.
• src/
  • index.js - move from the root.
  • dnslink/
    • cloudflare.js
    • sth-else.js
  • pinners/
    • infura.js
    • pinata.js
    • ipfs-cluster.js
    • sth-else.js

From dnslink, all files would export the same api:

updateSomethingDns(options)

From pinners, all files would export the same api:

// Setup Pinner
const pinner = setupPinnerName(options)

// Pin from file system
const hash = await pinner.pin(dir, options)

// Pin hash
await pinner.pinHash(hash, options)

Otherwise:

// Setup Pinner
const pin = setupPinnerName(options)

// Pin from file system
const hash = await pin(dir, options)

Some pinning services (such as Piñata or IPFS Cluster) allow us to just pin the hash instead of 'uploading' everything at the time. For instance, if I'd like to be able to 'upload' my files to IPFS Cluster and then use that hash to pin on Piñata via their API instead of 'uploading' to both services.

Perhaps we could use a different flag. Maybe, by default, we'd 'upload' it to all services and if we set --hash-from <service> it would use the hash given by that service and then pin on the others.

For people wanting to try out IPFS-deploy, a second, account-free default in addition to Infura can be IPFSpin twitter bot. Its IPFS cluster may seem to be overloaded but it still makes big promises to the world. Lately it always replys:

"IPFS Cluster has been pinning this for 10 mins. This is normal for big files. Otherwise, make sure there are providers for it. Don't worry, Cluster will keep at it for a week before giving up."

But as far as I can tell, you never know whether the pin was successful, as I never have seen a follow-up tweet to the above upon success (or failure). So a quiet request to ipfspin bot (or directly to its cluster?) via some less-noisy API might be a good thing.

Some details of the experience:
https://js.ipfs.io/ipns/QmZJBQBXX98AuTcoR1HBGdbe5Gph74ZBWSgNemBcqPNv1W/Zero-Config-CLI-to-Deploy-Static-Websites-to-IPFS.html

I have deployed my pet-shop using ipd -p infura -p pinata Here is the url:

https://ipfs.io/ipfs/QmT6SgXKChuzz9nCTu8ZNtc8y7wLR3DjvH3rsEkcreiSDG/

I am getting the following error:

invalid ipfs path: invalid path "/ipfs/pets.json": invalid CID: selected encoding not supported

Any ideas?

A more long-term idea: at some point it would be nice to have a monorepo with separate npm packages for the cli and the api. That way, heavier dependencies needed for better usability (say, CLI React?) can be segregated to the cli, leaving a lean, tree-shakeable api for other types of integrations.

Idea by @daviddias on scuttlebutt:

Are you planning on integrating a "health check" in which users can check the number of replicas available and confirm if it is still pinned?

It would be nice to have a way to monitor latency and uptime for pinning services, don't know how that would fit within ipfs-deploy yet or if a different service.

I do want to add more "deployment management" things like CID history, easy rollbacks, etc.

Summary

This is a revamp of issue #12 about supporting more pinning services. Here, we will track the support for all pinning services across many issues and PRs.

Status

• ❔ : Status unknown
• 🐣 : Work in progress
• 🐓 : Published

Makes it easier to copy and paste when using ipfs-deploy in places where automatic copying to clipboard and/or open on browser doesn't work, like on codesandbox.

If I upload a dir of a basic static website (a basic html, js and css file) with ipfs-deploy, even though the js/css files are relatively linked I get this error when viewing the resulting hash:

The resource from “https://ipfs.infura.io/ipfs/style.css” was blocked due to MIME type (“text/plain”) mismatch (X-Content-Type-Options: nosniff).

Same website works fine when I do ipfs add -r ..

I am using Infura and Pinata for my pin service.
But it always said "Found inconsistency in pinned hashes"
Pinata folder name is always Cloudflare DNS Record, or /Users/myusername/.config/yarn/global/node_modules/ipfs-deploy/src
Not the real folder name.

Summary

This is the main tracking issue for Naming Providers support. Naming providers can be either DNS - for use with DNSLink - or even naming services such as ENS and IPNS.

Ideally even doing things like detecting the provider from the domain name and doing the right thing.

Status

• ❔ : Status unknown
• 🐣 : Work in progress
• 🐓 : Published

Using Pastel (an Ink framework by Ink's author) would make it easier to add interactivity, like asking for credentials when they are missing, allowing the user to pick services from a list, and in general allowing an alternative UX to command line parameters. Also a great source of snazzy pretty things :)

https://github.com/vadimdemedes/pastel

A first PR would be to just set up a skeleton and convert what's already there to Pastel without changing anything user-facing.

As an idea for consideration: What if you don't have to convert your files into MerkleDAGs and just used the git structure directly?

One of the cool tricks that IPFS can do is resolve any kind of hash linked data structure. This means that you can save the users a step by avoiding the ipfs add and instead just resolve the git hash as a file system and resolve the published version of the website as the git hash.

With this, then to load any website, effectively you would just need to do something such as ipfs.io/git/<githash> and IPFS would resolve and distribute it through the network.

If this all sounds like jibba jabba at first glance, I do have some urls to clarify, namely:

• The IPLD Talk that explains the idea beyond the InterPlanetary Linked Data
• Traversing IPLD graphs examples. These examples show how to pick traverse nodes from a graph, including git graphs.
• IGiS, the InterPlanetary Git Service. A Web app that allows you to traverse git repos that are fetched through IPFS (note: apparently the version that is published has some spider webs to be cleaned up, I've ping @magik6k (the author) to see if he can check in) (note2: I also know that @magik6k has multiple recorded demos, once I find them, I'll link them here :))

They support both IPFS and IPNS (cc #20) pinning.

I recorded it with asciinema and converted to svg with svg-term. For some reason it's making the page very slow to respond for me. There's gotta be some parameter that can be tuned to improve that.

https://github.com/agentofuser/ipfs-deploy/blob/master/index.js#L147

If the request fail due to a timeout or another reason, ipfs-deploy use the last received hash as the root hash. However, there is no guarantee that it's actually the root hash, and it won't be if the response has been truncated.

When I try to deploy with Cloudflare to a subdomain, I get the following error:
Error: zone ipfs.mywebsite.tld couldn't be found

A standard create-react-app has both a "public" and "build" folder. The build folder is the one containing the actual build, while the "public" folder should be ignored.

Gguess-path.js just returns the first found matching folder, so you end up deploying the wrong "public" folder.

I think one possibility would be to just bail out when more than one candidate is found with an according message to the user. Something like "Found more than one path candidate to deploy, please specify the path to use.

Allow ipfs-deploy to update a DNSLink TXT record via the DNSimple api.

I'm happy to work on this as we need it for deploying sites for IPFS and Protocol Labs.

I'll add more details here when dig into it.

I don't have much experience testing JS, so suggestions of tools and practices are welcome.

Continuing from https://news.ycombinator.com/item?id=19808377

The idea is to make it possible to write the authentication key to the website's .env, then call ipd -p eternum like with -p pinata and -p infura to pin to that as well.

Refs #12

Still, it should be possible to call ipd ., so I'll look into that. Thank you for the helpful report!

Originally reported by @Exospheric in #22

• Tracking issue.
• Windows, macOS, Linux, and BSD should work as seamlessly as possible.
• Cf.: https://github.com/ehmicky/cross-platform-node-guide/blob/master/docs/summary.md

NPM has a dependency on XCODE, which is too huge for the casual user or someone with precious SSD space. So the NPX call is probably important to most people.

On MacOs, pretty much everything I try to do begets:

Unexpected token *
Some details of the experience:
https://js.ipfs.io/ipns/QmZJBQBXX98AuTcoR1HBGdbe5Gph74ZBWSgNemBcqPNv1W/Zero-Config-CLI-to-Deploy-Static-Websites-to-IPFS.html

versions:
node: 10.15.3
ipfs-deploy installed via yarn
hugo 0.55.5 generated website

I just tried deploying a WebGui build with react-script, and after telling me that everything was uploaded, I only get a string from the ipfs.infura.io Web gateway.

Using ipfs daemon all works fine.

The ora spinner is fine for things that take a couple of seconds, but when it's spinning for a minute it's hard to tell if things are still alive.

@olizilla, would you like to take this on? For Infura it seems simpler, as it's pure HTTP, but for Pinata it involves getting networking info from the in-process IPFS daemon and I don't know much about libp2p atm.

After experiencieng issues with infura hashes not matching the hashes from other pinners i looked a bit deeper into the issue to understand what is happening.

Test case is the following directory structure:

public/:
root.txt  sub

public/sub:
sub1.txt  sub2.txt  subsub

public/sub/subsub:
subsub1.txt  subsub2.txt

Then i tried to upload with infura and dappnode pinner, and check the response from api.addFromFs in pinners/infura.js and pinners/dappnode.js (via debugger):

Response from dappnode pinner:

[ { path: 'public/root.txt',
    hash: 'QmQJRuzSnVj3ZtyTkuLqdhBL7QBzpDMwUSsuX2ZvvZqK9X',
    size: 34 },
  { path: 'public/sub/sub1.txt',
    hash: 'QmYXAWSYoyDVAwGuUe5X9B4F9e4r1G8E66meVk12U6MZSQ',
    size: 44 },
  { path: 'public/sub/sub2.txt',
    hash: 'Qma8NBewQpX2GZafD71zYUjVX7Ve5LZ44bYaEcTCHYY3hE',
    size: 44 },
  { path: 'public/sub/subsub/subsub1.txt',
    hash: 'QmZxES63SLnndw2vASMwYpXggqpUJu1wNhbNLmMti1GGq1',
    size: 53 },
  { path: 'public/sub/subsub/subsub2.txt',
    hash: 'Qma9Y3hHFb7Xt1kYoLZ6emtGRZym8wooTHokJ9NYhD2DDN',
    size: 53 },
  { path: 'public/sub/subsub',
    hash: 'QmUnRVEPAPnsm3cTWLy6cA9RrgdHajSbuuVaJb8WRpvyXr',
    size: 216 },
  { path: 'public/sub',
    hash: 'Qmf9EDMfav9WBmM5reng5H4igoMnkFJP5eiN4pTUD12hV5',
    size: 457 },
  { path: 'public',
    hash: 'QmWJJU8G7SDBYvu3dYZ9qdDHNgLkbQLGwAREEx3Djc5LFd',
    size: 591 } ]

Response from infura pinner:

[ { path: 'public/root.txt',
    hash: 'QmQJRuzSnVj3ZtyTkuLqdhBL7QBzpDMwUSsuX2ZvvZqK9X',
    size: 34 } ]

Obviously, the responses are different. Dappnode response object contains all files, with the root folder element being the last element. Infura response object contains just one (random?) file from the directory.

While the root cause of this issue might be instability of Infura service (it also took about 20 seconds to get a response from infura API at all, so they clearly had problems at this time), this error needs to result in ipfs-deploy exiting with an error instead of providing wrong results.

In only had a brief look so far at the implementation, but I'm wondering if this needs to be investigated more inside ipfs-http-client? After all we're just using their "addFromFs" API.

Edit/addendum: Looking at https://github.com/ipfs/interface-js-ipfs-core/blob/master/SPEC/FILES.md#addfromfs there is no mention of a specifc ordering of the results. So the current approach of assuming that the last array element is the root node seems quite dangerous.

The rc module allows the user to store the config for ipfs-deploy either in the current project, as env does, or in standard locations in ~ or /etc as per https://github.com/dominictarr/rc#standards

For things like account credentials, it'd be reasonable to store them in one place for all sites you use ipfs-deploy for, rather than in each project.

How do I setup my domain to resolve when using www? This is probably a simple fix but I'm not super well versed in DNS. My site is currently working when using the root domain, but the www 404s. I tried to follow along with these instructions but might have missed something. Here are my current DNS records:

CNAME www cloudflare-ipfs.com DNS Only 
CNAME tatecarson.com cloudflare-ipfs.com DNS Only 
TXT _dnslink dnslink=/ipfs/QmRFciUgPdWYpvFWcMWHEhcMndSeACMUhwVgZVcAmd5RF3 DNS only

and the env setup. I see that I have it going to the root here but not sure what else to put.

IPFS_DEPLOY_CLOUDFLARE__ZONE=tatecarson.com
IPFS_DEPLOY_CLOUDFLARE__RECORD=_dnslink.tatecarson.com

Any ideas?

Thank you, and I realize this isn't the best place to ask but I have no other idea where to go.

Hi! Author here. 👋

This issue is where you can attest that ipfs-deploy was useful to you, if you so wish :) It makes a huge difference to my motivation.

Feel free to leave any kind of positive comment here, and please open a different issue for bug reports, feature requests, and telling me off for being a n00b. I will do my best to help <3

• Infura: 100MB https://community.infura.io/t/is-there-a-file-size-limit-for-files-attached-to-ipfs-infura-io/61/2
• Pinata: 1GB #52 (comment)

ref #52

I'm getting a port unreachable error when trying to use the the api. Looking for some pointers rather than reporting an issue here.

This is the call I'm making from a nodejs app:

async function puttoifps() {
    try {
      const deployOptions = {
        publicDirPath: 'public',
        // siteDomain: 'ipfs.infura.io/ipfs/',
        remotePinners: 'pinata',
       // port: '8080',
        credentials: {
          pinata: {
            apiKey: 'my api key',
            secretApiKey: 'my secret',
          },
        },
      }
      return deploy(deployOptions)
    } catch (e) { console.log('error wtf happened?', e)}
  };

Here is there error message:
✔ 🚚 public weighs 25.0 MiB.
✔ ☎️ Connected to temporary local IPFS node.
✖ 💔 Could not reach port 4002 from the outside :(
ℹ 💡 Please forward it or try a different one with the --port option.

Error: Could not reach port 4002 from the outside

• index.js:203 IPFS.node.on
  [gatsby-plugin-crypowerk-blockchain]/[@agentofuser]/ipfs-deploy/index.js:203 :16

• next_tick.js:81 processTicksAndRejections
  internal/process/next_tick.js:81:5

I've tried changing the port in the config but that hasn't worked. I don't think it's a firewall issue on my side as those ports are open.

Any suggestions or pointers would be appreciated. - Kevin

$ ipd
ℹ 🤔 No path argument specified. Looking for common ones…
✔ 📂 Found local public directory. Deploying that.
✔ 🚚 public weighs 133.3 MiB.
✖ 💔 Uploading to Infura didn't work.

  ┌─────────────────────────────────────────────────────────────────────────────────────────┐
  │                                                                                         │
  │ {                                                                                       │
  │    "name": "Error",                                                                     │
  │    "className": "Error",                                                                │
  │    "message": "payload too large\n",                                                    │
  │    "superclasses": ["Object"],                                                          │
  │    "enumerableFields": { "statusCode": 400 },                                           │
  │    "stack": [                                                                           │
  │      "Error: payload too large",                                                        │
  │      "at parseError                                                                     │
  │ (/usr/local/share/.config/yarn/global/node_modules/ipfs-http-client/src/utils/send-requ │
  │ est.js:22:17)",                                                                         │
  │      "at ClientRequest.<anonymous>                                                      │
  │ (/usr/local/share/.config/yarn/global/node_modules/ipfs-http-client/src/utils/send-requ │
  │ est.js:64:14)",                                                                         │
  │      "at Object.onceWrapper (events.js:288:20)",                                        │
  │      "at ClientRequest.emit (events.js:200:13)",                                        │
  │      "at HTTPParser.parserOnIncomingClient [as onIncoming] (_http_client.js:565:23)",   │
  │      "at HTTPParser.parserOnHeadersComplete (_http_common.js:116:17)",                  │
  │      "at TLSSocket.socketOnData (_http_client.js:452:22)",                              │
  │      "at TLSSocket.emit (events.js:200:13)",                                            │
  │      "at addChunk (_stream_readable.js:294:12)",                                        │
  │      "at readableAddChunk (_stream_readable.js:275:11)"                                 │
  │    ]                                                                                    │
  │ }                                                                                       │
  │                                                                                         │
  │                                                                                         │
  └─────────────────────────────────────────────────────────────────────────────────────────┘

  ┌─────────────────────────────────────────────────────────────────────────────────────────┐
  │                                                                                         │
  │ Failed to deploy.                                                                       │
  │                                                                                         │
  └─────────────────────────────────────────────────────────────────────────────────────────┘

Also happens with pinata.

• https://news.ycombinator.com/item?id=19803967
• ipfs/in-web-browsers#121

I just tried deploying a site with the default settings, and after telling me that everything was uploaded and pinned, I only get gateway timeouts from the ipfs.infura.io Web gateway. Anyone else had this issue before?

Cloudflare supports generating tokens with specific set of permissions to be used instead of the global api key. Perhaps a separate IPFS_DEPLOY_CLOUDFLARE__API_TOKEN variable?