feature: cli about scrape-it HOT 8 CLOSED

That would be great! ✨
For the cli module I'd recommend using tilda.

Also, I would say to create a different project for the cli tool (e.g. scrape-it-cli).

Let me know if you have any questions! 😄

from scrape-it.

I thought about docopt...
it's based on this original project

The idea is that it takes usage string as DSL of your CLI. So all you have to write is this:

docopt(`
Usage:
  scrape-it <url> [--some-option]
  scrape-it -
  scrape-it -h | --help | --version

Options:
  --config=FILE  Config file [default: ./scrape-it.conf.js]
`)

And all of sudden you get something like this:

{
  url: 'http://...',
  config: './cfg.js',
  some-option: true
}

If you're okay with that I'd be happy to do it (and yeah it can be in its own repo)

from scrape-it.

@cztomsik Sure, feel free to use whatever you're comfortable with and share a link once it's on GitHub. 🚀

I'll be happy to add a link to it. 😁

from scrape-it.

ok :)

from scrape-it.

Here's the very first version, it's not even published to npm yet, what do you think? Any ideas?

I was thinking about some default config or maybe an init command so people could start quickly?

Something like

npm i -g scrape-it-cli
scrape-it init
scrape-it http://github.com

and instantly get:

{ title: 'How people build software · GitHub' }

from scrape-it.

@cztomsik Cool! 👍

from scrape-it.

@cztomsik Hey! Did you do any progress on this? Maybe we can work together for a stable release of scrape-it-cli.

from scrape-it.

Hey, not exactly, basic version worked fine but I've actually found myself in need of something much more powerful (dozens of concurrent crawls backed in one big database) which I am working on and hopefully will open-source soon - I think you might like it and OFC we could join forces then :)

Anyway if you have ideas about scrape-it-cli I can certainly help you a little (or brainstorm) or even transfer the repo to you

from scrape-it.