Giter Club home page Giter Club logo

cloudquery's Introduction

CloudQuery

Turn any website to serverless API (with SPA support!)

Try online: https://2uvlp0y13b.execute-api.us-west-2.amazonaws.com/staging/

API

Example url: https://2uvlp0y13b.execute-api.us-west-2.amazonaws.com/staging/query?url=https://news.ycombinator.com/best&selectors=*:nth-child(1) > * > *:nth-child(2) > *:nth-child(2) > :nth-child(1),:nth-child(3) *:nth-child(1) > *:nth-child(3) > *:nth-child(1)

Query params:

  • url: url of the webpage you want to fetch
  • selectors: css selectors describing the elements interestes you

Returns:

  • innerText: element innerText
  • href: element href

Run locally

npm i
npm start

How to deploy your own CloudQuery to AWS lambda

1. Configuration

  • update profile(AWS Credential Profiles) in up.json to use your own aws account
  • update rateLimit in config.json to set your own rate limit (Default: 5 request/hour)

2. Deploy

  • Input up in terminal and hit enter :)
  • up url to see the API baseURL of your CloudQUery

Contribute

Backend

vim app.js

Frontend

cd frontend
# develop
npm start
# build
npm run build

Thanks

  • up for deploying serverless API to AWS with ease
  • serverless-chrome for running chrome on AWS lambda
  • finder for making it easy to select elements on webpage

cloudquery's People

Watchers

James Cloos avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.