computablelabs / crunky Goto Github PK

The queries that the backend system should run should be drawn from a limited subset that is known to be safe in order to avoid potential security vulnerabilities. For this purpose, we should assemble a whitelist of known "safe" queries and limit the user to running queries from this safe subset.

We'll want Computable backend systems like Crunky to support a REST API. The goal of the REST API is to provide a standard way for data market users to communicate with backends. Here's a first cut at some proposed REST API options:

Authentication

Description

• AUTHENTICATE user-credentials
• Allows a given user to authenticate itself to the backend
• Results of a spike outlining the Metamask authentication flow can be found here

Endpoints

• POST /signup creates a new user record
• POST /login creates a new session record for a given user

Marketplace Schema

Description

• GET_SCHEMA auth_token
• Returns the scheme of the dataset, likely in JSON format
• Can support different backend schemas (SQL/Spark/etc) with different JSON blogs

Endpoints

• GET /marketplaces/:marketplace_id/schema returns the schema for a data marketplace, given a marketplace ID

Data Queries

Description

• RUN_QUERY auth_token query-specification
• Runs the specified query
• Security is a key focus here. It's likely that we'll have a number of fixed "whitelisted" queries rather than allowing for arbitrary queries to prevent SQL injection attacks or dataset scraping attempts.

Endpoints

• POST /marketplaces/:marketplace_id/search returns the result of the query defined in the request body

Data Management

Description

• ADD_DATAPOINT auth_token datapoint
• Adds the specified datapoint from backend. One datapoint corresponds to one listing in market
• TODO: fix the terminology once we settle on it

Endpoints

• POST /marketplaces/:marketplace_id/data adds the associated data to the data marketplace, given a marketplace ID

Description

• REMOVE_DATAPOINT auth_token datapoint
• Removes the specified datapoint. One datapoint corresponds to one listing in market
• TODO: fix the datapoint/listing terminology later

Endpoints

• DELETE /marketplaces/:marketplace_id/data/:data_id removes the specified data from its data marketplace, given data and marketplace IDs

Marketplace Discovery

Description

• MARKETS_SUPPORTED
• Returns a list of the markets this backend system serves and their deployed contract addresses
• No auth token needed since public information

Endpoints

• GET /marketplaces returns a list of supported data marketplaces

Let's discuss the details of the REST API in the comments. As we gain clarity, I'll edit this top-level listing of the REST API to stay current.