... which raises the challenge that datasette p

Here's the full help for Cloud Run at the moment: <div class="highlight highlight-

Found more when I <a href="https://github.com/search?q=datasette+publish+extra-options

Maybe I should add --cors and <code class="notranslat

`datasette publish` needs support for the new config/metadata split about datasette HOT 9 OPEN

simonw commented on June 20, 2024

`datasette publish` needs support for the new config/metadata split

from datasette.

Comments (9)

simonw commented on June 20, 2024

As soon as datasette publish cloudrun has this I can re-enable this bit of the demo deploy:

datasette/.github/workflows/deploy-latest.yml

Lines 91 to 97 in 2da1a6a

  - name: Make some modifications to metadata.json 

 run: | 

  cat fixtures.json | \ 

  jq '.databases |= . + {"ephemeral": {"allow": {"id": "*"}}}' | \ 

  jq '.plugins |= . + {"datasette-ephemeral-tables": {"table_ttl": 900}}' \ 

  > metadata.json 

  cat metadata.json

Which should fix this broken demo from https://simonwillison.net/2022/Dec/2/datasette-write-api/

https://todomvc.datasette.io/

from datasette.

simonw commented on June 20, 2024

The @add_common_publish_arguments_and_options decorator described here is bad. If I update it to support a new config option all plugins that use it will break.

datasette/docs/plugin_hooks.rst

Lines 347 to 355 in f130c7c

  @hookimpl 

  def publish_subcommand(publish): 

  @publish.command() 

  @add_common_publish_arguments_and_options 

  @click.option( 

  "-k", 

  "--api_key", 

  help="API key for talking to my hosting provider", 

  )

I want to deprecate it and switch to a different, better design to address the same problem.

from datasette.

simonw commented on June 20, 2024

I think the actual design of this is pretty simple. Current help starts like this:

Usage: datasette publish cloudrun [OPTIONS] [FILES]...

  Publish databases to Datasette running on Cloud Run

Options:
  -m, --metadata FILENAME         Path to JSON/YAML file containing metadata
                                  to publish
  --extra-options TEXT            Extra options to pass to datasette serve

The -s and -c short options are not being used.

So I think -c/--config can point to a JSON or YAML datasette.yaml file, and -s/--setting key value can mirror the new -s/--setting option in datasette serve itself (a shortcut for populating the config file directly from the CLI).

Here's the relevant help section from datasette serve:

  -m, --metadata FILENAME         Path to JSON/YAML file containing
                                  license/source metadata
  -c, --config FILENAME           Path to JSON/YAML Datasette configuration
                                  file
  -s, --setting SETTING...        nested.key, value setting to use in
                                  Datasette configuration

from datasette.

simonw commented on June 20, 2024

Here's the full help for Cloud Run at the moment:

datasette publish cloudrun --help

Usage: datasette publish cloudrun [OPTIONS] [FILES]...

  Publish databases to Datasette running on Cloud Run

Options:
  -m, --metadata FILENAME         Path to JSON/YAML file containing metadata
                                  to publish
  --extra-options TEXT            Extra options to pass to datasette serve
  --branch TEXT                   Install datasette from a GitHub branch e.g.
                                  main
  --template-dir DIRECTORY        Path to directory containing custom
                                  templates
  --plugins-dir DIRECTORY         Path to directory containing custom plugins
  --static MOUNT:DIRECTORY        Serve static files from this directory at
                                  /MOUNT/...
  --install TEXT                  Additional packages (e.g. plugins) to
                                  install
  --plugin-secret <TEXT TEXT TEXT>...
                                  Secrets to pass to plugins, e.g. --plugin-
                                  secret datasette-auth-github client_id xxx
  --version-note TEXT             Additional note to show on /-/versions
  --secret TEXT                   Secret used for signing secure values, such
                                  as signed cookies
  --title TEXT                    Title for metadata
  --license TEXT                  License label for metadata
  --license_url TEXT              License URL for metadata
  --source TEXT                   Source label for metadata
  --source_url TEXT               Source URL for metadata
  --about TEXT                    About label for metadata
  --about_url TEXT                About URL for metadata
  -n, --name TEXT                 Application name to use when building
  --service TEXT                  Cloud Run service to deploy (or over-write)
  --spatialite                    Enable SpatialLite extension
  --show-files                    Output the generated Dockerfile and
                                  metadata.json
  --memory TEXT                   Memory to allocate in Cloud Run, e.g. 1Gi
  --cpu [1|2|4]                   Number of vCPUs to allocate in Cloud Run
  --timeout INTEGER               Build timeout in seconds
  --apt-get-install TEXT          Additional packages to apt-get install
  --max-instances INTEGER         Maximum Cloud Run instances
  --min-instances INTEGER         Minimum Cloud Run instances
  --help                          Show this message and exit.

from datasette.

simonw commented on June 20, 2024

I'd really like to remove --extra-options. I think the new design makes that completely obsolete?

Maybe it doesn't. You still need --extra-options for the --crossdb option for example.

from datasette.

simonw commented on June 20, 2024

https://github.com/search?q=datasette+publish+extra-options+language%3AShell&type=code&l=Shell shows 17 matches, I'll copy in illustrative examples here:

--extra-options="--setting sql_time_limit_ms 5000"
--extra-options="--config default_cache_ttl:3600 --config hash_urls:1"
--extra-options "--setting sql_time_limit_ms 3500 --setting default_page_size 20 --setting trace_debug 1"
--extra-options="--config default_page_size:50 --config sql_time_limit_ms:30000 --config facet_time_limit_ms:10000"
--extra-options="--setting sql_time_limit_ms 5000"
--extra-options "--setting suggest_facets off --setting allow_download on --setting truncate_cells_html 0 --setting max_csv_mb 0 --setting sql_time_limit_ms 2000"

from datasette.

simonw commented on June 20, 2024

Found more when I searched for YAML.

Here's the most interesting: https://github.com/labordata/warehouse/blob/0029a72fc1ceae9091932da6566f891167179012/.github/workflows/build.yml#L59

--extra-options="--crossdb --setting sql_time_limit_ms 100000 --cors --setting facet_time_limit_ms 500 --setting allow_facet off --setting trace_debug 1"

Uses both --cors and --crossdb.

from datasette.

simonw commented on June 20, 2024

Maybe I should add --cors and --crossdb to datasette publish cloudrun as well?

from datasette.

simonw commented on June 20, 2024

Worth noting that it already sets --cors automatically without you needing to specify it:

datasette/datasette/utils/__init__.py

Line 374 in d97e82d

cmd.extend(["--cors", "--inspect-file", "inspect-data.json"])

I wonder if that's actually surprising behaviour that we should change before 1.0.

from datasette.

`datasette publish` needs support for the new config/metadata split about datasette HOT 9 OPEN

Comments (9)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

	- name: Make some modifications to metadata.json
	run: \|
	cat fixtures.json \| \
	jq '.databases \|= . + {"ephemeral": {"allow": {"id": "*"}}}' \| \
	jq '.plugins \|= . + {"datasette-ephemeral-tables": {"table_ttl": 900}}' \
	> metadata.json
	cat metadata.json

	@hookimpl
	def publish_subcommand(publish):
	@publish.command()
	@add_common_publish_arguments_and_options
	@click.option(
	"-k",
	"--api_key",
	help="API key for talking to my hosting provider",
	)