Comments (7)
Some options for where this could go:
- Directly in the release notes? I'm not sure about that, those are getting pretty long already. I think the release notes should link to relevant upgrade guides.
- On a new page? We could have a "upgrade instructions" page in the documentation.
- At the bottom of the new https://docs.datasette.io/en/latest/configuration.html page
I'm leaning towards the third option at the moment.
But... we may also need to provide upgrade instructions for plugin authors. Those could live in a separate area of the documentation though, since issues affecting end-users who configure Datasette and issues affecting plugin authors are unlikely to overlap much.
from datasette.
Related idea: how about a datasette-upgrade
plugin which adds a datasette upgrade
command that can be used to automate this process?
Maybe something like this:
datasette install datasette-upgrade
datasette upgrade metadata-to-config metadata.json
This would output two new files: metadata.yaml
and datasette.yaml
. If files with those names existed already in the current directory they would be called metadata-new.yaml
and datasette-new.yaml
.
The command would tell you what it did:
Your metadata.json file has been rewritten as two files:
metadata-new.yaml
datasette.yaml
Start Datasette like this to try them out:
datasette -m metadata-new.yaml -c datasette.yaml
The command is datasette upgrade metadata-to-config
because metadata-to-config
is the name of the upgrade recipe. The first version of the plugin would only have that single recipe, but we could add more recipes in the future for other upgrades.
from datasette.
I dig it - I was thinking an Observable notebook where you paste your metadata.json
/metadata.yaml
and it would generate the new metadata + datasette.yaml files, but an extensible datasette upgrade
plugin would be nice for future plugins.
One thing to think about: If someone has comments in their original metadata.yaml
, could we preserve them in the new files? tbh maybe not too important bc if people cared that much they could just copy + paste, and it might be too distracting
from datasette.
I think I'm OK with not preserving comments, just because it adds a level of complexity to the tool which I don't think is worth the value it provides.
If people want to keep their comments I'm happy to leave them to copy those over by hand.
from datasette.
Started playing with this plugin idea, now tearing myself away to work on something more important:
from datasette import hookimpl
import click
import pathlib
@hookimpl
def register_commands(cli):
@cli.group()
def upgrade():
"""
Apply configuration upgrades to an existing Datasette instance
"""
pass
@upgrade.command()
@click.argument(
"metadata", type=click.Path(exists=True)
)
@click.option(
"new_metadata", "-m", "--new-metadata", help="Path to new metadata.yaml file", type=click.Path(exists=False)
)
@click.option(
"new_datasette", "-c", "--new-datasette", help="Path to new datasette.yaml file", type=click.Path(exists=False)
)
@click.option(
"output_dir", "-e", "--output-dir", help="Directory to write new files to", type=click.Path(), default="."
)
def metadata_to_config(metadata, new_metadata, new_datasette, output_dir):
"""
Upgrade an existing metadata.json/yaml file to the new metadata.yaml and
datasette.yaml split introduced prior to Datasette 1.0.
"""
print("Upgrading {} to new metadata.yaml format".format(metadata))
output_dir = pathlib.Path(output_dir)
if not new_metadata:
# Pick a filename for the new metadata.yaml file that does not yet exist
new_metadata = pick_filename("metadata", output_dir)
if not new_datasette:
new_datasette = pick_filename("datasette", output_dir)
print("New metadata.yaml file will be written to {}".format(new_metadata))
print("New datasette.yaml file will be written to {}".format(new_datasette))
def pick_filename(base, output_dir):
options = ["{}.yaml".format(base), "{}-new.yaml".format(base)]
i = 0
while True:
option = options.pop(0)
option_path = output_dir / option
if not option_path.exists():
return option_path
# If we ran out
if not options:
i += 1
options = ["{}-new-{}.yaml".format(base, i)]
from datasette.
Demo of that logic:
$ datasette upgrade metadata-to-config ../datasette/metadata.json
Upgrading ../datasette/metadata.json to new metadata.yaml format
New metadata.yaml file will be written to metadata-new-1.yaml
New datasette.yaml file will be written to datasette.yaml
$ touch metadata-new-1.yaml
$ datasette upgrade metadata-to-config ../datasette/metadata.json
Upgrading ../datasette/metadata.json to new metadata.yaml format
New metadata.yaml file will be written to metadata-new-2.yaml
New datasette.yaml file will be written to datasette.yaml
$ touch datasette.yaml
$ datasette upgrade metadata-to-config ../datasette/metadata.json
Upgrading ../datasette/metadata.json to new metadata.yaml format
New metadata.yaml file will be written to metadata-new-2.yaml
New datasette.yaml file will be written to datasette-new.yaml
from datasette.
Pushed that incomplete code here: https://github.com/datasette/datasette-upgrade
from datasette.
Related Issues (20)
- 'Datasette' object has no attribute 'table_metadata' HOT 6
- Database download missing content-length header and is sent chunked
- Run spell check against all of the documentation HOT 5
- Async example for track_event plugin hook HOT 1
- Test fail against Python 3.13-dev with a Pint error HOT 4
- 500 Error editing row with binary column using datasette-write-ui
- Canned queries don't appear in the app
- `json1` SQLite extension is not correctly detected in `versions.json`
- `Stable` documentation refers to `latest` demo instance
- get_metadata() plugin hook should support async await_me_maybe pattern HOT 2
- Datasette 1.0 rc tracking issue
- Usablity issue with need for root user
- Consider releasing a 0.65 with some forwards compatibility for 1.0 HOT 2
- Bug (in docs?): the "_internal" table on latest.datasette.io doesn't load HOT 1
- Consider adding a new plugin hook: "pre_query" or similar HOT 3
- Proposal - store metadata inside `internal.db` tables HOT 2
- Broken link in documention: fivethirtyeight.datasettes.com
- Fix font size on filter inputs
- base_url getting appended twice in redirects when applying filters?
- Accessibility: add a `lang` attribute to `html` HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from datasette.