gicentre / litvis Goto Github PK

With the introduction of Vega-loader-arrow, it is now possible to import binary data in apache arrow format. To do this, it requires including two extra packages, both available via npm:

https://cdn.jsdelivr.net/npm/apache-arrow
https://cdn.jsdelivr.net/npm/vega-loader-arrow

The elm-vegalite (and shortly, the elm-vega) APIs now have an arrow format function, so it would be good if litvis can embed the two packages to allow this to work.

Here is a vega-lite example for testing:

arrowExample : Spec
arrowExample =
    let
        pollData =
            dataFromUrl "https://gicentre.github.io/data/euPolls.arrow" [ arrow ]

        enc =
            encoding
                << position X [ pName "Answer", pMType Nominal ]
                << position Y [ pName "Percent", pMType Quantitative, pAggregate opMean ]
                << color [ mName "Pollster", mMType Nominal ]
                << column [ fName "Pollster", fMType Nominal ]
    in
    toVegaLite [ pollData, enc [], bar [] ]

Since the update to litvis 0.18.0, the default behaviour of rendered visualizations is now not to show tooltips. Previously, simply making a code block interactive showed a tooltip for all fields that were rendered with some kind of mark. Tooltips are rendered if they are set explicitly. For example:

```elm {v interactive}
barChart : Spec
barChart =
    let
        data =
            dataFromColumns []
                << dataColumn "a" (strs [ "A", "B", "C", "D", "E", "F", "G" ])
                << dataColumn "b" (nums [ 28, 55, 43, 91, 81, 53, 19 ])

        enc =
            encoding
                << position X [ pName "a", pMType Ordinal ]
                << position Y [ pName "b", pMType Quantitative ]
                << tooltips
                    [ [ tName "a", tMType Ordinal ]
                    , [ tName "b", tMType Quantitative ]
                    ]
    in
    toVegaLite [ data [], bar [], enc [] ]
```

The question is whether a specification without explicit tooltips should show them in interactive mode.

Arguments against the new default and attempting to revert to pre- 0.18.0 behaviour:

• This is technically a breaking change as default behaviour is now different for a spec in interactive mode without explicit tooltip setting.
• Having tooltips now requires more code in the spec than it did previously

Arguments for the new behaviour:

• Consistent with default behaviour of Vega-lite javascript and the official Vega-lite editor (no tooltips by default)
• Easier to enable other interaction without tooltips (previously to do this we had to set an empty tooltips list).
• Specification is semantically more correct (no tooltips unless we specify them).

I think on balance I prefer the new no-tooltips-by-default behaviour but would be interested in others' views. Perhaps one solution might be the ability to switch between the two behaviours, perhaps with a tooltips / t option in the code block header.

It would be helpful to be able to define a label in a narrative schema that inserts a checkbox in the formatted document, and a rule that requires the checkbox to be ticked for the litvis to 'pass' linting.

Issue #7 would allow this to be achieved by providing an appropriate regex, but given this might be quite a common task, perhaps it would be better to define it as simple label / rule attributes and for litvis to insert the relevant markdown and rule validation logic.

For example, something like this (see includeCheck, checkMustPass):

labels:
  - name: invariance
    includeCheck: true
    paired:
      htmlTemplate: 'Confirm that non-meaningful changes in data representation, such as table row order have no discernible effect on visualization.{{children}}</div>'

rules:
  - description: Kindlmann and Scheidegger's principle of data representation invariance.
    selector:
      label: invariance
    checkMustPass: true

There is an open question as to whether such a check should be attached to the 'question' or 'response' label (e.g. invariance or invarianceAssessment in albebra.yaml) and whether an explicit message is required to be given if the box is not checked (currently this would be the description field in the rule).

Markdown-preview-enhanced is currently broken with Atom 1.47.0 as is markdown-preview-enhanced-with-litvis.

Issue reported here,

shd101wyy/vscode-markdown-preview-enhanced#1119

Hopefully resolved when the fix us published in Atom, but we should check in case we need to make any local adjustments.

In the meantime, the workaround is to use the previous working version of Atom, which you can download from here: https://github.com/atom/atom/releases/tag/v1.46.0

I don't know if this is a consequence of the latest version of Atom as I have not noticed it before and I thought this was one of our test cases, but trying to display an empty list as markdown now crashes the preview pane in Atom.

Minimal working example:

```elm {m}
test : List String
test =
    []
```

The fix may arise naturally when addressing #15, but we should certainly include empty list output in our tests if it is not present.

It would appear that caching of previously compiled code blocks is not working correctly in Atom.

The following tested with Atom 1.5.0, VSCode 1.48.2 and markdown-preview-enhanced-with-litivs 0.28.0 (all the latest at time of writing). The problem was also present in the previous version of markdown-preview-enhanced-with-litivs.

I've added a caching test to demonstrate the problem. After clearing the cache, first previewing of the test file takes around 10 seconds to evaluate. Editing the markdown after this should re-render output in the preview window without re-evaluating the expensive function and so should be effectively instant. However, in Atom it appears to re-evaluate, taking another 10 seconds. Even pressing the 'refresh' icon in the preview window appears to force a re-evaluation.

The problem does not occur in VSCode where the document is relying on a cached version of the evaluated function as expected.

The following code is handled correctly in litvis:

selectProblem : Spec
selectProblem =
    let
        data =
            dataFromColumns []
                << dataColumn "category" (strs [ "a", "b" ])
                << dataColumn "value" (nums [ 1, 2 ])

        sel =
            selection
                << select "mySelection"
                    seSingle
                    [ seFields [ "category" ]
                    , seBind [ iSelect "category" [ inOptions [ "a", "b" ] ] ]
                    ]
    in
    toVegaLite [ sel [], circle [] ]

but if iSelect "category" is replaced with iSelect "" (which is valid, but not very useful), litvis generates the following parsing error in atom:

Could not parse "{ $schema = "ht..." (litvis:expression-value)

Related to this, if select "mySelection" is replaced with select "" or a string with some internal whitespace (e.g. select "my selection"), we get a similar parsing warning. In this case, the JSON created is not a valid Vega-Lite spec, so it should be returning the error message generated by the Vega-Lite runtime (InvalidCharacterError: The string contains invalid characters.)

Currently (as of [email protected]), narrative schemas can use rules that

• require the presence, or a number of instances, of a label (minimumOccurrences)
• require minimum content length within a paired label (minimumTrimmedTextLength)
• require that one label follows another, allowing label orders to be defined (followedBy)

What we are not able to do yet is validate against specific content within paired labels. One approach would be to add the ability to validate against a regex. This would give us some flexibility in validating content.

One open question is whether we simply allow a boolean regexMatch (as below), or whether we can process the returned matched values in some way within the YAML rule definition.

Some Examples

Vis Algebra

We might require that for a vis algebra schema, content must include reference to hallucinators, confusers, jumblers and misleaders under the relevant 'principle' assessment:

 - description: invariance assessment must make reference to hallucinators.
    selector:
      label: invarianceAssessment
    children:
      regexMatch: [hH]allucinator

 - description: unambiguity assessment must make reference to confusers.
    selector:
      label: unambiguityAssessment
    children:
      regexMatch: [cC]onfuser

 - description: data-visualization correspondence assessment must make reference to jumblers and misleaders.
    selector:
      label: correspondenceAssessment
    children:
      regexMatch: (?=.*[jJ]umbler)(?=[\s\S]*[mM]isleader).+

Unit Testing

Rather than just add a checkbox to some content that requires a subjective judgement as to whether it has passed, we might use regex to validate that the user has confirmed the test has taken place and has passed:

 - description: must include a '[ ] passed?' checkbox.
    selector:
      label: myTestableLabel
    children:
      regexMatch: \[.\] [pP]assed\?

 - description: test has not passed.
    selector:
      label: myTestableLabel
    children:
      regexMatch: \[x\] [pP]assed\?

These are only suggestions for the syntax. We might need to escape special characters depending on how tolerant YAML is.

The link to the scatter plots documents is missing an s both in the text and the link. incomeScatterplot.md => incomeScatterplots.md

The following code block generates the parsing error SyntaxError: Unexpected token < in JSON at position 588:

```elm {v interactive}
parserError : Spec
parserError =
    let
        data =
            dataFromUrl "https://gicentre.github.io/data/westMidlands/westMidsCrimesAggregated.tsv"

        enc =
            encoding
                << position X [ pName "month", pMType Temporal ]
                << position Y [ pName "reportedCrimes", pMType Quantitative ]
                << color [ mName "crimeType", mMType Nominal ]

        sel =
            selection
                << select "maxThreshold"
                    seSingle
                    [ seFields [ "reportedCrimes" ]
                    , seInit [ ( "reportedCrimes", num 14000 ) ]
                    , seBind [ iRange "reportedCrimes" [ inName "Max", inMax 14000 ] ]
                    ]

        trans =
            transform
                << filter (fiExpr "datum.reportedCrimes <= maxThreshold_reportedCrimes")
    in
    toVegaLite [ width 540, data [], trans [], sel [], enc [], circle [] ]
```

The error is specific to interactive mode and appears to be caused by a failure to parse <=. in the filter expression line. A single < does not generate the error, nor does > or >=. Presumably the expression parser is confusing the token with an HTML opening tag.

If there has been a problem installing Elm as part of Litvis, trying to preview a litvis document that contains some elm generates this error message:

spawn elm ENOENT (litvis:elmp-environment)

We should replace this with a more friendly message, especially as this is most likely to occur when using Litvis for the first time. I suggest

I am having trouble finding Elm on your machine. Is it installed? Check by opening a terminal window and typing "elm --version" (without quotation marks). If you have recently installed Elm, try restarting your machine.

See also related issue #14

Just so we have a record of it, currently, when setting a code block to raw / r, tuples are not displayed.

Minimal working example:

```elm {r}
tupleTest : ( Int, Int )
tupleTest =
    ( 99, 50 )
```

Should display (99,50) but nothing is displayed, nor is any warning generated.

The same problem occurs with a list of tuples:

```elm {r}
tupleListTest : List ( Int, Int )
tupleListTest =
    [ ( 1, 2 ), ( 3, 4 ) ]
```

Interestingly, if a tuple or list of tuples is embedded inside a record, it is displayed correctly, which may help to diagnose the cause of the problem:

```elm {r}
type alias MyRecord =
    { intItem : Int
    , strItem : String
    , booItem : Bool
    , lstItem : List Int
    , tplItem : ( Int, Int )
    , tp3Item : ( Bool, Int, String )
    , tLsItem : List ( Int, Int )
    , rcdItem : { nestedName : String, nestedValue : Int }
    }


recordTest : MyRecord
recordTest =
    MyRecord 1 "two" True [ 3, 4, 5 ] ( 6, 7 ) ( False, 8, "nine" ) [ ( 10, 11 ), ( 12, 13 ) ] { nestedName = "fourteen", nestedValue = 15 }
```

Generated the following (correct) raw output:

{ booItem = True, intItem = 1, lstItem = [3,4,5], rcdItem = { nestedName = "fourteen", nestedValue = 15 }, strItem = "two", tLsItem = [(10,11),(12,13)], tp3Item = (False,8,"nine"), tplItem = (6,7) }[(1,2),(3,4)]

Thanks to @nickysinclair for bringing this to my attention:

In VSCode, but not Atom, the following Litvis document will not preview the image when stored locally. Examples here use triple-hat notation, but also applies to full code blocks:

^^^elm m=(3+4)^^^
![Does image show?](myImage.png)

If output is changed to raw rather than markdown, there is no problem:

^^^elm r=(3+4)^^^
![Does image show?](myImage.png)

If the image is referenced via a URL to an external CORS-compliant sever, there is no problem:

^^^elm m=(3+4)^^^
![Does image show?](https://gicentre.github.io/data/images/LillyTarn.jpg)

The fact it is VSCode specific, and only to local files suggests it might be related to the cause of #49 although I don't know why it only applies to markdown output.

The following spec generates a Litvis parsing error Could not parse "{ $schema = "ht..." (litvis:expression-value) because the field name a-name contains a dash.

barChart : Spec
barChart =
    let
        data =
            dataFromColumns []
                << dataColumn "a-name" (strs [ "A", "B", "C" ])
                << dataColumn "b" (nums [ 28, 55, 43 ])

        enc =
            encoding
                << position X [ pName "a-name", pMType Ordinal ]
                << position Y [ pName "b", pMType Quantitative ]
    in
    toVegaLite [ data [], bar [], enc [] ]

This is a litvis error rather than a Vega-Lite one because the equivalent JSON spec (if it could be generated by litvis) is correctly handled by the VL runtime.

Suppose we have a function defined in a document:

parent.md

```elm{l=hidden}
myFunction : String
myFunction =
    "Hello from parent"
```

And if a child document that follows it makes a call to the parent via a triple hat expression:

child.md

---
follows: parent
---

^^^elm {r=myFunction}^^^

This results in the error I cannot find a 'myFunction' variable

However if some other function is declared in the child document, the function in the parent is called correctly:

---
follows: parent
---

^^^elm {r=myFunction}^^^

```elm {l=hidden}
anotherFunction : Int
anotherFunction =
    7
```

It would appear that there must be at lest one declared function within a document before a triple hat expression can reference a function in parent (or grandparent etc.) document. This will prevent 'clean' documents from being created that have no code but reference functions from other documents in the hierarchy.

Some errors passed to the Atom linter window appear 'escaped'. To reproduce:

• In an empty folder, create the following litvis document in Atom (noting that the src folder should not exist):

---
elm:
  source-directories:
    - src
---

```elm {l r}
value : String
value =
    "Hello"
```

This generates the following text in the linter window:

Error	Litvis	{"type":"error","path":"elm.json","title":"BAD JSON","message":["The \"source-directories\" in your elm.json lists the following directory:\n\n ",{"bold":false,"underline":false,"color":"yellow","string":"/Users/jwo/common/research/litvis/sandbox/scratch/src"},"\n\nI cannot find that directory though! Is it missing? Is there a typo?"]} (literate-elm:compile)	1:1
Info	Litvis	‘elm.source-directories[0]:’ src is not an existing directory and is therefore ignored. (litvis:elm-source-directories)

The info output would probably be better simply as src is not an existing directory and is therefore ignored.

But more significantly, the error output is shown as a literal JSON with escaped content rather than formatted output. We should probably extract the message content and remove the escape symbols.

See also related issue #13

If an Elm record is declared in a function with a type alias for the record in the type annotation, it is displayed as expected:

```elm {r}
type alias MySimpleRecord =
    { name : String }


recordTest1 : MySimpleRecord
recordTest1 =
    { name = "hello" }

(generates correct output of { name = "hello" } )

But if an anonymous declaration (i.e. not using a named type alias) is used in the type annotation, it is not displayed:

```elm {r}
recordTest2 : { name : String }
recordTest2 =
    { name = "hello" }
```

(it should be noted this is a perfectly legal construct in Elm)

If it is not straightforward to detect record types in type annotations for raw output, we should probably at least generate a warning.

If a code block in a siding is placed between another code block and its dependency, it appears to prevent the dependent block from compiling.

This example generates the error I cannot find a 'getValueFromBelow' variable:

```elm {l}
fn1 : Int
fn1 =
    getValueFromBelow
```

```elm {l siding}
fn2 : Int
fn2 =
    101
```

```elm {l}
getValueFromBelow : Int
getValueFromBelow =
    101
```

If the fn2 block is not a siding or if it is moved to above or below the other two code blocks, there is no error.

Since v.0.14.0, it has not been possible to load local files with dataFromUrl. Previously, placing a file in the same folder as the markdown document and specifying the filename without a path or file: prefix was compatible with browser security restrictions. Now, it does not appear to be possible to load them. The following example generates a valid spec, but the data file is not loaded (assuming there is a local copy of population.json in the same folder):

localFileTest : Spec
localFileTest =
    let
        data =
            --dataFromUrl "https://vega.github.io/vega-lite/data/population.json"
            dataFromUrl "population.json"

        enc =
            encoding
                << position X [ pName "age", pMType Ordinal ]
                << position Y [ pName "people", pMType Quantitative, pAggregate opSum ]
    in
    toVegaLite [ data [], enc [], bar [] ]

Replacing with a CORS hosted external URL works, and a temporary workaround is to make the code block header {interactive}.

Possibly the cause of the problem is the shift to Vega 5.x and a new fetch mechanism used be vega-loader.

I have found a 'false-positive' identification of elm code when elm-like text is embedded in a multiline string:

falsePositive : String
falsePositive =
    """Some multiline text that looks a bit like some elm code
thisIsNotAFunction : Int
thisIsNotAFunction =
    99
"""

It happens because text inside multiline quotes can (correctly) start a line.

While this is an unlikely scenario, it may help in refining the regexp used to identify type annotations.

Currently litvis visualization output can be in one of two modes.

The default mode, suitable for static views, generates a full SVG/Canvas spec for rendering in a web view. This allows static web pages and PDFs to be generated from within Atom or VS Code.

The interactive mode, necessary for enabling Vega and Vega-Lite interaction, must be specified with v interactive. This has the additional advantage that the (usually much smaller) Vega/Vega-Lite specification is sent to the runtime and the view is generated in the browser. For specifications with large datasets, this can be much faster, but has the disadvantage that web pages and PDFs cannot be generated directly.

On balance I think the need for interaction and reduced output file size, is probably more common and important than the need to generate PDFs, so I would like to propose we make 'interactive' mode the default, no longer requiring an explicit setting. We could still allow non-interactive (and therefore PDF/SVG/Canvas output) but via an explicit setting in the litvis document. I suggest we call this non-interactive mode archive mode. For backwards compatibility, we should deprecate but continue to allow (now redundant) v interactive but also v interactive=false (would be equivalent to v archive).

The question arises, at what level should we allow specification of archive mode. If possible I would like us to be able to specify this both at the code block level (equivalent to the current way we specify interactive mode, but also at the document level, in the page header. More local settings should override more global settings.

• Code block level: elm {v archive} or ^^^elm v=mySpec archive=true^^^
• Document level:

---
archive: true
---

Setting archive to false in any of these ways should allow interactive mode to be set (which would be redundant unless the mode has to archive mode in a document header.

Ideally, when one document follows another, we would be able to redefine archive in the header to override the setting in the parent, but if this is not possible, and only a single setting the root document is possible, this would be workable. Block level settings should always override document level settings.

For the record:

As indicated in #33, Atom was failing to use prettier-plugin-elm, in cases where npm was not installing packages in the default location. A common use-case is not wishing, or not being able, to use admin privileges and therefore setting the npm global location to be somewhere in user account space (e.g. via nvm or set prefix).

The result is that Atom defaults to its bundled version of prettier rather than the custom elm version and as consequence, elm-format would not be applied to elm code blocks.

The cause of the the problem can be traced back to prettier-atom using global-prefix to detect npm global package locations.

Described and fixed in prettier/prettier-atom#516

Interested in the possibilities of narrative schemas with litvis but found insuficient documentation to get started. Can you link to additional resources?

It can be helpful in a narrative to highlight arbitrary lines or blocks of code. It would be a useful feature in litvis if we could do this.

Such a feature should aim to

• allow a single line or continuous lines of code to be highlighted within a fenced code block.
• encode the highlight extent within the document itself (so not like GitHub highlighting for example)
• keep the syntax is simple as possible and readable within raw markdown
• be independent of the number of lines preceding the fenced code block
• allow external (re)styling of the code block via the normal css/less mechanism.

The best place to locate the highlight specification is probably within the curly braces of fence definition. Perhaps something like

```elm {l highlight=[10-14]}

@kachkaev Do you think this is practical?

If you create a document containing the following two functions

```elm {l}
myFunction : String
myFunction =
    let
        repeatedName =
            "Hello"
    in
    repeatedName
```

```elm {r}
repeatedName : String
repeatedName =
    "world"
```

the linter correctly reports the error The name 'repeatedName' is first defined here: repeatedName = But then it is defined AGAIN over here: repeatedName = , as Elm 0.19 does not allow shadowing of functions.

However, if the code block defining repeatedName sits instead in a separate document that follows the one that defines myFunction, the error persists (as we would expect), but no error message is reported (which is a litvis bug).

This is likely to be problematic for beginners or anyone following a document they have not authored themselves, as it is quite a hard error to spot without a linter message.

The problem is that a downstream modification can render an upstream local function in a separate document invalid. This can appear counterintuitive as local functions occupy a global namespace so making it doubly hard to spot without linter help.

We can reference data sources in three ways:

1. Inline directly encoded with dataFromColumns etc
2. Referencing a local file with dataFromUrl, e.g. dataFromUrl "myFile.csv"
3. Referencing an (usually remote) http/https file with dataFromUrl, e.g. dataFromUrl "https://vega.github.io/vega-lite/data/movies.json"

All three work in static visualization mode (elm {v})

However, in VS Code, when showing blocks in interactive mode (elm {v interactive}), option 2 is now failing. Presumably because of security, cross origin restrictions. But I note that it works without problems in Atom. Not sure what settings in VS Code (presumably the default) is preventing this from working.

Code block headers expressed as elm{l} and elm {l} (note the space in the second example) both work. In Atom syntax highlighting is applied to the block of code correctly, but in VSCode with the elm-tooling plugin (the one we recommend for litvis), only the second case with a space generates correct syntax colouring.

I assume the problem is with the elm-tooling plugin for VSCode, but could apply a Prettier type auto formatting to ensure a single space between the elm and { ?

Is there a way to view litvis files without downloading them? Something akin nbviewer but for litvis files?

If not could there be? Either something like nbviewer's web view for jupyter notebooks or a chrome extension which would allow you to view litvis examples as rendered litvis files in context.

Elm-format and Prettier will ensure correct indentation of markdown content and elm code within code blocks. However, it will not always automatically identify or correct poor indentation in yaml frontmatter. This can be a problem if the indentation prevents elm dependencies from being correctly identified and generates a slightly misleading error message.

For example, the frontmatter below has no indentation for dependencies: so the gicentre/elm-vegalite dependency is not registered.

---
id: litvis

elm:
dependencies:
  gicentre/elm-vegalite: latest
---

This results in Elm's error message of

The Programfdb02db87dec4c92c1adf27ecb5bc6246a917fd5 module has a bad import: import VegaLite I cannot find that module! Is there a typo in the module name? The "source-directories" field of your elm.json tells me to only look in the . directory, but it is not there. Maybe it is in a package that is not installed yet? (literate-elm:compile)

While the message is technically the correct one, exposing the cache hash is confusing and it is a little misleading for beginners, encouraging them to mess with elm.json. It would be great if we could check for poorly indented frontmatter (that escapes Prettier's auto-indenting) and either correct, or generate a more friendly error message that encourages user to check indentation of the yaml.

I have noted this problem with many beginners who copy / paste a litvis document into Atom and indentation gets altered/corrupted. It can happen if the first --- is pasted in any position other than the first column, resulting in all yaml being formatted flush left.

It looks like the changes in Atom from 1.37 -> 1.38 have broken our markdown-preview-enhanced-with-litvis plugin.

We need to realign our fork with markdown-preview-enhanced which does work in Atom 1.38.0.

Users of litvis sometimes need to flush the literate-elm cache if previously compiled code blocks have become corrupted or if dependencies have been updated. Currently to do this they have to remove the folder inside the hidden .mume folder: ~/.mume/literate-elm.

If they have hidden folders hidden in the file view of their operating system this is a little tricky for beginners. The instructions for doing so will depend on operating system, where the home directory is located, and whether hidden files are shown or not.

It would be more beginner-friendly if a flush the litvis cache option was available directly from within Atom and VS Code (probably via keyboard shortcut and editor menu option).

Possibly related to #26, data parsing of some CSV files is failing since v.0.14.0. The problem seems to be (at least) related to parsing of dates. The example below fails to parse the data from the CSV file. If encoding is limited to AvHireTime and NumberOfHires (both simple positive integer values), there is no problem. But Month is not read correctly. If the same data are loaded from a .json file there is no problem.

csvMonthParsing : Spec
csvMonthParsing =
    let
        data =
            --dataFromUrl "https://gicentre.github.io/data/bicycleHiresLondon.json"
            dataFromUrl "https://gicentre.github.io/data/bicycleHiresLondon.csv"

        enc =
            encoding
                << position X [ pName "Month", pMType Temporal ]
                --<< position X [ pName "AvHireTime", pMType Quantitative ]
                << position Y [ pName "NumberOfHires", pMType Quantitative ]
    in
    toVegaLite [ data [], enc [], circle [] ]

As with #26, a workaround is to make the code block interactive, which should at least help to isolate the cause of the problem.

For the record, we still need to implement markdown output for a wider range of types.

Currently some types can generate parsing warnings. For example

```elm {m}
myFunction : List ( Int, Int )
myFunction =
    [ ( 1, 2 ) ]
```

generates the warning : Could not parse "[(1,2)]" (litvis:expression-value) highlighting the entire code block.

We should also test against those examples listed in #13 and #14.

If this isn't fixable in the short term, we should probably generate a more beginner friendly and correct warning (the problem isn't with the contents of the function, but the output type of the code block) such as Markdown output not supported for this type. Could you use 'raw' (or 'r') output instead?

Currently narrative schemas can only be linked a 'root' document. If a document follows another, it cannot directly link to a schema in its own frontmatter.

A consequence of this is that any schema requirements apply to all documents in a linked or branching narrative, which limits the usefulness of the schema for guiding different branches of a narrative.

A common use case which we cannot currently support is to have a root document containing some literate data shaping and a following document containing a narrative about its visualization structured by a schema.

We need a mechanism for scoping schemas on a per-document basis. Ideally we would allow schemas to apply only in the document in which the schema is referenced in its frontmatter, and that we would allow schemas to be referenced in non-root documents as well as the root.

If this is not practical to implement, we could keep the current root-only schema referencing but allow a per-document ignoreSchema schemaName option to prevent schema warnings from appearing in documents in which we don't want them to be applied.

After releasing 0.12.0, I'm noticing this error every time something is being visualised with vega:

SyntaxError: Invalid regular expression: /[ªµºÀ-ÖØ-öø-Ë�ˆ-Ë‘Ë -ˤˬˮͰ-ʹͶͷͺ-ͽͿΆΈ-ΊΌΎ-ΡΣ-ϵϷ-Ò�ÒŠ-Ô¯Ô±-Õ–Õ™Õ¡-Ö‡×�-תװ-×²Ø -يٮٯٱ-Û“Û•Û¥Û¦Û®Û¯Ûº-Û¼Û¿Ü�Ü’-ܯÝ�-ޥޱߊ-ßªß´ßµßºà €-à •à šà ¤à ¨à¡€-ࡘࢠ-ࢲऄ-हऽà¥�क़-ॡॱ-ঀঅ-ঌà¦�à¦�ও-নপ-রলশ-হঽৎড়à§�য়-ৡৰৱਅ-ਊà¨�à¨�ਓ-ਨਪ-ਰਲਲ਼ਵਸ਼ਸਹਖ਼-ੜਫ਼ੲ-ੴઅ-àª�àª�-ઑઓ-નપ-રલળવ-હઽà«�ૠૡଅ-ଌà¬�à¬�ଓ-ନପ-ରଲଳଵ-ହଽଡ଼à­�à­Ÿ-ୡୱஃஅ-ஊஎ-à®�à®’-கஙசஜஞடணதந-பம-ஹà¯�à°…-ఌఎ-à°�à°’-నప-హఽౘౙౠౡಅ-ಌಎ-à²�ಒ-ನಪ-ಳವ-ಹಽೞೠೡೱೲഅ-ഌഎ-à´�à´’-ഺഽൎൠൡൺ-ൿඅ-ඖක-නඳ-රලව-à·†à¸�-ะาำเ-ๆàº�ຂຄງຈຊàº�ດ-ທນ-ຟມ-ຣລວສຫອ-ະາຳຽເ-ໄໆໜ-ໟༀཀ-ཇཉ-ཬྈ-ྌက-ဪဿá��-á�•á�š-á��á�¡á�¥á�¦á�®-á�°á�µ-á‚�á‚Žá‚ -ჅჇáƒ�áƒ�-ჺჼ-ቈቊ-á‰�á‰�-ቖቘቚ-á‰�በ-ኈኊ-áŠ�áŠ�-ኰኲ-ኵኸ-ኾዀዂ-ዅወ-ዖዘ-áŒ�ጒ-ጕጘ-á�šáŽ€-áŽ�Ꭰ-á�´á��-ᙬᙯ-ᙿáš�-áššáš -ᛪᛮ-ᛸᜀ-ᜌᜎ-ᜑᜠ-ᜱá�€-á�‘á� -á�¬á�®-á�°áž€-áž³áŸ—áŸœá  -ᡷᢀ-ᢨᢪᢰ-ᣵᤀ-ᤞá¥�-ᥭᥰ-ᥴᦀ-ᦫá§�-ᧇᨀ-ᨖᨠ-ᩔᪧᬅ-ᬳᭅ-ᭋᮃ-ᮠᮮᮯᮺ-ᯥᰀ-á°£á±�-á±�ᱚ-ᱽᳩ-ᳬᳮ-ᳱᳵᳶᴀ-ᶿḀ-ἕἘ-á¼�á¼ -ὅὈ-á½�á½�-ὗὙὛá½�Ὗ-ώᾀ-ᾴᾶ-ᾼιῂ-ῄῆ-á¿Œá¿�-á¿“á¿–-Ίῠ-Ῥῲ-ῴῶ-ῼâ�±â�¿â‚�-ₜℂℇℊ-â„“â„•â„™-â„�ℤΩℨK-ℭℯ-ℹℼ-â„¿â……-â…‰â…Žâ… -ↈⰀ-Ⱞⰰ-ⱞⱠ-ⳤⳫ-ⳮⳲⳳⴀ-ⴥⴧⴭⴰ-ⵧⵯⶀ-ⶖⶠ-ⶦⶨ-ⶮⶰ-ⶶⶸ-ⶾⷀ-ⷆⷈ-â·Žâ·�-â·–â·˜-ⷞⸯ々-〇〡-〩〱-〵〸-〼ã��-ã‚–ã‚�-ã‚Ÿã‚¡-ヺー-ヿㄅ-ㄭㄱ-ㆎㆠ-ㆺㇰ-ㇿã�€-䶵一-鿌ꀀ-ê’Œê“�-ꓽꔀ-ꘌê˜�-ꘟꘪꘫꙀ-ꙮꙿ-êš�êš -ꛯꜗ-ꜟꜢ-ꞈꞋ-ꞎêž�-ꞭꞰꞱꟷ-ê �ê ƒ-ê …ê ‡-ê Šê Œ-ê ¢ê¡€-ꡳꢂ-ꢳꣲ-ꣷꣻꤊ-ꤥꤰ-ê¥†ê¥ -ꥼꦄ-ꦲê§�ê§ -ꧤꧦ-ꧯꧺ-ꧾꨀ-ꨨꩀ-ê©‚ê©„-ê©‹ê© -ꩶꩺꩾ-ꪯꪱꪵꪶꪹ-ꪽꫀꫂꫛ-ê«�ê« -ꫪꫲ-ê«´ê¬�-ꬆꬉ-ꬎꬑ-ê¬–ê¬ -ꬦꬨ-ꬮꬰ-ê­šê­œ-ꭟꭤꭥꯀ-ꯢ가-힣ힰ-ퟆퟋ-ퟻ豈-ï©­ï©°-龎ff-stﬓ-ﬗï¬�ײַ-ﬨשׁ-זּטּ-לּמּנּï­�ףּפּצּ-ﮱﯓ-ï´½ïµ�-ï¶�ﶒ-ﷇﷰ-ﷻﹰ-ﹴﹶ-ﻼＡ-Ｚï½�-ｚｦ-ﾾￂ-ￇￊ-ï¿�ï¿’-ï¿—ï¿š-ï¿œ]/: Range out of order in character class

Will try investigating tomorrow morning.

Markdown linting rule MD030, which determines the number of spaces after a list marker, suggests 1 space by default. Validation of markdown on our GitHub setup suggests 2 and so generates listing warnings with a default formatted document.

MD030/list-marker-space Spaces after list markers [Expected: 2; Actual: 1]

Is there any reason why we shouldn't use the default of 1?

The website field for this repository is currently empty; it would be helpful for it to be set to https://www.gicentre.net/litvis, as that page has useful links to the paper, slides and videos that can't otherwise be reached by following links from this repository.