Giter Club home page Giter Club logo

outline's Introduction

outline

I spend a lot of time sketching out software packages. These sketches are often programming-language agnostic and written in lots of weird places (mainly my own notes, github issues, documentation, and source code). A while back @alandonovan posted what I felt is a clear, concise definition of a package: google/starlark-go#19 (comment). Since reading that package outline I've started to think and write in something like that format.

While working on a standard library for starlark I ran into the issue of needing a way to embed documentation about a package that's written in go, but targets another language (starlark). I figure with a little rigor it'd be easiest to formalize my preferred outlining format in a way that it can be embedded in a comment, riding with the source code itself. Using the "template" commands we can generate documentation markdown for our website

Project Status

use-at-your-own-risk alpha. I'm working on this with Qri's rfc and starlib projects as concrete use-cases to drive development.

Example

go install github.com/b5/outline

make a file: outline.txt:

  outline: geo
    geo defines geographic operations

    functions:
      point(lat,lng)
        Point constructor takes an x(longitude) and y(latitude) value and returns a Point object
        params
          lat float
          lng float
      within(geomA,geomB)
        Returns True if geometry A is entirely contained by geometry B
        params:
          geomA [point,line,polygon]
            maybe-inner geometry
          geomB [point,line,polygon]
            maybe-outer geometery
      intersects(geomA,geomB)
        Similar to within but part of geometry B can lie outside of geometry A and it will still return True

    types:
      point
        methods:
          buffer(x int)
            Generates a buffered region of x units around a point
          distance(p2 point)
            Euclidian Distance
          distanceGeodesic(p2 point)
            Distance on the surface of a sphere with the same radius as Earth
          KNN()
            Given a target point T and an array of other points, return the K nearest points to T
          greatCircle(p2 point)
            Returns the great circle line segment to point 2
      line
        methods:
          buffer()
          length()
          geodesicLength()
      polygon

Currently the only thing you can do out-of-the box with outline is parse documents & template them:

outline template ./outline.txt

This will by default spit out a markdown version of your outline. dope. The real upside of this format is it's designed to survive in weird places. try running the same command against this readme file:

git clone [email protected]:b5/outline.git
cd outline
outline template ./readme.md

And you'll get the same result. Lovely! You can supply custom templates with the template flag. The markdown template is here.

Maybe someday...

  • outline fmt <- "golint" style formatter
  • outline . <- validate any found outline documents in a given filepath
  • outline require . <- a command that requires at least one outline document present in the given filepath, useful for integration with CI
  • outline starter --language python . <- generate starter stub code for a given package based on templates

outline's People

Contributors

b5 avatar dustmop avatar mcuadros avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar

Forkers

mcuadros dustmop

outline's Issues

Errors when no `outline:` present in document

If no outline: present in document:

in template:
Parses the doc, but never finds DocumentTok. Continues to read until we get to EofTok, which returns. It returns no error and an empty Doc
We attempt to execute the template using the data in Doc. It's empty so it prints this error: # template: mdIndex:17:5: executing "mdIndex" at <.Name>: can't evaluate field Name in type *lib.Doc

in fmt:
doc is nil, and when we use MarshalIndent, it panics

potential solutions:

  1. if no DocumentTok is found, in other words, if parser.read() never finds tok.Type DocumentTok, error saying the docs are ill-formatted and link to the readme.
    or
  2. Check for doc == nil before attempting to execute the template or marshal the data

extra white space in code blocks is stripped 

            code:
              load("dataframe.star", "dataframe")
              # create a new DataFrame
              df = dataframe.DataFrame([["cat", "meow"],
                                        ["dog", "bark"],
                                        ["eel", "zap"]],
                                       columns=["name", "sound"])

The parser removes all of the whitespace from the left-hand side here. This is especially bad for python, which has semantically meaningful whitespace. Conditionals won't work at all:

  if a > b:
    print('ok')

Instead, I feel like the parser should be preserving all whitespace on the left-hand side, and can tell when the block is done by finding a line with the same indentation amount as the code: label.

need support for examples

Would be good to have the ability to include examples in the generated docs, and have them appear nicely rendered (copyable, syntax-highlighted, etc).

segfault if type is missing 

  outline: dataframe
    dataframe is a 2d columnar data structure that provides analysis and manipulation tools
    path: dataframe
    functions:
      DataFrame(data, index, columns, dtype) DataFrame
        constructs a DataFrame
        params:
          data any
            data for the content of the DataFrame
          index
            index index for the rows of the DataFrame

index not being followed by a type causes a segfault. The stack trace:

goroutine 1 [running]:
github.com/b5/outline/lib.(*parser).readParam(0xc00009fba8, 0x5, 0x0, 0x0, 0x1)
	/Users/me/code/github.com/b5/outline/lib/parser.go:231 +0x269
github.com/b5/outline/lib.(*parser).readParams(0xc00009fba8, 0x4, 0x8, 0x0, 0x0, 0xc0001921b0, 0x6)
	/Users/me/code/github.com/b5/outline/lib/parser.go:213 +0x5d
github.com/b5/outline/lib.(*parser).readFunction(0xc00009fba8, 0xc0001920a1, 0x9, 0x3, 0x9, 0xc0001920f1, 0xa)
	/Users/me/code/github.com/b5/outline/lib/parser.go:191 +0x327
github.com/b5/outline/lib.(*parser).readFunctions(0xc00009fba8, 0xc0001920a1, 0x9, 0x2, 0x0, 0xc000192120, 0x9, 0x0, 0x0)
	/Users/me/code/github.com/b5/outline/lib/parser.go:160 +0x77
github.com/b5/outline/lib.(*parser).readDocument(0xc00009fba8, 0x1, 0x2, 0x0, 0x0)
	/Users/me/code/github.com/b5/outline/lib/parser.go:136 +0x24f
github.com/b5/outline/lib.(*parser).read(0xc00009fba8, 0xc00019e000, 0x1000, 0xc00019c000)
	/Users/me/code/github.com/b5/outline/lib/parser.go:98 +0x75
github.com/b5/outline/lib.Parse(0x128eea0, 0xc00018e080, 0x0, 0x0, 0xc00018e080, 0x0, 0x0)
	/Users/me/code/github.com/b5/outline/lib/parser.go:29 +0xf6
github.com/b5/outline/cmd.glob..func4(0x13bdf00, 0xc00007ef30, 0x1, 0x3)
	/Users/me/code/github.com/b5/outline/cmd/template.go:107 +0x252
github.com/spf13/cobra.(*Command).execute(0x13bdf00, 0xc00007eed0, 0x3, 0x3, 0x13bdf00, 0xc00007eed0)
	/Users/me/go/qri/pkg/mod/github.com/spf13/[email protected]/command.go:844 +0x2c2
github.com/spf13/cobra.(*Command).ExecuteC(0x13bd9c0, 0xc000000180, 0x200000003, 0xc000000180)
	/Users/me/go/qri/pkg/mod/github.com/spf13/[email protected]/command.go:945 +0x336
github.com/spf13/cobra.(*Command).Execute(...)
	/Users/me/go/qri/pkg/mod/github.com/spf13/[email protected]/command.go:885
github.com/b5/outline/cmd.Execute()
	/Users/me/code/github.com/b5/outline/cmd/root.go:27 +0x31
main.main()
	/Users/me/code/github.com/b5/outline/main.go:8 +0x25

types without any methods will generate broken markdown 

  outline: dataframe
    dataframe description
    path: dataframe

    functions:
      DataFrame(data) DataFrame
        constructs a DataFrame containing the given data
        params:
          data any
            data for the content of the DataFrame. Can be a list, dict, Series, or another DataFrame

    types:
      DataFrame
        a dataframe
        methods:
          d()
            d method

      GroupByResult
        result of a group_by call

      AtIndexer
        result of using the at attribute

Here both GroupByResult and AtIndexer have no methods. The generated markdown for the "Types" section looks like this:

## Types

### 'AtIndexer'

result of using the at attribute### 'DataFrame'

a dataframe### 'GroupByResult'

result of a group_by call

types are sorted alphabetically, rather than the order they appear 

    types:
      DataFrame
        a dataframe
        methods:
          a()
            a method

      GroupByResult
        result of a group_by call
        methods:
          b()
            b method

      AtIndexer
        result of using the at attribute
        methods:
          c()
            c method

These types are mentioned in the order that makes sense to show them. The markdown output, however, orders them AtIndexer then DataFrame then GroupByResult.

methods on a type should become h3 instead of h3

Take the example:

  outline: xlsx
    xlsx implements excel file readers in starlark. currently a highly-experimental package
    that will definitely change at some point in the future
    path: xlsx
    functions:
      get_url(url string)
        fetch an excel file from a url
    types:
      File
        an excel file
        methods:
          get_sheets() dict
            return a dict of sheets in this excel file
          get_rows(sheetname) list
            get all populated rows / columns as a list-of-list strings

This yields the markdown:

## Functions



### get_url

\```
get_url(url string)
\```

fetch an excel file from a url



## Types

### File

an excel file

**Methods**

### get_sheets

\`\`\`
get_sheets() dict
\`\`\`

return a dict of sheets in this excel file


### get_rows

\`\`\`
get_rows(sheetname) list
\`\`\`

get all populated rows / columns as a list-of-list strings

In the qri website, this is rendered in the table of contents as:

image

File is an h3, but two methods of File are also h3. These should be h4 so they appear subordinate to File

We might also consider making methods an h4 so it will appear in the table of contents, and having the actual methods be h5, so the user can clearly see the hierarchy.

fields can't include examples

Trying to use examples for a field results in a parsing error:

unexpexted token: examples: examples 6 3

But even if it parsed, fields are displayed in a terse table with no room for examples. This is usually fine when fields are simple data values. However, for dataframes, I'm using fields to represent attributes which are actually pretty complex and require a bit of explanation. The proper fix is probably implementing a category for attributes, rather than trying and failing to use fields instead.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.