nordzilla / cargo-doc-gen Goto Github PK

Answer the following questions:

• How can we invoke rustc_ast from our crate?
• Do the AST nodes come bundled with their respective source code locations? (Used for injecting template info)
• Can we extract full module paths from referenced decls (modules, structs, etc)? Important for linking docs.

This will inject the generated text back into the source code files.

We should be able to handle custom attributes that will override the config for a particular documentable.

For example, if the config specifies that all public structs should have descriptions for each member, but you would like to disable that for one struct in particular, you could write something like:

#[docgen::exclude_member_docs]

A complete list of supported attributes, as well as the syntax is still TBD.

Depends on #7

We should look into parallelizing the generation of ASTs, similar to the way ripgrep walks the directory structure in parallel.

Example:

functions:
    description:      [ "all" ]
    describe_args:    [ "pub", "pub(crate)", "pub(super)" ]
    describe_return:  [ "pub", "pub(crate)", "pub(super)" ]
    describe_panics:  [ "all" ]
    examples_section: [ "pub", "pub(crate)" ]

structs:
    describe_fields: [ "all" ]

constants:
    ...

traits:
    ...

Create a help section with example configs and explanations that the CLI can point to when people need help setting up their config.

This can probably just live here on GitHub in a README.

From the function's representation in the AST, we should be able to determine if this function panics directly.

This is opposed to transitively panicking (see #9), in which a function either panics directly, or if any of the functions it calls may panic.

A tree-like structure whose nodes consist of:

• Absolute module path
• Source code location
• Access specifier

Small (incomplete) example of node types:

Nodes/
└── Module
    ├── Constants
    ├── Function
    ├── Struct
    │   ├── Fields
    │   └── Functions
    └── Trait
        └── Functions

The "doc tree" type will be consumed by some generator API.

We need to have a way to handle command-line arguments when people invoke cargo doc-gen.

We should probably use clap to implement this.

Initial Features:

• cargo doc-gen Runs the main application
• cargo doc-gen --help Displays the help menu
• cargo doc-gen -c|--check Validates the existing documentation
• cargo doc-gen -t|--threads{num-threads} Sets the number of threads that should be used

A roadmap of functionality to support before an initial release.

Misc.

• #18 Implement a CLI
• [#19] Check for config file
• [#3] Support deserializing config
  • [#20] Create config examples/help section

Generate an AST

• [#7] Generate AST(s) from relevant files in a rust project.
  • [#21] Parallelize AST generation

Generate a DocTree

• [#4] Create DocTree(s) from Rust AST(s)
  • [#26] Handle custom attributes
  • [#25] Determine if a function locally panics.
  • [#9] Determine if a function may transitively panic
  • [#23] Parallelize DocTree creation

Generate DocComments

• [#5] Generate DocComments
  • [#22] Merge DocComments

Insert DocComments into files

• [#17] Insert DocComments into files
  • [#24] Parallelize DocComment insertion

Depends on #4

We should look into parallelizing the insertion of DocComments into files.
I think it makes most sense to let each thread in the pool handle one file at a time.

Super rough outline:

Broad strokes:

Launch stage

Config module :: Config.{toml,yaml,etc} -> Config struct

Parsing stage

DocTree module :: rustc_ast -> DocTree

Generation stage

Generator module :: DocTree -> "Insertion Records"

Source code stage

Insertion Layer :: InsertionRecords -> Checks source code for conformity / Ensures doc comment exists
Insertion Layer :: InsertionRecords -> Reconcile (merge/complete) doc comments

Depends on #4

We should look into parallelizing the generation of DocTree(s) from the Rust AST(s).

There is enough data here that I think multiple threads would be a benefit.

Depends on #5

In order to handle updating existing documentation (rather than generating new documentation from scratch) we will need the capability for DocComments to generate the merged representation of two comments together.

Here are some tentative ideas about how two merged comments should behave:

Different sections

Different sections should be put together (deterministic order TBD). If one comment with an Examples section is merged with another comment that has a Safety section, the final representation should have both sections.

/// # Examples
/// example

combined with

/// #Safety
/// safety comment

produces

/// # Examples
/// example
///
/// # Safety
/// safety comment

Same sections, same content

If two DocComments have the same exact sections and content, the section should appear only once.

/// #Safety
/// safety comment

combined with

/// #Safety
/// safety comment

produces

/// # Safety
/// safety comment

Same sections, different content

If two DocComments have the same sections, but different content, the section should appear once with a diff of the content.

e.g. new documentation template for a function that now has one more argument than before

/// # Arguments
/// arg1: DocGen(TODO)
/// arg2: DocGen(TODO)

combined with

e.g. pre-existing documentation for a function that used to have only one argument.

/// # Arguments
/// arg1: Description of arg1

produces

/// # Arguments
<<<<<<< DocGen(OLD)
/// arg1: Description of arg1
=======
/// arg1: DocGen(TODO)
/// arg2: DocGen(TODO)
>>>>>>> DocGen(NEW)

Comments with these diffs will be flagged by doc-gen --check as needing manual resolution by the author. It is useful to show the previous state of the comment so that the author can reuse the content in the final resolution, which might look like this.

/// # Arguments
/// arg1: Description of arg1
/// arg2: Description of arg2

When doc-gen runs, the first thing it should do is check for the config file.

I imagine this would be called something like doc-gen.toml

If no such file exists, the program should output an error message and link to a help page on GitHub about how to set up your doc-gen config.

Blocked-by: #4