Raise events when parsing the document to open the door for parsing extensions or actions.

Similar to how an XML parser raises events when parsing an XML structure, raise events when parsing the AsciiDoc structure. Examples of events might be:

• before parsing
• after parsing
• begin/end section
• begin/end block
• parsing error
• include file
• etc.

This would open the door for extensions in an entirely new way. Here are some examples of what you might use an observer for:

• index the document
• build a catalog of links, images or references
• skip a block
• terminate rendering (or prevent recursion into a structure)
• validate text or code listings
• ...and the list goes on

As a reference, see the events that Nokogiri raises when parsing an XML structure.

We may also want to raise events during rendering. If necessary, we can create a separate issue to track that request.

Since AsciiDoc has a camel-case name, I think it's appropriate for Asciidoctor to follow suit. Plus, it draws out the "Doctor" part and, in a since, Asciidoctor is both a doctor for AsciiDoc and a remedy for the common "I hate documentation" problem :)

Just a thought.

Although we don't have native pdf generation yet in Asciidoctor, it would still be nice to offer this functionality through the asciidoctor cli and/or API.

For now, it will require having the AsciiDoc distribution available to access the XSL stylesheets and such (or we could pull them from the web and cache them locally).

If we need files from the AsciiDoc distribution, we could require the ASCIIDOC_HOME environment variable to point to the install location so that we can run a2x directly or the resources it uses.

Here's the commands I use to generate PDF from the DocBook output that Asciidoctor generates (for an article doctype)

a2x -k -fpdf -darticle --fop output.xml

Under the covers, here are the commands that a2x executes when the doctype is article (the paths here are obviously invocation specific):

xsltproc --stringparam callout.graphics 0 --stringparam navig.graphics 0 \
--stringparam admon.textlabel 1 --stringparam admon.graphics 0  \
--output "$HOME/projects/asciidoc-samples/output.fo" \
"$HOME/opt/asciidoc-8.6.8/docbook-xsl/fo.xsl" \
"$HOME/projects/asciidoc-samples/output.xml"

and

fop -fo "$HOME/projects/asciidoc-samples/output.fo" \
-pdf "$HOME/projects/asciidoc-samples/output.pdf"

We could use Nokogiri to replace the xsltproc call. There isn't yet a replacement for the fop invocation that I know of, but we could wrap it.

I tend to prefer unit tests to look for particular structures rather than comparing input vs output as it more clearly indicates intent.

However, we should still have end-to-end (functional) tests to detect regressions that slip between the cracks...or to alert us to where we need more test coverage. I would call this A/B testing because we are establishing a baseline based on the current state of things, not necessarily a perfectly rendered document (at least until we are there). It will also be dependent on whitespace.

powerman's cheatsheets (cloned by jakoch) provide a nice repository of inputs for this type of test.

https://github.com/jakoch/asciidoc-cheatsheet

Import these into the fixtures directory, generate a baseline and add those as well. Then setup a test that validates against the rendered output.

To demonstrate compliance with AsciiDoc, we've been testing the rendered HTML output against the default AsciiDoc stylesheet. However, I'll be the first to admit that the first time I saw that theme, I dismissed AsciiDoc for being stuck in the 90s.

Let's change that perception.

Asciidoctor should provide a professional, modern stylesheet for HTML output out of the box...something we can call our own.

We can draw inspiration from readthedocs.org, which recently took a fresh approach to a stylesheet for HTML-based documentation. We could also consider the look and feel of GitHub, Bootstrap, Foundation 3 and others.

Ideally, the stylesheet would coordinate with the Asciidoctor logo, issue #48.

Sections that are assigned the glossary and appendix styles get handled differently in the document structure. In particular, they are assigned special section numbers rather than continuing the main section numbering.

Add support for these special section blocks. It may be possible to reuse the existing section template for this purpose.

Note that the glossary style can also be applied to a dlist. For the HTML backend, this does nothing special. It does change the tags used in the dlist_template for the DocBook backend.

There are additional styles that affect the section tag used for DocBook but do not apply to HTML. Here's the complete list:

• abstract
• colophon
• preface
• appendix
• glossary
• bibliography
• index
• synopsis

Supporting these may just be a matter of mapping the style to the tag name 1-1 if the tag name is not otherwise recognized (or perhaps check a list using include?)

I've used rspec in other projects and I think it's BDD style expectations fits much better w/ a test suite that is focused on compliance.

Here's a nice example of an rspec test from awestruct

https://github.com/awestruct/awestruct/blob/master/spec/page_loader_spec.rb

Similar to that test, I think we can use input/output files to make assertions rather than all this crazy (and somewhat poke-a-stick-at-it) xpath.

We'll need to be able to report warnings (and other messages) to the user. Rather than printing these to stdout, we should pass them through a message facility which can queue them, print them with styling, etc.

I've setup the asciidoctor organization on GitHub, where I'm currently hosting the (very rough) project site. Can we move the main asciidoctor repository there as well?

In my experience, I've found organizations to be better suited for open source projects which have a development team, plus it encourages more developers to participate.

If we put the main repository under the asciidoctor organization, I can also move my asciidoctor-backends repository there as well.

In general, the ifeval macro has the potential to be vary dangerous. However, there is one purpose that is serves that is essential to support, attribute value comparison.

AsciiDoc provides the ifdef::[] macro to handle conditional inclusion of content based on the presence of attribute values, such as the backend, toc, icons, etc. However, knowing the attribute is defined often isn't enough. You need to know if it has, or doesn't have, a certain value.

Thus, we can overload the ifeval::[] macro to use it strictly for attribute value comparison.

Here's an example:

ifeval::["{source-highlighter}" == "coderay"]
...
endif::[]

Despite the name, we will not use Ruby eval to execute the code. Instead, we'll parse out the information we need and do the comparison in a controlled way.

lists: https://github.com/jakoch/asciidoc-cheatsheet/blob/master/asciidoc-866_lists.asciidoc
tables: https://github.com/jakoch/asciidoc-cheatsheet/blob/master/asciidoc-866_tables.asciidoc
you might consider using the asciidoc cheatsheet files as fixtures.

Detect the presence of docinfo files and incorporate them into the header of the document.

See http://asciidoc.org/userguide.html#X97

Enable the warning flag in the Rakefile so that we get immediate feedback on potention lint issues.

Add tests with bad input to make sure that Asciidoctor doesn't fall down when the user supplies a bad document. At worse, Asciidoctor should fail to render the document and spit out an error.

I'd like to avoid paranoid argument checking all over the place because I think that is going to unnecessarily slow down the engine. The way I see it, crap in, crap out. If the user has major syntax errors, then failing to render the document is the right thing to do.

Eventually we will have better diagnostics as to where the error occurred. In the short term, we just want to make sure that it dies gracefully and does not cause the program invoking Asciidoctor to crash.

Examples of tests that might be written:

• invalid attribute lists (unclosed, syntax errors)
• bad values for attributes that affect parsing and rendering
  ** for instance, when we assume we'll get an integer and we get something else instead
• invalid paths to images when data-uri is set
  ** also spaces in paths...make sure this doesn't cause us to resolve root level files
• a bogus doctype
• a bogus template directory (for Tilt)
• widths that are used for division (e.g., table parser)
• tables with too many or two few cells in a row
• unclosed delimited blocks

The key here is to be wise, but not paranoid. We want to give the user more leniency when using Asciidoctor in a local build than when running it in a server-side application. In those cases, we can put logic behind the 'safe' attribute (which can only be set through the constructor).

Technically, you don't need Tilt to use Asciidoctor. You have the option of sticking to the built-in templates. Therefore, to get us down to 0 dependencies, we should switch Tilt to being a development dependency.

I think the naming of the block templates is legacy. To be consistent with AsciiDoc, block-level elements (sidebar, example, listing, literal, etc) should be prefixed with block_ rather than section_. The only template that should be named section is the section template itself.

For instance, in the html5.conf file from AsciiDoc:

[sidebarblock]
<div class="sidebarblock{role? {role}}{unbreakable-option? unbreakable}"{id? id="{id}"}>
<div class="content">
<div class="title">{title}</div>
|
</div></div>

I do prefer putting the context first, as in block_sidebar.

I think this grammar spec from the AsciiDoc manual helps to clarify naming:

Document      ::= (Header?,Preamble?,Section*)
Header        ::= (Title,(AuthorInfo,RevisionInfo?)?)
AuthorInfo    ::= (FirstName,(MiddleName?,LastName)?,EmailAddress?)
RevisionInfo  ::= (RevisionNumber?,RevisionDate,RevisionRemark?)
Preamble      ::= (SectionBody)
Section       ::= (Title,SectionBody?,(Section)*)
SectionBody   ::= ((BlockTitle?,Block)|BlockMacro)+
Block         ::= (Paragraph|DelimitedBlock|List|Table)
List          ::= (BulletedList|NumberedList|LabeledList|CalloutList)
BulletedList  ::= (ListItem)+
NumberedList  ::= (ListItem)+
CalloutList   ::= (ListItem)+
LabeledList   ::= (ListEntry)+
ListEntry     ::= (ListLabel,ListItem)
ListLabel     ::= (ListTerm+)
ListItem      ::= (ItemText,(List|ListParagraph|ListContinuation)*)

If you make a sidebar like so:

.Sidebar title
****************
My content here.

My second paragraph.
****************

...it just spits the content directly into the sidebar container rather than processing it into <p> tags or what not. It should process all other kinds of blocks like normal (except other sidebars, which isn't allowed).

Although a deviation from AsciiDoc, I think we should include some semblance of a document title (h1 heading) in the embedded template by default. I think this is important because the current behavior of cropping the document title is not parallel with Markdown, and not what the user would expect.

Obviously, we want to leave off the document element (html) and document metadata element (head) because, after all, we are embedding the content. But I think the content of the header div is worthwhile to keep.

I am proposing including the following markup at the top of the embedded template:

<h1>Title</h1>
<span id="author">Dan Allen</span><br>

If the toc is enabled, then it would grow to:

<h1>Title</h1>
<span id="author">Dan Allen</span><br>
<div id="toc">
  <div id="toctitle">Table of Contents</div>
  <ol>
    ...
  </ol>
</div>

I'm not sure if we should wrap the header in the header div and the content in the content div. I'm kind of leaning towards the minimal approach.

Perhaps to be consistent w/ AsciiDoc, we should have an option for this behavior. Perhaps something like:

:embedded_content_header => true

(Of course, this can be overridden by creating a custom template for the embedded view).

I think it's time to update the description on the GitHub repository. I propose the following text:

Asciidoctor is an open source Ruby processor for converting AsciiDoc source files and strings into HTML 5, DocBook 4.5 and other formats.

wdyt?

Fenced code blocks are a popular syntax, appearing in GitHub-flavored Markdown and other Markdown variants. Although it would be a deviation from AsciiDoc, I think we should offer support for this syntax (and perhaps lobby to get it included in AsciiDoc).

The syntax is typically three backticks optionally followed by a language name.

```ruby
require 'asciidoctor'
puts Asciidoctor.render('Hello, World!', :header_footer => true)
```

The three backticks as a block delimiter would be a normal block extension in AsciiDoc. The trailing language name doesn't follow the AsciiDoc extension model, but we could still support it as a special exception.

Alternatively, we could require the language be set in AsciiDoc style using a block attribute.

[ruby]
```
require 'asciidoctor'
puts Asciidoctor.render('Hello, World!', :header_footer => true)
```

Three backticks does not conflict with any existing syntax in AsciiDoc.

It's a cinch to get Ruby gem projects running on Travis CI. Let's do it.

Currently, the parsing of inline content happens during rendering. This limits the information you have in the document after parsing. Instead, Asciidoctor should parse all text extents into inline nodes during parsing.

This requires moving the substitutions from the rendering phase to the parsing phase.

It also means that each line in the buffer will become an array of inline nodes that represent the chunked text, toggling between plain text and elements like links and images.

Since this has the potential to slow down a single pass parse/render, we may want to have a flag which controls the phase in which inline text is parsed.

This is a tracking issue for creating a Maven plugin that invokes Asciidoctor on source files in the documentation directory. This is an important feature for getting the Java community on board with Asciidoctor (and AsciiDoc in general).

According to the AsciiDoc documentation:

If the included file name is specified with a relative path then the
path is relative to the location of the referring document.

Currently, the include:: macro in Asciidoctor is looking for a file in the directory in which you launched Asciidoctor.

I raise this as an issue because we don't know the file for the data we are processing since Document accepts an array of lines.

I propose that we add an additional option for specifying the input file. It may be that this merely serves as a point of reference otherwise, but it's important for us to know the working directory.

Here's an example usage of the API I'm proposing:

doc = Asciidoctor::Document.new(lines, :input_file => input,
    :template_dir => "templates")

Then we can add the following logic around the Reader in document.rb.

# change to directory of input file, if specified,
# so includes can be relative to the document
if options.has_key?(:input_file) and File.exist? options[:input_file]
  oldpwd = Dir.pwd
  Dir.chdir File.dirname options[:input_file]
end

@reader = Reader.new(data, &block)

if !oldpwd.nil?
  Dir.chdir oldpwd
end

We may need this in other places, but this at least gets us off the ground. WDYT?

Currently, non-semantic blank lines are being removed in the output generated by the built-in templates. While care has been taken to only remove non-semantic endlines, there is still some risk involved in this feature. Once a configuration mechanism is in place, make this post-processing step an optional feature.

Atm, this feature is necessary to make the tests less sensitive to whitespace left behind by ERB.

Currently, we allow a block to be passed to the Document constructor to resolve include::[] macro references programmatically. This approach is too limiting because it's hard-wired to a single type of macro.

Instead, we should allow resolvers to be registered, which would get invoked for macros that use a path reference such as the include::[] macro. They could also be used for resolving paths for a source::[] macro, and other macros added through extensions.

This design is similar to what Sass offers for handling @import statements. See Importers for reference.

This also keeps the API simpler and cleaner.

Tracking issue for https://bugzilla.redhat.com/show_bug.cgi?id=889011

Create a man page for asciidoctor, specifically for the cli command, though it can cover use of the API as well.

Of course, the man page should be written in AsciiDoc (which supports creating man pages from plain text documents).

I used the version that exists in the hg repo from version 8.6.8:

`require 'asciidoctor'
lines = File.readlines('userguide.txt')
lines.size
doc = Asciidoctor::Document.new(lines)
html = doc.render``

Here's the output:

RuntimeError: Couldn't find a view in @views for section_oblock
from /home/jporter/.gem/ruby/1.9.1/gems/asciidoctor-0.0.7/lib/asciidoctor/renderer.rb:48:in `render'

In AsciiDoc, custom block names are the hook for plugins / filters. For example:

[ditaa]
----
fancy ascii diagram here
----

We should add support for these types of extensions using some sort of register API that sets up a callback into custom code. I'm imagining something like:

Asciidoctor.register(:ditaa, DitaaBlockFilter)

Drawing from Jekyll, Awestruct and Tilt, the plugin / filter API might be something like:

class DitaaBlockFilter
  # Public: Initialize a reusable instance of this block filter
  def initialize(options = {})
  end

  # Public: Parse the custom block and return an instance of Block
  #
  # reader     - A reader holding the lines of a delimited block
  # context    - The context of the delimited block (e.g., :literal, :listing, :open, etc)
  # parent     - The parent of the delimited block region
  # attributes - Block attributes the were captured in the lines above the block
  # 
  def parse(reader, context, parent, attributes)
    block = Block.new(:ditaa, parent)
    ...
    block
  end
end

We absolutely want to get away from the AsciiDoc way of using shell commands to handle filters. It's up to the filter author if they want to use a shell command inside their filter, but we should be using Ruby APIs to invoke these filters, absolutely.

For environments where includes are not needed, and are a security risk, make it possible to explicitly disable include:: entirely.

I'd like to introduce an additional safe model level, which I propose naming SERVER.

Some teams using Asciidoctor in a server-side application will want to support relative includes while others will not. Regardless, both will want to share the minimum level of security needed for a server-side deployment.

SERVER mode would be that shared base level. It would prevent the document from controlling how the document is rendered (backend, doctype, source-highlighter, etc) (essentially a lock-down mode). However, it would still allow for use of include::[] for paths within the base directory and embedding assets inside the document.

SECURE mode, which would still be the default, would additionally restrict the document from incorporating other files into the document (which means disabling include::[], data-uri and linkcss!).

By adding this extra safe mode option, I think we prevent crippling Asciidoctor for those wanting to use the include feature. (but it would not be the default, so we aren't changing current agreed upon behavior)

When the icons attribute is enabled, callout lists are not using numbered bullet icons (circles) like AsciiDoc. We should either add support for those icons, or do something with CSS that produces a similar look.

Callouts are a major feature of AsciiDoc over other lightweight markup languages, so we should give this feature high priority.

The following text should be converted to a link:

<http://github.com>

Currently, it is not being detected by the sub_macros operation.

Add a macro that can extract segments of code from source file and insert it into the document, passing it through the listing block (for source code highlighting and/or escaping).

The source macro should use the standard block macro syntax of AsciiDoc:

source::[language, filename, tag, ...]

There are three pieces of functionality for this macro:

• locating the source file, which may be within a source directory tree, a source archive or a remote resource (like a gist)
• isolate and extract the relevant section of code
• wrap the code in a listing block and insert it into the AST

The most straightforward way to extract code is to look for delimiters in the source file qualified with a tag. This is the approach that both Neo4j and codeextractor take in similar macros for AsciiDoc python. We could also support line number ranges, or other strategies.

For example, assume the source file has the following contents:

@RunWith(Arquillian.class)
pubic class SampleTest {
  // BEGIN: deployment
  @Deployment
  public static JavaArchive createDeployment() {
    return ShrinkWrap.create(JavaArchive.class)
      .addClass(Calculator.class);
  }
  // END: deployment
}

We want to extract the code between the BEGIN: deployment and END: deployment. Here's the macro we might use:

source::[java, org/arquillian/example/SampleTest.java, tag=deployment, indent=0]

The source directory would likely be set by a document attribute.

Of course, we'll try to trim that macro to allow the minimum information that is needed to track down the snippet. But that's the idea.

Create a cli for asciidoctor that works roughly the same as the asciidoc command.

At a minimum, I would expect it to support:

• input file (argument)
• output file (-o)
• attribute flags (-a)
• suppress header/footer (-s)
• template directory (-t)
• doctype (-d)
• help (-h)
• verbose mode (-v)

In the future we'll need backend and conf-file, but we don't have any implicit support for those atm, so no need to add them yet.

* There is currently a bin/asciidoctor file in the repository, but it isn't hooked up to any code

Every strong project needs a strong visual identity. Create a logo for Asciidoctor that will inspire contributors to write amazing, agile documentation.

Themes to play on are:

• Ascii (over XML) for doc writing
• Asciidoctor as a doctor for documentation malaise and XML suffering
• "AsciiDoc-in-R" where R is for Ruby and/or Rubygems.

Since Asciidoctor has proven to be so fast, "emergency room urgency" could be an inspiration as well.

If possible, please make available as SVG. License should be either MIT or CC-BY-SA.

Also, feel free to recommend a color palette for the project.

The constructor somewhere between ambiguous and limited since it can accept either a String or String array, but not a file.

I propose that we follow suit with YAML and other Ruby libraries and provide load methods on the Document class object.

Here's my proposal:

# Public: Load an AsciiDoc string as an Asciidoctor::Document
#
# Load the AsciiDoc document contained in +data+, which can either be a
# String or an Array of String lines. Returns the structure described in the
# document as a Ruby Asciidoctor:Document.
#
# data - The String or Array of String lines holding the AsciiDoc source document
# options - A Hash of options to control processing of the document
# block - A block that can be used to retrieve external AsciiDoc
#         data to include in this document (using the include macro)
def self.load(data, options = {})
  Document.new(data, options)
end

# Public: Load an AsciiDoc file as an Asciidoctor::Document
#
# Load the AsciiDoc document contained in +filename+. Returns the
# structure described in the document as a Ruby Asciidoctor:Document.
#
# filename - The filename holding the AsciiDoc source document
# options - A Hash of options to control processing of the document
# block - A block that can be used to retrieve external AsciiDoc
#         data to include in this document (using the include macro)
def self.load_file(filename, options = {}, &block)
  data = File.readlines(filename)
  launch_dir = Dir.getwd
  Dir.chdir(File.dirname(filename))
  doc = Document.new(data, options, &block)
  Dir.chdir(launch_dir)
  doc
end

This makes the experience for newcomers really nice:

puts Document.load_file('doc.asciidoc').render

To complement the load_file method, I suppose we could add render_to_file, render_file and render_file_to_output_file. For example:

puts Document.render_file('doc.asciidoc')

and

Document.render_file_to_output_file('doc.asciidoc', 'doc.html')

WDYT?

Notice in the load_file method, I am switching to the directory of the file. This allows Asciidoctor to be compliant w/ the behavior of AsciiDoc, which is to change to the directory of the file being processed (and similarly for include files). To handle this more generally, I'm considering adding an 'input_file' option that will allow you to specify the source even if you don't pass in the file directly.

Example:
https://github.com/jakoch/asciidoc-cheatsheet/blob/master/asciidoc-866_blocks.asciidoc

FYI, the previous rendering engine used by github/markup had the same issue, which was fixed a few months ago:
github/markup#108

When an attribute reference is resolved in the document, the lookup should not be case sensitive. Attribute keys are lowercased before they are stored in the hash. Therefore, the attribute reference should also be lowercased before looking up the value.

Example:

AsciiDoc: http://asciidoc.org

{AsciiDoc}[AsciiDoc]

Should produce a link to the AsciiDoc hompage.

For markup like this:

• foo
  +
  This is a paragraph.

The paragraph should be appended (as a block) to the list item for 'foo', so that it renders inside the

When processing a file as the Asciidoctor document, use that file's directory as the base directory for include:: macro.

When processing stdin or String or other IO input, treat include:: as unsafe and don't allow it.

Implement inline passthroughs by replacing content with temporary placeholders and then restoring that content once the substitutions are performed.

...or if you have a better idea :)

According the asciidoc man page, the NAME=VALUE@ syntax indicates that the attribute should not be overridden if defined in the document.

Parse the header information, which immediately follows the level 0 heading, and stuff the data into document attributes.

The information to focus on are as follows:

• first line: firstname[ [middlename ]lastname][ ]]
• line two: revision #, date, remark

The attributes that should be set (if found) are:

• author
• firstname
• middlename
• lastname
• authorinitials
• email
• revnumber
• revdate
• revremark

It might be nice to support multiple authors, but we can leave that for another issue. We'll also want to eventually have a docinfo include, but again, could be for another issue.

Icons really make the output look sharp, but they are a pain to manage. Asciidoctor should offer the option to "draw" the icons using Font Awesome, keeping the assets inside the page and avoiding the need for external resources.

I recommend the following mappings:

• tip - icon-pushpin (or icon-star or icon-plus)
• note - icon-info-sign
• warning - icon-warning-sign
• important - icon-exclamation-sign
• caution - icon-bolt (or icon-minus)

This feature would be something that perhaps we could make the default when using the built-in Asciidoctor stylesheet (#76).

According the AsciiDoc user guide, include1 macro differ from the include macro in that:

...files are not subject to tab expansion or system macro processing nor are attribute or lower priority substitutions performed. (Its) intended use is to include verbatim embedded CSS or scripts into configuration file headers.

Reading the lines for the section before parsing the section into blocks requires that we apply all the rules for parsing twice. The reason is that you have to apply the rules in order to know which lines to grab. The context (which block am I currently in?) affects how to interpret the line and it's relationship to adjacent lines.

This change should have no external impact on how the parser works...and will actually improve the accuracy of the parsing. While at first glace it may seem like reading the lines simplifies parsing, in reality it makes it more complex and fragile.

Granted, it will take a bit of effort to make this change, but we have plenty of tests in order to make sure the refactoring goes smoothly and we don't have any regressions.

The two methods to change, in particular, are next_section and next_list_item. This also applies to the processing done by the Reader (when it scans for attributes and include blocks), but I'd like to tackle that in a separate issue unless decided otherwise.

I'd like to request that we cut a 0.1.0 release so that we can enable integration and packaging to get underway.

Before cutting the release, I'd like to use this issue to discuss anything we think needs to be completed before we call it 0.1.0 (which shouldn't sound like too much of a risk).

I think we should be clear that the APIs are still subject to change. There are still some important refactoring that need to occur that might get squeezed if we are stuck with the current API.

The focus of this release is to get the code into the hands of alpha testers and early adopters so we can get some more feedback (and in turn write more tests!).

Discuss!

Tables have not yet been implemented in Asciidoctor. This issue will track the feature to add support for AsciiDoc tables.

For reference, see Tables in the AsciiDoc user guide for information.