tlienart / franklin.jl Goto Github PK

add possibility of an else statement. Suggested syntax would be

[[ if ... 
][ else ... 
]]

so the ][ would act as separator.

After extracting should probably do one last pass and flag commands that were not processed or whatever

• cannot use explicit \n
• cannot define a math environment in a newcommand so for example

\newcommand{\blah}{$x+y=1$}

would not be ok because you'd need to re-parse after the replacement to. (note that it could be done, but would require gymnastics)

• cannot use latex commands in escaped blocks such as code

In Markdown should allow something like

\html{
some html here that gets escaped then pugged back
}

would link to #13

currently triple quoted blocks are extracted but single ones should too.

Pages that would be visible locally but not pushed.

Need to think about how exactly they would be linked to. Maybe best is to just generate pages that don't get linked to as draft-....

currently in find_lxcoms! and also find_commands, the enumeration is on everything and we check if it's within the block, and do this multiple times. would be better to keep track.

1. test the standard syntax features from Base Markdown
2. document "extra features" such as div blocks and maths.

might be nice to use the proper tool for the job instead of the try loop. An example drawn from a Bukdu issue:

using FileWatching

dir = "."
fm = FolderMonitor(dir)

function reload_chrome()
   run(`osascript chrome.scpt`)
end

watching_files = ["test.css"]

loop = Task() do
    while fm.open
        (fname, events) = wait(fm)::Pair
        println("fname ", fname)
        fname in watching_files && reload_chrome()
    end
end
schedule(loop)

coauthor

also allows for a number of extensions to standard markdown. Some of them might be very useful to consider (and it has the regexes). It's all under MIT license so can easily mention & inspire.

While unclear how to do this with WebSocket etc, here are pieces to an alternative

installation of browser-sync

npm install -g browser-sync

in .bash_profile

export LOCAL_IP=`ipconfig getifaddr en0`
alias serve="browser-sync start -s -f . --no-notify --host $LOCAL_IP --port 9000"

background task in Julia

(the run syntax is likely only ok on unix)

try  
  tbs = @async run(`browser-sync start -s -f . --no-notify --logLevel silent`)
  # ... 
  Base.throwto(tbs, InterruptException())
catch x
  if isa(x, InterruptException)
    println("ok")
  end
end

or something of the form.

it should be doable to keep track of \label{...} and keep some form of count then restitute the whole thing when a \eqref or \ref (\\ref{([^{}]*)}) is seen.

In html processing, should use an anchor (see tutorial)

Actions

• implement something that processes \command{...} (see #5 also)
• have another dictionary (not JD_LOC_VARS because you don't need to keep track of types) to keep track of names->number pair.
• might want to keep macro variable such as section variables that are incremented every time a # or ## etc is seen. could also use that for section numbering (if required)

No need to browse those anymore. The line returns should be kept at least until after the blocks. then can be killed as well

The code uses a number of short circuits of the form

boolean && continue

this is potentially inconsequential. However for more complicated scenarios involving function calls, short circuits are probably best not used as CPUs may fail to optimise there (https://en.wikipedia.org/wiki/Short-circuit_evaluation).

Therefore their use seems justified either if

• the case is very simple (eg above)
• the secondary function is very slow (fast_check && slow_call)

and it should be documented as such.

Further, the use of

a = boolean ? v1 : v2

is probably best replaced by

a = ifelse(boolean, v1, v2)

again to help with CPU predictions. The difference being that in the first case v2 is only evaluated if boolean is false whereas in the second case both v1 and v2 are evaluated. So the ? construction should be used only if v2 is significantly slower than v1 and v1 should always be the fast operation.

Basic Image insertion should work out of the box with the usual markdown syntax.

In some future, it may be that we'd generate images directly from julia code (in SVG) for example from PGFTex or something and plug them in a-la weaver. Would be good to pre-test SVG and see how easily it handles that.

If an infrastructure HTML file is modified, all pages should be re-compiled.

Can use resolve_latex as well.

This is so that something like this works:

\newcommand{\ca}[1]{hello #1}
\newcommand{\cb}[1]{you must say \ca{#1}}
\cb{Idi Amin}

xb_i, xb_idx ... just use one standard

so now that we're working with block, we have to be a bit careful about the adding and the removing of <p> and </p>.

I'm not sure whether this is going to cause any bugs, but it may on more complex documents, should keep an eye on this and experiment.

this may be a bit tricky to do and would purely be for speed improvements.

Atm, when a command is seen, the definition is plugged in then the resulting plug is re-processed. If the same command is used mutliple times the same procedure is repeated.

This is a bit wasteful and could probably be optimised away by storing the information as we progress. No clear idea how to do this well and can probably be pushed back somewhat.

• comdefs --> lxdefs

So with pretty much every token, a full linear pass over the markdown is done. This means a lot of useless passes. While all of it still probably takes <10ms for a standard page, it feels like it's a slippery slope.

Building a more systematic parser is probably not trivial, using a generic parser is probably hard and overkill. Steps that could be done:

1. tree-based parsing to stop parsing bits where there's obviously nothing
2. doing a first pass to denote areas where there may be tokens of interest, using this for the tree.
3. using the matching token thing as ways to split the tree.

Probably good to first benchmark as it is with a test input file and then see whether another way is better.

Seems to work seamlessly with github...! every comment is a pullrequest that can be merged (= moderated)

https://staticman.net/

1. try to see how to set it up + whether it can be easily integrated
2. try to see how easily it would be for it to accept KaTeX.

JD_GLOB_VARS gets set explicitly after reading config.md and may even get set later on, so it's definitely not a const. It should be a global.

The JD_LOC_VARS is only used as a template that gets copied for every page.

Either way it seems that using const both times is a bit dodgy. May want to rethink this.

The target page variable is title which can be set in markdown via

@def title = "the title"

however if not given, there should be a default one extracted from the first # block.

should allow rudimentary variables in CSS (fill-type). For colours, ratios, etc

• would require tracking of modifications + processing

So there seems to be some exotic stuff going on when iterating through a string that may contain strange characters. See the use of length (https://docs.julialang.org/en/v0.6.1/manual/strings/) also maybe use prevind, nextind or something like that to iterate through the string rather than do index arithmetic.

This may not change anything for "non exotic" strings but in the case where someone would use an emojii or some symbol outside the standard pane, there could be problems...

Can use something like a generic minifyer to do the trick at the end.

Potential problem: the files should be replaced so that in the git-push to (for example) git pages, it's the minified version that is served.

Could also maybe directly apply it on every file that is compiled but that would make things slower. Could also minify stuff yourself removing whitespaces or whatever.

see also #5

should be able to deal with

\newcommand{...}[...]{...}

for basic replacement commands etc.

Instead of parsing @def vname expr, parse @def vname = expr

This kind of markdown:

* $f$ is a function that... 
* one can compute ...

leads to this kind of html

<ul>
<li></li>
</ul>
\(f\) is in the set \(\Gamma_0(X)\) of convex functions on \(X\) that are <em>proper</em> and <em>lower semi-continuous</em>,</p>
<ul>
<li><p>one can compute a gradient or subgradient of </p>
</li>
</ul>

The second one is fine but the first one is not. This may have to do with the order in which things are done and eventually md2html is called.

What it does:

• only parses a * then closes it as a standalone <ul></ul> with a single, empty, <li></li>
• then plug in the maths thing

we look for the first definition so if there's two it will ignore the second one. it'd be better to just fail when newcommands are found.

see whether it's still needed or not (via find_token etc + tests probably)

unclear what's failing but the reports are not analyzed. It does not look like it's an issue with the Travis CI. (though it only appeared after 0.7, maybe look at Diff*Eq.jl)

for example regex patterns should be called something like PAT_...

The lxdefs are already pointed to in lxcoms so in fact are not needed. Should be an easy fix after the substring changes.

The function is now too big and there are some parts that should be reasonably easy to cut out.

1. the continuous checking part
2. the markdown transformation part
3. going through files and populating a dictionary accordingly

also really need to write some doc or think in more details about issues (currently all NOTE and TODO in the code).

1. dealing with assets that are changing
2. dealing with infra files that are changing and must cause a re-compile
3. having some verbosity level indicating what's being seen and what's going on

if only go with match_curly_braces, could take braces that are not connected in case a command has too few args.

Can do this later and assume (for now) that commands are properly written.

A simple way could be to check that the {}{} are all contiguous, so opening braces must be adjacent to the previous closing brace. (Latex also doesn't allow spaces afaik)

using HTTP
HTTP.listen() do req::HTTP.Request
     req.target == "/" && return HTTP.Response(200, read("index.html"))
     file = HTTP.unescapeuri(req.target[2:end])
     return isfile(file) ? HTTP.Response(200, read(file)) : HTTP.Response(404)
end

does the job (needs a REPL).

• add a catch for InterruptException
• check if it does live updating of things?
• how to interweave with the judoc(single_pass=false) ?

move todo stuff to issues here,

move specific brainstorm stuff to relevant files.

• dates management, Dates.today() is no good, every time the file is recompiled (may be often). Or should just not re-interpret if there's no diff (then ok). The latter would require keeping track of the files + time of last modification in a hidden file that is gitignored.
• if empty, nothing should appear.
• dates should all have the same format (atm, it could be different).

Dates.format(now(), "U dd, yyyy")
Dates.format(Date(2018, 5, 15), "dd U yyyy")

See also formats.

• _html/
• _css/
• _libs/

should be kept somewhere, ideally an auxiliary repo (potentially with submodularity?). Ideally not kept in this one as it will clog the repo (and make it appear as a bloody js repo)

Instead of having one conv folder, use two folders\

1. one for everything markdown related
2. one for everything html related

basically should be related to the general workflow

1. markdown processing
2. (base markdown parsing)
3. html processing

For the sake of sticking with naming conventions, follow/adapt the rust-style guidelines

https://doc.rust-lang.org/1.0.0/style/style/naming/README.html

at the moments, re-creating allnewblocks several time which is obviously not very clean.

It doesn't really matter because it's small but really that stuff should be done with a LinkedList or something. Could use DataStructures.jl but it's a bit obscure. Might be just easier to do your own stuff here.

want

• easy insert at middle of list
• easy linear traversing of list

so this seems to work quite well, possibly because of pointer stuff:

t = raw"""
  this is some text blah
  and some more blah yes so blah
  and then more
  here blah"""
for m ∈ eachmatch(r"blah", t)
  t = replace(t, r"blah", "YA")
end
println(t)

returns the appropriate stuff (no messing with eachmatch on a transforming string:

this is some text YA
and some more YA yes so YA
and then more
here YA

that means that interweave_rep can be simplified using that trick (maybe).

• have a hidden file keeping track of all files (basically something corresponding to the watched_files dictionary).
• when compiling (without a forcing argument), check if there's a diff or new files
• recompile what has diff-ed.

Nesting is currently not allowed (in that it will cause things crash, it won't raise an error)

html

• for [[ if ... ]] blocks, they allow a single level of {{ }} nesting
• the {{ ... }} blocks do not allow nesting
• [[ ... ]] blocks do not allow nesting

md

• the @@... div blocks allow math blocks within them but no nesting of other div blocks
• \com{..} (future) should maybe allow nesting in that the inner, most nested, should be replaced first etc.

Actions

• either raise a proper error
• or allow nesting

it may require to actually implement a simple open/close function which counts and builds a stack which is then further processed or something. (that would also allow to handle problems by indicating when parens are not found)

Also nesting would require to use some form of recursive parsing. Having a generic function which extracts stuff based on delimiters is probably a good idea.

--> see README.md for a prototype of a function that does this in some level.

Since will look at allowing latex, the curly braces in code blocks (e.g. in types in Julia) would clash.

These should be extracted early and potentially fed back in just before the HTML parsing which handles block codes just fine.