statiqdev / statiq.web Goto Github PK
View Code? Open in Web Editor NEWStatiq Web is a flexible static site generator written in .NET.
Home Page: https://statiq.dev/web
License: Other
Statiq Web is a flexible static site generator written in .NET.
Home Page: https://statiq.dev/web
License: Other
Currently the ViewStart used by the Razor Module is magic filename "_ViewStart.cshtml". Rather than using a magic filename, have it set via the constructor. This allows the more than just directory level organizing of the cshtml files. For example, on your site, rather than having a separate _ViewStart.cshtml in each subdir, you could have PostViewStart.cshtml, and you can specify a path to it, so you don't have to have such long Where predicates in your FileReader Module since all those files can live in a different subdir.
Something like --metadata key1=value1;key 2=value 2
. This will be helpful in certain scenarios like excluding secrets from a config file.
Right now --watch monitors file system changes. This can be expanded to watching a certain URL and trigger a pipeline when it changes (check etag).
Not sure how deep this should go. A simple implementation would just send timings for each module and pipeline to trace. Maybe this should be done with verbose regardless (I.e., no flag to enable) - not sure the added overhead of setting timers adds enough to warrant turning on and off. If a flag is used, should it also be sent to modules or available in the execution context to modules can report in-module timing (I.e., compile vs. execute for Razor templates)? Maybe we need a new trace category for timing information.
Inspired from this PR in Jekyll: jekyll/jekyll#3762
Module to reuse documents from previous pipelines. Not sure about the name... Example use case is pulling in blog entries from multiple pipelines to create an RSS feed.
See discussion in #25
For example, outputting a trace message when a file can't be found would be helpful in pointing out missing resources.
Create a generic interface and class, IMetadata<T>
and Metadata<T>
that wraps Metadata
and returns strongly typed metadata values. Use type converters or some other mechanism that will work in more cases than simple casting. I.e., if we have an int coded as a string object like Metadata["Foo"] = "123"
, then Metadata<int>["Foo"]
should successfully return 123 as an int value. Create properties that create the wrapper to IDocument
, the Razor page, etc. This should help minimize awkward metadata casting like ((string)Metadata["FileBase"]) == "Foo"
.
A module that can parse HTML content and do something with it (like store results in metadata). Main use case is extracting the first paragraph from documents and storing it in metadata.
Should read Less CSS content and transform it to CSS content. Use DotLess.
Nice project! May I suggest using Command Line Parser library to simplify argument parsing? https://github.com/gsscoder/commandline
If there was a reason for creating the method to parse the command line args the way its done in Program.cs, pls. ignore!
This module is commonly used at the end of the pipeline, usually after ReadFiles()
or ForEachDocument
.
The ability to handle common network failure and resume uploading is critical. Consider using this FTP library http://netftp.codeplex.com/documentation.
To pull content from https://www.contentful.com
Should include NuGet packages as well. For example, if Newtonsoft.Json
is added as a NuGet package and the relevant namespaces are added in the config, then a Razor view should be able to use the JSON parser without any extra effort.
Takes input document content and splits it into multiple documents based on a delimiter.
Should work just like it does in ASP.NET MVC.
It was always the intent that modules could execute their internal behavior in parallel, that's why they get a collection of documents as input. Right now, the new cache support is not thread-safe, so if a module uses it during parallel operations there might be problems.
Module to perform concatenation on all input documents and generate a single output document (for example, to take a bunch of CSS files and output a single one). Allow and optional delimiter. Similar to string.Join(...)
(as opposed to LINQ .Join(...)
).
I'm still getting my head wrapped around using Wyam and what I'd like my workflow to be, but for now, I think I'd like to use Visual Studio to manage/edit my input project, using an empty asp.net 5 site.
With asp.net 5's project.json format having the ability to register commands, which can then be run from dnx or inside Visual Studio Code: Link. I think it would be helpful if Wyam could be packaged up and published as a DNX package.
That should let us register a command for wyam and integrate with the IDE or command-line a bit better.
All/most of the moving parts to do this might already exist, so this issue is either an enhancement request, or might just be a place to gather some of this documentation.
Instead of strings, use streams for binary content (see discussion in #25). This will allow documents to contain string or binary content. It should also yield better performance in some cases where transformations are optimized for streaming data (instead of having to read into a string). Will need to convert all existing modules over to streams. Should also add convenience getters and setters to convert from the stream from/to byte arrays and strings (in the case of getting data, handle reading the stream into the array and vise versa for setters).
Should be able to generate page redirects based on specified metadata values. Take a look at how Jekyll handles this in the jekyll-redirect-from plugin by using HTTP-REFRESH
meta tags. Also note that there is support for this in GitHub Pages.
Takes Liquid template content and transforms it to output content. Use DotLiquid. Make the metadata, completed documents, and execution context available similar to how the Razor module does.
NuGet handling still isn't quite where I'd like it to be. Consider creating a global packages folder where the command line application is, and pre-propulate it with latest releases of common modules (Razor, Yaml, Markdown, etc.). Check the global for a match first, then check local, then download if not available. Also, don't update by default, only download if no match exists locally. Use a flag like --update-packages
to actually update all packages (which might eliminate the need for the other control flag).
Convert HTML content to PDF
It was added to support testing, but should be removed and all tests converted over to mocks, etc. if possible.
Automatically extracts headings from a HTML document and generates a table of contents. How to specify where the ToC is inserted? How to identify whether a heading has an anchor (and what to do if it doesn't)? Should it also be added to the metadata? What if we just want it added to metadata without inserting the ToC (I.e., for display on another page)?
Possibly call this modules Headings instead of ToC (or maybe two different complimentary modules).
Not sure how this one would work, but it would be cool to create a search index that could be queried at the client. Is there a JavaScript compatible Lucene client? The module should add content (and metadata?) to the search index and be usable from multiple pipelines.
Actually searching is probably the job of hand-written javascript code (but include examples in the docs).
Constructs an RSS feed from the input documents. How to handle feed items from multiple pipelines (might not need to worry about it if there's also a module that can "reuse" documents from other pipelines)? How to extract the excerpt (pull from metadata or provide a Func<IDocument, string>
)?
Recipes should configure one or more pipelines. They should be contained in separate libraries and available via NuGet. Therefore, any interfaces need to go in Wyam.Abstractions
.
They should also allow additional configuration as needed. Consider how they'll be configured from the command line.
Create an IDocument.Id
property that persists with the document (similar to metadata). This should generally be unique and can be used for things like trace messages, cache keys, etc. ReadFiles
should set the ID to the RelativeFilePath
. Consider other modules that clone extra documents - they should also set an ID. Should ID be populated from a Clone(...)
overload? Log a warning if a duplicate ID is used. Find all trace message that refer to documents and use the ID (instead of path, etc.)
Read RSS files and produces IDocuments with related meta tags for further processing.
This will allow Wyam to publish blogs based on RSS wire format. Support the outline extension as well (http://scripting.com/essayblog.xml)
Allow a recipe to be selected from the command line. Should also require the name of the NuGet package where the recipe lives (which will be automatically downloaded). Should also allow the specification of an alternate package source, version, etc. Argument should look something like --recipe package name[@nuget feed]:recipe name
.
When a recipe is specified on the command line, and configuration files are ignored. Specifying a recipe and a configuration file on the command line should output an error.
Also need to consider how to configure recipes on the command line. This may impact how recipes are configured in general - for example, only configured in ways the command line can support like string key value pairs.
It should be able to convert images as well as possibly create images from scratch (using SVG? System.Drawing?). Would be good for things like image-based text headers, etc.
Should read JSON content and generate metadata. Probably based on Json.Net. Should also flatten the metadata like the YAML module does.
Allow specifying one or more folders to watch. Make the tray icon spin or something when building.
Compiling Razor views for every page is getting to be a little slow. This is especially evident when using a layout for multiple views.
Scans HTML content and automatically generates links given a mapping of keywords to urls. Can't just do a search and replace, have to only link text in the body and in elements where a tags are allowed. Provide a mapping of keywords as a dictionary and also make sure to have an overload that uses a func with a document for custom maps per document. Need to provide an override for content that shouldn't be linked, perhaps a user-defined CSS class. Also need to make sure not to double-link if keyword is already in an a tag.
Need to figure out how a module can trigger watch events as necessary. For example, ReadFiles module should trigger for changes to read files while a database module could trigger for changes to tables, etc.
Read OPML outlines and produce documents with content and metadata.
It's becoming clear that there may be additional code that module libraries might need beyond the interfaces currently in Wyam.Abstractions. Renaming will keep the spirit of the library while expanding the implied scope. Also better to do it early now then when there are actually third-party module libraries out there. The NuGet package under the old name will have to be delisted, or at least a comment added.
Should read Sass CSS content and transform it to CSS content. Not sure if there's a suitable library available (if not, this will have to wait - not building a new Sass parser).
Create a module that can take TypeScript content as input and generate JavaScript content as output.
There are several modules that would benefit from some sort of caching between executions:
The cache should be keyed to some sort of hash on the input content since not all inputs are going to come from files, and even if they do they might be manipulated by other modules before getting to the one that needs caching. The key needs to be unique enough to prevent false positives. Should we first check for a matching hash and if one is found, compare the content for fast hit but then guaranteed no false positive? This would require storing the old content along with the hash and might take long enough that the cache is a moot point... Another requirement is that we can only cache stuff that can't refer to other metadata because we have no way of knowing what that metadata will be on each run (and it might be different).
The cache should be passed in the execution context and be available for any module to use (somehow keyed to the module to limit the search space?). Provide methods to generate cache entries from the document (automatically calculate hash, store content, etc) and check the cache from a document. Cache value should be a raw object so that things other than content (like Razor compilations) can be stored.
Launch a simple web server. Should point to OutputFolder
. Allow specifying port to use.
Crawls through specified HTML files and checks relative links to make sure they're all valid. Outputs warnings to trace. Does not perform any transformation.
Look at html-proofer: https://github.com/gjtorikian/html-proofer
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.