Giter Club home page Giter Club logo

smeagol's Introduction

Smeagol - locally hosted wiki

The goal of this project is to create a wiki software with these properties:

  • Compatible with GitHub. This means:
    • Git is the backing store
    • Markdown is used to format pages
  • Easy to run on your local computer.
  • Really fast and easy to install.

Non goals include:

  • Support for multiple users.

Phrased in another way, the goal of this project is to create something roughly like Gollum, but does not take half an hour to gem install on a low-powered Chromebook.

Getting started

See the smeagol.dev website for more install options.

If you use the Rust programming language, you can also install this tool using Cargo:

cargo install smeagol-wiki

Download the latest release from GitHub. Extract the smeagol-wiki executable from the compressed archive. smeagol-wiki is a command line application. It needs a directory containing the Markdown files you want to serve. You can pass a command line argument to it to specify the directory:

smeagol-wiki ~/wiki

When run without arguments, the current directory is used.

Once started, it listens on http://127.0.0.1:8000 by default.

Configuration

There are a few command line options:

  • --host - takes an argument that specifies which IP address to bind to. By default this is 127.0.0.1, which means only users on your local computer can access the wiki. Set to 0.0.0.0 to let other computers on your network access it.
  • --port - takes an argument that specifies which port to listen on. 8000 by default.
  • --fs - instructs Smeagol to load and save using the file system. By default Smeagol uses Git to load files committed to a Git repository and saves them by committing them to the current branch.

Additionally, the following settings can be put in a smeagol.toml file in the root directory of the wiki:

  • index-page - By default README. When you browse to a directory, Smeagol will display a file whose name (not including the extension) is index-page. For example, when you navigate to /page/for/bar/, Smeagol will display the file at foo/bar/README.md.
  • h1-title - By default false. When false, Smeagol will use the file name as the title of the page. This title will be displayed at the top of the page and in the title bar. When this setting is true and a document starts with an h1 (written as a line that starts with # in Markdown), the text of this h1 will be used as the title of the page. It will not be rendered as a normal part of the document.

Differences from Gollum

  • index-page on Gollum defaults to Home. Smeagol defaults to README to be compatible with online code hosting systems such as GitHub and Azure Devops.
  • The default port is 8000 rather than 4567.
  • Support for transclusion. If a line contains {{file-name.md}}, the contents of file-name.md will replace that line.

Security

Smeagol is intended to be run on your local computer to read your own private data. It is not designed to be exposed to the public internet: there is no authentication.

That said, there is one class of security problem I would be interested in hearing about: opening a maliciously designed Wiki with Smeagol either causing code execution or writing to files outside the wiki directory. Please file an issue if you encounter such a problem.

Why Rust, please tell me more about why you love Rust

Rust makes it easy to ship cross-compiled executables that run without much fuss. As for why not some other language also shares this capability (Go or C#), I just want to get more experience working with Rust.

License

Licensed under the MIT License.

Note that some elements, specifically aspects of the visual design, have been copied from Gollum.

smeagol's People

Contributors

austinwise avatar dependabot[bot] avatar therealbstern avatar

Stargazers

Alexander Strogy avatar Dmytro Uhnichenko avatar  avatar  avatar CrimeFog avatar  avatar 7158 avatar  avatar Joseph from Granata avatar  avatar JM avatar Senne Hofman avatar  avatar Matthias Braun avatar Matthias Homann avatar  avatar Yukai Huang avatar Jeffrey van Hoven avatar  avatar Alexander Koz. avatar Federico Scodelaro avatar Sarath avatar

Watchers

 avatar James Cloos avatar  avatar

smeagol's Issues

Theme: good error messages

When an error happens, it should be clear what the cause was and how the user can fix the problem. This includes both command ling error messages and error messages sent over HTTP.

  • Error messages sent of HTTP should ideally look relatively nice. Like they should not be plain text; they should not be the output from a Debug trait impl.
  • Do we have to worry about that thing where IE only displays error pages greater than 512 bytes in size? It looks like Chrome may have copied this feature
  • Audit the different From implementations for MyError. Make sure the message matches the level of specificity of the error type they convert from.

Some specific cases that are known to not have good error messages:

  • When a markdown file is not valid UTF-8.
  • Failures to read wiki pages.
  • Failure to find wiki pages.

Consider using a more full-featured web framework

Currently Smeagol uses hyper for HTTP and askama for templates. I assume more full featured web frameworks exist for Rust.

A non-exhaustive listen of reasons why a web framework might be useful:

  • Built in form persistence.
  • Perhaps the ability to have static type checking when constructing links to different controllers or pages

Another approach would be to only write an API in Rust and make a fancy JavaScript frontend.

Edit screen should have a "Preview" button

When editing, it would be nice to preview what the output will look like. Markdown is pretty simple, but formatting mistakes can still be made and it doesn't feel right adding further commits to fix formatting errors.

Create an API

Currently this program is designed to be used as an executable. It could also be accessible as an API, like how gollum-lib works.

  • Figure out if it makes sense to have an API
  • Design a Rust API
  • Design an FFI-api
  • Use the FFI-api to create a Ruby API that is a drop-in replacement for the Ruby gollum-lib.

Theme: Performance

The software should not waste resources wantonly. The following dimensions should be considered, very roughly in order of importance:

  • Installation time
  • Latency (time to first byte)
  • Startup time (probably correlated with binary size)
  • Binary size
  • Memory usage
  • Throughput (requests per second)

The software is currently intended to be used by a single user at a time, so throughput and scaling is not a huge concern.

Some concrete performance things to investigate:

  • Render markdown into the template engine's buffer directly. That is, instead of rendering markdown into a string, and then feeding that into the templating engine, have the markdown render expose a Write that is called from the template system's Write implementation.
  • Instead of rending templates into strings and then sending the contents over the wire, use the Write interface on templates to send directly to the socket. Some investigation will be required to see how much this change effects allocations and copying, as this may just shift the allocation from the template engine to the network abstraction.

Theme: security

Security is not a huge issue, as the current intended use case is a single person viewing data they wrote. However there are some things we should watch out for even in the single user case:

  • Make sure by default we are only binding to localhost by default, so that the user's wiki is not exposed to other people on the network inadvertently.
  • #43

Some other security issue could become an issue in a multi-user environment

  • Path traversal. Ideally we could also use some OS sandbox like unvail(2) to help us ensure this works.
  • XSS
  • Sandboxing, using something like pledge(2) or SELinux

Generic MIME type support

The webserver needs to be able to map file extensions to MIME types so it can set the correct value for Content-Type headers. This will be useful when serving media like pictures or when allowing binary files to be stored and downloaded from the wiki.

Theme: editing support

The wiki should be writable from the website. Features include:

  • #16
  • #51
  • Some amount syntax highlighting?
  • Maybe embed the VS Code editor into the page?
  • Stretch goal that probably will never happen: a WYSIWYG editor

Theme: documentation

People should be able to discover Smeagol, understand it, and solve problems they encounter.

  • Have a getting started guide
  • Create a dedicated webpage that describes what problem this program even solves
  • Create a man(1) page.

If saving a page fails for some reason, ensure that the changes are not lost

There are two ways I can think of to do this:

  • Redirect back to the edit page with the modified content. Display an alert with the error message.
  • Instead of submiting the form with the tradtional post, use JavaScript to save the changes. Once the client side confirms that the page was saved, only then redirect to displaying the page.

I think we could also save the edited page in localstorage, so if the browser crashes, the changes are not lost.

Embed Primer CSS in application

Currently it is being served from unpkg.com.

This might part of a larger theme of having a fancier front end story. Like we could download the CSS files from NPM and user and bundler and whatnot.

Imrpove Git support

Support reading and writing files from a Git repository. This includes bare repos. Features include:

  • #34
  • #35
  • #36
  • Viewing file change history
  • View overall wiki recent changes
  • #39

Theme: distribution

The program should be easy to acquire.

  • Automatic CI builds for every release
  • #19
  • No internet connection should be required after the program is downloaded #9
  • #15
  • Code signing
  • macOS notarization?
  • macOS fat binaries?
  • #31
  • #40

It would be cool if this program was easy to install using operating system packages managers. Here are a few:

  • Debian packaging
  • Homebrew packaging
  • pkg-src packaging
  • FreeBSD Ports packaging
  • Nix packaging
  • Windows packaging (Choco, winget, Windows Store, Scoop, etc.)

Note that currently winget does not support standalone executables

Cache static assets

Currently half a megabyte of CSS is downloaded on every page view.

Ideally we could do something where we generate a URL for the static assets at build time that contains the hash of the file contents. This would make it easy to bust the cache when the file changes.

Improve search support

It should be easy to search for content in the wiki. Even basic searching is useful, but there is a lot that can be done to make the experience great:

  • #23
  • #41
  • In search UI, update search results on every key press
  • Support regex queries
  • Consider persisting search index to disk. Currently it is in-memory and recreated on every startup.
  • Speed up indexing. Currently it is a little slow when indexing tens of thousands of documents.

Copy-paste commands to easily download and install binaries

Some ideas are in the CLI book: https://rust-cli.github.io/book/tutorial/packaging.html

trust gets most of the way there. There are some tweaks I would like to make:

  • Instead of using rustc to detect the target triple, steal the target detection code from the rustup-init.sh script
  • Instead of parsing the message GitHub gives when doing a 302 redirect to find the the latest version, do something else. Either a lot of work to discover the latest version at download time, or include the version number in the install instructions.
  • Add a PowerShell script for Windows users

Also, a cargo install command should be listed, so people can install on targets I have not yet create binaries for.

Some more pie-in-the-sky ideas:

  • rustup already does a lot of what I want for binary distribution and installation. Perhaps a binary rust package manager could be built out of parts of it?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.