Giter Club home page Giter Club logo

algernon's Introduction

Web server with built-in support for HTTP/2, Lua, Markdown, Amber, GCSS, JSX, Bolt, Redis, MySQL, rate limiting, graceful shutdown, plugins, users and permissions.

Build Status GoDoc Gitter license

Technologies

Written in Go. Uses Bolt (built-in), MySQL or Redis (recommended) for the database backend, permissions2 for handling users and permissions, gopher-lua for interpreting and running Lua, http2 for serving HTTP/2, blackfriday for Markdown rendering, amber for Amber templates and GCSS for CSS preprocessing. logrus is used for logging, risotto for converting from JSX to JavaScript, tollbooth for rate limiting, pie for plugins and graceful for graceful shutdowns.

http2check can be used to confirm that the server is in fact serving HTTP/2.

Design decisions

  • HTTP/2 over SSL/TLS (https) is used by default, if a certificate and key is given.
    • If not, regular HTTP is used.
  • /data and /repos have user permissions, /admin has admin permissions and / is public, by default. This is configurable.
  • The following filenames are special, in prioritized order:
    • index.lua is interpreted as a handler function for the current directory.
    • index.md is rendered as HTML.
    • index.html is outputted as it is, with the correct Content-Type.
    • index.txt is outputted as it is, with the correct Content-Type.
    • index.amber is rendered as HTML.
    • data.lua is interpreted as Lua code, where the functions and variables are made available for Amber and Markdown pages in the same directory.
    • If a single Lua script is given as a commandline argument, it will be used as a standalone server. It can be used for setting up handlers or serving files and directories for specific URL prefixes.
    • style.gcss is used as the style for Amber and Markdown pages in the same directory.
  • The following filename extensions are handled by Algernon:
    • .md is interpreted as Markdown and rendered as a HTML page.
    • .amber is interpreted as Amber and rendered as a HTML page.
    • .gcss is interpreted as GCSS and rendered as a CSS file.
    • .jsx is interpreted as JSX and rendered as a JavaScript file.
    • .lua is interpreted as a Lua script that provides its own output and content type.
  • Other files are given a mimetype based on the extension.
  • Directories without an index file are shown as a directory listing, where the design is hardcoded.
  • UTF-8 is used whenever possible.
  • The server can be configured by commandline flags or with a lua script, but no configuration should be needed for getting started.

Features and limitations

  • Supports HTTP/2, with or without HTTPS.
  • Also supports regular HTTP.
  • Can use Lua scripts as handlers for HTTP requests.
  • Compiled to native.
  • Works on Linux, OS X and 64-bit Windows.
  • Is reasonably fast.
  • The Lua interpreter is compiled into the executable.
  • The use of Lua allows for short development cycles, where code is interpreted when the page is refreshed (there is an auto-refresh feature).
  • Self-contained Algernon applications can be zipped into an archive (ending with .zip or .alg) and be loaded at start.
  • Built-in support for Markdown, Amber, GCSS and JSX.
  • Redis is used for the database backend, by default.
  • Algernon will fall back to the built-in Bolt database if no Redis server is available.
  • The HTML title for a rendered Markdown page can be provided by the first line specifying the title, like this: title: Title goes here. This is a subset of MultiMarkdown.
  • No file converters needs to run in the background (like for SASS). Files are converted on the fly.
  • If -autorefresh is enabled, the browser will automatically refresh pages when the source files are changed. Works for Markdown, Lua error pages and Amber (including GCSS and data.lua). This only works on Linux and OS X, for now. If listening for changes on too many files, the OS limit for the number of open files may be reached.
  • Includes an interactive REPL.
  • If only given a Markdown filename as the first argument, it will be served on port 3000, without using any database, as regular HTTP. Handy for viewing README.md files locally.
  • Full multithreading. All available CPUs will be used.
  • Supports rate limiting, by using tollbooth.
  • The help command is available at the Lua REPL, for a quick overview of the available Lua functions.
  • Can load plugins written in any language. Plugins must offer the Lua.Code and Lua.Help functions and talk JSON-RPC over stderr+stdin. See pie for more information. Sample plugins for Go and Python are in the plugins directory.
  • Thread-safe file caching is built-in, with several available cache modes (for only caching images, for example).
  • Can read from and save to JSON documents. Supports simple JSON path expressions (like a simple version of XPath, but for JSON).
  • If cache compression is enabled, files that are stored in the cache can be sent directly from the cache to the client, without decompressing.
  • Files that are sent to the client are compressed with gzip, unless they are under 4096 bytes.

Quick Installation

OS X
  • brew install algernon
  • Install Homebrew, if needed.
Arch Linux
  • Install algernon from AUR, using your favorite AUR helper.
Any system where go is installed (and $GOPATH has been set)
  • go get github.com/xyproto/algernon
  • Add $GOPATH/bin to $PATH, if needed.

Overview

Running Algernon (screenshot from an earlier version):


The idea is that webpages can be written in Markdown, Amber, HTML or JSX (+React), depending on the need, and styled with CSS or GCSS, while data can be provided by a Lua script that talks to Redis, Bolt or MySQL.

Amber and GCSS is a good combination, that allows for more clarity and less repetition than HTML and CSS. It˙s also easy to use Lua for providing data for the Amber templates, which helps separate model, controller and view.

The auto-refresh feature is supported when using Markdown or Amber, and is useful to get an instant preview when developing.

The JSX to JavaScript (ECMAscript) transpiler is built-in.

Redis is fast, scalable and offers good data persistence. This should be the prefered backend.

Bolt is a pure key/value store, written in Go. It makes it easy to run Algernon without having to set up a database host first.

MySQL support is included because of its widespread availability.

Screenshots

Markdown can easily be styled with GCSS.


This is how errors in Lua scripts are handled, when Debug mode is enabled.


One of the poems of Algernon Charles Swinburne, with three rotating tori in the background. Uses CSS3 for the gaussian blur and three.js for the 3D graphics.


Screenshot of the prettify sample. Served from a single Lua script.


JSX transforms are built-in. Using React together with Algernon is easy.

Getting started

Run Algernon in "dev" mode

This enables debug mode, uses the internal Bolt database, uses regular HTTP instead of HTTPS+HTTP/2 and enables caching for all files except: Amber, Lua, GCSS, Markdown and JSX.

  • algernon -e
Enable HTTP/2 in the browser
  • Chrome: go to chrome://flags/#enable-spdy4, enable, save and restart the browser.
  • Firefox: go to about:config, set network.http.spdy.enabled.http2draft to true. You might need the nightly version of Firefox.
Configure the required ports for local use
  • You may need to change the firewall settings for port 3000, if you wish to use the default port for exploring the samples.
  • For the auto-refresh feature to work, port 5553 must be available (or another host/port of your choosing, if configured otherwise).
Prepare for running the samples
  • cd $GOPATH/src/github.com/xyproto/algernon
  • go build
The "bob" sample, over https
  • Run ./bob.sh to start serving the "bob" sample.
  • Visit https://localhost:3000/ (note: it's https)
All the samples, over http, with auto-refresh enabled
  • Run ./samples.sh to start serving the sample directory.
  • Visit http://localhost:3000/ (note: it's http)
Create your own Algernon application, for regular HTTP
  • mkdir mypage
  • cd mypage
  • Create a file named index.lua, with the following contents: print("Hello, Algernon")
  • Start algernon --httponly --autorefresh.
  • Visit http://localhost:3000/.
  • Edit index.lua and refresh the browser to see the new result.
  • If there were errors, the page will automatically refresh when index.lua is changed.
  • Markdown and Amber pages will also refresh automatically, as long as -autorefresh is used.
Create your own Algernon application, for HTTP/2 + HTTPS
  • mkdir mypage
  • cd mypage
  • Create a file named index.lua, with the following contents: print("Hello, Algernon")
  • Create a self-signed certificate, just for testing:
  • openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 3000 -nodes
  • Press return at all the prompts, but enter localhost at Common Name.
  • Start algernon.
  • Visit https://localhost:3000/.
  • If you have not imported the certificates into the browser, nor used certificates that are signed by trusted certificate authorities, perform the necessary clicks to confirm that you wish to visit this page.
  • Edit index.lua and refresh the browser to see the result (or a Lua error message, if the script had a problem).

Basic Lua functions

// Return the version string for the server.
version() -> string

// Sleep the given number of seconds (can be a float).
sleep(number)

// Log the given strings as information. Takes a variable number of strings.
log(...)

// Log the given strings as a warning. Takes a variable number of strings.
warn(...)

// Log the given strings as an error. Takes a variable number of strings.
err(...)

Lua functions for handling requests

// Set the Content-Type for a page.
content(string)

// Return the requested HTTP method (GET, POST etc).
method() -> string

// Output text to the browser/client. Takes a variable number of strings.
print(...)

// Return the requested URL path.
urlpath() -> string

// Return the HTTP header in the request, for a given key, or an empty string.
header(string) -> string

// Set an HTTP header given a key and a value.
setheader(string, string)

// Return the HTTP headers, as a table.
headers() -> table

// Return the HTTP body in the request (will only read the body once, since it's streamed).
body() -> string

// Set a HTTP status code (like 200 or 404). Must come before other output.
status(number)

// Set a HTTP status code and output a message (optional).
error(number[, string])

// Return the directory where the script is running. If a filename (optional) is given, then the path to where the script is running, joined with a path separator and the given filename, is returned.
scriptdir([string]) -> string

// Return the directory where the server is running. If a filename (optional) is given, then the path to where the server is running, joined with a path separator and the given filename, is returned.
serverdir([string]) -> string

// Serve a file that exists in the same directory as the script.
serve(string)

// Return a table with keys and values as given in a posted form, or as given in the URL (`/some/page?x=7` makes the key `x` with the value `7` available).
formdata() -> table

// Redirect to a different URL
redirect(string)

// Transmit what has been outputted so far, to the client.
flush()

Lua functions for formatted output

// Output Markdown to the browser/client. The given text is converted from Markdown to HTML. Takes a variable number of strings.
mprint(...)

// Output Amber to the browser/client. The given text is converted from Amber to HTML. Takes a variable number of strings.
aprint(...)

// Output GCSS to the browser/client. The given text is converted from GCSS to CSS. Takes a variable number of strings.
gprint(...)

// Output JSX to the browser/client. The given text is converted from JSX to JavaScript. Takes a variable number of strings.
jprint(...)

Lua functions related to JSON

Tips:

  • Use JFile(filename) to use or store a JSON document in the same directory as the Lua script.
  • A JSON path is on the form x.mapkey.listname[2].mapkey, where [, ] and . have special meaning. It can be used for pinpointing a specific place within a JSON document. It's a bit like a simple version of XPath, but for JSON.
  • Use tostring(userdata) to fetch the JSON string from the JFile object.
// Use, or create, a JSON document/file.
JFile(filename) -> userdata

// Takes a JSON path. Returns a string value, or an empty string.
jfile:get(string) -> string

// Takes a JSON path (optional) and JSON data to be added to the list.
// The JSON path must point to a list, if given, unless the JSON file is empty.
// Returns true on success.
jfile:add([string, ]string) -> bool

// Take a JSON path and a string value. Changes the entry. Returns true on success.
jfile:set(string, string) -> bool

// Remove a key in a map. Takes a JSON path, returns true on success.
jfile:delkey(string) -> bool

// Convert a Lua table with strings or ints to JSON.
// Takes an optional number of spaces to indent the JSON data.
toJSON(table[, number]) -> string
  • ToJSON was renamed to toJSON, but ToJSON is kept as a valid alias, for backward compatibility.

Lua functions for plugins

// Load a plugin given the path to an executable. Returns true on success. Will return the plugin help text if called on the Lua prompt.
Plugin(string)

// Returns the Lua code as returned by the Lua.Code function in the plugin, given a plugin path. May return an empty string.
PluginCode(string) -> string

// Takes a plugin path, function name and arguments. Returns an empty string if the function call fails, or the results as a JSON string if successful.
CallPlugin(string, string, ...) -> string

Lua functions for code libraries

These functions can be used in combination with the plugin functions for storing Lua code returned by plugins when serverconf.lua is loaded, then retrieve the Lua code later, when handling requests. The code is stored in the database.

// Create or uses a code library object. Optionally takes a data structure name as the first parameter.
CodeLib([string]) -> userdata

// Given a namespace and Lua code, add the given code to the namespace. Returns true on success.
codelib:add(string, string) -> bool

// Given a namespace and Lua code, set the given code as the only code in the namespace. Returns true on success.
codelib:set(string, string) -> bool

// Given a namespace, return Lua code, or an empty string.
codelib:get(string) -> string

// Import (eval) code from the given namespace into the current Lua state. Returns true on success.
codelib:import(string) -> bool

// Completely clear the code library. Returns true on success.
codelib:clear() -> bool

Lua functions for file uploads

// Creates a file upload object. Takes a form ID (from a POST request) as the first parameter.
// Takes an optional maximum upload size (in MiB) as the second parameter.
// Returns nil and an error string on failure, or userdata and an empty string on success.
UploadedFile(string[, number]) -> userdata, string

// Return the uploaded filename, as specified by the client
uploadedfile:filename() -> string

// Return the size of the data that has been recevied
uploadedfile:size() -> number

// Return the mime type of the uploaded file, as specified by the client
uploadedfile:mimetype() -> string

// Save the uploaded data locally. Takes an optional filename. Returns true on success.
uploadedfile:save([string]) -> bool

// Save the uploaded data as the client-provided filename, in the specified directory.
// Takes a relative or absolute path. Returns true on success.
uploadedfile:savein(string)  -> bool

Lua functions for the file cache

// Return information about the file cache.
CacheInfo() -> string

// Clear the file cache.
ClearCache()

Lua functions for data structures

Set
// Get or create a database-backed Set (takes a name, returns a set object)
Set(string) -> userdata

// Add an element to the set
set:add(string)

// Remove an element from the set
set:del(string)

// Check if a set contains a value
// Returns true only if the value exists and there were no errors.
set:has(string) -> bool

// Get all members of the set
set:getall() -> table

// Remove the set itself. Returns true on success.
set:remove() -> bool

// Clear the set
set:clear() -> bool
List
// Get or create a database-backed List (takes a name, returns a list object)
List(string) -> userdata

// Add an element to the list
list:add(string)

// Get all members of the list
list:getall() -> table

// Get the last element of the list
// The returned value can be empty
list:getlast() -> string

// Get the N last elements of the list
list:getlastn(number) -> table

// Remove the list itself. Returns true on success.
list:remove() -> bool

// Clear the list. Returns true on success.
list:clear() -> bool
HashMap
// Get or create a database-backed HashMap (takes a name, returns a hash map object)
HashMap(string) -> userdata

// For a given element id (for instance a user id), set a key
// (for instance "password") and a value.
// Returns true on success.
hash:set(string, string, string) -> bool

// For a given element id (for instance a user id), and a key
// (for instance "password"), return a value.
// Returns a value only if they key was found and if there were no errors.
hash:get(string, string) -> string

// For a given element id (for instance a user id), and a key
// (for instance "password"), check if the key exists in the hash map.
// Returns true only if it exists and there were no errors.
hash:has(string, string) -> bool

// For a given element id (for instance a user id), check if it exists.
// Returns true only if it exists and there were no errors.
hash:exists(string) -> bool

// Get all keys of the hash map
hash:getall() -> table

// Remove a key for an entry in a hash map
// (for instance the email field for a user)
// Returns true on success
hash:delkey(string, string) -> bool

// Remove an element (for instance a user)
// Returns true on success
hash:del(string) -> bool

// Remove the hash map itself. Returns true on success.
hash:remove() -> bool

// Clear the hash map. Returns true on success.
hash:clear() -> bool
KeyValue
// Get or create a database-backed KeyValue collection (takes a name, returns a key/value object)
KeyValue(string) -> userdata

// Set a key and value. Returns true on success.
kv:set(string, string) -> bool

// Takes a key, returns a value.
// Returns an empty string if the function fails.
kv:get(string) -> string

// Takes a key, returns the value+1.
// Creates a key/value and returns "1" if it did not already exist.
// Returns an empty string if the function fails.
kv:inc(string) -> string

// Remove a key. Returns true on success.
kv:del(string) -> bool

// Remove the KeyValue itself. Returns true on success.
kv:remove() -> bool

// Clear the KeyValue. Returns true on success.
kv:clear() -> bool

Lua functions for handling users and permissions

// Check if the current user has "user" rights
UserRights() -> bool

// Check if the given username exists
HasUser(string) -> bool

// Get the value from the given boolean field
// Takes a username and fieldname
BooleanField(string, string) -> bool

// Save a value as a boolean field
// Takes a username, fieldname and boolean value
SetBooleanField(string, string, bool)

// Check if a given username is confirmed
IsConfirmed(string) -> bool

// Check if a given username is logged in
IsLoggedIn(string) -> bool

// Check if the current user has "admin rights"
AdminRights() -> bool

// Check if a given username is an admin
IsAdmin(string) -> bool

// Get the username stored in a cookie, or an empty string
UsernameCookie() -> string

// Store the username in a cookie, returns true on success
SetUsernameCookie(string) -> bool

// Clear the login cookie
ClearCookie()

// Get a table containing all usernames
AllUsernames() -> table

// Get the email for a given username, or an empty string
Email(string) -> string

// Get the password hash for a given username, or an empty string
PasswordHash(string) -> string

// Get all unconfirmed usernames
AllUnconfirmedUsernames() -> table

// Get a confirmation code that can be given to a user,
// or an empty string. Takes a username.
ConfirmationCode(string) -> string

// Add a user to the list of unconfirmed users
// Takes a username and a confirmation code
AddUnconfirmed(string, string)

// Remove a user from the list of unconfirmed users
// Takes a username
RemoveUnconfirmed(string)

// Mark a user as confirmed
// Takes a username
MarkConfirmed(string)

// Removes a user
// Takes a username
RemoveUser(string)

// Make a user an admin
// Takes a username
SetAdminStatus(string)

// Make an admin user a regular user
// Takes a username
RemoveAdminStatus(string)

// Add a user
// Takes a username, password and email
AddUser(string, string, string)

// Set a user as logged in on the server (not cookie)
// Takes a username
SetLoggedIn(string)

// Set a user as logged out on the server (not cookie)
// Takes a username
SetLoggedOut(string)

// Log in a user, both on the server and with a cookie
// Takes a username
Login(string)

// Log in a user, both on the server and with a cookie
// Takes a username. Returns true if the cookie was set successfully.
CookieLogin(string) -> bool

// Log out a user, on the server (which is enough)
// Takes a username
Logout(string)

// Get the current username, from the cookie
Username() -> string

// Get the current cookie timeout
// Takes a username
CookieTimeout(string) -> number

// Set the current cookie timeout
// Takes a timeout number, measured in seconds
SetCookieTimeout(number)

// Get the current password hashing algorithm (bcrypt, bcrypt+ or sha256)
PasswordAlgo() -> string

// Set the current password hashing algorithm (bcrypt, bcrypt+ or sha256)
// ‘bcrypt+‘ accepts bcrypt or sha256 for old passwords, but will only use
// bcrypt for new passwords.
SetPasswordAlgo(string)

// Hash the password
// Takes a username and password (username can be used for salting sha256)
HashPassword(string, string) -> string

// Check if a given username and password is correct
// Takes a username and password
CorrectPassword(string, string) -> bool

// Checks if a confirmation code is already in use
// Takes a confirmation code
AlreadyHasConfirmationCode(string) -> bool

// Find a username based on a given confirmation code,
// or returns an empty string. Takes a confirmation code
FindUserByConfirmationCode(string) -> string

// Mark a user as confirmed
// Takes a username
Confirm(string)

// Mark a user as confirmed, returns true on success
// Takes a confirmation code
ConfirmUserByConfirmationCode(string) -> bool

// Set the minimum confirmation code length
// Takes the minimum number of characters
SetMinimumConfirmationCodeLength(number)

// Generates a unique confirmation code, or an empty string
GenerateUniqueConfirmationCode() -> string

Lua functions that are available for server configuration files

// Set the default address for the server on the form [host][:port].
// May be useful in Algernon application bundles (.alg or .zip files).
SetAddr(string)

// Reset the URL prefixes and make everything *public*.
ClearPermissions()

// Add an URL prefix that will have *admin* rights.
AddAdminPrefix(string)

// Add an URL prefix that will have *user* rights.
AddUserPrefix(string)

// Provide a lua function that will be used as the permission denied handler.
DenyHandler(function)

// Return a string with various server information.
ServerInfo() -> string

// Direct the logging to the given filename. If the filename is an empty
// string, direct logging to stderr. Returns true on success.
LogTo(string) -> bool

// Returns the version string for the server.
version() -> string

// Logs the given strings as INFO. Takes a variable number of strings.
log(...)

// Logs the given strings as WARN. Takes a variable number of strings.
warn(...)

// Logs the given string as ERROR. Takes a variable number of strings.
err(...)

// Provide a lua function that will be run once, when the server is ready to start serving.
OnReady(function)

// Use a Lua file for setting up HTTP handlers instead of using the directory structure.
ServerFile(string) -> bool

Functions that are only available for Lua server files

This function is only available when a Lua script is used instead of a server directory, or from Lua files that are specified with the ServerFile function in the server configuration.

// Given an URL path prefix (like "/") and a Lua function, set up an HTTP handler.
// The given Lua function should take no arguments, but can use all the Lua functions for handling requests, like `content` and `print`.
handle(string, function)

// Given an URL prefix (like "/") and a directory, serve the files and directories.
servedir(string, string)

Commands that are only available in the REPL

  • help displays a syntax highlighted overview of most functions.
  • webhelp displays a syntax highlighted overview of functions related to handling requests.
  • confighelp displays a syntax highlighted overview of functions related to server configuration.
  • pprint can be used for displaying a description, or contents, of Lua values.

Releases

Requirements

  • go >= 1.4
  • readline, for UNIX-like systems

General information

algernon's People

Contributors

xyproto avatar

Watchers

 avatar  avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.