Giter Club home page Giter Club logo

train's Introduction

Train

Build Status

Assets Management Package for web app in Go. The main purpose of it is to introduce some good practices already existed in Ruby on Rails' Assets Pipeline.

Main features

Installation

Get the package:

$ go get github.com/shaoshing/train

Install the command-line tool:

$ go build -o $GOPATH/bin/train github.com/shaoshing/train/cmd

Install node-sass, CoffeeCcript

$ npm install -g [email protected]
$ npm install -g [email protected]

Prepare for the Pipeline feature

If planning to use SASS or CoffeeScript, you should run the diagnose command to see whether your environment is fit for the feature. Otherwise, skip to the next section.

# Diagnose and follow the instructions to get your environment prepared
$ train diagnose

# If you experience `command not found` error, you should add $GOPATH/bin to $PATH
# or run the command as follow:
$ $GOPATH/bin/train

Quick Example

$ cd $GOPATH/src/github.com/shaoshing/train
$ go run example/main.go
# Visit localhost:8000 and play with the `include` directive and the SASS and CoffeeScript Pipeline.

In the example page, you can toggle the Include Directive feature, or try out the Pipeline feature by requesting a sass or coffee file.

Use it in your project

First, allow train to handle assets requests by adding handler to the http.ServeMux:

import "github.com/shaoshing/train"

...

// Adding handler to the http.DefaultServeMux
train.ConfigureHttpHandler(nil)
http.ListenAndServe(":8000", nil)

For custom ServeMux that overwrites the DefaultServeMux, you will need to pass the mux to train.ConfigureHttpHandler:

mux := http.NewServeMux()

...

train.ConfigureHttpHandler(mux)
http.ListenAndServe(":8000", mux)

Next, add the helper functions to templates so that Train can generate assets links for you:

import "github.com/shaoshing/train"
import "html/template"

...

tpl := template.New("home")
// Adding helpers
tpl.Funcs(template.FuncMap{
  "javascript_tag":            train.JavascriptTag,
  "stylesheet_tag":            train.StylesheetTag,
  "stylesheet_tag_with_param": train.StylesheetTagWithParam,
})
tpl.ParseFiles("home.html")
tpl.Execute(wr, nil)

Now in your template file, you can use the above helpers to include your assets:

(example: home.html)

<html>
  <head>
  {{stylesheet_tag "main"}}
  {{stylesheet_tag "home"}}

  {{javascript_tag "main"}}
  {{javascript_tag "home"}}
  ...
  </head></html>

Train enforce the following assets hierarchy and generate asset paths accordingly:

Project Root
├── assets
│   ├── javascripts // put js and coffee scripts here
│   │   ├── main.js
│   │   ├── home.coffee
│   └── stylesheets // put css and sass here
│       ├── main.sass
│       ├── home.css

Include Directive

Train allows you specify dependency inside asset file by using the include directive, and when you include the file using Train's helper, Train will check the dependency and expand the file into related files.

Say you have the following files:

├── assets
│   ├── javascripts
│   │   ├── base.js
│   │   ├── app.js   // depends on base.js

The regular way of insuring the dependency would be including both javascripts in the html file, something like this:

<script src="/assets/javascripts/basic.js"></script>
<script src="/assets/javascripts/app.js"></script>

In the Train way, you can do it by specifying the dependency in app.js:

//= require javascripts/base
...

And then use the helper to include app.js:

{{javascript_tag "app"}}

When request for the html, the content will become:

<script src="/assets/javascripts/basic.js?3392212"></script>
<script src="/assets/javascripts/app.js?3392212"></script>

To use the include directive in css is similar to js:

/*
 *= require stylesheets/base
 */
...

SASS and CoffeeScript

The Include Directive is only available for js and css. However, SASS already has the @import directive, which is doing the same thing. For CoffeeScript, you will have to manage the dependencies in a regular way.

Pipeline

When handling js or css request, Train will first look for the asset file with the same extension in the assets folder. If the file cannot be found, it will keep searching for a alternative extension, which is .sass/.scss for .css and .coffee for .js . When found, Train will convert the file into the desired extension.

Take a look at an simple example:

├── assets
│   ├── stylesheets
│   │   ├── app.sass

In the html, you include the sass file as if it is a css file:

{{stylesheet_tag "app"}}

Configuration

There are several configuration options related to the Pipeline feature:

// From SASS's doc:
// When set to true, causes the line number and file where a selector is defined to be
// emitted into the compiled CSS in a format that can be understood by the browser. Useful in
// conjunction with [the FireSass Firebug extension](https://addons.mozilla.org/en-US/firefox/addon/103988)
// for displaying the Sass filename and line number.
train.Config.SASS.DebugInfo = true // false by default


// From SASS's doc:
// When set to true, causes the line number and file where a selector is defined to be emitted
// into the compiled CSS as a comment. Useful for debugging, especially when using imports and mixins.
train.Config.SASS.LineNumbers = true // false by default


// Show SASS and CoffeeScript errors.
train.Config.Verbose = true // false by default

Bundling and Fingerprinting Assets

You probably want to merge or convert the assets in production site for performance concern. This is done by running Train's command-line tool train without any option:

$ cd project/root
$ train
-> clean bundled assets
-> copy assets from assets
-> bundle and compile assets
-> compress assets
-> Fingerprinting Assets

The following example is what were generated after running the train command:

Project Root
├── assets
│   ├── javascripts
│   │   ├── main.js
│   │   ├── app.js
│   │   ├── home.coffee
│   └── stylesheets
│       ├── main.sass
│       ├── home.css
├── public
│   ├── assets // generated by train
│   │   ├── manifest.txt
│   │   ├── javascripts
│   │   │   ├── main.js
│   │   │   ├── main-223e2f3f9ca508630ead4db28042cc42.js
│   │   │   ├── app.js
│   │   │   ├── app-c5d14af50112f85c0aee9181b14f02e4.js
│   │   │   ├── home.js
│   │   │   ├── home-c471ecdacdaf77f591100c4cffd51f41.js
│   │   └── stylesheets
│   │       ├── main.css
│   │       ├── main-d208d2ef0e80f9a7d372f0bd681f8ade.css
│   │       ├── home.css
│   │       ├── home-924c344bccc46742a90835cc104dbe20.css

When Train detect the public/assets folder, it will disable the Include Directive and Pipeline features and serve from these static files directly. Template helpers will also stop expanding assets and generate with fingerprinted paths:

{{javascript_tag "app"}}
{{stylesheet_tag "home"}}

// to

<script src="/assets/javascripts/app-c5d14af50112f85c0aee9181b14f02e4.js"></script>
<link rel="stylesheet" href="/assets/stylesheets/home-924c344bccc46742a90835cc104dbe20.css">

Why Fingerprinting?

From Rails' Assets Pipeline Document:

Fingerprinting is a technique that makes the name of a file dependent on the contents of the file.
When the file contents change, the filename is also changed. For content that is static or infrequently
changed, this provides an easy way to tell whether two versions of a file are identical, even across
different servers or deployment dates.

Checkout its document for more details about this technique.

Deploy to Production Server

There are two ways to deploy the Bundled and Fingerprinted assets to your server:

  1. Run the train command in the production server after each deployment. By doing this you can make sure to update public/assets to the latest. This is the simples way, but it requires your server have NodeJS and required npm if you are using the Pipeline feature.

  2. Run the train command in your local machine and upload the assets to the production server. With this way, the production server doesn't need to have NodeJS and required npm for the command.

Here is bash snippet to deploy assets using the second way:

SERVER="replace to your server's ssh address"
SERVER_PUBLIC="replace to your server's public path"

echo "Bundle assets"
$GOPATH/bin/train

if [[ $? != 0 ]] ; then
  echo "== fail to bundle assets"
  exit 1
fi

echo "Copy assets to $SERVER"
cd public
tar zcf assets.zip assets
scp assets.zip "$SERVER":assets.zip
ssh $SERVER "tar mxf assets.zip && sudo cp -r assets/* $SERVER_PUBLIC/assets/ && rm assets.zip"
rm -f assets.zip assets
cd -

Status

Train is production ready, and has been used in our production site Qortex. You are very welcome to report usage in your project.

Tested language / lib versions:

  • Go: go1.2.1 darwin/amd64
  • node-sass: 2.0.1
  • CoffeeScript: 2.2.0

Contribution

  • Fork & Clone
  • Make awesome changes (as well as tests)
  • Run the tests
  • Pull Request

Run the tests

  • Install Go (1.2.1)
  • Run all the tests ./test_all.sh

License

Train is released under the MIT License.

train's People

Contributors

bom-d-van avatar echlebek avatar huacnlee avatar imadha avatar jabley avatar jcamenisch avatar kenegozi avatar krasnoukhov avatar mrjbq7 avatar nilbus avatar shaoshing avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

train's Issues

Cache-Control header?

Does it make sense to add a Cache-Control header, or is there a way I can currently add one myself? Right now I'm serving my assets manually to be able to do this.

`couldn't determine public URL for ` issue

Could not compile sass:
Error generating source map: couldn't determine public URL for "assets/stylesheets/app.sass".
Without a public URL, there's nothing for the source map to link to.
Custom importers should define the #public_url method.

25655 2014/07/31 11:11:03 Failed to deliver asset

GET /assets/stylesheets/iphone.css

Could not compile sass:
Error generating source map: couldn't determine public URL for "assets/stylesheets/iphone.sass".
Without a public URL, there's nothing for the source map to link to.
Custom importers should define the #public_url method.

25655 2014/07/31 11:11:03 Failed to deliver asset

GET /assets/stylesheets/ipad.css

Could not compile sass:
Error generating source map: couldn't determine public URL for "assets/stylesheets/ipad.sass".
Without a public URL, there's nothing for the source map to link to.
Custom importers should define the #public_url method.

Production mode path issues

I recently upgraded train to latest from a version that was at least a few months old. After the upgrade, javascript_tag and stylesheet_tag helpers appeared to be broken in production mode. Not sure if this is an issue with train or the way I am using train.

I was able to fix my issues by changing flag.StringVar(&outPath, "out", "./public", "") in cmd.go to flag.StringVar(&outPath, "out", "public", ""). I think the ./ prefix was causing the strings.Replace calls in WriteToManifest to fail. Is this a bug with train or do you think there's something wrong with my local setup?

As a 2nd request, would you consider changing HasPublicAssets() to look for public/assets/manifest.txt instead of public/assets/? This would be useful because I'm storing other files in public/assets besides the files that train generates so I don't delete the entire folder when switching between development/production modes.

Thanks for your help and building this package. Keep up the good work!

cryptic panic when there are no CSS or no JS files

When train is executed when there are no CSS or no JS files, YUI compressor fails, and the error reporting is unclear:

YUI Compressor error:
...
panic: exit status 1
...
stacktraces, stacktraces everywhere!

suggested solution is to check for len() of []files, skip YUI if 0, and report.

Use wellington?

I see that you are calling out to a node-sass binary for Sass compiling. Any interest in using a Go wrapper to libsass instead Wellington? I can PR the needed changes to make make the switch.

Public Panic

When running the command generated by cmd package, it will panic if the public folder is not existed.

It would be nice if it automatically create the public when it is not exited, or at least suggest what to do in order to get train work.

-> clean bundled assets
-> copy assets from assets
panic: exit status 1

goroutine 1 [running]:
main.copyAssets()
    /Users/bom_d_van/Code/go/workspace/src/github.com/shaoshing/train/cmd/bundle.go:39 +0x1fd
main.main()
    /Users/bom_d_van/Code/go/workspace/src/github.com/shaoshing/train/cmd/bundle.go:21 +0x1d

goroutine 2 [syscall]:
created by runtime.main
    /usr/local/go/src/pkg/runtime/proc.c:221

goroutine 3 [syscall]:
syscall.Syscall6()
    /usr/local/go/src/pkg/syscall/asm_darwin_amd64.s:38 +0x5
syscall.wait4(0x16057, 0xf84007d2a0, 0x0, 0xf840085750, 0x1, ...)
    /usr/local/go/src/pkg/syscall/zsyscall_darwin_amd64.go:32 +0x81
syscall.Wait4(0x16057, 0x23dee1c, 0x0, 0xf840085750, 0x39006, ...)
    /usr/local/go/src/pkg/syscall/syscall_bsd.go:136 +0x6a
os.(*Process).wait(0xf8400e45e0, 0xf84007d268, 0x0, 0x0, 0x392a8, ...)
    /usr/local/go/src/pkg/os/exec_unix.go:22 +0xe1
os.(*Process).Wait(0xf8400e45e0, 0x0, 0x0, 0x23deef0)
    /usr/local/go/src/pkg/os/doc.go:43 +0x25
os/exec.(*Cmd).Wait(0xf8400b3100, 0x0, 0x0, 0x0)
    /usr/local/go/src/pkg/os/exec/exec.go:308 +0x1b7
os/exec.(*Cmd).Run(0xf8400b3100, 0x0, 0x0, 0x0)
    /usr/local/go/src/pkg/os/exec/exec.go:232 +0x6d
github.com/shaoshing/train/interpreter._func_001(0xf840066600, 0x0)
    /Users/bom_d_van/Code/go/workspace/src/github.com/shaoshing/train/interpreter/bridge.go:57 +0x29
created by github.com/shaoshing/train/interpreter.NewInterpreter
    /Users/bom_d_van/Code/go/workspace/src/github.com/shaoshing/train/interpreter/bridge.go:61 +0x16b

goroutine 4 [syscall]:
syscall.Syscall()
    /usr/local/go/src/pkg/syscall/asm_darwin_amd64.s:14 +0x5
syscall.Read(0x3c00000004, 0xf8400f6000, 0x800000008000, 0x100000001, 0x0, ...)
    /usr/local/go/src/pkg/syscall/zsyscall_darwin_amd64.go:905 +0x78
os.(*File).read(0xf84007d250, 0xf8400f6000, 0x800000008000, 0x800000008000, 0x0, ...)
    /usr/local/go/src/pkg/os/file_unix.go:174 +0x58
os.(*File).Read(0xf84007d250, 0xf8400f6000, 0x800000008000, 0xf8400f6000, 0x0, ...)
    /usr/local/go/src/pkg/os/file.go:95 +0x83
io.Copy(0xf840066750, 0xf84007d208, 0xf84006ab70, 0xf84007d250, 0x0, ...)
    /usr/local/go/src/pkg/io/io.go:360 +0x20c
os/exec._func_003(0xf840075a70, 0xf84007d248, 0x3b065, 0x0, 0x0, ...)
    /usr/local/go/src/pkg/os/exec/exec.go:207 +0x5a
os/exec._func_004(0xf84007d300, 0xf840064f40, 0x0, 0x0)
    /usr/local/go/src/pkg/os/exec/exec.go:274 +0x1e
created by os/exec.(*Cmd).Start
    /usr/local/go/src/pkg/os/exec/exec.go:275 +0x65c

CDN Service

How can one use Train with a CDN service? (Eg. S3 or Fastly).
Do we just have to point the AssetsUrl to the right location?

Memory Leaking problem caused by interpreter package

Each time when we start running the train package, and after it invoke interpreter package, it will start a ruby process and don't try to close it even after the train sever been shut down. That causes piling of ruby process.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.