Giter Club home page Giter Club logo

statik's Introduction

statik

Build Status

statik allows you to embed a directory of static files into your Go binary to be later served from an http.FileSystem.

Is this a crazy idea? No, not necessarily. If you're building a tool that has a Web component, you typically want to serve some images, CSS and JavaScript. You like the comfort of distributing a single binary, so you don't want to mess with deploying them elsewhere. If your static files are not large in size and will be browsed by a few people, statik is a solution you are looking for.

Usage

Install the command line tool first.

go get github.com/rakyll/statik

statik is a tiny program that reads a directory and generates a source file that contains its contents. The generated source file registers the directory contents to be used by statik file system.

The command below will walk on the public path and generate a package called statik under the current working directory.

$ statik -src=/path/to/your/project/public

The command below will filter only files on listed extensions.

$ statik -include=*.jpg,*.txt,*.html,*.css,*.js

In your program, all your need to do is to import the generated package, initialize a new statik file system and serve.

import (
  "github.com/rakyll/statik/fs"

  _ "./statik" // TODO: Replace with the absolute import path
)

  // ...

  statikFS, err := fs.New()
  if err != nil {
    log.Fatal(err)
  }
  
  // Serve the contents over HTTP.
  http.Handle("/public/", http.StripPrefix("/public/", http.FileServer(statikFS)))
  http.ListenAndServe(":8080", nil)

Visit http://localhost:8080/public/path/to/file to see your file.

You can also read the content of a single file:

import (
  "github.com/rakyll/statik/fs"

  _ "./statik" // TODO: Replace with the absolute import path
)

  // ...

  statikFS, err := fs.New()
  if err != nil {
    log.Fatal(err)
  }
  
  // Access individual files by their paths.
  r, err := statikFS.Open("/hello.txt")
  if err != nil {
    log.Fatal(err)
  }    
  defer r.Close()
  contents, err := ioutil.ReadAll(r)
  if err != nil {
    log.Fatal(err)
  }

  fmt.Println(string(contents))

There is also a working example under example directory, follow the instructions to build and run it.

Note: The idea and the implementation are hijacked from camlistore. I decided to decouple it from its codebase due to the fact I'm actively in need of a similar solution for many of my projects.

Deterministic output

By default, statik includes the "last modified" (mtime) time on files that it packs. This allows an HTTP FileServer to present the correct file modification times to clients.

However, if you have a continuous integration task that checks that your checked-in static files in a git repository match the code that is generated on your CI system, you'll run into a problem: The mtime on the git checkout does not match what you have locally, causing tests to fail.

You can fix the test in one of two ways:

  1. In CI, manually set the mtime on the freshly checked out tree: here's a stackoverflow answer that provides a shell command to do that; or,
  2. Instruct statik not to store the "last modified" time.

To ignore the last modified time, use the -m to statik, like so:

$ statik -m -include=*.jpg,*.txt,*.html,*.css,*.js

Note that this will cause http.FileServer to consider the file to always have changed & serve it with a "Last-Modified" of the time of the request.

statik's People

Contributors

a-h avatar ahmetb avatar aleksi avatar antifuchs avatar areski avatar coreyog avatar devfacet avatar dmitshur avatar dolmen avatar dvrkps avatar gfrey avatar grubernaut avatar jcchavezs avatar keats avatar matthewdale avatar mattn avatar mschoebel avatar rakyll avatar ribice avatar rogpeppe avatar schivei avatar shumon84 avatar songmu avatar tgulacsi avatar toddboom avatar vdobler avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

statik's Issues

Possibility to add a package comment

Several tools complain if a package has no package level comment at all,
eg. staticcheck:

staticcheck ./...
statik/statik.go:3:1: at least one file in a package should have a package comment (ST1000)

I'd like to propose the addition of a new command line flag with a
useful default, e.g.

flagPkgCmt     = flag.String("c", "Package statik contains static assets.", "Package comment")

(An empty value for *flagPgkCmt would omit the comment.)

If this sounds sensible I could provide a PR.

flag: include flag is not present in the latest version.

It seems like there has been a couple of changes, one of which is addition of -include flag, since the last release v0.1.6. In module-aware set-up, go get fetches and installs the latest version found where -include is missing for the time being. Would you mind bumping a release with the latest changes?

please tag and version this project

Hello,

Can you please tag and version this project?

I am the Debian Maintainer for statik and versioning would help Debian keep up with development.

On the problem of statik

Hello, can statik compress and render Vue projects?
I want to realize the following ideas on your basis.

-f flag to overwrite existing output file

I'm using your project to include shader language files in my Go-GL project. I'm not serving the files over HTTP, I just like having the resources embedded into the binary so it's all in one single package. It works great but I find that I'm modifying my external shaders quite often as I develop my app. I have to delete statik/statik.go each time before I run statik again. I can see the value in not overwriting files on the disk but I think a -f force flag would make my life a lot easier.

Can not handle the path containing dot

(*StatikFS).Open() can not handle the path containing dot.

See below

$ statik -src foo/
$ tree -F
.
├── foo/
│   ├── bar/
│   └── foo.txt
├── main.go
└── statik/
    └── statik.go

3 directories, 3 files
$ cat main.go
// main.go
package main

import (
	_ "./statik"
	"fmt"
	"github.com/rakyll/statik/fs"
	"os"
)

func main() {
	statikFS, _ := fs.New()
	path := "/./bar/../foo.txt"
	_, err1 := statikFS.Open(path)   // os.ErrNotExists occurred
	_, err2 := os.Open("foo" + path) // no errors occur
	fmt.Printf("err1 = %#v\n", err1)
	fmt.Printf("err2 = %#v\n", err2)
}
$ go run main.go
err1 = &errors.errorString{s:"file does not exist"}
err2 = <nil> 

Is this behavior expected? or a bug?

How to read files by FileSystem

I have a file named "sarasa-mono-j-medium.ttf" packaged in statik.go

But when I used codes like this:

	statikFS, err1 := fs.New()
	if err1 != nil {
		glog.Fatal(err1)
	}
	fontBytes, err2 := fs.ReadFile(statikFS, "sarasa-mono-j-medium.ttf")
	if err2 != nil {
		glog.Fatal(err2)
	}
	fontObj, err2 = freetype.ParseFont(fontBytes)
	if err2 != nil {
		glog.Fatal(err2)
	}

And the result told me

[FTAL] [2019-09-25 15:48:31] file does not exist

Does there has any problem in my codes? I don't want to open the file on a httpServer, I want to open the file directly, how?

Help running on Windows

I want to convert some static files into a go binary but i don't know how to run your program, can you help?

Multiple static file packages

I'd like to produce two (or more) different statik fs's in different import paths. Due to relying on init() and no public API from the package, it's impossible to use both filesystems at the same time; concurrent access is also impossible as fs.Register is package-scoped.

I'd propose modifying the API:

fs.New would take zipDatas ...string and fallback on whatever was put in fs.Register (backwards compatible),
The generated filesystem package could add Data() string for this purpose in addition to keeping init() behaviour.

error

linux
go version
go version go1.9.2 linux/amd64

go get github.com/rakyll/statik

github.com/rakyll/statik

../../../github.com/rakyll/statik/statik.go:162:11: fHeader.Modified undefined (type *zip.FileHeader has no field or method Modified)

Overwrite flag does not work on OS X

In the rename function the force overwrite flag is tested after a call to os.Rename, which succeeds in overwriting a file on OS X but not linux. (incoming PR to fix)

want add some function to list the dirs

once i get a file with name .
it returns not exist.
i checked a long time,but don't know why.
at last, i see something in your source.
the root path must start with /...

Generated file was considered changed by git but source file didn't.

I have a file public/index.html(no other files under public/), and statik generated file is statik/statik.go. My colleagues complain that statik/statik.go was considered changed by git after executing go generate command, but they didn't touch the public/index.html file. Is there any way to avoid this?

fallback for SPA applications

Hi. Is there a way to achieve fallback to some specific file if route is not found?
I would like to serve web application with html5 router supported. Actually it works until you hit refresh button on the route that does not exists in the statik filesystem. In that case it would be cool that fallback would be configurable, ie: ./index.html.
I did a PR for similar library here, but it is unmaintained so I would switch to statik.
Thanks!

Deterministic file generation

When using statik with default settings, and zipping the data the resulting file is different every time. Is it possible having a deterministic output for each run?

The goal is committing statik generated files in the VCS, and having them differ each run complicates things.

A command used is:

    statik -src=${src} -p ${component} -dest ${dest} -f -c ''

Embedding fs package directly in output

Thoughts on adding the fs package to the output? This would remove the need for the generated package to have non std-lib imports, thus removing the need to vendor the fs package at all. I'd be happy to do the work if you think the design is good.

Readdir returns sub-subfolders

Suppose the following folder structure:

test.txt
folder1/
   test2.txt

By default, http.File.Readdir in golang returns direct child files, so gives [test.txt, folder1/]

However, when I pack the files, File.Readdir seems to perform a recursive directory listing, and returns [test.txt,folder1,test2.txt] Note that test2.txt behaves as if it were in the root folder.

This causes issues in code using Readdir to recursively copy a directory from an embedded filesystem, and causes http.FileServer to give an incorrect listing when served without index.html

This issue is caused by https://github.com/rakyll/statik/blob/master/fs/fs.go#L163 , which does not check whether the file is in a subpath.

On a related note, empty directories are not listed at all, meaning that it would seem like they are non-existent.

Panics if src directory doesn't exist

statik -src=./fgd
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x20 pc=0x2d3c]

goroutine 1 [running]:
runtime.panic(0xd5820, 0x1fb379)
    /usr/local/go/src/pkg/runtime/panic.c:266 +0xb6
main.func·001(0x7fff5fbffb24, 0x5, 0x0, 0x0, 0x26c1b8, ...)
    /Users/burcu/src/github.com/rakyll/statik/statik.go:87 +0x4c
path/filepath.Walk(0x7fff5fbffb24, 0x5, 0x221040ce20, 0x26c140, 0x2102a5060)
    /usr/local/go/src/pkg/path/filepath/path.go:378 +0x81
main.generateSource(0x7fff5fbffb24, 0x5, 0x0, 0x0, 0x0)
    /Users/burcu/src/github.com/rakyll/statik/statik.go:104 +0x27f
main.main()
    /Users/burcu/src/github.com/rakyll/statik/statik.go:45 +0x48

Prefix example dir with "_"

If you update your gopath with go get ..., there is an error in your repository:

cannot find package "github.com/rakyll/statik/example/statik" in any of:
    /home/pierre/Logiciels/go/src/github.com/rakyll/statik/example/statik (from $GOROOT)
    /home/pierre/Go/src/github.com/rakyll/statik/example/statik (from $GOPATH)

Yes, this package does not exist.

Can you prefix your example dir with "_"?
It prevent the automatic installation with go get ....

Directory listing disabled?

I'm probably obtuse and missing something, but this package (and similar packages) all seem to disable directory listing. I understand opening an index file instead of directory upon request, but I'm a bit confused as to why you wouldn't want to implement ReadDir or ReadDirNames?

My thinking is that I want to use this to bundle files, but then I want to walk those files and add a handler for each to my muxer so I can put them at / and not interfere with another handler.

Before I go off and add that, am I just missing something?

use static files in html templates

Hi, i use Statik for css and js files and wondering if it possible to use compiled html files in templates. Like loginfile, _ := statikFS.Open("/tmpl/login.html")
tHeader, err := tHeader.ParseFiles(loginfile.data)
Thanks.

namespaces feature brokes compatibility with v0.1.6

master version of statik cmd generates code that uses fs.RegisterWithNamespace. But latest fs package from statik module version 0.1.6 doesn't have RegisterWithNamespace function.

Possible solution: statik cmd without -ns should generate statik.go with fs.Register()

Compilation error in main.go

The line 9 in rakyll/statik/example/main.go
_ "github.com/rakyll/statik/example/statik"

is giving an error during our build :
_vendor/github.com/rakyll/statik/example/main.go:9:2: cannot find package "github.com/rakyll/statik/example/statik" in any of: /root/go/src/github.com/***/go/vendor/github.com/rakyll/statik/example/statik (vendor tree) /usr/local/go/src/github.com/rakyll/statik/example/statik (from $GOROOT) /root/go/src/github.com/rakyll/statik/example/statik (from $GOPATH)_
Deleting that line fixes the issue.

The package was installed using
_go get -v -u -f github.com/rakyll/statik_

Looking forward to trying v0.1.4 :)

Jaana, if you have a minute, could you push v0.1.4 tag?
I see that there was a go.mod file added 6h ago and I'm looking forward to trying it :)

Thank you!

Accessing the directory directly?

I'm using golang-migrate, which has the following API usage:

migration, err := migrate.NewWithDatabaseInstance("file://migrations/", "postgres", driver)

I want to use statik to embed a ./migrations/ directory in the binary, and then use that in the place of file://, but I'm not sure how I can achieve this because file://migrations either exists or it doesn't, and this package I'm using doesn't look internally to find that file. It's literally looking for ./migrations as I tell it to and statik isn't intervening in filesystem calls to place it's own filesystem there is it?

Is it possible to do this with statik, or with any tool for that matter?

Seems to ignore the src argument

I'm not sure what I'm doing wrong, but statik always includes all files of the current working dir and seems to ignore my src parameter.

Usage:
statik -src=assets

Readdir should treat zero for count value as it was negative to be fully compatible with the interface

Hi @rakyll

Thanks for the statik, it's a great lib:)

I found an issue in the Readdir implementation while I was using another library that called statik's fs.Readdir(0) and it resulted in a problem. No file was found in this case.

It should treat the zero value and the negatives the same as the GoDoc recommends: https://golang.org/pkg/os/#File.Readdir

The problematic Readdir implementation:

statik/fs/fs.go

Line 190 in 3bac566

// If dirIdx reaches the end and the count is a positive value,

I fould the issue when I was using this library: shurcooL/httpfs#8

All best,
@shakahl

statik: Command not found

Hello,

I executed the following command

go get github.com/rakyll/statik

No errors after the above command. However, when I try the following command, I get the error. "statik: Command not found".

statik -src=/path/to/your/project/public

How do I solve this?

Regards,
Harish

Tests error with Golang 1.13 beta 1

I don't know if some rounding changed but I'm experiencing issues with Golang 1.13:

Testing    in: /builddir/build/BUILD/statik-0.1.6/_build/src
         PATH: /builddir/build/BUILD/statik-0.1.6/_build/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin
       GOPATH: /builddir/build/BUILD/statik-0.1.6/_build:/usr/share/gocode
  GO111MODULE: off
      command: go test -buildmode pie -compiler gc -ldflags "-X github.com/rakyll/statik/version=0.1.6 -extldflags '-Wl,-z,relro -Wl,--as-needed  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld '"
      testing: github.com/rakyll/statik
github.com/rakyll/statik/fs
--- FAIL: TestOpen (0.00s)
    --- FAIL: TestOpen/Files_should_retain_their_original_file_mode_and_modified_time (0.00s)
        fs_test.go:195: ModTime(/file.txt) = 2019-03-20 18:56:11 +0000 UTC; want 2019-03-20 18:56:10 +0000 UTC
    --- FAIL: TestOpen/Images_should_successfully_unpack (0.00s)
        fs_test.go:195: ModTime(/pixel.gif) = 2019-03-20 18:56:11 +0000 UTC; want 2019-03-20 18:56:10 +0000 UTC
    --- FAIL: TestOpen/'index.html'_files_should_be_returned_at_their_original_path_and_their_directory_path (0.00s)
        fs_test.go:195: ModTime(/index.html) = 2019-03-20 18:56:11 +0000 UTC; want 2019-03-20 18:56:10 +0000 UTC
        fs_test.go:195: ModTime(/sub_dir/index.html) = 2019-03-20 18:56:11 +0000 UTC; want 2019-03-20 18:56:10 +0000 UTC
    --- FAIL: TestOpen/listed_all_sub_directories_in_deep_directory (0.00s)
        fs_test.go:195: ModTime(/a) = 2019-03-20 18:56:11 +0000 UTC; want 2019-03-20 18:56:10 +0000 UTC
        fs_test.go:195: ModTime(/aa/bb/c) = 2019-03-20 18:56:11 +0000 UTC; want 2019-03-20 18:56:10 +0000 UTC
FAIL
exit status 1
FAIL	github.com/rakyll/statik/fs	0.011s

Reproductible output

Hello,

Is there any way to have reproductible output from statik?

As a context, this is needed for Debian. One of the packages I manage (InfluxDB) uses statik and I would like for it to be reproductible.

MIME types support

If you're serving, for example, js files with statik - it will return them with mime type text/plain, which will cause not working for some browser

Provide a NewReader on httpFile

I'm interested in using statik for a non-HTTP-serving use case, and one thing that'd make it a bit easier is if I could get a fresh bytes.Reader on the file:

// NewReader returns a new bytes.Reader reading from f.
func (f *httpFile) NewReader() *bytes.Reader {
	return bytes.NewReader(f.data)
}

The reason is because I need to use the file with something that wants an io.ReaderAt. I opted to use io.ReadFull, but that takes a copy of the underlying bytes which I'd like to avoid.

Offical exmple does not work?

image

As you see , when i copy the exmple from offical packages, and run go run main.go, access browser just show 404. What is the mistake I got from , please help! Thanks in advance!

> go env
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\Vector\AppData\Local\go-build
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=D:\GoProject
set GOPROXY=
set GORACE=
set GOROOT=D:\Dev\go
set GOTMPDIR=
set GOTOOLDIR=D:\Dev\go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\Vector\AppData\Local\Temp\go-build291331690=/tmp/go-build -gno-record-
gcc-switches

How do i check if directory exists ?

for files, following works

		file, err := statikFS.Open(relativeRequestPath)
		if err == nil {
			if fileInfo, err := file.Stat(); err == nil {
				isValidPath = true
				if !fileInfo.IsDir() {
					isValidFile = true
				}
			}
			file.Close()
		}

Hash in file name?

Is there a way to serve the same files with hash in their name for aggressive caching?

Relative path not working

I modified the example, relative path seems not working.

Code snippet:

func main() {
	statikFS, err := fs.New()
	...
	for _, name := range []string{"hello.txt", "/hello.txt"} {
		_, err := statikFS.Open(name)
		if err != nil {
			log.Printf("file %s: %s", name, err)
		} else {
			log.Printf("file %s exists", name)
		}
	}
        ...
}
$ go run main.go
2020/03/11 07:50:11 file hello.txt: file does not exist
2020/03/11 07:50:11 file /hello.txt exists

undefined: fs.RegisterWithNamespace

I am trying to build (GOOS=linux GOARCH=amd64 go build app/main.go) but getting this error:
app/statik/statik.go:12:2: undefined: fs.RegisterWithNamespace

code from auto generated statik: fs.RegisterWithNamespace("default", data)

how to resolve this?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.