This would remove the need for checking to ensure that a dataset is of the proper type before a certain method is called. (example: can only send snapshot: https://github.com/mistifyio/go-zfs/blob/master/zfs.go#L93-L95)

I think the Go technique of embedding is a good fit here: https://golang.org/doc/effective_go.html#embedding

And here is a quick example I threw together of how it could potentially look:

http://play.golang.org/p/mt4tgTtrk2

Overall, I think this approach could create a cleaner API. Please feel free to share your thoughts or concerns!

zfsonlinux changed its format and illumos/omnios is broken also

vagrant@omnios-vagrant:/export/home/vagrant/src/github.com/mistifyio/go-zfs$ sudo ./go-zfs.test -test.v
=== RUN   TestError
--- PASS: TestError (0.00s)
=== RUN   TestDatasets
--- PASS: TestDatasets (0.13s)
=== RUN   TestSnapshots
--- PASS: TestSnapshots (0.13s)
=== RUN   TestFilesystems
--- PASS: TestFilesystems (0.18s)
=== RUN   TestCreateFilesystemWithProperties
--- PASS: TestCreateFilesystemWithProperties (0.15s)
=== RUN   TestVolumes
--- PASS: TestVolumes (1.15s)
=== RUN   TestSnapshot
--- PASS: TestSnapshot (0.17s)
=== RUN   TestClone
--- PASS: TestClone (0.19s)
=== RUN   TestSendSnapshot
--- PASS: TestSendSnapshot (0.17s)
=== RUN   TestChildren
--- PASS: TestChildren (0.20s)
=== RUN   TestListZpool
--- PASS: TestListZpool (0.12s)
=== RUN   TestRollback
--- PASS: TestRollback (0.24s)
=== RUN   TestDiff
zfs_test.go:361:

        exp: "/test/origin/i ❤ unicode"

        got: "/test/origin/i \xff77777742\xff77777635\xff77777644 unicode"

--- FAIL: TestDiff (0.10s)
FAIL

Hi,

I'm going to use the mystifyio/go-zfs package in a cobra based command line application for filesystem and snapshot management. Thanks for your work so far!

I have implemented a straight-forward solution to support remote execution of the ZFS commands via SSH. I've tried to do this in a way, which has minimum impact on the core system and all your tests are running in a local environment as well as remotely against the Vagrant Ubuntu box. Pls. see ssh-remote-command branch of my fork for details. If you think, the implementation is of general interest for the project, I will be happy to review the code and create a pull request.

Additional features from my backlog are:

• advanced filtering of filesystem/snapshot lists with glob patterns
• func (d *Dataset) LastCommonSnapshot(target *Dataset) (*Dataset, error)

Thanks for your reply and comments,
Bernd

When getting a size related attribute via dataset.GetProperty because is not a field of the dataset struct (e.g. refquota), it is returned with K, M, `G suffix. Bytes would probably be preferable in most cases and you could still convert them afterwards if you wanted to.

Hi,

This issue is in conjunction with #39.
In an effort to come up with a PR for Solaris support I investigated performance on using zfs list instead of zfs get and came up with the following numbers on Linux.
In short zfs list is quicker, especially when retrieving a specific set of properties.

I wrote a small timer function using the go "time" package that would time zfs Init().
On a Ubuntu system that had 40 containers on it: (As the number of containers on the system increases the time difference also increases drastically)
Without my changes daemon startup showed ZFS init took:

root:~/docker# ./docker-1.11.0-dev-baseline daemon --storage-driver=zfs
ZFS driver init took 716.156429ms time
INFO[0000] Graph migration to content-addressability took 0.00 seconds 
INFO[0000] Firewalld running: false  
INFO[0001] Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address 
WARN[0001] Your kernel does not support swap memory limit. 
INFO[0001] Loading containers: start.  
INFO[0002] Loading containers: done.  
INFO[0002] Daemon has completed initialization  
INFO[0002] Docker daemon                                 commit=d687023-unsupported execdriver=native-0.2 graphdriver=zfs version=1.11.0-dev
INFO[0002] API listen on /var/run/docker.sock           

With my changes:

root@:~/docker# bundles/1.11.0-dev/binary/docker-1.11.0-dev daemon --storage-driver=zfs
ZFS driver init took 282.353825ms time
INFO[0000] Graph migration to content-addressability took 0.00 seconds 
INFO[0000] Firewalld running: false  
INFO[0000] Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address 
WARN[0001] Your kernel does not support swap memory limit. 
INFO[0001] Loading containers: start.  
INFO[0001] Loading containers: done.  
INFO[0001] Daemon has completed initialization  
INFO[0001] Docker daemon                                 commit=d687023-unsupported execdriver=native-0.2 graphdriver=zfs version=1.11.0-dev
INFO[0001] API listen on /var/run/docker.sock           

Also zfs_test,go does a good job of running all the ZFS commands supported and in all we saved almost 6s on the test run with the changes:

root@:~/vendor/src/github.com/mistifyio/go-zfs# go test
PASS
ok      github.com/mistifyio/go-zfs 20.738s
root@:~/vendor/src/github.com/amitkris/go-zfs# go test
PASS
ok      github.com/amitkris/go-zfs  14.677s

Container startup time is still comparable:

root@:~/docker# time bundles/1.11.0-dev/binary/docker-1.11.0-dev run -i ubuntu /bin/true

real    0m2.368s
user    0m0.104s
sys 0m0.008s
root@:~/docker# time ./docker-1.11.0-dev-baseline run -ti ubuntu /bin/true

real    0m2.039s
user    0m0.076s
sys 0m0.020s

CHANGES made:

1. Replace zfs/zpool get calls with zfs/zpool list
2. Specify the list of relevant properties using -o instead of just calling all
3. Changing parseLine to handle output of list instead of get
4. Make all the dataset and zpool struct elements strings instead of (strings + integers) because they are being consumed as strings in the graphdriver in Docker anyway( we're going from string->uint64->string, why not just store them as strings..)

Hi,

This is part of a larger effort to develop a native port of Docker on Solaris.
go-zfs is great but has the following issues on Solaris:

1. On Solaris zfs get doesn't support the -t option (only zfs list does).
2. zpool doesn't support -t.
3. zfs get doesn't support -r|-d after all
   i.e. the invocation zfs get all -d will error out.

As a part of coming up with a PR for this fix I investigated whether this should be resolved by creating a separate invocation for Solaris which would call zfs list but I propose a generic change due to performance improvements that it will bring to go-zfs.
Please see the issue titled "switch from zfs get to zfs list calls for better performance" (#40)

I propose to submit a commom PR to fix both these issues.

Hello all,

I have done a function SendSnapshotIncremental that add the options "-i" et "-I". As I don't know if this project is till active, before to do a pull request, I open this issue.

// SendFlag is the options flags passed to SendSnapshot
type SendFlag int

// Valid send options
const (
	SendDefault              SendFlag = 1 << iota
	IncrementalStream                 = 1 << iota
	IncrementalPackage	          = 1 << iota
)

func SendSnapshotIncremental(output io.Writer, d1 *Dataset, d2 *Dataset, flags SendFlag) error {
	if d1.Type != DatasetSnapshot || d2.Type != DatasetSnapshot {
		return errors.New("can only send snapshots")
	}

	// Flags for SendSnapshot
	option := ""
	if flags&IncrementalStream !=0 {
		option = "-i"
	}
	if flags&IncrementalPackage  !=0 {
		option = "-I"
	}
	c := command{Command: "zfs", Stdout: output}
	_, err := c.Run("send", option, d1.Name, d2.Name)
	return err
}

Anyway, If there are anybody who can improve it, this will be perfect !

I can open PR if no one working on this yet.

Due to fix for openzfs/zfs#3417

zfs get -H creation disk4T/code

returns: disk4T/code creation mer. mai 4 15:59 2016 -

but

func main() {
	ds, _ := zfs.GetDataset("disk4T/code")
	prop, _ := ds.GetProperty("creation")
	fmt.Printf("%#v\n", prop)
}

returns:

"mer."

This is because in utils.go:Run() use:

for i, l := range lines {
    output[i] = strings.Fields(l)
}

to split fields. But stings.Fields() doesn't make difference between space and tabulation.

As we will be iterating on this frequently.

#45 added fields, we should have some tests in place

If an error occurs when running zfs or zpool, and a non-zero exit code is returned, the following fmt.Errorf() statement is used to create an error message:

https://github.com/mistifyio/go-zfs/blob/master/utils.go#L43

To enable better error handling in client code, I propose that this be replaced with a new struct type, and an Error() method on that struct which can generate a string in the same format as the existing one.

This struct could look something like the following:

http://play.golang.org/p/ctPnZgy2UG

I would be happy to submit a PR if you'd like to move in this direction! Thanks for providing this excellent package!

Hi,

When I try to test your porject it fails with

• go test -compiler gc -ldflags '-X github.com/mistifyio/go-zfs/version.GitSHA=166add352731e515512690329794ee593f1aaff2 -extldflags '''-Wl,-z,relro ''''
  zfs_test.go:74: unexpected error: exec: "zpool": executable file not found in $PATH: "zpool zpool create test /tmp/zfs-015623521 /tmp/zfs-936753996 /tmp/zfs-071489595" =>

--- FAIL: TestDatasets (0.00s)
zfs_test.go:74: unexpected error: exec: "zpool": executable file not found in $PATH: "zpool zpool create test /tmp/zfs-911153758 /tmp/zfs-579910181 /tmp/zfs-122527040" =>

--- FAIL: TestDatasetGetProperty (0.00s)
zfs_test.go:74: unexpected error: exec: "zpool": executable file not found in $PATH: "zpool zpool create test /tmp/zfs-352642975 /tmp/zfs-305956978 /tmp/zfs-140087081" =>

--- FAIL: TestSnapshots (0.00s)
zfs_test.go:74: unexpected error: exec: "zpool": executable file not found in $PATH: "zpool zpool create test /tmp/zfs-392938612 /tmp/zfs-191840067 /tmp/zfs-120972230" =>

--- FAIL: TestFilesystems (0.00s)
zfs_test.go:74: unexpected error: exec: "zpool": executable file not found in $PATH: "zpool zpool create test /tmp/zfs-363306605 /tmp/zfs-084848360 /tmp/zfs-820738855" =>

--- FAIL: TestCreateFilesystemWithProperties (0.00s)
zfs_test.go:74: unexpected error: exec: "zpool": executable file not found in $PATH: "zpool zpool create test /tmp/zfs-288479322 /tmp/zfs-461731313 /tmp/zfs-613427356" =>

--- FAIL: TestVolumes (0.00s)
zfs_test.go:74: unexpected error: exec: "zpool": executable file not found in $PATH: "zpool zpool create test /tmp/zfs-507569483 /tmp/zfs-925299246 /tmp/zfs-713572789" =>

--- FAIL: TestSnapshot (0.00s)
zfs_test.go:74: unexpected error: exec: "zpool": executable file not found in $PATH: "zpool zpool create test /tmp/zfs-108708752 /tmp/zfs-225214383 /tmp/zfs-188771138" =>

--- FAIL: TestClone (0.00s)
zfs_test.go:74: unexpected error: exec: "zpool": executable file not found in $PATH: "zpool zpool create test /tmp/zfs-074942905 /tmp/zfs-070772164 /tmp/zfs-521692243" =>

--- FAIL: TestSendSnapshot (0.00s)
zfs_test.go:74: unexpected error: exec: "zpool": executable file not found in $PATH: "zpool zpool create test /tmp/zfs-795910550 /tmp/zfs-294920701 /tmp/zfs-851205944" =>

--- FAIL: TestChildren (0.00s)
zfs_test.go:74: unexpected error: exec: "zpool": executable file not found in $PATH: "zpool zpool create test /tmp/zfs-919999799 /tmp/zfs-514381098 /tmp/zfs-689129601" =>

--- FAIL: TestListZpool (0.00s)
zfs_test.go:74: unexpected error: exec: "zpool": executable file not found in $PATH: "zpool zpool create test /tmp/zfs-367396844 /tmp/zfs-327222363 /tmp/zfs-453601790" =>

--- FAIL: TestRollback (0.00s)
zfs_test.go:74: unexpected error: exec: "zpool": executable file not found in $PATH: "zpool zpool create test /tmp/zfs-592104261 /tmp/zfs-918564832 /tmp/zfs-517285311" =>

--- FAIL: TestDiff (0.00s)
FAIL
exit status 1
FAIL github.com/mistifyio/go-zfs 0.005s

Thank you for your help

dataset.GetProperty uses the first line of zfs get without any switches.

At time the zfs commands stuck for a long time. Is there a way to kill the commands after a specified time.

ps -ef | grep 'zfs list' gives me a long list of commands.

Hey, great lib 👍

Is it possible to access the "creation date"?
Its possible to list this information using:

zfs list -o name,creation -S creation

The last part (-S creation) is not necessary, its just sorting the output already.

If its not yet possible, how could we add this?
Thx!

Compare to https://github.com/bakins/zfsd/blob/2b704f2e27703fc90d2dd6bd865c528e35ecd72f/utils.go#L124

Is "brute-force", but no magic, etc.

we always call create with -p which will cause it to ignore already existing datasets. Should this be a flag?

code:
go fmt.Println(go-zfs.Snapshots("")
output:
[] Output does not match what is expected on this platform

While installing docker I had a serious problem with my ZFS-Installation. My installation had no fragmentation determination enabled and therefore the command

root@atomic:~# zpool get -p name,health,allocated,size,free,readonly,dedupratio,fragmentation,freeing,leaked tank
NAME  PROPERTY       VALUE    SOURCE
tank  name           tank     -
tank  health         ONLINE   -
tank  allocated      2098558267392  -
tank  size           2989297238016  -
tank  free           890738970624  -
tank  readonly       off      -
tank  dedupratio     1.00x    -
tank  fragmentation  -        -
tank  freeing        0        default
tank  leaked         0        default

can not be parsed. the lib struggles with the "-" in fragmentation. This results in the following error message: "Zpool: error while getting pool information strconv.ParseUint: parsing "": invalid syntax"

Original issue in Docker issue trakcer: moby/moby#30551

Hello,
I'd like to be able to create sparse volumes using this library. To do that with zfs command one need to specify -s flag so a function like this would work (it's just a copy of CreateVolume with additional flag in args[2]):

// CreateVolume creates a new ZFS volume with the specified name, size, and
// properties.
// A full list of available ZFS properties may be found here:
// https://www.freebsd.org/cgi/man.cgi?zfs(8).
func CreateSparseVolume(name string, size uint64, properties map[string]string) (*Dataset, error) {
	args := make([]string, 4, 5)
	args[0] = "create"
	args[1] = "-p"
	args[2] = "-sV"
	args[3] = strconv.FormatUint(size, 10)
	if properties != nil {
		args = append(args, propsSlice(properties)...)
	}
	args = append(args, name)
	_, err := zfs(args...)
	if err != nil {
		return nil, err
	}
	return GetDataset(name)
}

In some cases zpool reterns '-' in 'fragmentation' field

$ sudo zpool get -p name,health,dedupratio,fragmentation rpool
NAME   PROPERTY       VALUE   SOURCE
rpool  name           rpool   -
rpool  health         ONLINE  -
rpool  dedupratio     1.00x   -
rpool  fragmentation  -       -

parseLine doesn't check if there is trainling '%' in the string before removing it, and this leads to empty string passed to setUint.

This leads to broken output of docker info

$ docker info
Storage Driver: zfs
 Zpool: error while getting pool information strconv.ParseUint: parsing "": invalid syntax
 Zpool Health: not available

Opening an issue for adding support for zfs set command. This would allow setting various properties on zfs filesystems/volumes/snapshots (such as quota). I'm going to start working on a pull request for this.

This could allow for easier checks of common zpool states, and for more specific logic in the case of a pool which is degraded, offline, etc.

http://play.golang.org/p/_bSlCjlTWZ

Perhaps the zpool.Healthy() method is not entirely necessary, but it's a lot nicer than doing if z.Health == "ONLINE".

Similar to zpool states, string constants would be less fragile and error-prone for various dataset types.

http://play.golang.org/p/tAHoCFRN5F

In addition, these constants could be used at several places within the existing library:

https://github.com/mistifyio/go-zfs/blob/master/zfs.go#L64-L66
https://github.com/mistifyio/go-zfs/blob/master/zfs.go#L93-L95
https://github.com/mistifyio/go-zfs/blob/master/zfs.go#L189-L191

Hello all,

I have a question. When I use the function ReceiveSnapshot(), it does not end to read of conn net.Conn if there is not an error. So, if the delivery is good, I don't know how to stop the function and I don't want to use conn.Close because I want to continue sending information...

Thank you.