rasky / go-lzo Goto Github PK

Native LZO implementation in Go

License: GNU General Public License v2.0

Go 100.00%

go-lzo's Introduction

go-lzo

Native LZO1X implementation in Golang

This code has been written using the original LZO1X source code as a reference, to study and understand the algorithms. Both the LZO1X-1 and LZO1X-999 algorithms are implemented. These are the most popular of the whole LZO suite of algorithms.

Being a straightforward port of the original source code, it shares the same license (GPLv2) as I can't possibly claim any copyright on it.

I plan to eventually reimplement LZO1X-1 from scratch. At that point, I will be also changing license.

Benchmarks

These are the benchmarks obtained running the testsuite over the Canterbury corpus for the available compressor levels:

Compressor	Level	Original	Compressed	Factor	Time	Speed
LZO1X-1	-	18521760	8957481	51.6%	0.16s	109MiB/s
LZO1X-999	1	18521760	8217347	55.6%	1.38s	13MiB/s
LZO1X-999	2	18521760	7724879	58.3%	1.50s	12MiB/s
LZO1X-999	3	18521760	7384377	60.1%	1.68s	10MiB/s
LZO1X-999	4	18521760	7266674	60.8%	1.69s	10MiB/s
LZO1X-999	5	18521760	6979879	62.3%	2.75s	6.4MiB/s
LZO1X-999	6	18521760	6938593	62.5%	4.53s	3.9MiB/s
LZO1X-999	7	18521760	6905362	62.7%	6.94s	2.5MiB/s
LZO1X-999	8	18521760	6713477	63.8%	20.96s	863KiB/s
LZO1X-999	9	18521760	6712069	63.8%	22.82s	792KiB/s

go-lzo's People

Contributors

Stargazers

Watchers

Forkers

pmezard liwei-link avan06 getstream rminnich ascend-io rocky1001 kaiarvin

go-lzo's Issues

Can't decompress LZO compressed in Python

Hi Giovanni,

I have a Python application that compresses data using python-lzo and then decompresses when needed in another part of application.

I'm currently porting decompressing part of the application to golang and for some reason when I'm trying to use your library it always throws LookBehindUnderrun error at me. Here is the code I'm using:

import (
    "bytes"
    "fmt"

    "github.com/rasky/go-lzo"
)

outData, err = lzo.Decompress1X(bytes.NewReader(inData), 0, 0)
if err != nil {
    fmt.Println(err)
}

Any ideas ?

Thanks in advance.

$ go version
$ go version devel +431b5c69ca Tue Apr 16 21:54:01 2019 +0000 darwin/amd64
$ go test
--- FAIL: TestDecompInlen (0.00s)
panic: runtime error: index out of range [0] with length 0 [recovered]
        panic: runtime error: index out of range [0] with length 0 [recovered]
        panic: runtime error: index out of range [0] with length 0

goroutine 20 [running]:
testing.tRunner.func1(0xc0000e8100)
        /Users/herbie/work/go113/go/src/testing/testing.go:830 +0x3a3
panic(0x115b440, 0xc00010a020)
        /Users/herbie/work/go113/go/src/runtime/panic.go:619 +0x1b2
github.com/rasky/go-lzo.Decompress1X.func1(0xc000106e70)
        /Users/herbie/work/google3-go/lzo/decompress.go:165 +0xfa
panic(0x115b440, 0xc00010a020)
        /Users/herbie/work/go113/go/src/runtime/panic.go:619 +0x1b2
github.com/rasky/go-lzo.(*reader).ReadU16(...)
        /Users/herbie/work/google3-go/lzo/decompress.go:94
github.com/rasky/go-lzo.Decompress1X(0x1199c40, 0xc0000c2040, 0x31, 0x0, 0xc0000d6090, 0xf, 0x10, 0x0, 0x0)
        /Users/herbie/work/google3-go/lzo/decompress.go:234 +0x944
github.com/rasky/go-lzo.TestDecompInlen(0xc0000e8100)
        /Users/herbie/work/google3-go/lzo/compress_test.go:126 +0x20c
testing.tRunner(0xc0000e8100, 0x1177800)
        /Users/herbie/work/go113/go/src/testing/testing.go:865 +0xbf
created by testing.(*T).Run
        /Users/herbie/work/go113/go/src/testing/testing.go:916 +0x350
exit status 2
FAIL    github.com/rasky/go-lzo 0.024s

I'll send a PR to update the check.