multiformats / go-varint Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
Hey there,
So I was working on attempting to port this lib into another language, and I saw that UvarintSize
is dividing the number of bits by 7 to compute the size in bytes of a 64-bit unsigned integer (
Line 19 in 9946570
I'm trying to understand why did you consider 7 bits per byte, instead of 8 bits per byte.
Thanks in advance
This says that a varint can have at most 9 bytes. In go, a varint can have 10 bytes as long as the last byte is exactly 0x1
. This allow specifying 64bit numbers.
I'm happy to change this implementation, but I want to make sure nobody knows of anything it might break.
This ReadUvarint() method is time-sensitive, because it's running inside a mutex. It's called very often.
I'd like to offer a more performant version, that will improve latencies:
Previous version:
func ReadUvarint(r io.ByteReader) (uint64, error) {
// Modified from the go standard library. Copyright the Go Authors and
// released under the BSD License.
var x uint64
var s uint
for i := 0; ; i++ {
b, err := r.ReadByte()
if err != nil {
if err == io.EOF && i != 0 {
// "eof" will look like a success.
// If we've read part of a value, this is not a
// success.
err = io.ErrUnexpectedEOF
}
return 0, err
}
if (i == 8 && b >= 0x80) || i >= MaxLenUvarint63 {
// this is the 9th and last byte we're willing to read, but it
// signals there's more (1 in MSB).
// or this is the >= 10th byte, and for some reason we're still here.
return 0, ErrOverflow
}
if b < 0x80 {
if b == 0 && s > 0 {
return 0, ErrNotMinimal
}
return x | uint64(b)<<s, nil
}
x |= uint64(b&0x7f) << s
s += 7
}
}
New version:
func ReadUvarint(r io.ByteReader) (uint64, error) {
// Modified from the go standard library. Copyright the Go Authors and
// released under the BSD License.
var x uint64
var s uint
for s = 0; ; s+=7 {
b, err := r.ReadByte()
if err != nil {
if err == io.EOF && i != 0 {
// "eof" will look like a success.
// If we've read part of a value, this is not a
// success.
err = io.ErrUnexpectedEOF
}
return 0, err
}
if (s == 56 && b >= 0x80) || s >= (7 * MaxLenUvarint63) {
// this is the 9th and last byte we're willing to read, but it
// signals there's more (1 in MSB).
// or this is the >= 10th byte, and for some reason we're still here.
return 0, ErrOverflow
}
if b < 0x80 {
if b == 0 && s > 0 {
return 0, ErrNotMinimal
}
return x | uint64(b)<<s, nil
}
x |= uint64(b&0x7f) << s
s += 7
}
}
The 'i == 8' is replaced with 's == 56'. MaxLenUvarint63 is '9', currently, so that's not an overflow problem.
This change removes the unnecessary addition operation for 'i'.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.