Comments (7)
Thanks Zond.
This may be due to possible overflow issues, because Go's time.Time uses Jan 1, Year 1 as epoch, while binc/msgpack/java/python/etc use Jan 1, 1970 as epoch.
I wouldn't be able to look into it till next week at the earliest. If you can, please look at codec/time.go (a 100-line file which included encodeTime and decodeTime functions) to start debugging.
Thanks.
from go.
#6 created for this issue.
from go.
Hi Zond,
I looked into this issue.
I looked at your pull request solution also. Part of the reason for pruning the sign ext is to make storage of time super-compact. With your pull request, we lost that. So I tried to look at the solution while maintaining the compact storage for time.Time.
The problem as you figured out was that we didn't handle negative tsecs component during decoding. Thus, times before Unix EPOCH was broken. The fix is to pre-populate sign-extend the byte array with 0xff if a negative time.
///////////////// UGORJI NOTE: FIRST IF BLOCK IN DECODETIME
if bd&(1<<7) != 0 {
var btmp [8]byte
n = ((bd >> 2) & 0x7) + 1
i2 = i + n
copy(btmp[8-n:], bs[i:i2])
////////////// UGORJI NOTE: ADD BLOCK BELOW
//if first bit of bs[i] is set, then fill btmp[0..8-n] with 0xff (ie sign extend it)
if bs[i] & (1 << 7) != 0 {
var timeBs0xff = []byte{0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff}
copy(btmp[0:8-n], timeBs0xff)
////// UGORJI NOTE: OR USE LOOP BELOW. STILL HAVE TO TEST WHICH IS FASTER.
//for j,k := byte(0), 8-n; j < k; j++ { btmp[j] = 0xff }
}
I've tested this and it works. If you can verify that it works on your use-cases, that would be great. I'm not comfortable releasing it with just my limited testing, so need you to co-sign off on it, or wait for a few days when I can focus work on it.
from go.
Yeah, I realized that my solution wasn't the most compact, but I am too inexperienced when it comes to bit fiddling with signed numbers to make anything else work in the time I had :/
If my test (just copy it temporarily from https://github.com/zond/go/blob/master/codec/time_test.go if you don't want it in the repo) runs I should be fine.
from go.
Yes, I understand.
I tested with the code you put in the issue description, which is basically same as your time_test.go. So I think it's fine.
I'd do a push within the next few days. I want to tackle the nil slices/maps/pointers-to-structs thing which you alluded to in the other issue and push solutions to both together.
Thanks for your patience.
from go.
Great!
Thanks so much for looking into this - I really enjoy the possibility to have a compact and fast codec that doesn't assume stream processing (like gob).
I'm looking forward to the other solution as well, but don't sweat it - until you have your commits done I will just use github.com/zond/go/codec - the beauty of go dependency management is sweet :)
from go.
This should be fixed now with the latest commit.
Thanks.
from go.
Related Issues (20)
- codec.MsgpackHandle.RawToString is confusingly named HOT 2
- Go 1.20 - invalid memory address or nil pointer dereference HOT 5
- Msgpack deserialization with codecgen produces wrong result HOT 3
- Unexpected encoding with interface{} key using Canonical and MapKeyAsString.
- Decoding a slice of structs encoded as arrays into a slice of structs without the same number of fields HOT 2
- Duplicate key inside of a map triggers a "cannot decode unsigned integer" msgpack decode error HOT 1
- Efficient direct conversion between supported formats? HOT 2
- Expose Handle.Name() in some form through Encoder and Decoder for use by codec.Selfer HOT 1
- `io.ErrUnexpectedEOF` should be returned instead of `io.EOF` when only partial data is present
- Add `DecodeOptions.IntType` and `DecodeOptions.FloatType` ? HOT 1
- Docs do not seem to mention case-sensitive field handling
- mem leak? HOT 1
- Tests use rand.Intn(1) for boolean rand, but always returns 0
- Decoder accepts indefinite-length CBOR strings containing chunks with mismatched major types HOT 1
- UTF-8 validation isn't applied to CBOR text strings nested inside an indefinite-length text string HOT 2
- Add support for github.com/go-json-experiment/json / UnmarshalerV2 ? HOT 1
- runtime error with golang master (furture go 1.22)
- Leading zeros is not allowed in rfc7159
- attr.go
- [Question] Big Endian architecture
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from go.