Comments (8)
warpfork
commented on August 24, 2024
That sounds really weird. I'll take a look. FWIW, there are no mutexes at all in refmt -- I've intentionally kept them out, with the idea that if the application needs them, then everything can be sync.Pool'd at higher level and that should solve stuff -- so concurrency should be pretty transparent...
from refmt.
tobowers
commented on August 24, 2024
I'm not a go-internals expert, but I was wondering if it was related to interfaces and reflection... and/or use of channels... but I'm not sure.
from refmt.
warpfork
commented on August 24, 2024
I took your gist and ran with it a bit, and came up with this: https://gist.github.com/warpfork/ac63075a5d2996e011822423db7e2b1c
I don't think the parallelism is the problem here.
A reproducer with some of those more complicated objects might be worth trying to fabricate. And that's something I'd really like to merge more of also; the current benchmarks in the refmt repo are a little too narrow in what they exercise.
from refmt.
tobowers
commented on August 24, 2024
Ok. Thanks for looking! Let me take my real-world app and see if I can't make a simpler repo that's not as simple as that :)
from refmt.
tobowers
commented on August 24, 2024
I know this is way too complicated for a repro... but it's what I have at hand that led me down this path... maybe it's only about go context switching... but: https://github.com/QuorumControl/encodeexplore
That will produce output for how long each decode took (encode doesn't seem to suffer the same problem).
There are occasional huge spikes:
INFO [07-17|14:20:27.608] decode took=2.12226ms
INFO [07-17|14:20:27.608] decode took=2.810995ms
INFO [07-17|14:20:27.608] decode took=2.108842ms
INFO [07-17|14:20:27.609] decode took=2.153836ms
INFO [07-17|14:20:27.609] decode took=780.894544ms
INFO [07-17|14:20:27.610] decode took=2.485473ms
INFO [07-17|14:20:27.610] decode took=1.997425ms
INFO [07-17|14:20:27.611] decode took=2.244557ms
INFO [07-17|14:20:27.611] decode took=4.895766ms
INFO [07-17|14:20:27.611] decode took=2.243713ms
INFO [07-17|14:20:27.611] decode took=4.29352ms
INFO [07-17|14:20:27.611] decode took=2.551357ms
INFO [07-17|14:20:27.612] decode took=2.30946ms
INFO [07-17|14:20:27.613] decode took=3.84674ms
INFO [07-17|14:20:27.613] decode took=2.265534ms
INFO [07-17|14:20:27.613] decode took=2.205228ms
INFO [07-17|14:20:27.613] decode took=2.192477ms
INFO [07-17|14:20:27.613] decode took=2.282205ms
INFO [07-17|14:20:27.615] decode took=2.228272ms
INFO [07-17|14:20:27.615] decode took=4.539288ms
INFO [07-17|14:20:27.615] decode took=2.551364ms
INFO [07-17|14:20:27.615] decode took=3.87301ms
INFO [07-17|14:20:27.615] decode took=2.228045ms
INFO [07-17|14:20:27.616] decode took=2.411149ms
INFO [07-17|14:20:27.616] decode took=2.268814ms
INFO [07-17|14:20:27.616] decode took=2.295646ms
INFO [07-17|14:20:27.617] decode took=762.974013ms
INFO [07-17|14:20:27.617] decode took=791.030098ms
from refmt.
tobowers
commented on August 24, 2024
FWIW - switching the call to a time.Sleep(2 * time.Milisecond) does not have wild swings like that.
from refmt.
warpfork
commented on August 24, 2024
Okay, I definitely have something to keep investigating there 🤔
from refmt.
tobowers
commented on August 24, 2024
I know that's not ideal code to work with and it's nested go routines... let me know how I can help.
from refmt.
Related Issues (20)
- Things fail with weird errors if your atlas doesnt know about all your types
- cbor output is not canonical HOT 7
- Is it possible to defer unmarshaling? HOT 1
- [cbor] Unable to roundtrip map[string]*CustomStruct HOT 2
- Panics and OOMs in cbor decoding HOT 8
- Option to encode nil slice/map as []/{}
- The handling of nil byte slices is different than the handling of other types of nil slices HOT 2
- Allow pointers when using UseTag HOT 3
- [cbor] Unable to use numbers as map keys HOT 5
- Using a transform function to/from an interface cause a nil pointer dereference
- Can't round trip nil slice HOT 1
- Encoding zero values as null HOT 5
- Fails to decode a valid CBOR object HOT 2
- Remove random binaries from git repo HOT 1
- Support for arbitrary structs without explicitly registering them HOT 2
- Possible to create my own raw marshal/unmarshal? HOT 6
- Support skipping unknown fields
- No go.mod, so modern idioms for building and testing fail?
- .gitmodules has extra trailing slashes that break "git submodule update --init"
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from refmt.