Comments (8)
Hello etcimon,
The choice of GPL-3 was deliberate and before I abandoned the repository, the plan was to provide a shared library build that closed-source software could link to at the cost of losing CTFE functionality. I'm open to you taking over the repository and changing the licensing to your liking though, if that is an option, as I wont continue its development.
from fast.
I think that would be nice, if it can be MIT licensed. I think currently the more popular json serialization libraries in D are asdf and in vibe.d, but for high performance big data operations this is definitely the go-to. There's a lot of value in the processor-level optimizations you found that makes this library so great, I think the only hickup is the ~"000"
padding that triggers a GC allocation which could be solved by providing the length beforehand so an array resize can be done. Anyways, if you intend on letting me manage the repository I would merge a few of my changes myself and play around with the programming language benchmarks the way I still need to do with the BigInt in Botan. Mainly, it would save me some work in libwasm because my other option is to integrate mir-ion which has a lot of dependencies to adjust to betterC
Thanks
from fast.
If you rename your fork I can transfer it directly to your account and turn this into an automatic redirect.
from fast.
Ok it's renamed
from fast.
GitHub still complains. Apparently you cannot even have a secondary fork.
etcimon already has a repository in the mleise/fast network
Is it feasible for you to delete your fork from GH and merge it back in when the transfer is complete?
About the GC allocation, there were two major implications. I wanted to be able to return proper immutable string
s from the JSON without reallocation in most cases. So when the constructor is called with a const(char)[]
, a side effect of the potential GC reallocation (only if the capacity was too small) is that it turns the const(char)[] into a unique new string
and string returning functions can just return a slice of it from there on. That's what the m_isString
flag keeps track of.
On the CPU side, there is also a version of the string scan functionality that takes an explicit length and works without appending zeroes. It was 2-3 times slower on my Haswell CPU though IIRC, so that wasn't an option.
But you can go full @nogc
by manually preparing the input char[]
with the 16 zeroes padding and setting the simdPrep
flag in the constructor to No. Then, whenever you need to read a string, use borrowString()
instead. This will either return a slice of the original text or - in case there were escape sequences that had to be converted - a slice into a temporary buffer that will be overwritten next time (so make a copy if you need to keep it around).
from fast.
It should be deleted now.
I was thinking of using this to estimate the serialization length for numbers a little faster:
https://github.com/ssvb/speedy-stdio/blob/main/speedy/stdio.d#L241
And this part is a CT conversion table for numbers:
https://github.com/ssvb/speedy-stdio/blob/main/speedy/stdio.d#L272
Perhaps that would speed it up further.
from fast.
Pretty cool bit hack! I have no idea what's going on, but that's usually expected until you run them step by step. ;-)
I already initiated the repo transfer by the way.
from fast.
When it comes to huge CTFE loops like the conversion table, I used to prefer code generators, because there was no way to cache the result and avoid the compile time hit. In any case, it will be interesting to see how the lookup for batches of 4 digits compares to the traditional loop that converts one digit at a time.
from fast.
Related Issues (18)
- Last commit missing fast/buffer.d? HOT 2
- parse stream? HOT 1
- fast.math: Is inline assembler really faster than compiler builtin? HOT 1
- failure on os x HOT 4
- How to use fast's jsonparser HOT 7
- build break on dmd v2.092.0 HOT 1
- how to read single value of a key? /fast/json.d(1208): Assertion failure HOT 1
- Pull out custom File from fast.json? HOT 1
- Accurate number parsing HOT 1
- Consider Boost License HOT 3
- No support for Apple Silicon and fails on virtualized x86_64 (EC2) HOT 1
- More examples in read me please HOT 1
- json: C strings concept is not valid HOT 5
- json: wrong utf validation HOT 1
- Please drop me an email
- Missed optimization opportunity
- GDC compile failure (gdc-5.2.0-1 arch linux)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fast.