One man's weekend project to scratch a very random programming itch.
Inspired by others' attempts at the 1 Billion Row Challenge, I thought I'd have a stab at it... in C...despite not being much of a C developer.
Anyway...
It works.
It's not the fastest by a sizable margin... but it works.
On my gaming rig it does it in ~200 seconds.
(No idea what that translates to on an officially sanctioned machine.)
No parallelism. Single core all the way.
Single pass parsing and a simple arena bump style allocator for memory management.
It swaps the cost of hashtables for the cost of a directed graph. Whether this is a net win or loss will come back to the architecture its running on.
Maybe.
The current approach wasn't built to be thread safe so it may involve some drastic surgery.
My allocator doesn't do any form of bucketing so any arena growth requires a malloc and a full mem copy from the older smaller instance. Plus pointers don't survive between resizes so coding style takes a hit.