Concurrency
At the moment my main problem is the way the bot runs.
it starts streaming HTTP messages, which sometimes report some events like new challenge, game start, etc.
Start point is at state.rs
start()
function.
In order to be able to have multiple threads spawn and access/modify the same resources, I've used Arc<Mutex>. When stream_incoming_events
receives something, it clones the whole bot struct and lets a thread do something with it.
The main 2 points are event_stream_handler()
and game_stream_handler()
.
Now it bottlenecks when several threads want to play on games. I've done this ugly src/bot/state.rs:35
:
// TODO: Find if we can improve this.
pub games: Arc<Mutex<Vec<Arc<Mutex<BotGame>>>>>,
I've let the bot accept up to 3 games concurrently, but I see it times out on games whenever the time control is slow because it plays on 1 game at at time really, where I wish it could just spawn threads and do it concurrently for each game. There is one engine instance per game, and the engine need to be "blocking" running the go() function when it searches.
When trying to play on games : play_on_game()
at src/bot/state.rs:437
: typically we get a mutable reference to a bot game, and it blocks the other games.
Perhaps one way to solve it is to do like in engine_go_and_stop()
in src/chess/engine/test/engine_test.rs
, and then just monitor when the thread handle is_finished, so we do not need to hold a mutable reference and block the rest of the games.
Ideally, I would like to have another layout where I can concurrently block individual mutable references of a Vec/Slice individually.
Perhaps I could Box a slice/array of games pre-allocated to work around this.
Memory consumption
The bot seems to keep memory allocated after games, sometimes.
If I just fire it up and play games, memory allocated by the bot gets released at the end of the game. All good.
But if it plays several days, I see that sometimes it just keeps like 25 GB of RAM allocated. (I configured the cache size that a game should not use much more than 2GB)
It looks like the Drop is being called and our list of games from the bot state is empty (vector length = 0), but it kind of looks like that the engine cache tables (see EngineCache in src/chess/engine/cache/engine_cache.rs
is not being freed)
If I run tools like heaptrack, it tells me that the only leaked memory is from reqwest related to SSL stuff. I have no idea why this happens.
Benches
I've set some benches to try to see how fast we can iterate position and assess them. I used the divan crate because it looked nicely suited.
For example, I get the following output:
Running benches/chess_library.rs (target/release/deps/chess_library-73fdce3e15a9c9b3)
Pinned thread to core 0
chess_library fastest โ slowest โ median โ mean โ samples โ iters
โโ apply_moves_on_a_game_state 135 ns โ 2.082 ยตs โ 180.1 ns โ 197.9 ns โ 10000 โ 80000
โโ apply_moves_on_the_board 92.45 ns โ 2.794 ยตs โ 114.9 ns โ 117.4 ns โ 10000 โ 160000
โโ compute_legal_moves 368 ns โ 3.804 ยตs โ 543.2 ns โ 546 ns โ 10000 โ 40000
โโ determine_board_pins 2.15 ns โ 20.2 ns โ 2.327 ns โ 2.333 ns โ 10000 โ 10240000
โฐโ find_attackers 6.709 ns โ 30.97 ns โ 6.769 ns โ 6.817 ns โ 10000 โ 5120000
Running benches/engine.rs (target/release/deps/engine-4ba7a83b91aa9c70)
Pinned thread to core 0
engine fastest โ slowest โ median โ mean โ samples โ iters
โโ board_evaluation 429.7 ns โ 5.77 ยตs โ 440.7 ns โ 449.9 ns โ 10000 โ 10000
โโ board_evaluation_2 341.6 ns โ 2.226 ยตs โ 348 ns โ 350.1 ns โ 10000 โ 80000
โโ board_generic_evaluation 386.6 ns โ 3.174 ยตs โ 390.5 ns โ 393.8 ns โ 10000 โ 80000
โโ board_material_score 9.306 ns โ 60.61 ns โ 9.388 ns โ 9.463 ns โ 10000 โ 2560000
โโ cache_for_evals 10.91 ns โ 93.49 ns โ 11.38 ns โ 11.44 ns โ 10000 โ 2560000
โโ cache_for_game_status 9.974 ns โ 75.29 ns โ 10.59 ns โ 10.63 ns โ 10000 โ 2560000
โโ cache_for_moves 51.41 ns โ 452 ns โ 52.04 ns โ 52.46 ns โ 10000 โ 640000
โโ compute_game_phase 1.418 ns โ 8.056 ns โ 1.423 ns โ 1.431 ns โ 10000 โ 20480000
โโ detect_board_game_over 49.76 ns โ 3.837 ยตs โ 49.76 ns โ 52.01 ns โ 10000 โ 10000
โโ endgame_evaluation 349.7 ns โ 5.069 ยตs โ 360.7 ns โ 365 ns โ 10000 โ 10000
โโ endgame_piece_square_table_lookup 40.29 ns โ 412.5 ns โ 41.4 ns โ 42.45 ns โ 10000 โ 640000
โโ file_half_open_detection 0.943 ns โ 8.707 ns โ 0.948 ns โ 0.954 ns โ 10000 โ 20480000
โโ file_open_detection 1.183 ns โ 11.52 ns โ 1.183 ns โ 1.194 ns โ 10000 โ 20480000
โโ file_state_detection 1.031 ns โ 13.33 ns โ 1.046 ns โ 1.057 ns โ 10000 โ 20480000
โโ middlegame_evaluation 420.7 ns โ 23.67 ยตs โ 430.7 ns โ 439.4 ns โ 10000 โ 10000
โโ middlegame_piece_square_table_lookup 38.1 ns โ 338.5 ns โ 39.83 ns โ 40.12 ns โ 10000 โ 640000
โโ nnue_board_evaluation 36.33 ยตs โ 364.4 ยตs โ 62.8 ยตs โ 62.19 ยตs โ 10000 โ 10000
โโ nnue_input_layer_conversion 169.4 ns โ 3.121 ยตs โ 172.5 ns โ 175.1 ns โ 10000 โ 160000
โโ opening_evaluation 409.7 ns โ 6.952 ยตs โ 420.7 ns โ 427.8 ns โ 10000 โ 10000
โโ opening_piece_square_table_lookup 32.32 ns โ 433.6 ns โ 36.08 ns โ 36.54 ns โ 10000 โ 640000
โฐโ passed_pawn_detection 6.627 ns โ 69.93 ns โ 6.724 ns โ 6.77 ns โ 10000 โ 5120000
I tend to run release because the optimization level really makes a difference on the search speed. I am not sure if I set my benchs work, but I know that a lot of speed can be gained from pre-computing all possiblities and then make a lookup in an array rather than computing stuff manually... Like e.g. Rook moves. If we start here: src/chess/model/tables/rook_destinations.rs
, I have this 2-dimensional array: ROOK_DESTINATION_TABLE
We initialize it once with the values, then anytime we need to compute where the rook can move from a position we make a lookup, see get_rook_destinations()
at rook_destinations.rs:340
If we do
unsafe {ROOK_DESTINATION_TABLE
.get_unchecked(square)
.get_unchecked(blockers_key)
& !same_side_pieces }
Instead of
ROOK_DESTINATION_TABLE[square][blockers_key] & !same_side_pieces
I can see a speed increase. Great ! However, if I start replacing array lookup with .get_unchecked everywhere, it does not always speed up, sometimes get_unchecked is slower... I think it has to do with de-referencing, but I am not sure and it's kind of frustrating not to be able to predict.
Perhaps it is also that I did not set my benchmarks very nicely
Multi-threaded search
I think one of the next thing I'll try to implement is to have the bot search function to be multi-threaded.
I've read that the most efficient way to do this is to implement it like this: The APHID Parallel alpha beta Search Algorithm
So here I would have the search function (src/chess/engine/mod.rs:943
), which is recursive, that would decide at certain depth would split the search into N threads. (N being a pre-configured number)
They need to share an alpha/beta table that they read and update regularly. I am guessing to implement it a little like the engine cache, which is defined here: src/chess/engine/cache/engine_cache.rs
Any input on how you would do it ?