bcollazo / catanatron Goto Github PK

It would be nice to have algorithms so that players can keep track of enemy cards. This should make bots stronger.

It should be parameterized say by "CARDS_IN_MIND" so that we can play with how "strong" the inference is and how does it affect game play. I myself can't memorize too many cards on enemy hands. :)

I am not completely sure that if a player ties for longest road, the code correctly doesn't award the 2 victory points, but only does so when the player surpasses longest road by at least 1.

The node production for a spot that has two tiles with the same resource, will only have the production from one of those tiles.

Doing a copy.deepcopy of the complete game state is expensive. If we could find a way to either copy game state faster, or somehow use a fast immutable data structure for game state; or have game-state be c-powered numpy arrays, it might allow us to compute "later-in-time" calculations (explore the decision tree) faster.

Seems like this would always be a bad move.

Right now, this button appears as soon as you buy a dev card. It should appear only when one is playable, and a card is not playable if it was just bought or if the player already has played a dev card this turn.

If I try to run catanatron-play -players=VP,AB --num=2 -o sims --csv I get the following errors, which I believe are due to the cli only picking up one player code:

(venv) paul@DESKTOP-65EVJAG:~/environments/catanatron$ catanatron-play -players=VP,AB --num=2 -o sims --csv
Playing 2 games...                                            ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% -:--:--
AlphaBetaPlayer:BLUE(depth=2,value_fn=base_fn,prunning=False) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0%
Traceback (most recent call last):
  File "/home/paul/environments/catanatron/venv/bin/catanatron-play", line 11, in <module>
    load_entry_point('catanatron-experimental', 'console_scripts', 'catanatron-play')()
  File "/home/paul/environments/catanatron/venv/lib/python3.8/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/home/paul/environments/catanatron/venv/lib/python3.8/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/home/paul/environments/catanatron/venv/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/paul/environments/catanatron/venv/lib/python3.8/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/paul/environments/catanatron/catanatron_experimental/catanatron_experimental/play.py", line 184, in simulate
    play_batch(
  File "/home/paul/environments/catanatron/catanatron_experimental/catanatron_experimental/play.py", line 314, in play_batch
    for i, game in enumerate(
  File "/home/paul/environments/catanatron/catanatron_experimental/catanatron_experimental/play.py", line 245, in play_batch_core
    game.play(accumulators)
  File "/home/paul/environments/catanatron/catanatron_core/catanatron/game.py", line 103, in play
    self.play_tick(decide_fn=decide_fn, accumulators=accumulators)
  File "/home/paul/environments/catanatron/catanatron_core/catanatron/game.py", line 124, in play_tick
    else player.decide(self, actions)
  File "/home/paul/environments/catanatron/catanatron_experimental/catanatron_experimental/machine_learning/players/minimax.py", line 70, in decide
    result = self.alphabeta(
  File "/home/paul/environments/catanatron/catanatron_experimental/catanatron_experimental/machine_learning/players/minimax.py", line 121, in alphabeta
    result = self.alphabeta(
  File "/home/paul/environments/catanatron/catanatron_experimental/catanatron_experimental/machine_learning/players/minimax.py", line 121, in alphabeta
    result = self.alphabeta(
  File "/home/paul/environments/catanatron/catanatron_experimental/catanatron_experimental/machine_learning/players/minimax.py", line 100, in alphabeta
    value = value_fn(game, self.color)
  File "/home/paul/environments/catanatron/catanatron_experimental/catanatron_experimental/machine_learning/players/value.py", line 63, in fn
    enemy_production = value_production(enemy_production_sample, "P1", False)
  File "/home/paul/environments/catanatron/catanatron_experimental/catanatron_experimental/machine_learning/players/value.py", line 133, in value_production
    prod_sum = sum([sample[f] for f in features])
  File "/home/paul/environments/catanatron/catanatron_experimental/catanatron_experimental/machine_learning/players/value.py", line 133, in <listcomp>
    prod_sum = sum([sample[f] for f in features])
KeyError: 'EFFECTIVE_P1_WHEAT_PRODUCTION'

If I try to run with 4 players, I only get 3 (and with 3, I get 2).

If I try to run with 5 players, I also get an out of bounds error which I assume is expected.

It seems using a pure-python dictionary-based data structure for graph operations makes catanatron simulate games faster than using the nxgraph library. nxgraph is mostly used today just to consult neighbors for a given node, and so building a lookup table at game initialization (or even at simulation initialization) could make catanatron faster.

Unsure what algorithm could handle the high-branching factor Catan actually has. Up to now, we have just been truncating the DISCARD possibilities. Can we overcome this?

Not sure a player can steal "largest army" by just tie-ing the number of knights. Should need 1 more than current largest army holder. This issue asks to write a test to ensure the code deals with this correctly.

Having issues with unpacking certain things when I try to just run the default example on the readme. I think it has to do with the change to gymnasium as well as the addition of "truncated". Would appreciate if you could look into it, thanks!

Right now, even using the --help flag takes a long time. I am assuming it's because we are eagerly importing libraries (for example, numpy, tensorflow) which may not be needed until the actual code is being executed.

This may be a symptom of a broader need for import / loading hygiene.

time catanatron-play --help gave 20s on my MacBook Pro:

Create an endpoint with which we can get the MCTS Score out of a given position.

This could be used to show in the UI a stacked bar with the probability of each player winning. We can improve the evaluator in the future.

For now, we can have it just simulate 100 Random Games from the given state and return the % of games each player won.

The endpoint probably looks like GET /games/:uid/states/:uid/mcts-score

Right now, this is a noop in the UI. Implement this. Should probably ask what card player wants to monopolize.

Still unsure how implementation should work, but it seems the tick_queue approach would allow for the trading conversation to happen (e.g. P0 offers X, P1 decides against, P1 counter-offers, P0 counter-offers). Important for Random Bots not to get stuck here.

To make the UI more expressive, it'd be nice if with every roll the tiles that yield play some sort of animation. It'd be nice if its either from https://www.framer.com/motion/ or https://react-spring.io/. A slight "shake" or opacity animation might be cool.

Create functionality to allow replaying a game from start to finish ala "chess.com". This will be super helpful for debugging bot decision logic and extracting specific middle-of-the-game game states for further analysis.

When running it on Collab, we get:

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
[<ipython-input-3-e16933b01ac9>](https://localhost:8080/#) in <module>
      1 from catanatron.json import GameEncoder
      2 from catanatron_gym.features import create_sample_vector, create_sample
----> 3 from catanatron_experimental.machine_learning.board_tensor_features import (
      4     create_board_tensor,
      5 )

ModuleNotFoundError: No module named 'catanatron_experimental.machine_learning.board_tensor_features'

---------------------------------------------------------------------------
NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.

To view examples of installing some common dependencies, click the
"Open Examples" button below.
---------------------------------------------------------------------------

Right now it seems tensorflow, numpy, and pandas are only needed for CsvDataAccumulator which seems to be a Machine Learning inclined feature. It would be nice if the catanatron could come with the simulator without having to include heavy dependencies like these (tensorflow, numpy, and pandas). Actually, maybe even without including rich or click.

This issue is to study and think about how could we structure the codebase so that people could use the simulator by just pip install catanatron and something like:

from catanatron import RandomPlayer, Color, play_batch

# Play a simple 4v4 game
players = [
    RandomPlayer(Color.RED),
    RandomPlayer(Color.BLUE),
    RandomPlayer(Color.WHITE),
    RandomPlayer(Color.ORANGE),
]
results = play_batch(100, players)  # simulates 100 games

or so.

Maybe have a catanatron_cli package that uses this core play_batch and includes the click and rich libraries to do its thing? Maybe another catanatron_ai that includes the ML-based features like the CsvDataAccumulator and the AI-based players that depend on tensorflow, numpy, and pandas?

This would help with slow startup time #208 and would make it easier to adopt in M1 machines where tensorflow is a little more involved to install. Open to thoughts.

It doesnt perform well. Speed up please!

I'm interested in generating player statistics randomly. It'd be much better for me to know the points of all players instead of the winner

Right now trade actions look like:

Action(Color.BLUE, ActionType.MARITIME_TRADE, ("WOOD", "WOOD", None, None, "SHEEP")) # 2 woods for a sheep
Action(Color.BLUE, ActionType.MARITIME_TRADE, ("BRICK", "BRICK", "BRICK", "BRICK", "WHEAT")) # 4 bricks for a wheat

Seems better to encode these as a union of two "freqdecks". This will make it more vector-friendl, potentially faster, and more interoperable with the rest of the code.

Action(Color.BLUE, ActionType.MARITIME_TRADE, (2,0,0,0,0,0,0,1,0,0)) # 2 woods for a sheep
Action(Color.BLUE, ActionType.MARITIME_TRADE, (0,4,0,0,0,0,0,0,1,0)) # 4 bricks for a wheat

Right now the UI skips the MOVE_ROBBER action by deciding on behalf of the human player. Allow for human players to select the tile where they want to move the robber.

Right now this is not implemented. If a player clicks on "Use > Year of Plenty" it does nothing.

Every now and then one gets a:

del max_paths_by_player[action.player.color]
KeyError: <Color.BLUE: 'BLUE'>

Right now, this button does nothing. Implement this. Should probably start highlighting roads player can build right after clicking?

Right now, clicking on "Use > Knight" does nothing.

Would be helpful to have the dots for the numbers in the UI so you can easily tell which are the higher probability rolls

Keeping track of your enemies' cards is usually helpful. Implementing this could:

• Make Catanatron stronger.
• Allow this project be part of https://github.com/google-deepmind/open_spiel (since for that we would have to be able to explicitly create "Chance Nodes" in the tree).
• Improve the fact that right now, the AlphaBetaPlayer when searching a tree might imagine impossible "stealing card" outcomes (see: https://github.com/bcollazo/catanatron/blob/master/catanatron_experimental/catanatron_experimental/machine_learning/players/tree_search_utils.py#L107), but with perfect card counting it sounds like this might not be needed (and the player can keep a perfect distribution of the chances of stealing a card from any given enemy).

Would be cool to have this "card counting capability" be parametrized by a number K that represents how many cards from its enemy can a player keep "in its mind". I myself as a player, can probably just keep the last 2 - 3 cards of my enemies hands 😅 .

There is a question whether to implement this as part of the Game, or as a capability in the Players. Intuitively, sounds like it should be the Player (since some may have the capabilities, but others not); but I am not sure how the API would like. Do we have to now feed every "tick" to every player(?) Do we make it so that the player has a pointer to the log of ticks/actions and knows how many it has "consumed", so that the next time it is its turn, he "updates" its card counting distribution/believe? In that later approach, we would have to make sure reading the log doesn't give the player more information than what he should have.

This sounds like could be developed as a fairly independent module (say counter = CardCounter(k=5) and counter.update(...)), that players may or may not use in their API.

I am not sure that is working correctly. Check and potentially solve.