tiloccas / computational-intelligence-2023-24 Goto Github PK

Hi pal! Just checked your code and I want to give you my 5 cents.

• The overall code is really clear and well written: the classes you defined make the code pretty straightforward.
• I appreciated the various attempt for different mutation / xover mechanisms, especially the 1 cut with inversion of position, being all the dimensions binary and maybe somehow position related. It was a quite fancy but powerful way to try the same pattern in different dimensions of your solution.
• The training algorithm was also efficient wrt the number of calls.
• Running the code multiple times I noticed high variability in the solution, by the way, so probably the whole procedure would benefit from higher exploration and a more loose terminal condition. For example, if you won't find any better for 1000 steps you could increase exploration, till you reach 10k: then you stop.
• Another improvement might be the random xover: useful for sure but you may improve it by passing a probability of using each method, such that you could easily tune their weight, depending on your problem.

I really hope I helped you somehow,
best regards,
Andrea Panuccio

Hi salvo,
I appreciated the Q-learning code that you wrote! 👏
I liked the fact that you considered the symmetry of the Tic Tac Toe board , the fact that you have evaluated different parameters of the Q-Learning agent and the fact that you evaluated the agent to play as second player.
A quick improvement could be training the agent also in the case in which it is starting second, this could improve the results if the agent is starting as second :).

I think the code is easily readable: it's written in a standard way, it's compact and the name of the variables helps in understand what the code is going to do.
I appreciate the fact that both the + and , strategies can be made Adaptive only by passing the sigma parameter or not, so the code it's very optimized.
In addition, the code is modular, thanks to the use of a relevant number of small functions that do a precise job while long functions are avoided.
The Readme file is clear, so it's easy to understand the strategy used, how they are implemented and how they have been tested.
One note I can make is that in the code is used predominantly the english for the name of the variables but some functions use names in italian, so I suggest to use the same language.

Hi Salvatore,
Kudos to you for your presentation and documentation, at first sight can be grasped the work you've put on it and it shows.
The Q-Learning algorithm you've implemented seems flawless, taking into account also the fact that we exploit the values learned while playing on the other side
The only thing I don't understand is why you chose to go through the hassle of writing everything up instead of using the Professor's Game and Player class which i believe would've made your life easier. Please be sure to do so on the project assignment!
I also appreciated your approach on managing symmetries, using a "canonical" representation of the board to index the Q-Learning Table
From what I grasped it also seems that you chose to use this approach also on testing, meaning that you really test only on canonical games, avoiding the need of "transparently playing a game as it was the canonical one"
To better explain myself:
If you had gone with the Game-Player class, you would have had to implement a move method which had to return a move based on a Game board, in that method (while testing) to exploit the canonical representation there is the need of translating the move from the canonical representation to the "original" one (as in the one passed to the function. This will come in handy for the project assignment, since it's structured similarly (though the symmetries will be a bit more)
I feel I explained it really badly but I don't find better words, so please ping me anytime if needed

To have better understanding of the results I'd suggest a percentage of not lost: ((win+draw)/total); and a visualization of these values based on player position (if playing as X or O)

Hi!
Your approach to the evolutionary algorithm really hits the mark. The early stopping in find_best_fitness is a smart play—it cuts down on unnecessary computation and gets you to the goal faster. Your choice of crossover functions is a game-changer. Mixing up one-cut, two-point, and uniform crossovers keeps things diverse, giving your algorithm a solid edge in exploring solutions. And the results speak for themselves! It's clear you've not only tackled the problem head-on but also added your own creative twist to it. This kind of thinking is what pushes the boundaries in problem-solving – it's not just about finding a solution, but finding it smartly and efficiently. Your work here is a great example of that.

Hi Salvatore, the code is well written and understandable at first sight, the README helps a lot too!
I see you've implemented multiple crossover strategies and that's nice. One thing that could improve your results is replacing the early stopping with extinction or maybe with some selection on the offspring (like ignoring the children that have the same genome of one individual in the population). I believe that these improvements could improve your exploration and to avoid a convergence (which i believe you encountered in problem instances > 2)

The code is well structured, the use of @DataClass also allows you to save in terms of writing code. The methods highlighted in the first part are also clear from the name given to the methods themselves. The code is very clear even if there are no additional comments. In the Mutation Crossover part, the methods introduced to develop the crossover are also clear in the writing of the code and introduce the variability required by the EA. The part relating to the implementation is written with the help of some comments that clarify implementation doubts and the graph shown allows us to graphically understand what has been written in the code. Furthermore, it also shows the results obtained and their progress.