taylorhansen / pokemonshowdown-ai Goto Github PK
View Code? Open in Web Editor NEWReinforcement learning for Pokemon.
License: MIT License
Reinforcement learning for Pokemon.
License: MIT License
A couple major statuses are temporary or have an effect that depends on the amount of turns the pokemon was active:
encodePokemon()
's status field.|-setboost|POKEMON|STAT|AMOUNT
|-swapboost|SOURCE|TARGET|STATS
|-invertboost|POKEMON
|-clearboost|POKEMON
|-clearallboost|
|-clearpositiveboost|TARGET|POKEMON|EFFECT
|-clearnegativeboost|POKEMON
|-copyboost|SOURCE|TARGET
About a month ago |callback|
messages were replaced with normal |error|
's, followed by a |request|
if previously unknown info (trapping ability, imprison, etc) was the cause.
Reflect this change in PSBattle (replace #callback()
with #error()
) and anywhere else that had to do with |callback|
messages.
In handling the error messages, the response depends on the reason:
[Invalid choice]
: Eliminate last choice and re-choose.[Unavailable choice]
: Compare last request with the next one to see what changed and act accordingly. E.g. if trapped
is true we can infer a trapping ability.Sample:
|move|p1a: Wigglytuff|Wish|p1a: Wigglytuff
next turn...
|-heal|p1a: Wigglytuff|93/100|[from] move: Wish|[wisher] Wigglytuff
|upkeep
|turn|4
Whenever Wish is used successfully, set a boolean in TeamStatus, which should be reset the following turn.
Gen>=5: Also remember the user, since the user's max HP affects how much is restored rather than the recipient's
Move these methods into a separate file in the ai/
folder, since that's where they're actually used.
Sample messages:
|move|p1a: Lileep|Ingrain|p1a: Lileep
|-start|p1a: Lileep|move: Ingrain
Should be more transparent.
The move Hidden Power has a different base power (gen<6) and type based on the user's IVs.
Listen for |-resisted|
, |-immune|
, and |-supereffective|
messages after a |move|
was processed to narrow it down.
One thing to watch out for is that some moves and abilities temporarily change the type of a pokemon or certain types of moves (e.g. Soak and Pixilate respectively).
Some moves hit the target two turns after it was used.
Sample:
|move|p2a: Jirachi|Future Sight|p1a: Alakazam
|-start|p2a: Jirachi|Future Sight
...
|
|-end|p1a: Alakazam|move: Future Sight
|-damage|p1a: Alakazam|189/251
|upkeep
|turn|5
scripts/build-dex.ts
to provide functionality to encode the move name.isFutureMove
set to true
.I've been writing // tslint:disable-next-line:no-unused-expression
in test code too many times now.
Take a more functional approach to the MessageParser. This should allow for some useful refactors, reduce code duplication in error logging, and make extending and unit testing the parser much easier.
Sample messages:
|move|p1a: Blissey|Substitute|p1a: Blissey
|-start|p1a: Blissey|Substitute
...
|move|p2a: Chansey|Seismic Toss|p1a: Blissey
|-activate|p1a: Blissey|Substitute|[damage]
...
|move|p2a: Chansey|Seismic Toss|p1a: Blissey
|-end|p1a: Blissey|Substitute
Since stat numbers aren't known, this can be represented by a boolean for now.
When a pokemon uses the Transform move, it takes on some features (and volatile statuses) of its target while it's out.
Sample:
|move|p1a: Ditto|Transform|p2a: Magikarp
|-transform|p1a: Ditto|p2a: Magikarp
|-endability|p1a: Ditto|Limber|[from] move: Transform
When a |-transform|
message is encountered (and the pokemon is still out afterwards), the last |request|
message will provide info on the characteristics that were taken from the opponent.
These can also be used to infer characteristics of the opponent like stats, moveset, hidden power type (gen<5), and ability. All effects are reverted when switching out.
Add (or utilize) some VolatileStatus fields to include overridden species, ability, moveset, types, etc.
The overrideAbility
field and its usage in the parent Pokemon object can be used as a model. Moves are set to 5 pp remaining out of the target's maximum.
For example, in a scenario where an opponent Pokemon has revealed a single move that was just used until depleted, then uses Struggle, one or both of the following inferences can be made, in order of plausibility:
To be completely correct, both scenarios must be considered, discarding one or the other once further information is revealed later. This can be done by managing an array of BattleState
s instead of just one, using an operation similar to Array#flatMap()
to handle diverging ambiguous states when needed and discarding invalid states when a mutator throws.
Creating multiple BattleState
clones can be inefficient (and it would be even more painful to read a log bloated with BattleState#toString()
s), especially when other possibilities aren't even likely to be considered, so the class should be restructured to be more of a persistent data structure to deal with that. This would dramatically change the interface between PSEventHandler
and other classes that mutate the BattleState
, but could help contain all the game logic that happens in its individual mutator methods.
TODO: look into other options.
prefix()
method that returns a new Logger that appends a prefix to every message it sends. This way it's easier to tell where a log message came from.undefined
would mute certain message types.Refactor the BattleAgent
interface to only have a #decide()
method. This way it doesn't need to keep state and can be reused for multiple battles at the same time. A couple things to keep in mind though:
#acceptChoice()
, Network#decision
, etc., implement #42 first to eliminate most of the state that the Network
needs.Network.loadNetwork()
return a Promise<BattleAgent>
rather than its constructor.PSBot#formats
to use BattleAgent
instead of BattleAgentCtor
.BattleAgentCtor
interface altogether.Sample messages:
|move|p2a: Magnezone|Magnet Rise|p2a: Magnezone
|-start|p2a: Magnezone|Magnet Rise
...
|-end|p2a: Magnezone|Magnet Rise
Gen>=5: Can also be removed by Smack Down (but the |-end|
message won't be sent).
Gastro acid should be baton passable and should be removed if passed onto a Multitype pokemon.
When a pokemon is not grounded, certain things can be prevented from happening. This can happen due to an item, ability, field condition, volatile status, or move. Track all of these carefully and add a helper property Pokemon.grounded
to check this. May also be useful to add this to an active pokemon's array data when being processed by the neural network.
Add a method to PSBot (e.g. #login()
) that takes in a username (and optional password and/or avatar id) and attempts to log in to the official servers.
Add the above three fields to src/config.ts
and copy that file to src/config.example.ts
. The original config file will be .gitignore
'd and imported by src/index.ts
to be used when logging in on connect. Keep in mind that the string from the last |challstr|
message also has to be used.
Better naming.
Sample messages:
|move|p1a: Sableye|Taunt|p2a: Ninetales
|-start|p2a: Ninetales|move: Taunt
...(3-5 turns)
|-end|p2a: Ninetales|move: Taunt
Less ambiguity in editor tabs.
E.g. an Arcanine can have either Flash Fire or Intimidate. When the species is revealed, the possible abilities it can have should be looked up and a boolean should be set in the array data sent to the neural network, indicating that it can have either of the two abilities. These booleans should be translated into the reciprocal of the amount of true values so that it's not mistaken for having both at once.
If something happens where both sides of the Battle have not sent any messages to the BattleStream and are instead waiting to read from it, this causes the Promise.all
statement in the play()
function to never resolve, causing an immediate termination of the process. Either figure out a way to detect and handle this, or just cover all corner cases where this may happen. Doing both would be preferable here.
Sample:
|move|p2a: Forretress|Spikes|p1a: Nidoking
|-sidestart|p1: taylor108|Spikes
A different format for referencing a player's team is being used here, which will probably be used to handle other team statuses.
Each entry hazard type should have a counter to indicate how many layers of that hazard are stacked.
The move will |-fail|
if no |-sidestart|
message is parsed.
Not sure how hazard removal would work.
Likely it would be using a similar |-sideend|
message, but experimental logs should be gathered first.
Each species/form has a defined type, which can be temporarily changed while active through the following messages:
|-start|<id>|typechange|<Type>
|-start|<id>|typechange|<Type>/<Type>
|-start|<id>|typeadd|<Type>
Where <Type>
is a type.
The typeadd
argument sets a temporary third type in VolatileStatus called the added type.
The typechange
arguments should override the two primary types and reset the added type in the VolatileStatus object (if the second type is unspecified, assume typeless, i.e. ???
), similar to how overrideAbility
was done.
|request|
message as a sanity check where possible.Certain abilities keep the opponent from being able to switch, but is only partially revealed when the opponent tries to switch out.
Sample:
<< |request|{...(in RequestPokemon obj: maybeTrapped=true)}
>> /choose switch 2
<< |callback|trapped|0
<< |error|[Invalid choice] Can't switch: The active Pokémon is trapped
Parse the |callback|
message and send it to the Battle instance so it can use that info to send a new Choice. The network won't need to know whether its choices are restricted, since it already ranks all of its available choices.
Adding onto #5, which should be implemented first before this is considered.
When a |callback|trapped|0
message is encountered, this usually means that the opponent's pokemon has a trapping ability (there are only three so this can be hardcoded for now). The Battle instance should call a method in its EventProcessor field to store that data in the BattleState.
Gather info on how to parse and handle |detailschange|
and |formechange|
messages.
Some changes are temporary (e.g. Castform) while others can be permanent (e.g. freezing a Shaymin-Sky).
Currently, Network
s create Decision
objects whenever it makes a decision. This is only necessary during training, so the code that currently pertains to that should be removed.
Instead, in train.ts
, provide a means to subclass into a TrainingNetwork
that's able to properly create those Decision
objects with the help of having full control over the simulation in the play()
function.
The additional control can also be used to hook into events and replace the purpose of BattleAgent#acceptChoice()
, #onFaint()
, and #onTurn()
, as well as moving the RewardTracker
class into scripts/
and not as a field of Network
. As such, the BattleState#agent
field will be useless so that can be removed.
Sample:
|cant|p2a: Slaking|ability: Truant
The above message replaces every other move made by the pokemon that has the ability. Track this in VolatileStatus so we know when this ability will activate.
Activation of this ability also resets multiturn effects like |-mustrecharge|
, |prepare|
, lockedmove
, stalling moves, etc.
https://github.com/CrazyGuy108/pokemonshowdown-ai/blob/490ba3cd7bf2d641e6fe7814ea05e7fef4c5b6ef/src/bot/battle/Battle.ts#L73-L90
On line 88, postAction() is supposed to setup everything for the next turn. However, there are two scenarios where this could be called prematurely:
In both cases, the AI would have to make a second choice based on the postAction-changed data, which was meant to be called afterwards.
First, rename postAction()
back to postTurn()
to prevent confusion, since this function is supposed to handle per-turn statuses.
After that, in the battleprogress
handler, before handling new events, call postTurn()
if the events that were handled previously contained a |turn|
event. This can be done by exposing a getter to EventProcessor's newTurn
field.
Some statuses span multiple turns, and there are no in-game indicators to know when these statuses will take effect or disappear. Correct the behavior and usage of PSEventHandler#postTurn()
to properly handle all the applicable statuses.
Sample message:
|-enditem|p1a: Dialga|Lum Berry|[eat]
Since items can be gained and removed all the time, the current model for PossibilityClass won't work for Pokemon#item
. A subclass or a PossibilityClass#reset()
method should do, which should reset the object to when it was first constructed.
Item-related things that need to be handled specially:
[from] stealeat
suffix in |-enditem|
event)[eat]
suffix)lastItem
field when it can be recycled, which persists through switch. The field should be a PossibilityClass like the regular item
field since some items are consumed silently (e.g. via Natural Gift) and thus can't be revealed.|-enditem|
messages count for this, but the item-disabling Knock Off ([from] move: Knock Off
) as well as moves that transfer items, like Thief, do not count. These cases must be filtered out before determining whether to set lastItem
.unburden: boolean
in VolatileStatus that will be set accordingly.Moves that cause the lockedmove
volatile status should also activate a turn counter to determine when that status will end with confusion fatigue.
They last 2-3 turns, so that means the 2nd turn has a half chance of stopping afterwards and the 3rd turn will always stop.
It will also stop if the move fails somehow, e.g. by an immunity or a move like Protect/Disable/Imprison/Torment.
|-weather|
events.Example messages:
On move:
|move|p1a: Sunflora|Sunny Day|p1a: Sunflora
|-weather|SunnyDay
On switch:
|switch|p1a: Groudon|Groudon, L71|100/100
|-weather|SunnyDay|[from] ability: Drought|[of] p1a: Groudon
End of turn:
|-weather|SunnyDay|[upkeep]
none
(default), SunnyDay
, RainDance
, Sandstorm
, and Hail
.In order to login to an account, the PSBot makes some HTTP requests to https://play.pokemonshowdown.com/~~showdown/action.php
(assuming default config). In order to test that this works, replace the domain with a local server instance. This will require either a mock server or use of the official source for setting up a local server that serves a custom client for handling logins.
To make tests more consistent, replace init-ps.sh
(which always uses the latest commit), with an actual git submodule
or subtree
(which uses a specific commit). The submodule can be placed in the root directory of the project rather than in scripts/
.
When training, target tensors are calculated during play and added to an array. This leads to old predictions being used to train a network that has already gone through an epoch. Instead of Decisions, store Experience objects which contain state, action, reward, and next state, and calculate target Q-values the moment they're needed.
Since Team objects represent unrevealed Pokemon as just null
, the species' PossibilityClass object will only be narrowed when the Pokemon is revealed, in which case it will be fully narrowed. Add species: string
to the Pokemon constructor and use that instead of a PossibilityClass.
Also, Pokemon#data
can be the new species field. May want to rename PokemonData#species
to name
so the usage looks like mon.species.name
instead of mon.species.species
.
Sample messages:
|move|p1a: Pidgeot|Roost|p1a: Pidgeot
|-heal|p1a: Pidgeot|286/307
|-singleturn|p1a: Pidgeot|move: Roost
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.