We should start to think about other songs with 3 notes to add.

A good goal is 3 songs.

TODO: Compile a list of songs with 3 notes.

Add their corresponding classes.

We could add a difficulty level to the game.

Easy

• Shorter songs, and you only increment by 1 note at a time.

Medium

• Longer songs, and you initially increment by 1 note, then increment by 2 notes 1/3 of the way through, and 3 notes 2/3 of the way through.

Hard

• Same as medium, but the cubes no longer flash when the notes are played at the beginning of the round.

NOTE: These are rough ideas for difficulty levels. Please input your ideas and help me refine this.

This is a good issue to work on because we have to support a wide age range: Ages 3 - 6.

Currently if a player gets a correct sequence Cozmo always performs the MemoryMatchPlayerWinHand animation.

We should add in logic to perform the MemoryMatchPlayerWinHandLong animation when the player gets a long sequence correct.

See the docs for details:

http://cozmosdk.anki.com/docs/generated/cozmo.anim.html#cozmo.anim.Triggers.MemoryMatchPlayerWinHandLong

It would be simpler only complete one iteration of the game loop first.

Start with an arbitrary position, for example 3.

1. Play 3 notes of the song.
2. Wait for the player to match those 3 notes.
3. Wait for Cozmo to match those 3 notes.

If either the player or Cozmo gets the sequence wrong, then exit the game.

The interaction for playing an incorrect note is taken care of by issue #7 and can be completed later.

In large groups, playing the game can get boring for kid's watching.

We should add support for turn-based style gameplay with multiple players.

Here's what a round of gameplay would consist of:

1. A series of notes is played
2. Player 1 matches the notes
3. Player 2 matches the notes
4. Cozmo matches the notes

Then the next round would commence.

We should animate Cozmo when the game is over based upon who won the game.

See MemoryMatchCozmoWinGame and MemoryMatchPlayerWinGame.

He can also pop wheelies and roll cubes.

Within SongMatch is the following code:

    def __check_for_game_over(self) -> None:
        if self._num_player_wrong == MAX_STRIKES or self._num_cozmo_wrong == MAX_STRIKES:
            exit(0)

Right now, there's no logic in place for determining who won the game.

Currently we only support piano.

We could have guitar, saxophone, maybe even drums, etc.

Outstanding Questions

• What instruments do we want to support?
• Which instrument is used?
  • Does the player select an instrument in the beginning of the game?
  • Or does each song have a pre-configured instrument associated with it?

This might be a good task for the next group.

As a user, I want blocks to light up when I tap them, so that I get a visual indication when blocks are tapped.

Not all colors are created equal.
Some colors show up better than others when displayed on the cubes.
The colors that appear the brightest should be chosen to properly visually represent each note.
If possible, Red and Green should be excluded since they'll be reserved for representing the events associated with winning and losing.

Colors' brightness can be properly assessed by creating an RGB loop that shows every possible color variation for the cubes.

We should add the idea of duration to the Note class.

For example, half-notes, quarter notes, eighth notes, etc.

Then when we play the song sequence, we should respect each note's duration.

This will make the notes sound more like songs when we play the sequence.

TECHNICAL IMPLEMENTATION DETAILS

We have the following method in SongMatch

    async def __play_notes(self, notes: List[Note]) -> None:
        for note in notes:
            await self.__play_note(note)
            await sleep(TIME_BETWEEN_NOTES)

Instead of sleep()ing a fixed amount of time between each note, we would instead sleep for note.duration.

Also, the duration for each note should be relative to a single base duration.

For example, 1 second for quarter notes.

Then the duration for half notes would be calulcuated as 2 seconds.

Eighth notes are calculated as 0.5 seconds. Et cetera.

We need a way to tell the game how many players there are.

This could be by:

• Tapping a cube in a pre-game setup phase
• A voice command
• Python command line argument
• Desktop GUI window (I think I like this the best)

Several things need to be updated on the documentation site:

• Add information to introduction
  • Possibly add our game flow diagram to it.
• Change API to Song Match API
• Add Troubleshooting page
• Add "Future Plans" / "Moving Forward" page.
• Add Virtual Environments page and mention it in Setup
• Add "Codebase Conventions" page explaining some of the common conventions and design patterns found within the codebase.
• Add Dependencies page
• Mention how to find usages of functions within PyCharm
• Add page about package structure, and give overview of project
• Add page about how our documentation is built and setup

TROUBLESHOOTING PAGE
Mention replacing the batteries on the cube if you notice flickering or other weird behavior.

Probably more stuff I'm not thinking of.

@kb5 @bpowe1991 @rlwprog Any suggestions?

As a user, I want to emit a note when a block is tapped, so that I can play a song with Cozmo.

We need to add our names and User guide to the documentation site.

If a player or Cozmo taps the wrong cube, i.e. plays the wrong note, then the following should happen:

1. The cube should flash red.
2. Some sound should be emitted.
3. Cozmo should display some kind of animation.

This issue should implement a 3 strikes system. This means you can get 3 notes wrong before the game is over.

If you get a note wrong, then cozmo should start matching the notes.

Currently each cube is tied to a particular note and light.

Because of this, the cubes have to be in the right order when the game starts up.

This requires users to match the symbols to the corresponding spot on the cube mat.

It would be great to dynamically assign the cubes a note / color upon starting up the game.

This way when the game starts, it figures out what note / color should be played when the cube is tapped based upon their starting orientation.

As a user, I want the concept of a song, to be able to play a lot of different songs with Cozmo.

A song is a series of notes.

Each specific song could be a class, that inherits from a abstract base Song class.

Our init_game_loop(Song song) function could take in a song as an argument.

Here's what it might look like in Python code:

class MaryHadALittleLamb(Song):
    notes = [E, D, C, D, E, E, E, D, D, D, E, E, E, ..., E, D, C]

init_game_loop(MaryHadALittleLamb());

As a user, I want Cozmo to be able to play a note, so that the game is more engaging.

This would consist of Cozmo turning towards the appropriate block, raising and lowering his arm bar, and a note being emitted from the phone.

Right now the amount of players is designated during the invocation of main on the command line.

This makes it impossible for the user to choose how many players will play by clicking an icon and only can be done by starting the game from the command line.

This also prevents reevaluating how many players can play if songs are played back to back, necessitating a complete restart of the program.

Instead, the amount of players needs to be set with a song_match class method so that it can be instantiated before each song_match game is begun so that a new song can be played, potentially with a different amount of players.

As it stands, the user interface for each round, the game ending and checking who won only support either Cozmo and a single player.

If more than one player is playing with Cozmo, it will be hard to tell whose turn it is during each round.

If a player reaches three strikes during the game, that doesn't necessarily mean the game is over. This player should be skipped next round.

Checking who won as well as the subsequent reactions from Cozmo need to reflect how many strikes each person (and Cozmo) have when the game is finished.

Cozmo's chance for tapping the wrong note is too high.

This means he consistently doesn't make it to the end of the game and only last a few rounds.

We need to refactor how this works.

If Cozmo gets out in an early round, then two players get out in the same later round.

Cozmo says "... won. I Lost". He should probably say just "I lost."

Unless we want to rub in the fact that all the kids lost.

The banner and user guide should be updated with the name "Song Match" rather than "Cube Jam" since we're no longer pursuing the mini-game idea.

This involves replacing to files in the docs folder:

• https://github.com/gbroques/cozmo-song-match/blob/master/docs/_static/banner.png
• https://github.com/gbroques/cozmo-song-match/blob/master/docs/_static/cube_jam.pdf

Let's rename the PDF for consistency sake too.

To verify your changes locally before pushing, see the following guide:
https://github.com/gbroques/cozmo-song-match/tree/master/docs

Currently Mary had a little lamb has 25 notes. Starting with 3 notes, and incrementing by 1 note each round takes forever.

We have a few ideas on how to improve this:

• Increment by more than 1 note each round
  • This could make the game too difficult
• After a certain threshold, end the game and play the whole song
  • For example, after 10 notes of Mary had a little lamb, end the game and play the whole song.
• Shorten the songs
  • Have songs that are shorter like 10 - 15 notes in length. Play only the most memorable part.

Per @bpowe1991 comment:

• Increase the number of notes we increment by as the song progresses
  • For example. start by incrementing by 1 note, then 1/3 of the way through the song, increment by 2 notes, then 2/3 of the way through increment by 3 notes.

When a player taps a cube, Cozmo should turn towards the cube to make it seem like he's watching what the player does.

This will provide a more engaging experience.

Another idea is that Cozmo only turns to the cube on tap sometimes.

This would make Cozmo seem more life-like.

IMPLEMENTATION DETAILS

We should add a turn_to_cube(cube_id) method to the SongRobot class.

NOTE: This is different than Cozmo turning to TAP the cubes.

This issue closely relates to issue #22.

The following link may prove helpful:

http://cozmosdk.anki.com/docs/generated/cozmo.anim.html#cozmo.anim.Triggers.MemoryMatchCozmoFollowTapsSoundOnly

Right now when Cozmo plays a long series of notes he eventually falls out of alignment.

This could be due to using animation triggers, which does unpredictable animations, rather than a specific animation.

This would involve replacing the animation triggers with the animation names:

    # Source: SongRobot.py
    def __get_tap_animation(self, cube_id) -> Triggers:
        """Returns a tap animation based upon the current and previously tapped cubes."""
        key = (cube_id, self._prev_cube_id)
        return {
            (LightCube1Id, LightCube1Id): Triggers.MemoryMatchPointCenter,
            (LightCube1Id, LightCube2Id): Triggers.MemoryMatchPointRightSmall,
            (LightCube1Id, LightCube3Id): Triggers.MemoryMatchPointRightBig,
            (LightCube2Id, LightCube1Id): Triggers.MemoryMatchPointLeftSmall,
            (LightCube2Id, LightCube2Id): Triggers.MemoryMatchPointCenter,
            (LightCube2Id, LightCube3Id): Triggers.MemoryMatchPointRightSmall,
            (LightCube3Id, LightCube1Id): Triggers.MemoryMatchPointLeftBig,
            (LightCube3Id, LightCube2Id): Triggers.MemoryMatchPointLeftSmall,
            (LightCube3Id, LightCube3Id): Triggers.MemoryMatchPointCenter,
        }[key]

For example, change Triggers.MemoryMatchPointLeftSmall to the string 'anim_memorymatch_pointsmallleft_01'.

You can find these animation names by printing out self._robot.anim_names.

Then changing this function from play_anim_trigger to play_anim.

    # Source: SongRobot.py
    def __play_animation(self, animation_name: str):
        self._animation_complete = False
        self._robot.play_anim(name=animation_name, in_parallel=True)

Verify that this improves the alignment when playing a long series of notes.

If it doesn't improve his alignment, then we have to think of an alternative solution.

We could add a mini-game where a note is played and the player has to guess what the note was.

Cozmo would also participate.

This game could also train relative pitches, the distance is between notes. For example, a C and E is played, and the player has to answer a major third.

How does this connect to Song Match?

Song Match could have a hard difficulty where the cubes no longer flash when the notes are played at the beginning of a round.

Then the player has to rely on their hearing without a visual cue.

The Ear Training mini-game can be a way to train the player for this harder difficulty of Song Match.

The Note unit test on Travis CI fails because there's no audio device on the server.

This will need to be fixed somehow.

ERROR: setUpClass (test_note.TestNote)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/gbroques/cozmo-song-match/test/test_note.py", line 11, in setUpClass
    Note.init_mixer()
  File "/home/travis/build/gbroques/cozmo-song-match/song_match/song/note.py", line 33, in init_mixer
    init(frequency=44100, size=-16, channels=1, buffer=1024)
pygame.error: No available audio device
----------------------------------------------------------------------
Ran 0 tests in 0.009s
FAILED (errors=1)
The command "python -m unittest discover test" exited with 1.
Done. Your build exited with 1.

We could add a conductor's baton / magic wand prop, basically a stick, to make Cozmo tapping the block more noticeable.

This could make it more obvious for bystanders watching the game at a distance what Cozmo is doing.

This would probably be 3D printed and hook onto Cozmo's front arm-bar hooks.

Cozmo needs to play the wrong note sometimes.

When Cozmo plays a note, there should be a small probability to play a random note.

This probability should increase with the length of the series to simulate the sequence becoming harder to remember.

This issue should take care verifying he played the correct notes, based upon the current place in the main game loop.

Cozmo playing a series of notes will be handled in issue #6.

The wrong note interaction will be taken care of by issue #7.

We need to add the game loop.

Here's some pseudo code of what it might look like:

current_position = 1
while song.is_not_finished(current_position):
    note_sequence = song.get_sequence(current_position)
    play_sequence(note_sequence)
    wait_for_player_to_match_sequence()
    wait_for_cozmo_to_match_sequence()
    current_position += 1

UPDATE: Issue #13 is responsible for completing 1 iteration of the loop. That should hopefully make this issue trivial.

A sphinx extension will allow us to create UML, and other visual diagrams on the documentation site.

See the following links:

• http://build-me-the-docs-please.readthedocs.io/en/latest/Using_Sphinx/UsingGraphicsAndDiagramsInSphinx.html
• http://plantuml.com/class-diagram

Why create the visual diagrams with Sphinx?

Sphinx creates diagrams that have hyperlinks to the classes and code they reference.

See inheritance diagrams for an example.

NOTE: This extension also allows us to create flow charts, data flow diagrams, or any other visual explanation of the code. I think UML might be the easiest to satisfy this project requirement.

Post a comment if you have other ideas.

How do I update the docs?

See How to Update the Docs.

Now that we have three songs, we need to randomly select one at the start of the game.

This issue depends upon issue #26.

To support selecting multiple options we could have a pagination-like cube menu.

For example, selecting which song you want to play.

• Each song could have a color associated with it.
• If you tap the left most cube, the colors cycle to the left.
• If you tap the right most cube, the colors cycle to right.
• If you tap the middle cube, then that's the song you want to play.

This should be generically made, as it could support selecting anything, not just songs.

This could be augmented by add a Next and Previous label to the left most and right most cubes on the cube mat.

Currently the game can start if Cozmo is slightly off (not directly centered and looking at the middle cube).

This causes Cozmo's taps to be off the rest of the game, since he moves a fixed amount each tap.

We need to come up with a way for Cozmo to realign himself by turning towards the middle cube.

Something like...

# song_robot.py
def turn_to_cube(cube_id):

Then we could just pass in the middle cube's ID and have Cozmo turn to it in between rounds.

There's rudimentary code for this already...

    async def turn_back_to_center(self, in_parallel=False) -> None:
        """Turn Cozmo back to the center.

        :param in_parallel: Whether to do the action in parallel or wait until it's completed.
        :return: None
        """
        if in_parallel:
            self._robot.turn_in_place(self._initial_angle, is_absolute=True)
        else:
            await self._robot.turn_in_place(self._initial_angle, is_absolute=True).wait_for_completed()
        self._prev_cube_id = LightCube2Id

But it just turns Cozmo back to where he was initially. It has no knowledge where the cubes are.

This issue closely relates to #8.

There should be some kind of transition to the next round in the game.

This would consist of some sort of sound and pattern of the flashing with the cubes.

This will happen once at the bottom of the main game loop once:

1. The series is played
2. The player matches the series
3. And Cozmo matches the series

When a player or Cozmo gets a series correct, then the following should happen:

1. All the cubes flash a few times
2. Some sound is emitted
3. Cozmo gives some reaction
   • His reaction should vary based upon who got the series right
     • He might look surprised when the player gets it right
     • And look quite pleased when he gets it right

In multiplayer mode, when a player gets 3 strikes, there should be some indication.

For example, Cozmo could say "Player 1 is out of the game"

Maybe the WrongNoteEffect handles this behavior with a new boolean last_strike.

Cozmo needs to be able to play a series of notes.

We'll add sometimes playing a wrong note in issue #11.

This will involve adding a play_notes() method in the SongRobot class.

Something like:

def play_notes(notes: List[Note]) -> None:
    for note in notes:
        await self.play_note(note)