danielslater / pygameplayer Goto Github PK

hi,
when i look at line 100 at https://github.com/DanielSlater/PyGamePlayer/blob/master/examples/deep_q_pong_player.py#L100 i see

• current_state = np.append(self._last_state[:, :, 1:], screen_resized_binary, axis=2)

when i use this test code:

import numpy as np

a = np.array([[1,2,3],[4,5,6]])
print a.shape
print a
b = np.stack(tuple(a for _ in range(4)), axis=2)
print b.shape
print b
c = np.array([[7,8,9], [10,11,12]])
print c.shape
print c
d = np.reshape(c, (2,3,1))
print d.shape
e = np.append(b[:,:,1:], d, axis=2)
print e.shape
print e
f = np.array([[13,14,15],[16,17,18]])
g = np.reshape(f, (2,3,1))
h = np.append(b[:,:,1:], g, axis=2)
print h.shape
print h

it prints:

(2, 3)
[[1 2 3]
 [4 5 6]]
(2, 3, 4)
[[[1 1 1 1]
  [2 2 2 2]
  [3 3 3 3]]

 [[4 4 4 4]
  [5 5 5 5]
  [6 6 6 6]]]
(2, 3)
[[ 7  8  9]
 [10 11 12]]
(2, 3, 1)
(2, 3, 4)
[[[ 1  1  1  7]
  [ 2  2  2  8]
  [ 3  3  3  9]]

 [[ 4  4  4 10]
  [ 5  5  5 11]
  [ 6  6  6 12]]]
(2, 3, 4)
[[[ 1  1  1 13]
  [ 2  2  2 14]
  [ 3  3  3 15]]

 [[ 4  4  4 16]
  [ 5  5  5 17]
  [ 6  6  6 18]]]

but should it not be shifted-like inserted, so that the old values (frames) will all get lost after time ? or do i think wrong ?

Could you add a license file for PyGamePlayer?

The derivative Pong games will need to be GPLv2. The Tetris game is originally BSD. The rest appears not to have a license.

Thanks!

Hi Daniel - we talked briefly at pydata last week.

I've got your deep q pong player running on a 4 core Macbook pro.

• Can the clock speed / frame rate of pygame be increased? I am new to this package.

A quick back of the envelope calculation shows that about 20 hours game play is needed before the 500000 observation steps will be completed and training starts. I'm wondering how the wall clock time can be reduced.

By the way, pygame itself has a few dependencies which I managed to install using

brew install sdl
brew install sdl_ttf

I did a training session that ended with network-480000.meta.
The deep_q_pong_networks folder has all the files you would assume it has after training that many iterations.

However, when resuming the training, the AI seems to be starting over from scratch.
Is this a bug, user error, or intended?

Loving this tutorial!
Thanks!

hi,
i am new to python and i dont get it to install and run your PyGamePlayer ? I did:

• mkdir ~/test_dir
• cd ~/test_dir
• virtualenv venv
• source venv/bin/activate
• pip install numpy

now i am stuck ? I simply git cloned PyGamePlayer repository and copied it into ~/test_dir/venv/

When i then run "python examples/deep_q_pong_player.py" it returns:

Traceback (most recent call last):
  File "examples/deep_q_pong_player.py", line 5, in <module>
    from pong_player import PongPlayer
  File "/home/ros/Documents/python-virt-env/venv/examples/pong_player.py", line 1, in <module>
    from pygame.constants import K_DOWN
ImportError: No module named pygame.constants

this is because i did not installed pygame into the virtual environment, but how do i do that ?

thanks

Hi,

First, I would like to thanks you about your job and yours tutorials about Q learning method with tensorflow. I learn lots of things thanks to you !

I would like to report a bug. I prepared a little code to show you what happening. Maybe I did something wrong, but according to me, this is a normal behavior which should work.

`class PongPlayer(PyGamePlayer):
def init(self, force_game_fps=60, run_real_time=False):
"""
Example class for playing Pong
"""
super(PongPlayer, self).init(force_game_fps=force_game_fps, run_real_time=run_real_time)
self.last_bar1_score = 0.0
self.last_bar2_score = 0.0
self.it = 0

def get_keys_pressed(self, screen_array, feedback, terminal):
    # TODO: put an actual learning agent here
    self.it += 1
    if self.it < 100:  
        print "K_DWON"
        return [K_DOWN]
    else:
        print "K_UP"
        return [K_UP]`

When I execute this code, the agent going down, but when a new action is returned (K_UP), the agent stops working and don't move anymore. Even If continue to return K_UP.
Here, I made a simple example, but this cause lot of problems when I try to launch my real code.

Here, a gif showing the problem.

Thanks you for your help !

Is the left side or the right side of the video running are the AI racket? Is the serving of another racket random or how is it generated?

It would be good to have an option which would allow me to take control of the non-trained paddle and play against the AI agent.

hi,
in line 84 you wrote as comment:

set the pixels to all be 0. or 1.

but with the following code

cv2.threshold(screen_resized_grayscaled, 1, 255, cv2.THRESH_BINARY)

you set them to 0 or 255. So the input data is not normalized, or? So , if you mostly pipe in zeros, wouldnt then the neurons not all get also zero ? x*W+b ?

https://github.com/DanielSlater/PyGamePlayer/blob/master/examples/deep_q_pong_player.py#L84

Currently if this method is used it will run with the real pygame timer, not our intercepting one, which may cause synchronization issues.