danielslater / pygameplayer Goto Github PK
View Code? Open in Web Editor NEWModule to help with running learning agents against PyGame games
License: MIT License
Module to help with running learning agents against PyGame games
License: MIT License
Hi Daniel - we talked briefly at pydata last week.
I've got your deep q pong player running on a 4 core Macbook pro.
A quick back of the envelope calculation shows that about 20 hours game play is needed before the 500000 observation steps will be completed and training starts. I'm wondering how the wall clock time can be reduced.
By the way, pygame itself has a few dependencies which I managed to install using
brew install sdl
brew install sdl_ttf
hi,
in line 84 you wrote as comment:
set the pixels to all be 0. or 1.
but with the following code
cv2.threshold(screen_resized_grayscaled, 1, 255, cv2.THRESH_BINARY)
you set them to 0 or 255. So the input data is not normalized, or? So , if you mostly pipe in zeros, wouldnt then the neurons not all get also zero ? x*W+b ?
https://github.com/DanielSlater/PyGamePlayer/blob/master/examples/deep_q_pong_player.py#L84
hi,
when i look at line 100 at https://github.com/DanielSlater/PyGamePlayer/blob/master/examples/deep_q_pong_player.py#L100 i see
when i use this test code:
import numpy as np
a = np.array([[1,2,3],[4,5,6]])
print a.shape
print a
b = np.stack(tuple(a for _ in range(4)), axis=2)
print b.shape
print b
c = np.array([[7,8,9], [10,11,12]])
print c.shape
print c
d = np.reshape(c, (2,3,1))
print d.shape
e = np.append(b[:,:,1:], d, axis=2)
print e.shape
print e
f = np.array([[13,14,15],[16,17,18]])
g = np.reshape(f, (2,3,1))
h = np.append(b[:,:,1:], g, axis=2)
print h.shape
print h
it prints:
(2, 3)
[[1 2 3]
[4 5 6]]
(2, 3, 4)
[[[1 1 1 1]
[2 2 2 2]
[3 3 3 3]]
[[4 4 4 4]
[5 5 5 5]
[6 6 6 6]]]
(2, 3)
[[ 7 8 9]
[10 11 12]]
(2, 3, 1)
(2, 3, 4)
[[[ 1 1 1 7]
[ 2 2 2 8]
[ 3 3 3 9]]
[[ 4 4 4 10]
[ 5 5 5 11]
[ 6 6 6 12]]]
(2, 3, 4)
[[[ 1 1 1 13]
[ 2 2 2 14]
[ 3 3 3 15]]
[[ 4 4 4 16]
[ 5 5 5 17]
[ 6 6 6 18]]]
but should it not be shifted-like inserted, so that the old values (frames) will all get lost after time ? or do i think wrong ?
Is the left side or the right side of the video running are the AI racket? Is the serving of another racket random or how is it generated?
Hi,
First, I would like to thanks you about your job and yours tutorials about Q learning method with tensorflow. I learn lots of things thanks to you !
I would like to report a bug. I prepared a little code to show you what happening. Maybe I did something wrong, but according to me, this is a normal behavior which should work.
`class PongPlayer(PyGamePlayer):
def init(self, force_game_fps=60, run_real_time=False):
"""
Example class for playing Pong
"""
super(PongPlayer, self).init(force_game_fps=force_game_fps, run_real_time=run_real_time)
self.last_bar1_score = 0.0
self.last_bar2_score = 0.0
self.it = 0
def get_keys_pressed(self, screen_array, feedback, terminal):
# TODO: put an actual learning agent here
self.it += 1
if self.it < 100:
print "K_DWON"
return [K_DOWN]
else:
print "K_UP"
return [K_UP]`
When I execute this code, the agent going down, but when a new action is returned (K_UP), the agent stops working and don't move anymore. Even If continue to return K_UP.
Here, I made a simple example, but this cause lot of problems when I try to launch my real code.
Here, a gif showing the problem.
Thanks you for your help !
It would be good to have an option which would allow me to take control of the non-trained paddle and play against the AI agent.
Currently if this method is used it will run with the real pygame timer, not our intercepting one, which may cause synchronization issues.
I did a training session that ended with network-480000.meta.
The deep_q_pong_networks folder has all the files you would assume it has after training that many iterations.
However, when resuming the training, the AI seems to be starting over from scratch.
Is this a bug, user error, or intended?
Loving this tutorial!
Thanks!
Could you add a license file for PyGamePlayer?
The derivative Pong games will need to be GPLv2. The Tetris game is originally BSD. The rest appears not to have a license.
Thanks!
hi,
i am new to python and i dont get it to install and run your PyGamePlayer ? I did:
now i am stuck ? I simply git cloned PyGamePlayer repository and copied it into ~/test_dir/venv/
When i then run "python examples/deep_q_pong_player.py" it returns:
Traceback (most recent call last):
File "examples/deep_q_pong_player.py", line 5, in <module>
from pong_player import PongPlayer
File "/home/ros/Documents/python-virt-env/venv/examples/pong_player.py", line 1, in <module>
from pygame.constants import K_DOWN
ImportError: No module named pygame.constants
this is because i did not installed pygame into the virtual environment, but how do i do that ?
thanks
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.