ppaquette / gym-super-mario Goto Github PK

nes_env.py use os.mkfifo(self.path_pipe_out)
mkfifo is running on linux. windows didn't have?
Have any solution?

Fceux emulator keeps freezing seconds after the environment is deployed and it wont work after that. The provided script nes_env.py doesnt really have any impact on the emulator.
any solution would be very appreciated

Hi. gym don't make supermario

gym.error.UnregisteredEnv: No registered env with id: ppaquette/SuperMarioBros-7-2-v0

all version same.

I've noticed something strange happens when Mario dies after falling off a cliff. From the tile view, it looks like Mario gets hidden, but the underlying object persists for a few seconds and keeps falling. I'm concerned that this behavior may be interfering with reinforcement learning, since the agent continues to take actions and receive rewards even after Mario dies. What do you think?

I'm pretty sure on this line you want [2] rather than [1] which gives the vector two states per action, 0 and 1.

Great work on the gym, super fun.

Im getting this error while running

env = gym.make('SuperMarioBros-1-1-v0')

Error :

[2018-08-20 08:38:48,607] Making new env: SuperMarioBros-1-1-v0
Traceback (most recent call last):
File "", line 1, in
File "/anaconda3/envs/universe-starter-agent/lib/python3.5/site-packages/gym/envs/registration.py", line 161, in make
return registry.make(id)
File "/anaconda3/envs/universe-starter-agent/lib/python3.5/site-packages/gym/envs/registration.py", line 119, in make
env = spec.make()
File "/anaconda3/envs/universe-starter-agent/lib/python3.5/site-packages/gym/envs/registration.py", line 86, in make
env = cls(**self._kwargs)
File "/anaconda3/envs/universe-starter-agent/lib/python3.5/site-packages/gym/envs/ppaquette_gym_super_mario/super_mario_bros.py", line 256, in init
num_levels=32)
File "/anaconda3/envs/universe-starter-agent/lib/python3.5/site-packages/gym/envs/ppaquette_gym_super_mario/nes_env.py", line 548, in init
NesEnv.init(self)
File "/anaconda3/envs/universe-starter-agent/lib/python3.5/site-packages/gym/envs/ppaquette_gym_super_mario/nes_env.py", line 48, in init
self.action_space = spaces.MultiDiscrete([1] * NUM_ACTIONS)
File "/anaconda3/envs/universe-starter-agent/lib/python3.5/site-packages/gym/spaces/multi_discrete.py", line 29, in init
print(array_of_param_array)
File "/anaconda3/envs/universe-starter-agent/lib/python3.5/site-packages/gym/spaces/multi_discrete.py", line 29, in
print(array_of_param_array)
TypeError: 'int' object is not subscriptable

When running the program
`from nes_py.wrappers import BinarySpaceToDiscreteSpaceEnv
import gym_super_mario_bros
from gym_super_mario_bros.actions import SIMPLE_MOVEMENT
env = gym_super_mario_bros.make('SuperMarioBros-v0')
env = BinarySpaceToDiscreteSpaceEnv(env, SIMPLE_MOVEMENT)

done = True
for step in range(5000):
if done:
state = env.reset()
state, reward, done, info = env.step(env.action_space.sample())
env.render()

env.close()`
I am getting the error

Traceback (most recent call last): File "/home/dileep/.local/lib/python3.6/site-packages/pyglet/init.py", line 378, in getattr return getattr(self._module, name) AttributeError: 'NoneType' object has no attribute '_create_shadow_window' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "", line 5, in File "/home/dileep/.local/lib/python3.6/site-packages/gym/core.py", line 237, in render return self.env.render(mode, **kwargs) File "/home/dileep/.local/lib/python3.6/site-packages/gym/core.py", line 237, in render return self.env.render(mode, **kwargs) File "/home/dileep/.local/lib/python3.6/site-packages/nes_py/nes_env.py", line 364, in render from ._image_viewer import ImageViewer File "/home/dileep/.local/lib/python3.6/site-packages/nes_py/_image_viewer.py", line 2, in from pyglet.window import Window File "/home/dileep/.local/lib/python3.6/site-packages/pyglet/window/init.py", line 1896, in gl._create_shadow_window() File "/home/dileep/.local/lib/python3.6/site-packages/pyglet/init.py", line 384, in getattr import(import_name) File "/home/dileep/.local/lib/python3.6/site-packages/pyglet/gl/init.py", line 100, in from pyglet.gl.lib import GLException File "/home/dileep/.local/lib/python3.6/site-packages/pyglet/gl/lib.py", line 143, in from pyglet.gl.lib_glx import link_GL, link_GLU, link_GLX File "/home/dileep/.local/lib/python3.6/site-packages/pyglet/gl/lib_glx.py", line 50, in gl_lib = pyglet.lib.load_library('GL') File "/home/dileep/.local/lib/python3.6/site-packages/pyglet/lib.py", line 158, in load_library raise ImportError('Library "%s" not found.' % names[0]) ImportError: Library "GL" not found.

I'm getting this error after importing the env directly onto my computer, and running the train script.

I have SDL installed and installed this using pip. I'm running python 3.6 and ubuntu 18.04. When I try

env = gym.make('ppaquette/SuperMarioBros-1-1-v0')

done = True
for step in range(5000):
    if done:
        state = env.reset()
    state, reward, done, info = env.step(env.action_space.sample())
    env.render()

env.close()

I get

  File "/home/vivnp/Python Projects/super-mario-ai/test.py", line 15, in <module>
    env.render()
  File "/home/vivnp/.local/lib/python3.6/site-packages/gym/core.py", line 284, in render
    return self.env.render(mode)
  File "/home/vivnp/.local/lib/python3.6/site-packages/gym/core.py", line 111, in render
    raise NotImplementedError
NotImplementedError
Exception ignored in: <module 'threading' from '/usr/lib/python3.6/threading.py'>
Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 1294, in _shutdown
    t.join()
  File "/usr/lib/python3.6/threading.py", line 1056, in join
    self._wait_for_tstate_lock()
  File "/usr/lib/python3.6/threading.py", line 1072, in _wait_for_tstate_lock
    elif lock.acquire(block, timeout):
KeyboardInterrupt 

I have tried installing pyglet 1.2.4, and making a new python environment and installing it there, to no avail.
Any suggestions?

I'm noticing that the environment gives an unusually large reward on the first step and wondering why?

Here's some rollouts. The 5 element list on the line above are the rewards given at each of the first 5 steps. See how the first one is very big.

[4.0918, 0.0, 0.10219999999999985, 0.10229999999999961, 0.1023000000000005]
Episode:     1   Avg:   26.085   BestAvg:     -inf   σ:    0.000  |  Steps:       81   Reward:   26.085

[34.2685, 0.0, 0.1021999999999963, 0.10229999999999961, 0.10229999999999961]
Episode:     2   Avg:   26.136   BestAvg:     -inf   σ:    0.051  |  Steps:       73   Reward:   26.187

[64.5474, 0.0, 0.10229999999999961, 0.10229999999999961, 0.0]
Episode:     3   Avg:   39.724   BestAvg:     -inf   σ:   19.216  |  Steps:      186   Reward:   66.900

[105.3627, 0.0, 0.0, 0.0, 0.0]
Episode:     4   Avg:   36.519   BestAvg:     -inf   σ:   17.543  |  Steps:       84   Reward:   26.903

[105.9764, 0.0, -0.10229999999999961, -0.20459999999999923, -0.2044999999999959]
Episode:     5   Avg:   42.493   BestAvg:     -inf   σ:   19.722  |  Steps:      226   Reward:   66.389

[105.465, 0.0, 0.1021999999999963, 0.10229999999999961, 0.10229999999999961]
Episode:     6   Avg:   48.862   BestAvg:     -inf   σ:   22.956  |  Steps:      272   Reward:   80.710

FYI, I'm currently setting the first reward to 0 in my calculations, which is reflected in the output of Reward.

I installed the code using pip install ppaquette_gym_super_mario and I'm running gym 0.10.5

So I have been trying to initialize the environment for the Mario package but I keep getting the same error over and over again. I have already done the pulling of the environment and I have already registered it.
this is a part of the error:
File "/usr/local/lib/python2.7/dist-packages/gym/envs/registration.py", line 161, in make
return registry.make(id)
File "/usr/local/lib/python2.7/dist-packages/gym/envs/registration.py", line 118, in make
spec = self.spec(id)
File "/usr/local/lib/python2.7/dist-packages/gym/envs/registration.py", line 147, in spec
raise error.UnregisteredEnv('No registered env with id: {}'.format(id))
gym.error.UnregisteredEnv: No registered env with id: ppaquette/SuperMarioBros-1-1-v0

I was trying to install your environment via gym_pull without success, caused by changes to gym itself. Please see openai/gym#750

Reason: gym/scoreboard has changed 21 days ago, and not only gym_pull but also manual installation uses for example gym.scoreboard.client (see #6), which was excluded by previously mentioned merge, 21 days ago.

In order to achieve a proper environment install, I had to revert* gym to the state where gym.scoreboard was still exsisting with previous state (see following commit under gym b5576dc23a5fcad0733042ab2ad440200ebb6209).
*To revert, use git checkout <commit hash> .

After reverting, steps written by Jackiexiao (see #6 (comment)) resulted in proper installation.

Sometimes the agent gets stuck. So if the agent does not make any improvement in say 200 frames, I want to reset the game. But doing so closes the fceux and opens it again. This reduces the speed. I am using the gabegrand branch.

Thank you

I am trying to run the FCEUX emulator on AWS without rendering so that training can occur faster. So far I have tried setting no_render to False, and adding --nogui 1 to cmd_args in nes_env.py, but have had no luck. Any help is appreciated.

How do I get the tile data if the agent is placed in an environment for tiled data?

In level 1-1, we started our Mario agent at 0% of the map, but once the agent went over 40% of the map, it always restart at 40% not 0%.
How can i modify the code that it can always restart at the beginning of the map(0%)?

Thanks.

when running this basic import code I get error below

import gym
import gym_pull
gym_pull.pull('github.com/ppaquette/gym-super-mario')        # Only required once, envs will be loaded with import gym_pull afterwards
env = gym.make('ppaquette/SuperMarioBros-1-1-v0')

Traceback (most recent call last):
File "super_mario_1.py", line 2, in
import gym_pull
File "/usr/local/lib/python2.7/dist-packages/gym_pull/init.py", line 40, in
import gym_pull.scoreboard.api
File "/usr/local/lib/python2.7/dist-packages/gym_pull/scoreboard/api.py", line 3, in
from gym.scoreboard.client import resource, util
ImportError: No module named client

Hello, I can't seem to get gym_pull working properly with SMB. It seems like there is an issue with the install location. Any ideas? I'm on Mac OSX Yosemite with a Python 2.7 anaconda distribution.

>>> import gym, gym_pull
>>> gym_pull.pull('github.com/ppaquette/gym-super-mario')
[2016-10-23 19:09:50,870] Installing pip package from "https://github.com/ppaquette/gym-super-mario.git"
Collecting git+https://github.com/ppaquette/gym-super-mario.git
  Cloning https://github.com/ppaquette/gym-super-mario.git to /var/folders/p5/m1z33npj37z93fws69vxykv80000gn/T/pip-MThd2b-build
Requirement already up-to-date: gym>=0.2.3 in /Users/ggrand/Desktop/gym (from ppaquette-gym-super-mario==0.0.1)
Requirement already up-to-date: numpy>=1.10.4 in /anaconda/lib/python2.7/site-packages (from gym>=0.2.3->ppaquette-gym-super-mario==0.0.1)
Requirement already up-to-date: requests>=2.0 in /anaconda/lib/python2.7/site-packages (from gym>=0.2.3->ppaquette-gym-super-mario==0.0.1)
Requirement already up-to-date: six in /anaconda/lib/python2.7/site-packages/six-1.10.0-py2.7.egg (from gym>=0.2.3->ppaquette-gym-super-mario==0.0.1)
Requirement already up-to-date: pyglet>=1.2.0 in /anaconda/lib/python2.7/site-packages (from gym>=0.2.3->ppaquette-gym-super-mario==0.0.1)
Installing collected packages: ppaquette-gym-super-mario
  Found existing installation: ppaquette-gym-super-mario 0.0.1
    Uninstalling ppaquette-gym-super-mario-0.0.1:
      Successfully uninstalled ppaquette-gym-super-mario-0.0.1
  Running setup.py install for ppaquette-gym-super-mario ... done
Successfully installed ppaquette-gym-super-mario-0.0.1
[2016-10-23 19:09:53,381] The user environments for "github.com/ppaquette/gym-super-mario" are already up-to-date (no new version detected).
>>> env = gym.make('ppaquette/SuperMarioBros-1-1-v0')
[2016-10-23 19:10:20,645] Making new env: ppaquette/SuperMarioBros-1-1-v0
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "gym/envs/registration.py", line 126, in make
    return registry.make(id)
  File "gym/envs/registration.py", line 90, in make
    spec = self.spec(id)
  File "gym/envs/registration.py", line 112, in spec
    raise error.UnregisteredEnv('No registered env with id: {}'.format(id))
gym.error.UnregisteredEnv: No registered env with id: ppaquette/SuperMarioBros-1-1-v0

raise error.UnregisteredEnv('No registered env with id: {}'.format(id))

gym.error.UnregisteredEnv: No registered env with id: ppaquette/SuperMarioBros-1-1-v0

Once I run env.reset(), the fceux emulator opens up, and I am no longer able to enter commands into the terminal. Can you give me a code sample for how to take actions in the environment?

I also noticed that the README says "The game will only accept inputs after the timer has started to decrease (i.e. it will automatically move through the menus and animations)". The game does indeed move automatically through the menus and animations, but the time never starts to decrease. In fact, the time never appears in the first place (see below). Is this normal?

latest gym have no scoreboard.

I have read the lua script, and tried to implement this function via curr_page and curr_x_position. However, whatever value I set to curr_page, mario always move to 0 position. If I set curr_x_position a value more than 232, mario will just move to the edge of screen and do not update the screen. I am very confused about these, do you have some ideas about this function?

I did some gym training what need a long time for training.

my notebook is a new macbook 12,

Such as training mario game,

what is your hardware , and how long is your time for training?

what is the good way to accelerate it?

Hi Philip, I was wondering whether it's possible to manually set the emulator speed. It'd be nice to further increase the speed, say to 5000%, during training. Additionally, when demoing the RL agent, it'd be great to set it to normal speed, so that the sound works properly.

On a related note, in the interest of increasing training efficiency, is there any way to skip the lengthy intro screens (e.g., "World 1-1") that occur prior to the level? Currently, it takes about 20 mins to do 100 training iterations, which is pretty slow, since our goal is to train for several thousand iterations.

gym-pull is broken. Is there a way to install this without it?

Im running windows and i've got the env installed into gym but when i go to run:

import gym
env = gym.make('SuperMarioBros-1-1-v0')
observation = env.reset()
for _ in range(1000):
    env.render()
    action = env.action_space.sample() # your agent here (this takes random actions)
    observation, reward, done, info = env.step(action)

I receive the following error:

[2017-08-16 21:29:14,763] Making new env: SuperMarioBros-1-1-v0
Traceback (most recent call last):
  File "C:/Users/username/PycharmProjects/AI/tutorials.py", line 3, in <module>
    observation = env.reset()
  File "C:\Users\username\Anaconda3\envs\Python352\lib\site-packages\gym-0.9.2-py3.5.egg\gym\core.py", line 104, in reset
    return self._reset()
  File "C:\Users\username\Anaconda3\envs\Python352\lib\site-packages\gym-0.9.2-py3.5.egg\gym\envs\ppaquette_gym_super_mario\nes_env.py", line 653, in _reset
    self._launch_fceux()
  File "C:\Users\username\Anaconda3\envs\Python352\lib\site-packages\gym-0.9.2-py3.5.egg\gym\envs\ppaquette_gym_super_mario\nes_env.py", line 194, in _launch_fceux
    temp_lua_file = open(temp_lua_path, 'w', 1)
OSError: [Errno 22] Invalid argument: '\tmp\\3628161587.lua'
'C:\Users\usernm~1\AppData\Local\Temp\smb-fifo-out.684933541' is not recognized as an internal or external command,
operable program or batch file.
'C:\Users\usernm~1\AppData\Local\Temp\smb-fifo-in.684933541' is not recognized as an internal or external command,
operable program or batch file.