I am trying to create a Q-learning agent I am having a problem with rendering the environment. I want to render the environment every thousand episodes but due to env.reset() calling render internally the rendering is continuous and the training is so slow.

I'm trying to build a simple pygame renderer following the guidelines at https://www.gymlibrary.ml/content/environment_creation/#rendering
however the function Renderer is not available from gym.utils.renderer.
I have installed gym version 0.23.1.

Part of the VectorEnv subsection in the API Reference section is incorrect. See everything between ### VectorEnv and ### Reset

With a number of the new features coming to environment, I think it would make sense for us to have our own custom tutorials for these features. Otherwise we are waiting for other authors to do the work that we understand best already

See here: openai/gym#2825 (comment)

I believe that the first code snippet in the "Interacting with the Environment" section of the API page is quite important because it serves to illustrates how gym is supposed to be used. I think there are a few changes that could make some points a bit clearer at a quick glance.

• The output of step should be assigned to variables with meaningful names
• Seeding should be demonstrated
• The environment should be reset after a done signal has been emitted

Therefore, I would suggest something like this:

import gym

env = gym.make("CartPole-v0")
env.seed(42)
observation = env.reset()

for _ in range(1000):
  env.render()
  observation, reward, done, info = env.step(env.action_space.sample())

  if done:
    observation = env.reset()
env.close()

This snippet would demonstrate all essential methods of the API.
Of course, the animation won't be as pretty as the current one, but I think this snippet would do a better job of illustrating the usage of gym.

I would also suggest that each method is covered in a small subsection that covers parameters, effects and return values.

One thing that was quite unclear to me when I started using gym was how to tell whether a done signal was generated by the environment or a time limit. This is quite important in many RL algorithms, e.g. SAC. Although one might cover this in the Wrappers section, I think it is worth mentioning in the API section as well.

• Atari Age website linked with instruction page scans rather than fully digitized info (is this okay): #61 (comment)

• gym 0.21.0 error: #60 (comment)

• Where should giant Atari info table go: #59 (comment)

As the title says, the links are not clickable. This would probably be quite an annoyance.
They seem to be bare links that would work in normal Markdown. The problem could be fixed by using [<url>](<url>). However, I'm not raising a pull request yet in case anybody knows whether this can be fixed globally through changes to some configuration.

The documentation is partly in 2 and 4 space intent.

It would be better if it would be consistently. Would make copying examples easier, too.

For the vector API, reset method (https://www.gymlibrary.ml/content/vector_api/#reset), currently it shows

Note that for the parameter "options" - If to return the options
I doubt the actually meaning is

that is options actually is the reset option

Currently, the individual wrappers are documented in a gigantic table here: Wrappers - Gym Documentation. The table is confusing. Should we have one auto-generated page per wrapper (not including the abstract super-classes) and list them on the main page?

• @Markus28

We should add release notes to the website

• Inconsistent subheaders (## vs ###). Fix in gym.
• Wrong subheaders (**stuff**)

e.g. the content from "https://www.gymlibrary.ml/environments/toy_text/blackjack/" perhaps should also be accessible by typing "https://www.gymlibrary.ml/blackjack/"

The problem was raised on discord. The method needs a docstring and entry on environment page.

It was never updated after moving to the new sphinx site.
@kkahadze I believe you may be working on this.