OpenAI Gym is a open-source Python toolkit for developing and comparing reinforcement learning algorithms. This R package is a wrapper for the OpenAI Gym API, and enables access to an ever-growing variety of environments.


You can install the latest development version from CRAN:


Or from GitHub with:

if (packageVersion("devtools") < 1.6) {
devtools::install_github("paulhendricks/gym-R", subdir = "R")

If you encounter a clear bug, please file a minimal reproducible example on GitHub.

Getting started

Setting up the server

To download the code and install the requirements, you can run the following shell commands:

git clone
cd gym-http-api
pip install -r requirements.txt

This code is intended to be run locally by a single user. The server runs in python.

To start the server from the command line, run this:


For more details, please see here:

Running an example in R

In a separate R terminal, you can then try running the example agent and see what happens:


remote_base <- ""
client <- create_GymClient(remote_base)

# Create environment
env_id <- "CartPole-v0"
instance_id <- env_create(client, env_id)

# List all environments
all_envs <- env_list_all(client)

# Set up agent
action_space_info <- env_action_space_info(client, instance_id)
agent <- random_discrete_agent(action_space_info[["n"]])

# Run experiment, with monitor
outdir <- "/tmp/random-agent-results"
env_monitor_start(client, instance_id, outdir, force = TRUE, resume = FALSE)

episode_count <- 100
max_steps <- 200
reward <- 0
done <- FALSE

for (i in 1:episode_count) {
  ob <- env_reset(client, instance_id)
  for (i in 1:max_steps) {
    action <- env_action_space_sample(client, instance_id)
    results <- env_step(client, instance_id, action, render = TRUE)
    if (results[["done"]]) break

# Dump result info to disk
env_monitor_close(client, instance_id)


To cite package ‘gym’ in publications use:

Paul Hendricks (2016). gym: Provides Access to the OpenAI Gym API. R package version 0.1.0.

A BibTeX entry for LaTeX users is

  title = {gym: Provides Access to the OpenAI Gym API},
  author = {Paul Hendricks},
  year = {2016},
  note = {R package version 0.1.0},
  url = {},