Refactor CLI/interface: init command, factory field, remove extras_size

- Rename `retro-gamer create` to `retro-gamer init` with positional
  args (GAME OUTPUT) instead of --game/--output flags
- Add [tool.retro-gamer].factory = "module:attr" to declare the game
  factory function from pyproject.toml instead of relying on a
  hard-coded create_game attribute
- Remove extras_size from user-facing config; it is now measured
  automatically from a sample observation and never declared
- Update docs throughout: create→init, runs/→training/, add factory
  field documentation, clarify [model] vs [training] hyperparameter
  sections, remove stale version-history explanations
- Bump version to 0.3.0; require retro-games>=2.5.0
This commit is contained in:
Chris Proctor
2026-06-26 06:55:30 -04:00
parent 0cd3c3b488
commit c89609fe77
12 changed files with 217 additions and 136 deletions

View File

@@ -7,7 +7,7 @@ Game description
----------------
.. autoclass:: retro_gamer.GameMetadata
:members: from_pyproject, from_dict, validate, resolve_observation_function
:members: from_pyproject, from_dict, validate, resolve_factory, resolve_observation_function
Training
--------

View File

@@ -58,7 +58,7 @@ Verify the installation by checking the command-line tool:
Train and run RL agents for retro games.
Commands:
create Create a new training run directory with config.toml.
init Initialize a new training run directory with config.toml.
info Print a summary of a training run.
play Watch a trained agent play the game.
train Train (or resume training) a DQN agent.

View File

@@ -19,14 +19,14 @@ Both approaches start by creating a :class:`retro_gamer.TrainedPolicy`:
from retro_gamer import TrainedPolicy
ai = TrainedPolicy("runs/snake/")
ai = TrainedPolicy("training/snake/")
This reads ``config.toml``, rebuilds the network, and loads the latest
checkpoint. To load a specific checkpoint instead:
.. code-block:: python
ai = TrainedPolicy("runs/snake/", checkpoint="ep_0500")
ai = TrainedPolicy("training/snake/", checkpoint="ep_0500")
PolicyInput: model as player
----------------------------
@@ -40,7 +40,7 @@ it to ``game.play()`` and everything else works exactly as usual:
from retro.examples.snake import create_game
from retro_gamer import TrainedPolicy, PolicyInput
ai = TrainedPolicy("runs/snake/")
ai = TrainedPolicy("training/snake/")
game = create_game()
game.play(input_source=PolicyInput(ai, game))
@@ -62,7 +62,7 @@ loaded from disk once — not once per episode.
from retro.examples.snake import Apple, SnakeHead
from retro_gamer import TrainedPolicy
_ai = TrainedPolicy("runs/snake/")
_ai = TrainedPolicy("training/snake/")
class AISnake(SnakeHead):
def handle_keystroke(self, k, game): pass # ignore keyboard
@@ -112,9 +112,8 @@ from the game state. To train an enemy:
.. code-block:: console
% retro-gamer create --game my_game:create_enemy_training_game \
--output runs/enemy/
% retro-gamer train runs/enemy/
% retro-gamer init games/my_game training/enemy
% retro-gamer train training/enemy/
3. **Embed the trained model in your main game** using ``get_action``, exactly
as shown above.
@@ -145,7 +144,7 @@ once per episode:
# enemy_training_game.py
from retro_gamer import TrainedPolicy
_player = TrainedPolicy("runs/player/") # loaded once when the module is imported
_player = TrainedPolicy("training/player/") # loaded once when the module is imported
def create_game():
enemy = EnemyAgent()
@@ -156,9 +155,9 @@ You then alternate training runs:
.. code-block:: console
% retro-gamer train runs/player/ # train player against current enemy
% retro-gamer train runs/enemy/ # train enemy against updated player
% retro-gamer train runs/player/ # train player again
% retro-gamer train training/player/ # train player against current enemy
% retro-gamer train training/enemy/ # train enemy against updated player
% retro-gamer train training/player/ # train player again
# ...
How many episodes to run before switching is itself a design decision: too

View File

@@ -58,6 +58,7 @@ A typical workflow looks like this. First, describe your game in the
.. code-block:: toml
[tool.retro-gamer]
factory = "snake:create_game"
actions = ["KEY_RIGHT", "KEY_UP", "KEY_LEFT", "KEY_DOWN"]
reward = "score"
character_set = ["@", "*", ">", "<", "^", "v"]
@@ -66,13 +67,13 @@ Then create a training run, train, and watch the result:
.. code-block:: console
% retro-gamer create --game my_game --output runs/snake/
% retro-gamer init games/snake training/snake
% retro-gamer train runs/snake/
% retro-gamer train training/snake
% retro-gamer play runs/snake/ --checkpoint ep_0500
% retro-gamer play training/snake --checkpoint ep_0500
The ``create`` command sets up the training run directory; ``train``
The ``init`` command sets up the training run directory; ``train``
runs the learning algorithm; ``play`` loads a checkpoint and lets you
watch the trained agent live in the terminal.

View File

@@ -5,7 +5,7 @@ Game description fields
-----------------------
Game descriptions are written in the ``[tool.retro-gamer]`` section of
your game project's ``pyproject.toml``. ``retro-gamer create`` reads
your game project's ``pyproject.toml``. ``retro-gamer init`` reads
this section and copies the metadata into the training run's
``config.toml``, where it can also be inspected or hand-edited.
@@ -14,6 +14,7 @@ A complete example for the Snake game:
.. code-block:: toml
[tool.retro-gamer]
factory = "snake:create_game"
actions = ["KEY_RIGHT", "KEY_UP", "KEY_LEFT", "KEY_DOWN"]
reward = "score"
character_set = ["@", "*", ">", "<", "^", "v"]
@@ -23,6 +24,17 @@ directly from your game's ``board_size`` attribute.
The fields are described below.
``factory``
~~~~~~~~~~~
**Required.** A ``"module:attr"`` string naming the function that creates
a fresh game instance. The function must take no arguments and return a
new :class:`retro.game.Game`.
.. code-block:: toml
factory = "snake:create_game"
``actions``
~~~~~~~~~~~
@@ -69,7 +81,7 @@ Preprocessing options
Preprocessing options live in the ``[preprocessing]`` section of a run's
``config.toml``. They control how the game's board and state are
transformed into the observation vector that the neural network sees.
``retro-gamer create`` writes sensible defaults; you can edit them by
``retro-gamer init`` writes sensible defaults; you can edit them by
hand before running ``retro-gamer train``.
.. note::
@@ -119,18 +131,17 @@ element each; list or tuple values are flattened.
observe_state = ["apple_dx", "apple_dy"]
The keys must be present in ``game.state`` at every step, initialized
in ``create_game()`` before the game starts. All values that are lists
or tuples must always have the same length from episode to episode.
before the game starts. All values that are lists or tuples must always
have the same length from episode to episode.
.. warning::
``observe_state`` keys must be initialized to their final shape in
``create_game()`` before the game starts. If a key is absent or its
list length changes between episodes, training will crash with an
error explaining which key changed and by how much. This happens
because the neural network's input layer has a fixed size determined
at the start of training; it cannot adapt to a changing observation
shape mid-run.
``observe_state`` keys must be initialized to their final shape before
the game starts. If a key is absent or its list length changes between
episodes, training will crash with an error explaining which key
changed and by how much. This happens because the neural network's
input layer has a fixed size determined at the start of training; it
cannot adapt to a changing observation shape mid-run.
Always initialize every observed key with a placeholder of the
correct type and length before the first ``game.step()`` call.
@@ -160,14 +171,20 @@ one vector. ``character_set`` and ``board_size`` stay required either way,
because that's what lets ``observation_function`` also use a spatial
(``spatial = true``) network — the trainer slices the flat vector back into
a board tensor using exactly those two fields, the same way it does for the
built-in encoder. The size of whatever comes after the board (``extras_size``)
is not configured; it's measured automatically from one sampled observation
when training starts.
built-in encoder.
This is also how you get an egocentric (cropped, player-centered) board now —
there's no longer a built-in flag for it. Call ``egocentric_board()`` and
``encode_board()`` yourself, from :mod:`retro_gamer.observation`, inside your
own function, and declare ``board_size`` to match your crop:
``observation_function`` is the training-side complement to the game's
state dict. Use ``observe_state`` when the game already computes the
features you want and stores them in ``game.state``; use
``observation_function`` when you want to transform the game's board or
state into a representation that is more useful for learning — for example,
mapping all obstacle types to a single character, or cropping the board to
an egocentric window centred on the player.
This is how you get an egocentric (cropped, player-centered) board —
call ``egocentric_board()`` and ``encode_board()`` yourself, from
:mod:`retro_gamer.observation`, inside your own function, and declare
``board_size`` to match your crop:
.. code-block:: python
@@ -194,7 +211,7 @@ own function, and declare ``board_size`` to match your crop:
board_size = [17, 17] # 2*RADIUS + 1
observation_function = "my_module:egocentric_observation"
Outside DQN training — for example, BabySnake's tabular Q-learning lab, which
Outside DQN training — for example, a tabular Q-learning lab that
uses :class:`~retro_gamer.GameEnvironment` directly without
:class:`~retro_gamer.DQNTrainer` — there's no 1-D requirement at all.
``observation_function`` can return anything you want to use as your
@@ -210,10 +227,11 @@ Hyperparameters are split across two sections of ``config.toml``:
- ``[model]`` — network architecture (changing these requires starting fresh)
- ``[training]`` — learning algorithm parameters (safe to change at any time)
Both sections can be set via ``retro-gamer create`` options or edited directly.
Both sections can be set via ``retro-gamer init`` options or edited directly
in ``config.toml`` between ``init`` and ``train``.
Learning and optimization
~~~~~~~~~~~~~~~~~~~~~~~~~
Learning and optimization (``[training]``)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
``learning_rate`` (default: ``0.0001``)
The step size used by the Adam optimizer when updating network
@@ -232,8 +250,8 @@ Learning and optimization
agent value all future rewards equally; smaller values make the
agent increasingly myopic.
Exploration
~~~~~~~~~~~
Exploration (``[training]``)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
``epsilon`` (default: ``1.0``)
The initial exploration rate. At each turn, the agent takes a
@@ -248,8 +266,8 @@ Exploration
continued exploration prevents the agent from becoming permanently
committed to a suboptimal policy.
Memory and sampling
~~~~~~~~~~~~~~~~~~~
Memory and sampling (``[training]``)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
``batch_size`` (default: ``64``)
The number of experiences sampled from the replay buffer per
@@ -265,11 +283,11 @@ Memory and sampling
This often improves sample efficiency at a modest computational
cost.
Model architecture
~~~~~~~~~~~~~~~~~~
Model architecture (``[model]``)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
These live in the ``[model]`` section. Changing them requires starting fresh
(run ``retro-gamer clean`` before retraining).
Changing any ``[model]`` option requires starting fresh (run
``retro-gamer clean`` before retraining).
``hidden_sizes`` (default: ``[128, 64]``)
A list of integers giving the size of each hidden layer in the MLP
@@ -278,8 +296,8 @@ These live in the ``[model]`` section. Changing them requires starting fresh
network. Larger or deeper networks can represent more complex
Q-functions but train more slowly and may need more episodes.
Training duration
~~~~~~~~~~~~~~~~~
Training duration (``[training]``)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
``training_episodes`` (default: ``20000``)
The total number of game episodes to run. Each episode runs until
@@ -302,8 +320,8 @@ Training duration
experience. The default of 4 is a good balance for most games;
set to 1 to train on every step.
Character discovery
~~~~~~~~~~~~~~~~~~~
Character discovery (``[training]``)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
``exploration_turns`` (default: ``200``)
When ``character_set`` is not specified, the number of random
@@ -319,8 +337,8 @@ Character discovery
CLI reference
-------------
``retro-gamer create``
~~~~~~~~~~~~~~~~~~~~~~
``retro-gamer init``
~~~~~~~~~~~~~~~~~~~~~
Create a new training run directory with ``config.toml``. Game metadata
is read automatically from the ``[tool.retro-gamer]`` section of your
@@ -328,19 +346,20 @@ game's ``pyproject.toml``; you do not pass it on the command line.
.. code-block:: console
% retro-gamer create --game GAME --output DIR [OPTIONS]
% retro-gamer init GAME OUTPUT [OPTIONS]
**Required options:**
**Required arguments:**
- ``--game GAME`` — Your game, specified as a file path or a Python
- ``GAME`` — Your game, specified as a directory path or a Python
module name:
- File path: ``--game my_game.py`` or ``--game my_game/``
- Module name: ``--game retro.examples.snake``
- Directory: ``games/snake``
- Module name: ``retro.examples.snake``
The ``[tool.retro-gamer]`` section is read from the ``pyproject.toml``
found in or above the game file.
- ``--output DIR`` — Directory to create for this training run.
found in or above the game directory.
- ``OUTPUT`` — Directory to create for this training run
(e.g. ``training/snake``).
**Hyperparameter options** (all optional; see :ref:`hyperparameters`):
@@ -369,7 +388,7 @@ Train a DQN agent.
% retro-gamer train RUN_DIR
``RUN_DIR`` must contain a ``config.toml`` generated by ``retro-gamer
create``. If checkpoints already exist in ``RUN_DIR``, training
init``. If checkpoints already exist in ``RUN_DIR``, training
automatically resumes from the latest one so prior work is never lost.
If all configured episodes have already been completed, the command
@@ -381,8 +400,8 @@ unusable. If you change any of the following, ``retro-gamer train`` will
detect the mismatch and refuse to resume, with a clear explanation:
- ``actions``, ``reward``, ``character_set``, ``board_size``,
``observation_function``, ``extras_size`` (``[metadata]``) — game
description and observation shape
``observation_function`` (``[metadata]``) — game description and
observation shape
- ``spatial``, ``board``, ``observe_state`` (``[preprocessing]``) —
observation encoding
- ``hidden_sizes`` (``[model]``) — network architecture
@@ -439,7 +458,7 @@ contents:
.. code-block:: text
runs/snake/
training/snake/
├── config.toml # game description + hyperparameters
├── training.log # architecture rationale + per-episode log
└── checkpoints/
@@ -447,11 +466,11 @@ contents:
├── ep_0200.pt
└── ... # one file saved every 100 episodes
``config.toml`` is written by ``retro-gamer create`` and updated (with
``config.toml`` is written by ``retro-gamer init`` and updated (with
the discovered character set and resolved hyperparameters) when
``retro-gamer train`` begins. It has five sections: ``[game]``,
``[metadata]``, ``[preprocessing]``, ``[model]``, and ``[training]``.
Editing ``config.toml`` between ``create`` and ``train`` is the
Editing ``config.toml`` between ``init`` and ``train`` is the
recommended way to adjust hyperparameters.
``training.log`` begins with the full network architecture description,
@@ -489,5 +508,5 @@ library. See the :doc:`api` reference for full details.
from retro.examples.snake import create_game
metadata = GameMetadata.from_pyproject("retro.examples.snake")
trainer = DQNTrainer(create_game, metadata, "runs/snake/")
trainer = DQNTrainer(create_game, metadata, "training/snake/")
trainer.train()

View File

@@ -21,32 +21,29 @@ You will need:
Preparing your game
-------------------
``retro-gamer`` loads your game by calling a function named
``create_game``. The function must take no arguments and return a new
``Game`` instance.
``retro-gamer`` loads your game by calling a *factory function* — a
function that takes no arguments and returns a new ``Game`` instance.
You declare this function in ``[tool.retro-gamer]``:
Here is the ``create_game`` function for Snake:
.. code-block:: toml
[tool.retro-gamer]
factory = "snake:create_game"
The factory function itself lives in your game's Python module:
.. code-block:: python
def create_game():
head = SnakeHead()
apple = Apple()
game = Game([head, apple], {'score': 100}, board_size=(32, 16), framerate=12)
game = Game([head, apple], {'score': 0}, board_size=(32, 16), framerate=12)
apple.relocate(game)
return game
If your game file does not already have a ``create_game`` function, add
one following this pattern.
When you run ``retro-gamer create``, you can point to your game file
directly by path or by Python module name:
.. code-block:: console
% retro-gamer create --game my_game.py --output runs/my_game/
% retro-gamer create --game retro.examples.snake --output runs/snake/
The ``"snake:create_game"`` string follows the ``"module:attr"`` format
used throughout Python's packaging ecosystem: ``snake`` is the importable
module name and ``create_game`` is the attribute within it.
Describing your game
--------------------
@@ -62,12 +59,21 @@ Here is the ``[tool.retro-gamer]`` section for the Snake example:
.. code-block:: toml
[tool.retro-gamer]
factory = "snake:create_game"
actions = ["KEY_RIGHT", "KEY_UP", "KEY_LEFT", "KEY_DOWN"]
reward = "score"
character_set = ["@", "*", ">", "<", "^", "v"]
Let's go through each field.
``factory``
~~~~~~~~~~~
The ``"module:attr"`` string naming the function that creates a fresh
game instance. ``retro-gamer`` calls this function at the start of each
training episode and whenever it needs to inspect the game (for example,
to discover the board size).
``actions``
~~~~~~~~~~~
@@ -129,7 +135,7 @@ The ``[tool.retro-gamer]`` section describes the game. Preprocessing
options—such as ``spatial`` (whether to use a CNN or MLP, default:
``false``) and ``observe_state``—live in the ``[preprocessing]`` section of
the generated ``config.toml``. You can edit them there after running
``retro-gamer create``. For full control over the observation (for example,
``retro-gamer init``. For full control over the observation (for example,
a cropped/egocentric board), write an ``observation_function`` instead — see :ref:`observation-function` in the
reference docs for details.
@@ -151,7 +157,7 @@ board encoding (or uses them as the entire observation when
``board = false``).
These values must be set in ``game.state`` at the start of every
episode—typically inside ``create_game()``—and must keep the same
episode—typically inside the factory function—and must keep the same
type and length from episode to episode.
.. warning::
@@ -171,21 +177,19 @@ Once you have written this section, create the training run directory:
.. code-block:: console
% retro-gamer create \
--game retro.examples.snake \
--output runs/snake/
% retro-gamer init games/snake training/snake
Created training run at runs/snake/config.toml
game : retro.examples.snake
Initialized training run at training/snake/config.toml
game : games/snake
board_size : 32×16
actions : ['KEY_RIGHT', 'KEY_UP', 'KEY_LEFT', 'KEY_DOWN']
reward : score
characters : ['@', '*', '>', '<', '^', 'v']
architecture: MLP
``retro-gamer create`` reads your game metadata directly from
``retro-gamer init`` reads your game metadata directly from
``pyproject.toml`` and writes it—along with all hyperparameters—to
``runs/snake/config.toml``.
``training/snake/config.toml``.
Training the agent
------------------
@@ -194,16 +198,16 @@ With the ``config.toml`` in place, start training:
.. code-block:: console
% retro-gamer train runs/snake/
% retro-gamer train training/snake
100%|████████████████████| 1000/1000 [12:34<00:00, 1.32ep/s, reward=9.0, eps=0.007, loss=0.0003]
Done. Checkpoints saved in runs/snake/checkpoints/
Done. Checkpoints saved in training/snake/checkpoints/
A progress bar shows how far training has gone, along with the most
recent episode's reward, the current exploration rate (``eps``), and
the average prediction error (``loss``).
Training saves a checkpoint every 100 episodes to
``runs/snake/checkpoints/``. You can stop training at any time with
``training/snake/checkpoints/``. You can stop training at any time with
Ctrl-C and resume it later—the next ``retro-gamer train`` command will
automatically pick up from the latest checkpoint.
@@ -215,7 +219,7 @@ log:
.. code-block:: console
% cat runs/snake/training.log
% cat training/snake/training.log
The log begins with the full network architecture, followed by one line
per checkpoint (every 100 episodes):
@@ -264,7 +268,7 @@ checkpoint is always available immediately:
.. code-block:: console
% retro-gamer play runs/snake/
% retro-gamer play training/snake
This loads the most recent checkpoint and runs the agent in your
terminal. Press Enter or Escape to quit.
@@ -280,7 +284,7 @@ To watch an earlier stage of training, use ``--checkpoint``:
.. code-block:: console
% retro-gamer play runs/snake/ --checkpoint ep_0100
% retro-gamer play training/snake --checkpoint ep_0100
Comparing what the agent at episode 100 does versus the agent at episode
500 can reveal exactly what the agent has (and has not) learned. For
@@ -298,7 +302,7 @@ command you used before:
.. code-block:: console
% retro-gamer train runs/snake/
% retro-gamer train training/snake
``retro-gamer`` automatically detects and resumes from the latest
checkpoint. No extra flags are needed. If all configured episodes have
@@ -309,7 +313,7 @@ already been completed, it prints a message and exits:
Training already complete (1000 episodes). To keep training,
increase training_episodes in config.toml.
To continue training, open ``runs/snake/config.toml``, increase the
To continue training, open ``training/snake/config.toml``, increase the
``training_episodes`` value, and run ``retro-gamer train`` again.
Watching a trained agent play
@@ -319,15 +323,15 @@ Once training is complete, watch the final agent:
.. code-block:: console
% retro-gamer play runs/snake/
% retro-gamer play training/snake
By default the latest checkpoint is loaded. You can also compare the
agent's performance at different stages of training:
.. code-block:: console
% retro-gamer play runs/snake/ --checkpoint ep_0100
% retro-gamer play runs/snake/ --checkpoint ep_0500
% retro-gamer play training/snake --checkpoint ep_0100
% retro-gamer play training/snake --checkpoint ep_0500
Press Enter or Escape to quit.
@@ -338,8 +342,8 @@ To review the configuration and recent training progress for a run:
.. code-block:: console
% retro-gamer info runs/snake/
Game module : retro.examples.snake
% retro-gamer info training/snake
Game module : snake
Metadata : {'actions': ['KEY_RIGHT', ...], 'reward': 'score', 'board_size': [32, 16], ...}
Preprocessing : {'spatial': False, 'board': True, 'observe_state': ['apple_dx', 'apple_dy'], ...}
Model : {'hidden_sizes': [128, 64]}
@@ -358,8 +362,8 @@ Adjusting hyperparameters
--------------------------
The training hyperparameters can be changed by editing ``config.toml``
before training, or by passing them as options to ``retro-gamer
create``. Common adjustments and their effects:
before training, or by passing them as options to ``retro-gamer init``.
Common adjustments and their effects:
**``training_episodes``** — How long to train. More episodes give the
agent more time to learn, but also take longer to run. This is always
@@ -411,7 +415,7 @@ game or the shape of the network. The saved model weights are
incompatible with the new configuration:
- ``actions``, ``reward``, ``character_set``, ``board_size``,
``observation_function``, ``extras_size`` (``[metadata]``) — These define
``observation_function`` (``[metadata]``) — These define
what the agent perceives and what it can do. Changing them changes the
size of the network's input or output layers; the existing weights no
longer fit.
@@ -443,18 +447,18 @@ To clear out the old checkpoints and begin again:
.. code-block:: console
% retro-gamer clean runs/snake/
Will remove 5 checkpoint(s) and training log from runs/snake/:
% retro-gamer clean training/snake
Will remove 5 checkpoint(s) and training log from training/snake/:
checkpoints/ep_0100.pt
checkpoints/ep_0200.pt
...
training.log
Proceed? [y/N]: y
Cleaned. Run 'retro-gamer train runs/snake/' to start fresh.
Cleaned. Run 'retro-gamer train training/snake/' to start fresh.
The ``config.toml`` is always preserved so you do not need to run
``retro-gamer create`` again.
``retro-gamer init`` again.
Reasoning about training from the log
--------------------------------------
@@ -528,4 +532,3 @@ concepts underlying the training algorithm.
episode 1000 and watch each play the same game. What has the later
agent learned that the earlier one has not? How would you describe
this difference to someone who does not know about neural networks?