mwc/retro-gamer
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Refactor CLI/interface: init command, factory field, remove extras_size

Browse Source 
- Rename `retro-gamer create` to `retro-gamer init` with positional
  args (GAME OUTPUT) instead of --game/--output flags
- Add [tool.retro-gamer].factory = "module:attr" to declare the game
  factory function from pyproject.toml instead of relying on a
  hard-coded create_game attribute
- Remove extras_size from user-facing config; it is now measured
  automatically from a sample observation and never declared
- Update docs throughout: create→init, runs/→training/, add factory
  field documentation, clarify [model] vs [training] hyperparameter
  sections, remove stale version-history explanations
- Bump version to 0.3.0; require retro-games>=2.5.0
This commit is contained in:
Chris Proctor
2026-06-26 06:55:30 -04:00
parent 0cd3c3b488
commit c89609fe77
12 changed files with 217 additions and 136 deletions
Download Patch File Download Diff File Expand all files Collapse all files

119
docs/reference.rst
View File

@@ -5,7 +5,7 @@ Game description fields
-----------------------
Game descriptions are written in the ``[tool.retro-gamer]`` section of
your game project's ``pyproject.toml``. ``retro-gamer create`` reads
your game project's ``pyproject.toml``. ``retro-gamer init`` reads
this section and copies the metadata into the training run's
``config.toml``, where it can also be inspected or hand-edited.
@@ -14,6 +14,7 @@ A complete example for the Snake game:
.. code-block:: toml
[tool.retro-gamer]
factory = "snake:create_game"
actions = ["KEY_RIGHT", "KEY_UP", "KEY_LEFT", "KEY_DOWN"]
reward = "score"
character_set = ["@", "*", ">", "<", "^", "v"]
@@ -23,6 +24,17 @@ directly from your game's ``board_size`` attribute.
The fields are described below.
``factory``
~~~~~~~~~~~
**Required.** A ``"module:attr"`` string naming the function that creates
a fresh game instance. The function must take no arguments and return a
new :class:`retro.game.Game`.
.. code-block:: toml
factory = "snake:create_game"
``actions``
~~~~~~~~~~~
@@ -69,7 +81,7 @@ Preprocessing options
Preprocessing options live in the ``[preprocessing]`` section of a run's
``config.toml``. They control how the game's board and state are
transformed into the observation vector that the neural network sees.
``retro-gamer create`` writes sensible defaults; you can edit them by
``retro-gamer init`` writes sensible defaults; you can edit them by
hand before running ``retro-gamer train``.
.. note::
@@ -119,18 +131,17 @@ element each; list or tuple values are flattened.
observe_state = ["apple_dx", "apple_dy"]
The keys must be present in ``game.state`` at every step, initialized
in ``create_game()`` before the game starts. All values that are lists
or tuples must always have the same length from episode to episode.
before the game starts. All values that are lists or tuples must always
have the same length from episode to episode.
.. warning::
``observe_state`` keys must be initialized to their final shape in
``create_game()`` before the game starts. If a key is absent or its
list length changes between episodes, training will crash with an
error explaining which key changed and by how much. This happens
because the neural network's input layer has a fixed size determined
at the start of training; it cannot adapt to a changing observation
shape mid-run.
``observe_state`` keys must be initialized to their final shape before
the game starts. If a key is absent or its list length changes between
episodes, training will crash with an error explaining which key
changed and by how much. This happens because the neural network's
input layer has a fixed size determined at the start of training; it
cannot adapt to a changing observation shape mid-run.
Always initialize every observed key with a placeholder of the
correct type and length before the first ``game.step()`` call.
@@ -160,14 +171,20 @@ one vector. ``character_set`` and ``board_size`` stay required either way,
because that's what lets ``observation_function`` also use a spatial
(``spatial = true``) network — the trainer slices the flat vector back into
a board tensor using exactly those two fields, the same way it does for the
built-in encoder. The size of whatever comes after the board (``extras_size``)
is not configured; it's measured automatically from one sampled observation
when training starts.
built-in encoder.
This is also how you get an egocentric (cropped, player-centered) board now —
there's no longer a built-in flag for it. Call ``egocentric_board()`` and
``encode_board()`` yourself, from :mod:`retro_gamer.observation`, inside your
own function, and declare ``board_size`` to match your crop:
``observation_function`` is the training-side complement to the game's
state dict. Use ``observe_state`` when the game already computes the
features you want and stores them in ``game.state``; use
``observation_function`` when you want to transform the game's board or
state into a representation that is more useful for learning — for example,
mapping all obstacle types to a single character, or cropping the board to
an egocentric window centred on the player.
This is how you get an egocentric (cropped, player-centered) board —
call ``egocentric_board()`` and ``encode_board()`` yourself, from
:mod:`retro_gamer.observation`, inside your own function, and declare
``board_size`` to match your crop:
.. code-block:: python
@@ -194,7 +211,7 @@ own function, and declare ``board_size`` to match your crop:
board_size = [17, 17] # 2*RADIUS + 1
observation_function = "my_module:egocentric_observation"
Outside DQN training — for example, BabySnake's tabular Q-learning lab, which
Outside DQN training — for example, a tabular Q-learning lab that
uses :class:`~retro_gamer.GameEnvironment` directly without
:class:`~retro_gamer.DQNTrainer` — there's no 1-D requirement at all.
``observation_function`` can return anything you want to use as your
@@ -210,10 +227,11 @@ Hyperparameters are split across two sections of ``config.toml``:
- ``[model]`` — network architecture (changing these requires starting fresh)
- ``[training]`` — learning algorithm parameters (safe to change at any time)
Both sections can be set via ``retro-gamer create`` options or edited directly.
Both sections can be set via ``retro-gamer init`` options or edited directly
in ``config.toml`` between ``init`` and ``train``.
Learning and optimization
~~~~~~~~~~~~~~~~~~~~~~~~~
Learning and optimization (``[training]``)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
``learning_rate`` (default: ``0.0001``)
The step size used by the Adam optimizer when updating network
@@ -232,8 +250,8 @@ Learning and optimization
agent value all future rewards equally; smaller values make the
agent increasingly myopic.
Exploration
~~~~~~~~~~~
Exploration (``[training]``)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
``epsilon`` (default: ``1.0``)
The initial exploration rate. At each turn, the agent takes a
@@ -248,8 +266,8 @@ Exploration
continued exploration prevents the agent from becoming permanently
committed to a suboptimal policy.
Memory and sampling
~~~~~~~~~~~~~~~~~~~
Memory and sampling (``[training]``)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
``batch_size`` (default: ``64``)
The number of experiences sampled from the replay buffer per
@@ -265,11 +283,11 @@ Memory and sampling
This often improves sample efficiency at a modest computational
cost.
Model architecture
~~~~~~~~~~~~~~~~~~
Model architecture (``[model]``)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
These live in the ``[model]`` section. Changing them requires starting fresh
(run ``retro-gamer clean`` before retraining).
Changing any ``[model]`` option requires starting fresh (run
``retro-gamer clean`` before retraining).
``hidden_sizes`` (default: ``[128, 64]``)
A list of integers giving the size of each hidden layer in the MLP
@@ -278,8 +296,8 @@ These live in the ``[model]`` section. Changing them requires starting fresh
network. Larger or deeper networks can represent more complex
Q-functions but train more slowly and may need more episodes.
Training duration
~~~~~~~~~~~~~~~~~
Training duration (``[training]``)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
``training_episodes`` (default: ``20000``)
The total number of game episodes to run. Each episode runs until
@@ -302,8 +320,8 @@ Training duration
experience. The default of 4 is a good balance for most games;
set to 1 to train on every step.
Character discovery
~~~~~~~~~~~~~~~~~~~
Character discovery (``[training]``)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
``exploration_turns`` (default: ``200``)
When ``character_set`` is not specified, the number of random
@@ -319,8 +337,8 @@ Character discovery
CLI reference
-------------
``retro-gamer create``
~~~~~~~~~~~~~~~~~~~~~~
``retro-gamer init``
~~~~~~~~~~~~~~~~~~~~~
Create a new training run directory with ``config.toml``. Game metadata
is read automatically from the ``[tool.retro-gamer]`` section of your
@@ -328,19 +346,20 @@ game's ``pyproject.toml``; you do not pass it on the command line.
.. code-block:: console
% retro-gamer create --game GAME --output DIR [OPTIONS]
% retro-gamer init GAME OUTPUT [OPTIONS]
**Required options:**
**Required arguments:**
- ``--game GAME`` — Your game, specified as a file path or a Python
- ``GAME`` — Your game, specified as a directory path or a Python
module name:
- File path: ``--game my_game.py`` or ``--game my_game/``
- Module name: ``--game retro.examples.snake``
- Directory: ``games/snake``
- Module name: ``retro.examples.snake``
The ``[tool.retro-gamer]`` section is read from the ``pyproject.toml``
found in or above the game file.
- ``--output DIR`` — Directory to create for this training run.
found in or above the game directory.
- ``OUTPUT`` — Directory to create for this training run
(e.g. ``training/snake``).
**Hyperparameter options** (all optional; see :ref:`hyperparameters`):
@@ -369,7 +388,7 @@ Train a DQN agent.
% retro-gamer train RUN_DIR
``RUN_DIR`` must contain a ``config.toml`` generated by ``retro-gamer
create``. If checkpoints already exist in ``RUN_DIR``, training
init``. If checkpoints already exist in ``RUN_DIR``, training
automatically resumes from the latest one so prior work is never lost.
If all configured episodes have already been completed, the command
@@ -381,8 +400,8 @@ unusable. If you change any of the following, ``retro-gamer train`` will
detect the mismatch and refuse to resume, with a clear explanation:
- ``actions``, ``reward``, ``character_set``, ``board_size``,
``observation_function``, ``extras_size`` (``[metadata]``) — game
description and observation shape
``observation_function`` (``[metadata]``) — game description and
observation shape
- ``spatial``, ``board``, ``observe_state`` (``[preprocessing]``) —
observation encoding
- ``hidden_sizes`` (``[model]``) — network architecture
@@ -439,7 +458,7 @@ contents:
.. code-block:: text
runs/snake/
training/snake/
├── config.toml # game description + hyperparameters
├── training.log # architecture rationale + per-episode log
└── checkpoints/
@@ -447,11 +466,11 @@ contents:
├── ep_0200.pt
└── ... # one file saved every 100 episodes
``config.toml`` is written by ``retro-gamer create`` and updated (with
``config.toml`` is written by ``retro-gamer init`` and updated (with
the discovered character set and resolved hyperparameters) when
``retro-gamer train`` begins. It has five sections: ``[game]``,
``[metadata]``, ``[preprocessing]``, ``[model]``, and ``[training]``.
Editing ``config.toml`` between ``create`` and ``train`` is the
Editing ``config.toml`` between ``init`` and ``train`` is the
recommended way to adjust hyperparameters.
``training.log`` begins with the full network architecture description,
@@ -489,5 +508,5 @@ library. See the :doc:`api` reference for full details.
from retro.examples.snake import create_game
metadata = GameMetadata.from_pyproject("retro.examples.snake")
trainer = DQNTrainer(create_game, metadata, "runs/snake/")
trainer = DQNTrainer(create_game, metadata, "training/snake/")
trainer.train()