mwc/lab_reinforcement_learning
Template
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Give forager an energy mechanic so episodes terminate naturally

Browse Source 
Previously the game ran indefinitely and relied entirely on
max_turns_per_episode to end an episode. Add start/food energy values
so an episode ends on its own when energy runs out, and mention
retro-gamer plot in the lab's training log instructions.
This commit is contained in:
Chris Proctor
2026-06-24 07:50:14 -04:00
parent 42bc2e7a50
commit 3d42de7b12
2 changed files with 13 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
forager/__init__.py
View File

@@ -1,7 +1,8 @@
"""Forager: an 8×8 grid game where an agent collects food. """Forager: an 8×8 grid game where an agent collects food.
The agent moves in four directions collecting food that respawns on collection. The agent moves in four directions collecting food that respawns on collection.
The game runs indefinitely; retro-gamer's max_turns_per_episode controls episode length. The agent starts with 100 energy. Each step costs 1 energy. Collecting food
restores 40 energy. The episode ends when energy reaches 0.
Observation features for retro-gamer: Observation features for retro-gamer:
food_dx: (food_x - agent_x) / board_width (positive = food is to the right) food_dx: (food_x - agent_x) / board_width (positive = food is to the right)
@@ -12,6 +13,8 @@ from random import randint
from retro.game import Game from retro.game import Game
BOARD_SIZE = 8 BOARD_SIZE = 8
START_ENERGY = 100
FOOD_ENERGY = 40
class Forager: class Forager:
@@ -46,11 +49,13 @@ class Forager:
if game.on_board(new_pos): if game.on_board(new_pos):
self.position = new_pos self.position = new_pos
game.state['energy'] -= 1
game.state['reward'] -= 0.01 game.state['reward'] -= 0.01
food = game.get_agent_by_name("Food") food = game.get_agent_by_name("Food")
if self.position == food.position: if self.position == food.position:
food.relocate(game) food.relocate(game)
game.state['energy'] += FOOD_ENERGY
game.state['score'] += 1 game.state['score'] += 1
game.state['reward'] += 1.0 game.state['reward'] += 1.0
@@ -60,6 +65,9 @@ class Forager:
game.state['food_dx'] = (fx - ax) / bw game.state['food_dx'] = (fx - ax) / bw
game.state['food_dy'] = (fy - ay) / bh game.state['food_dy'] = (fy - ay) / bh
if game.state['energy'] <= 0:
game.end()
class Food: class Food:
"""The food item. Respawns when collected.""" """The food item. Respawns when collected."""
@@ -85,7 +93,7 @@ def create_game():
bw = bh = BOARD_SIZE bw = bh = BOARD_SIZE
game = Game( game = Game(
[forager, food], [forager, food],
{'score': 0, 'reward': 0.0, 'food_dx': 0.0, 'food_dy': 0.0}, {'score': 0, 'reward': 0.0, 'energy': START_ENERGY, 'food_dx': 0.0, 'food_dy': 0.0},
board_size=(bw, bh), board_size=(bw, bh),
framerate=12, framerate=12,
) )

5
training_log.md
View File

@@ -3,8 +3,9 @@
Document each training attempt below. For each attempt, write your hypothesis Document each training attempt below. For each attempt, write your hypothesis
before you run the experiment, then fill in the evidence and analysis after. before you run the experiment, then fill in the evidence and analysis after.
Use `retro-gamer info runs/forager/` to see a summary of your run, and Use `retro-gamer info runs/forager/` to see a summary of your run,
`cat runs/forager/training.log` to see the full log. `cat runs/forager/training.log` to see the full log, and
`retro-gamer plot runs/forager/ -o runs/forager/training.png` to graph it.
--- ---