e8a24ae7be
q_learning.py mixed a bespoke BabySnake environment wrapper, the two functions students implement, a training loop, and a terminal watch routine in one file, with no tests and no single command to run training. - q_learning.py / q_learning_solution.py now hold only choose_action and update_q, with actions as an explicit parameter instead of a module-global, so the file has no babysnake/retro/retro_gamer imports at all. - train_babysnake.py builds GameEnvironment directly from babysnake's pyproject metadata, trains, and watches the trained agent in one command: `python train_babysnake.py`. It also caps steps per episode, since a lucky random walk that keeps finding food can otherwise make an episode run unboundedly long. - test_q_learning.py adds unittest coverage for both functions with no game dependency. - questions.md adds a checkpoint instructing students to get the tests passing before training, and points the post-training step at train_babysnake.py.
19 lines
673 B
Python
19 lines
673 B
Python
"""Solution for q_learning.py — remove before publishing to students."""
|
|
|
|
import random
|
|
|
|
|
|
def choose_action(q_table, state, actions, epsilon):
|
|
if random.random() < epsilon:
|
|
return random.choice(actions)
|
|
q_values = [q_table.get((state, a), 0.0) for a in actions]
|
|
return actions[q_values.index(max(q_values))]
|
|
|
|
|
|
def update_q(q_table, state, action, reward, next_state, actions, alpha, gamma):
|
|
old_q = q_table.get((state, action), 0.0)
|
|
next_q_values = [q_table.get((next_state, a), 0.0) for a in actions]
|
|
best_next_q = max(next_q_values)
|
|
new_q = old_q + alpha * (reward + gamma * best_next_q - old_q)
|
|
q_table[(state, action)] = new_q
|