e8a24ae7be
q_learning.py mixed a bespoke BabySnake environment wrapper, the two functions students implement, a training loop, and a terminal watch routine in one file, with no tests and no single command to run training. - q_learning.py / q_learning_solution.py now hold only choose_action and update_q, with actions as an explicit parameter instead of a module-global, so the file has no babysnake/retro/retro_gamer imports at all. - train_babysnake.py builds GameEnvironment directly from babysnake's pyproject metadata, trains, and watches the trained agent in one command: `python train_babysnake.py`. It also caps steps per episode, since a lucky random walk that keeps finding food can otherwise make an episode run unboundedly long. - test_q_learning.py adds unittest coverage for both functions with no game dependency. - questions.md adds a checkpoint instructing students to get the tests passing before training, and points the post-training step at train_babysnake.py.
1.4 KiB
Questions
BabySnake
Checkpoint 1: Before training
-
How do you decide where to move in BabySnake? Explain how to choose moves in enough detail that someone else could follow your instructions.
-
How many distinct states are there for BabySnake? If we assume that all four arrow keys are valid actions in every state, how many rows would the full Q-table contain?
-
The discount factor γ (gamma) can range from 0 to 1. What would be the effect of setting γ to 0? What about 1?
-
The learning rate α (alpha) can also range from 0 to 1. What would be the effect of setting α to 0? What about 1?
-
Calculate the new Q-value for the situation described. Explain your answer.
-
Implement
choose_actionandupdate_qinq_learning.py, then runpython test_q_learning.pyGet every test passing before moving on — errors are much easier to track down here than during training.
Checkpoint 2: After training
Train your Q-learning agent to consistently score 3 or more food items per episode, then watch it play:
python train_babysnake.py
At what episode did the agent start reliably finding food?
Print q_table after training. Can you read the policy? For a state you
pick, does the highest Q-value point toward the food?
How does the trained agent's behavior compare to the reasoning you wrote down in Checkpoint 1?