Split q_learning.py into algorithm, environment glue, and a training script
q_learning.py mixed a bespoke BabySnake environment wrapper, the two functions students implement, a training loop, and a terminal watch routine in one file, with no tests and no single command to run training. - q_learning.py / q_learning_solution.py now hold only choose_action and update_q, with actions as an explicit parameter instead of a module-global, so the file has no babysnake/retro/retro_gamer imports at all. - train_babysnake.py builds GameEnvironment directly from babysnake's pyproject metadata, trains, and watches the trained agent in one command: `python train_babysnake.py`. It also caps steps per episode, since a lucky random walk that keeps finding food can otherwise make an episode run unboundedly long. - test_q_learning.py adds unittest coverage for both functions with no game dependency. - questions.md adds a checkpoint instructing students to get the tests passing before training, and points the post-training step at train_babysnake.py.
This commit is contained in:
51
questions.md
Normal file
51
questions.md
Normal file
@@ -0,0 +1,51 @@
|
||||
# Questions
|
||||
|
||||
## BabySnake
|
||||
|
||||
## Checkpoint 1: Before training
|
||||
|
||||
1. How do you decide where to move in BabySnake? Explain how to choose moves
|
||||
in enough detail that someone else could follow your instructions.
|
||||
|
||||
2. How many distinct states are there for BabySnake? If we assume that all four
|
||||
arrow keys are valid actions in every state, how many rows would the full Q-table contain?
|
||||
|
||||
3. The discount factor γ (gamma) can range from 0 to 1. What would be the effect of setting
|
||||
γ to 0? What about 1?
|
||||
|
||||
4. The learning rate α (alpha) can also range from 0 to 1. What would be the effect of setting
|
||||
α to 0? What about 1?
|
||||
|
||||
5. Calculate the new Q-value for the situation described. Explain your answer.
|
||||
|
||||
6. Implement `choose_action` and `update_q` in `q_learning.py`, then run
|
||||
|
||||
```
|
||||
python test_q_learning.py
|
||||
```
|
||||
|
||||
Get every test passing before moving on — errors are much easier to track
|
||||
down here than during training.
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Checkpoint 2: After training
|
||||
|
||||
Train your Q-learning agent to consistently score 3 or more food items per
|
||||
episode, then watch it play:
|
||||
|
||||
```
|
||||
python train_babysnake.py
|
||||
```
|
||||
|
||||
**At what episode did the agent start reliably finding food?**
|
||||
|
||||
|
||||
**Print `q_table` after training. Can you read the policy?** For a state you
|
||||
pick, does the highest Q-value point toward the food?
|
||||
|
||||
|
||||
**How does the trained agent's behavior compare to the reasoning you wrote
|
||||
down in Checkpoint 1?**
|
||||
|
||||
Reference in New Issue
Block a user