e8a24ae7be
q_learning.py mixed a bespoke BabySnake environment wrapper, the two functions students implement, a training loop, and a terminal watch routine in one file, with no tests and no single command to run training. - q_learning.py / q_learning_solution.py now hold only choose_action and update_q, with actions as an explicit parameter instead of a module-global, so the file has no babysnake/retro/retro_gamer imports at all. - train_babysnake.py builds GameEnvironment directly from babysnake's pyproject metadata, trains, and watches the trained agent in one command: `python train_babysnake.py`. It also caps steps per episode, since a lucky random walk that keeps finding food can otherwise make an episode run unboundedly long. - test_q_learning.py adds unittest coverage for both functions with no game dependency. - questions.md adds a checkpoint instructing students to get the tests passing before training, and points the post-training step at train_babysnake.py.
52 lines
1.4 KiB
Markdown
52 lines
1.4 KiB
Markdown
# Questions
|
||
|
||
## BabySnake
|
||
|
||
## Checkpoint 1: Before training
|
||
|
||
1. How do you decide where to move in BabySnake? Explain how to choose moves
|
||
in enough detail that someone else could follow your instructions.
|
||
|
||
2. How many distinct states are there for BabySnake? If we assume that all four
|
||
arrow keys are valid actions in every state, how many rows would the full Q-table contain?
|
||
|
||
3. The discount factor γ (gamma) can range from 0 to 1. What would be the effect of setting
|
||
γ to 0? What about 1?
|
||
|
||
4. The learning rate α (alpha) can also range from 0 to 1. What would be the effect of setting
|
||
α to 0? What about 1?
|
||
|
||
5. Calculate the new Q-value for the situation described. Explain your answer.
|
||
|
||
6. Implement `choose_action` and `update_q` in `q_learning.py`, then run
|
||
|
||
```
|
||
python test_q_learning.py
|
||
```
|
||
|
||
Get every test passing before moving on — errors are much easier to track
|
||
down here than during training.
|
||
|
||
|
||
---
|
||
|
||
## Checkpoint 2: After training
|
||
|
||
Train your Q-learning agent to consistently score 3 or more food items per
|
||
episode, then watch it play:
|
||
|
||
```
|
||
python train_babysnake.py
|
||
```
|
||
|
||
**At what episode did the agent start reliably finding food?**
|
||
|
||
|
||
**Print `q_table` after training. Can you read the policy?** For a state you
|
||
pick, does the highest Q-value point toward the food?
|
||
|
||
|
||
**How does the trained agent's behavior compare to the reasoning you wrote
|
||
down in Checkpoint 1?**
|
||
|