mwc/lab_reinforcement_learning
Template
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Refactor lab

Browse Source
This commit is contained in:
Chris Proctor
2026-06-25 21:10:13 -04:00
parent 8294311d4b
commit e752bb848b
4 changed files with 74 additions and 95 deletions
Download Patch File Download Diff File Expand all files Collapse all files

43
questions.md
View File

@@ -1,13 +1,13 @@
# Questions
## BabySnake
## Checkpoint 1: Before training
## Checkpoint 1
1. How do you decide where to move in BabySnake? Explain how to choose moves
in enough detail that someone else could follow your instructions.
2. How many distinct states are there for BabySnake? If we assume that all four
## Checkpoint 2
2. How many distinct states are there for BabySnake on a 4×4 grid? If we assume that all four
arrow keys are valid actions in every state, how many rows would the full Q-table contain?
3. The discount factor γ (gamma) can range from 0 to 1. What would be the effect of setting
@@ -16,36 +16,13 @@
4. The learning rate α (alpha) can also range from 0 to 1. What would be the effect of setting
α to 0? What about 1?
5. Calculate the new Q-value for the situation described. Explain your answer.
5. Calculate the new Q-value for ((2, 2, 3, 3), RIGHT). Explain your answer.
6. Implement `choose_action` and `update_q` in `q_learning.py`, then run
## Checkpoint 3
```
python test_q_learning.py
```
6. At what episode did the agent start reliably finding food?
Get every test passing before moving on — errors are much easier to track
down here than during training.
---
## Checkpoint 2: After training
Train your Q-learning agent to consistently score 3 or more food items per
episode, then watch it play:
```
python train_babysnake.py
```
**At what episode did the agent start reliably finding food?**
**Print `q_table` after training. Can you read the policy?** For a state you
pick, does the highest Q-value point toward the food?
**How does the trained agent's behavior compare to the reasoning you wrote
down in Checkpoint 1?**
7. Add `print(Q)` to `train_babysnake.py` before the `watch` call and run it again. Can you
read the policy? For a given state, does the highest Q-value point toward the food?
8. How does the trained agent's behavior compare to the reasoning you wrote down in question 1?