29 lines
1.0 KiB
Markdown
29 lines
1.0 KiB
Markdown
# Questions
|
||
|
||
## Checkpoint 1
|
||
|
||
1. How do you decide where to move in BabySnake? Explain how to choose moves
|
||
in enough detail that someone else could follow your instructions.
|
||
|
||
## Checkpoint 2
|
||
|
||
2. How many distinct states are there for BabySnake on a 4×4 grid? If we assume that all four
|
||
arrow keys are valid actions in every state, how many rows would the full Q-table contain?
|
||
|
||
3. The discount factor γ (gamma) can range from 0 to 1. What would be the effect of setting
|
||
γ to 0? What about 1?
|
||
|
||
4. The learning rate α (alpha) can also range from 0 to 1. What would be the effect of setting
|
||
α to 0? What about 1?
|
||
|
||
5. Calculate the new Q-value for ((2, 2, 3, 3), RIGHT). Explain your answer.
|
||
|
||
## Checkpoint 3
|
||
|
||
6. At what episode did the agent start reliably finding food?
|
||
|
||
7. Add `print(Q)` to `train_babysnake.py` before the `watch` call and run it again. Can you
|
||
read the policy? For a given state, does the highest Q-value point toward the food?
|
||
|
||
8. How does the trained agent's behavior compare to the reasoning you wrote down in question 1?
|