mwc/lab_reinforcement_learning
Template
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Updates

Browse Source
This commit is contained in:
Chris Proctor
2026-06-26 20:59:25 -04:00
parent a9385f8296
commit 482f4f6cfa
5 changed files with 191 additions and 86 deletions
Download Patch File Download Diff File Expand all files Collapse all files

40
questions.md
View File

@@ -29,37 +29,41 @@
## Checkpoint 4
9. **Full board (runs/snake-v1, ep_5000):** Describe the agent's behavior. Does it seem to know
where the apple is? Does it move randomly or with some purpose?
9. In Attempt 1, the agent sees the full 32×16 board as 3,072 numbers—the apple's location
is already in there somewhere. In Attempt 2, we supplemented the board with just two extra
numbers: the direction to the apple. Performance tripled. Why did two extra numbers make
such a large difference when the board already contained the apple's location?
10. **Features only (runs/snake-v2, ep_3000):** How does this agent differ from the v1 agent?
What is it doing better? What is it doing that leads to shorter episodes?
10. Attempt 3 added the full board back and switched to a CNN—a more powerful
architecture—yet performance was worse than Attempt 1. Why didn't more information
and a more powerful model help?
11. **Final run, early (runs/snake, ep_1300):** This agent uses the egocentric view plus
apple_dx/apple_dy. What has it learned that neither v1 nor v2 showed?
11. The only difference between Attempt 3 and Attempt 4 is that Attempt 4 shows the agent a
17×17 window centered on its own head, rather than the full board. Why did this single
change make such a large difference?
12. **Final run, mature (runs/snake, ep_20000):** What does this agent do well? Where does it
still make mistakes?
12. When the snake's body gets very long, it becomes important to plan your route so you don't
get trapped inside your own body. None of our training attempts was very successful at
learning this behavior. Which of the approaches do you think would be most promising for
learning it? Why?
13. In the features-only run (v2), reward rose as episodes got shorter. Why does a snake agent
that is getting better at finding apples end up with shorter episodes?
14. The egocentric view crops the observation to a 17×17 window centered on the snake's head.
What did the agent gain from this change, and what information did it lose access to?
13. The reward function gives the snake +1 for each step it moves toward the apple and −1 for
each step away. Can you think of a way this reward signal might accidentally encourage bad
behavior—especially as the snake grows longer?
## Checkpoint 5
Answer these questions after completing both training experiments in "Training Frogger."
15. **Hypothesis (Attempt 1):** Before training, predict what will happen. Will the agent learn to
14. **Hypothesis (Attempt 1):** Before training, predict what will happen. Will the agent learn to
reach the top of the board? What challenge do you think it will face?
16. **Evidence (Attempt 1):** Copy the first three and last three lines of `runs/frogger/training.log`.
15. **Evidence (Attempt 1):** Copy the first three and last three lines of `runs/frogger/training.log`.
Did training go as expected?
17. **Analysis (Attempt 1):** What did the agent learn to do? Where did it struggle?
16. **Analysis (Attempt 1):** What did the agent learn to do? Where did it struggle?
18. **Experiment (Attempt 2):** What one thing did you change? Write your prediction, show the
17. **Experiment (Attempt 2):** What one thing did you change? Write your prediction, show the
evidence (first and last few log lines), and describe what happened.
19. Which attempt produced the best agent? What would you try next if you had more time?
18. Which attempt produced the best agent? What would you try next if you had more time?