4.7 KiB
Digit Classifier: Questions
Checkpoint 1: Hand-Picked Features
1. What features did your group design? List each one and what it measures:
| Feature | What it measures |
|---|---|
2. Record your results:
| Metric | Value |
|---|---|
| Test accuracy | |
| F1 (macro) |
3. What accuracy did your hand-designed features achieve? How does that compare to random guessing across ten digits (10%), and to your results in the spam lab?
Your answer:
4. Which digits were classified correctly most often? Which were most often confused for each other? (Use digits models.features.FeatureClassifier -a to see misclassified examples.)
Your answer:
5. What makes it harder to design features for digits than for spam messages?
Your answer:
Checkpoint 2: Every Pixel as a Feature
6. Record your results:
| Metric | Value |
|---|---|
| Test accuracy | |
| F1 (macro) |
7. How did the pixel classifier's accuracy compare to your hand-designed features? What do you think accounts for the difference?
Your answer:
8. The pixel classifier learns 784 independent weights, one per pixel position—the same way FeatureClassifier learned one weight per hand-picked feature. Why might treating pixels as independent features cause trouble for a photo where the digit isn't perfectly centered?
Your answer:
Checkpoint 3: Single Hidden Layer
9. Record your results for at least three combinations of hidden layer size and number of epochs:
| Hidden size | Epochs | Test accuracy |
|---|---|---|
10. What happened with a very small hidden layer (8 or 16 neurons)? With a very large one (512)? What does that suggest about what the hidden layer is doing?
Your answer:
11. Each hidden neuron is a learned feature. How does the number of features available to this network compare to the one or two you designed by hand in Checkpoint 1—and how does that help explain the difference in accuracy?
Your answer:
Checkpoint 4: Multi-Layer Perceptron
12. Sketch your MLP architecture here (fill in the layer sizes you used):
Input layer: _____ neurons (one per pixel)
Hidden layer 1: _____ neurons, ReLU activation
Hidden layer 2: _____ neurons, ReLU activation [if you used one]
Output layer: _____ neurons (one per digit)
13. Record your best results:
| Hidden sizes | Epochs | Val accuracy (final) | Test accuracy | F1 (macro) |
|---|---|---|---|---|
14. Both the MLP and the pixel classifier see the same 784 numbers. What does the MLP do with them that the pixel classifier cannot?
Your answer:
15. The MLP still flattens the image into a vector of 784 numbers before its first layer ever sees it—it has no idea that pixel 0 and pixel 28 are vertical neighbors. Did stacking layers fix the limitation you identified in Checkpoint 2, or just hide it better?
Your answer:
Final Questions
16. Sketch the CNN architecture (label each layer with its type and dimensions):
Input: ___x___x___ (height × width × channels)
↓
Conv layer 1: ___ filters, ___x___ kernel, stride ___ → output: ___x___x___
↓
Conv layer 2: ___ filters, ___x___ kernel, stride ___ → output: ___x___x___
↓
Flatten: _____ values
↓
Fully connected: _____ → 10
↓
Output: 10 class probabilities (softmax)
17. Record your CNN results:
| Epochs | Val accuracy (final) | Test accuracy | F1 (macro) |
|---|---|---|---|
18. Fill in the final comparison table with every classifier you built in this lab:
| Classifier | Hyperparameters | Test accuracy | F1 (macro) | Notes |
|---|---|---|---|---|
| Hand-picked features | ||||
| Every pixel | ||||
| MLP | hidden= | |||
| CNN |
19. Architecture comparison: the MLP and CNN both ultimately process the same 784-pixel images, but the CNN reliably outperforms the MLP. What does the CNN know about images that the MLP does not?
Your answer:
20. Model selection: if you needed to deploy a digit classifier on a device with very limited memory and compute (e.g., a microcontroller), which classifier would you choose, and why? (Consider model size, prediction speed, and accuracy.)
Your answer:
21. Real-world applications: CNNs are used for object detection, face recognition, and medical imaging. What properties of CNNs make them well suited for these applications—and what would have to change to handle images that aren't neatly centered and cropped, the way MNIST's are?
Your answer: