130 lines
4.1 KiB
Markdown
130 lines
4.1 KiB
Markdown
# Digit Classifier: Questions
|
||
|
||
---
|
||
|
||
## Checkpoint 1: Hand-Picked Features
|
||
|
||
**1. What features did your group design? List each one and what it measures:**
|
||
|
||
| Feature | What it measures |
|
||
|---------|-----------------|
|
||
| | |
|
||
| | |
|
||
|
||
**2. Record your results:**
|
||
|
||
| Metric | Value |
|
||
|--------|-------|
|
||
| Test accuracy | |
|
||
| F1 (macro) | |
|
||
|
||
**3. What accuracy did your hand-designed features achieve? How does that compare to random guessing across ten digits (10%), and to your results in the spam lab?**
|
||
|
||
*Your answer:*
|
||
|
||
**4. Which digits were classified correctly most often? Which were most often confused for each other? (Use `digits models.handpicked.HandPickedClassifier -a` to see misclassified examples.)**
|
||
|
||
*Your answer:*
|
||
|
||
**5. What makes it harder to design features for digits than for spam messages?**
|
||
|
||
*Your answer:*
|
||
|
||
---
|
||
|
||
## Checkpoint 2: Every Pixel as a Feature
|
||
|
||
**6. Record your results:**
|
||
|
||
| Metric | Value |
|
||
|--------|-------|
|
||
| Test accuracy | |
|
||
| F1 (macro) | |
|
||
|
||
**7. How did the pixel classifier's accuracy compare to your hand-designed features? What do you think accounts for the difference?**
|
||
|
||
*Your answer:*
|
||
|
||
**8. The pixel classifier learns 784 independent weights, one per pixel position—the same way `FeatureClassifier` learned one weight per hand-picked feature. Why might treating pixels as independent features cause trouble for a photo where the digit isn't perfectly centered?**
|
||
|
||
*Your answer:*
|
||
|
||
---
|
||
|
||
## Checkpoint 3: Multi-Layer Perceptron
|
||
|
||
**9. Sketch your MLP architecture here (fill in the layer sizes you used):**
|
||
|
||
```
|
||
Input layer: _____ neurons (one per pixel)
|
||
Hidden layer 1: _____ neurons, ReLU activation
|
||
Hidden layer 2: _____ neurons, ReLU activation [if you used one]
|
||
Output layer: _____ neurons (one per digit)
|
||
```
|
||
|
||
**10. Record your best results:**
|
||
|
||
| Hidden sizes | Epochs | Val accuracy (final) | Test accuracy | F1 (macro) |
|
||
|-------------|--------|---------------------|--------------|------------|
|
||
| | | | | |
|
||
| | | | | |
|
||
|
||
**11. Both the MLP and the pixel classifier see the same 784 numbers. What does the MLP do with them that the pixel classifier cannot?**
|
||
|
||
*Your answer:*
|
||
|
||
**12. The MLP still flattens the image into a vector of 784 numbers before its first layer ever sees it—it has no idea that pixel 0 and pixel 28 are vertical neighbors. Did stacking layers fix the limitation you identified in Checkpoint 2, or just hide it better?**
|
||
|
||
*Your answer:*
|
||
|
||
---
|
||
|
||
## Final Questions
|
||
|
||
**13. Sketch the CNN architecture (label each layer with its type and dimensions):**
|
||
|
||
```
|
||
Input: ___x___x___ (height × width × channels)
|
||
↓
|
||
Conv layer 1: ___ filters, ___x___ kernel → output: ___x___x___
|
||
↓
|
||
Pooling: ___x___ max pool → output: ___x___x___
|
||
↓
|
||
Conv layer 2: ___ filters, ___x___ kernel → output: ___x___x___
|
||
↓
|
||
Pooling: ___x___ max pool → output: ___x___x___
|
||
↓
|
||
Flatten: _____ values
|
||
↓
|
||
Fully connected: _____ → 10
|
||
↓
|
||
Output: 10 class probabilities (softmax)
|
||
```
|
||
|
||
**14. Record your CNN results:**
|
||
|
||
| Epochs | Val accuracy (final) | Test accuracy | F1 (macro) |
|
||
|--------|---------------------|--------------|------------|
|
||
| | | | |
|
||
|
||
**15. Fill in the final comparison table with every classifier you built in this lab:**
|
||
|
||
| Classifier | Hyperparameters | Test accuracy | F1 (macro) | Notes |
|
||
|-----------|----------------|--------------|------------|-------|
|
||
| Hand-picked features | | | | |
|
||
| Every pixel | | | | |
|
||
| MLP | hidden= | | | |
|
||
| CNN | | | | |
|
||
|
||
**16. Architecture comparison: the MLP and CNN both ultimately process the same 784-pixel images, but the CNN reliably outperforms the MLP. What does the CNN know about images that the MLP does not?**
|
||
|
||
*Your answer:*
|
||
|
||
**17. Model selection: if you needed to deploy a digit classifier on a device with very limited memory and compute (e.g., a microcontroller), which classifier would you choose, and why? (Consider model size, prediction speed, and accuracy.)**
|
||
|
||
*Your answer:*
|
||
|
||
**18. Real-world applications: CNNs are used for object detection, face recognition, and medical imaging. What properties of CNNs make them well suited for these applications—and what would have to change to handle images that aren't neatly centered and cropped, the way MNIST's are?**
|
||
|
||
*Your answer:*
|