K-Nearest Neighbors
Train K-Nearest Neighbors to predict categorical outcomes
Classifies based on the K closest training examples.
When to use:
- Small datasets
- Irregular decision boundaries
- Need simple baseline
- No training phase needed
Strengths: Simple, no training time, naturally handles multiclass, non-parametric Weaknesses: Slow prediction, requires feature scaling, curse of dimensionality
Model Parameters
N Neighbors (default: 5) Number of neighbors to consider. Larger = smoother boundary.
- 3-5: Flexible boundary
- 5-20: Standard
- 20+: Very smooth, may underfit
Weights
- uniform: All neighbors equal (default)
- distance: Closer neighbors more important
Algorithm
- auto: Choose automatically (default)
- ball_tree: Good for low dimensions
- kd_tree: Fast for low dimensions
- brute: Exhaustive search (slow but exact)
P (default: 2) Power parameter for distance metric (1 = Manhattan, 2 = Euclidean).