K-Nearest Neighbors

Predicts based on the average of the K closest training examples.

When to use:

• Small datasets
• Irregular patterns
• Need simple baseline
• No training phase needed

Strengths: Simple, no training time, non-parametric, naturally handles complex patterns Weaknesses: Slow prediction, requires feature scaling, curse of dimensionality, memory intensive

Model Parameters

N Neighbors (default: 5) Number of neighbors to average. Larger = smoother predictions.

• 3-5: Flexible
• 5-20: Standard
• 20+: Very smooth, may underfit

Weights

• uniform: All neighbors equal (default)
• distance: Closer neighbors more important

Algorithm

• auto: Choose automatically (default)
• ball_tree: Good for low dimensions
• kd_tree: Fast for low dimensions
• brute: Exhaustive search (slow but exact)

P (default: 2) Distance metric (1 = Manhattan, 2 = Euclidean).