LightGBM
Fast gradient boosting with leaf-wise tree growth
LightGBM uses a leaf-wise growth strategy and histogram-based splitting, making it significantly faster than traditional boosting on large datasets while often achieving better accuracy.
When to use:
- Large datasets where XGBoost or Gradient Boosting are too slow
- High-cardinality categorical features (handled natively)
- When speed and memory efficiency are priorities alongside accuracy
Input: Tabular data with the feature columns defined during training Output: Predicted class label and class probabilities
Model Settings (set during training, used at inference)
N Estimators (default: 100) Number of boosting rounds.
Max Depth (default: -1 — unlimited) Maximum tree depth. Leaf-wise growth often doesn't require depth limits, but setting this can prevent overfitting.
Learning Rate (default: 0.1) Shrinkage rate per boosting step.
Num Leaves (default: 31) Maximum number of leaves per tree. Key parameter for LightGBM — increase for more complex models.
Min Child Samples (default: 20) Minimum data in a leaf. Higher values regularize the model.
Subsample (default: 1.0) Row sampling fraction per iteration.
Col Sample By Tree (default: 1.0) Feature sampling fraction per iteration.
Class Weight (default: null)
Set to balanced for imbalanced datasets.
Inference Settings
No dedicated inference-time settings. The trained LightGBM model produces predictions directly.