Fast, scalable, and scikit-learn compatible optimization for machine learning
ReHLine-Python is the official Python implementation of ReHLine, a powerful solver for large-scale empirical risk minimization (ERM) problems with convex piecewise linear-quadratic (PLQ) loss functions and linear constraints. Built with high-performance C++ core and seamless Python integration, ReHLine delivers exceptional speed while maintaining ease of use.
See more details in the ReHLine documentation.
- 🚀 Blazing Fast: Linear computational complexity per iteration, scales to millions of samples
- 🎯 Versatile: Supports any convex PLQ loss (hinge, check, Huber, and more)
- 🔒 Constrained Optimization: Handle linear equality and inequality constraints
- 📊 Scikit-Learn Compatible: Drop-in replacement with
GridSearchCV,Pipelinesupport - 🐍 Pythonic API: Both low-level and high-level interfaces for flexibility
pip install rehlineRelease wheels and CI cover standard CPython 3.10–3.14 on these platforms:
| Operating system | Architecture | Wheel family |
|---|---|---|
| Linux with glibc | x86-64 | manylinux |
| macOS (Apple Silicon) | ARM64 | macosx |
| Windows | x86-64, with 64-bit Python | win_amd64 |
We do not publish wheels for 32-bit systems, Alpine/musl, Intel macOS, Linux ARM, Windows ARM, or free-threaded Python. Alpine/musl lacks compatible scikit-learn wheels, and the extension has not been validated for free-threaded Python. Intel macOS and the other ARM targets are outside the current CI matrix. Source distributions remain available; builds on other targets are not covered by the release tests.
For contributors and developers:
git clone https://github.com/softmin/ReHLine-python.git
cd ReHLine-python
pip install -e ".[test]"To run tests:
pytest tests/
ReHLine provides plq_Ridge_Classifier and plq_Ridge_Regressor that work seamlessly with scikit-learn:
from rehline import plq_Ridge_Classifier
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
# Generate dataset
X, y = make_classification(n_samples=1000, n_features=20, random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
# Simple usage
clf = plq_Ridge_Classifier(loss={'name': 'svm'}, C=1.0)
clf.fit(X_train, y_train)
print(f"Accuracy: {clf.score(X_test, y_test):.3f}")
# Use in Pipeline
pipeline = Pipeline([
('scaler', StandardScaler()),
('classifier', plq_Ridge_Classifier(loss={'name': 'svm'}))
])
pipeline.fit(X_train, y_train)
# Hyperparameter tuning with GridSearchCV
param_grid = {
'C': [0.1, 1.0, 10.0],
'loss': [{'name': 'svm'}, {'name': 'sSVM'}]
}
grid_search = GridSearchCV(plq_Ridge_Classifier(loss={"name": "svm"}), param_grid, cv=5)
grid_search.fit(X_train, y_train)
print(f"Best params: {grid_search.best_params_}")See more details in ReHLine with Scikit-Learn.
from rehline import ReHLine
import numpy as np
# Generate sample data
np.random.seed(42)
X = np.random.randn(100, 5)
y = np.random.choice([-1, 1], size=100)
n, d = X.shape
C = 1.0
# Define custom PLQ loss parameters
clf = ReHLine()
# Set custom U, V matrices for ReLU loss
# and S, T, tau for ReHU loss
## U
clf._U = -(C*y).reshape(1,-1)
## V
clf._V = (C*np.ones(n)).reshape(1,-1)
# Set custom linear constraints A*beta + b >= 0
X_sen = X[:,0] - X[:,0].mean()
tol_sen = 0.1
clf._A = np.repeat([X_sen @ X], repeats=[2], axis=0) / n
clf._A[1] = -clf._A[1]
clf._b = np.full(2, tol_sen)
clf.fit(X)See more detailed in Manual ReHLine Formulation.
ReHLine excels at solving a wide range of machine learning problems:
| Problem | Description | Key Benefits |
|---|---|---|
| Support Vector Machines | Binary and multi-class classification | 100-400× faster than CVXPY solvers |
| Fair Machine Learning | Classification with fairness constraints | Bounds sensitive-attribute/score covariance |
| Quantile Regression | Robust conditional quantile estimation | 2800× faster than general solvers |
| Huber Regression | Outlier-resistant regression | Superior to specialized solvers |
| Sparse Learning | Feature selection with L1 regularization | Scales to high dimensions |
| Custom Optimization | Any PLQ loss with linear constraints | Flexible framework for research |
ReHLine delivers exceptional speed compared to state-of-the-art solvers. Here are speed-up factors on real-world datasets:
| Task | vs. ECOS | vs. MOSEK | vs. SCS | vs. Specialized Solvers |
|---|---|---|---|---|
| SVM | 415× faster | ∞ (failed) | 340× faster | 4.5× vs. LIBLINEAR |
| Fair SVM | 273× faster | 100× faster | 252× faster | ∞ vs. DCCP (failed) |
| Quantile Regression | 2843× faster | ∞ (failed) | ∞ (failed) | — |
| Huber Regression | ∞ (failed) | 452× faster | ∞ (failed) | 2.4× vs. hqreg |
| Smoothed SVM | — | — | — | 1.6-2.3× vs. SAGA/SAG/SDCA/SVRG |
Note: "∞" indicates the competing solver failed to produce a valid solution or exceeded time limits. Results from NeurIPS 2023 paper.
All benchmarks are reproducible via benchopt at our ReHLine-benchmark repository.
| Problem | Benchmark Code | Interactive Results |
|---|---|---|
| SVM | Code | 📊 View |
| Smoothed SVM | Code | 📊 View |
| Fair SVM | Code | 📊 View |
| Quantile Regression | Code | 📊 View |
| Huber Regression | Code | 📊 View |
We welcome contributions! Whether it's bug reports, feature requests, or code contributions:
- 🐛 Open an issue
- 💬 Start a discussion
- 🔀 Submit a pull request
If you use ReHLine in your research, please cite our NeurIPS 2023 paper:
@inproceedings{dai2023rehline,
title={ReHLine: Regularized Composite ReLU-ReHU Loss Minimization with Linear Computation and Linear Convergence},
author={Dai, Ben and Qiu, Yixuan},
booktitle={Thirty-seventh Conference on Neural Information Processing Systems},
year={2023}
}
|
|
Source distributions include Eigen 5.0.1 headers and licenses, so compiling a published source package does not require an Eigen download. A C++ compiler and the Python build dependencies are still required.
Git checkouts do not include Eigen. Editable installs, wheel builds and source
distribution builds automatically download the pinned release when needed and
check its SHA-256 and individual file checksums against tools/eigen-5.0.1.json.
No separate preparation command is required. The files in
vendor/eigen-5.0.1/ are ignored by Git and reused after verification on later
builds. For offline preparation, use
python tools/prepare_eigen.py --archive /path/to/eigen-5.0.1.zip; the same
checksums are required. Running python tools/prepare_eigen.py without
--archive is an optional way to download Eigen ahead of time. Builds from
published source packages use only the bundled headers and do not download
Eigen, including when their contents are missing or corrupted.
Set EIGEN3_INCLUDE_DIR to a local directory containing Eigen/Core to compile
against a different local Eigen installation without preparing the default
headers. Creating an sdist still prepares the pinned headers automatically so
the resulting package remains self-contained, regardless of this override.
To build an sdist:
python -m build --sdistThe sdist command verifies all pinned headers and licenses before packaging. An incomplete or modified existing copy fails verification instead of being silently reused. CI exercises automatic preparation and rebuilds the sdist with Python network access disabled.