himalaya.kernel_ridge.MultipleKernelRidgeCV

class himalaya.kernel_ridge.MultipleKernelRidgeCV(kernels=['linear', 'polynomial'], kernels_params=None, solver='random_search', solver_params=None, cv=5, random_state=None, Y_in_cpu=False, force_cpu=False)

Multiple-kernel ridge regression with cross-validation.

Solve the kernel ridge regression:

w* = argmin_w ||K @ w - Y||^2 + (w.T @ K @ w)

where the kernel K is a weighted sum of multiple kernels Ks:

K = sum_i exp(deltas[i]) Ks[i]

The solver optimizes the log kernel weight deltas over cross-validation, using random search (solver="random_search"), or hyperparameter gradient descent (solver="hyper_gradient").

Parameters

kernelslist of (str or callable), default=[“linear”, “polynomial”]

List of kernel mapping. Available kernels are: ‘linear’, ‘polynomial, ‘poly’, ‘rbf’, ‘sigmoid’, ‘cosine’. Set to ‘precomputed’ in order to pass a precomputed kernel matrix to the estimator methods instead of samples. A callable should accept two arguments and the keyword arguments passed to this object as kernel_params, and should return a floating point number.

kernels_paramslist of dict, or None

Additional parameters for the kernel functions. See more details in the docstring of the function: MultipleKernelRidgeCV.ALL_KERNELS[kernel]

solverstr

Algorithm used during the fit, “random_search”, or “hyper_gradient”.

solver_paramsdict or None

Additional parameters for the solver. See more details in the docstring of the function: MultipleKernelRidgeCV.ALL_SOLVERS[solver]

cvint or scikit-learn splitter

Cross-validation splitter. If an int, KFold is used.

random_stateint, or None

Random generator seed. Use an int for deterministic search.

Y_in_cpubool

If True, keep the target values y in CPU memory (slower).

force_cpubool

If True, computations will be performed on CPU, ignoring the current backend. If False, use the current backend.

Examples

>>> from himalaya.kernel_ridge import MultipleKernelRidgeCV
>>> from himalaya.kernel_ridge import ColumnKernelizer
>>> from himalaya.kernel_ridge import Kernelizer
>>> from sklearn.pipeline import make_pipeline

>>> # create a dataset
>>> import numpy as np
>>> n_samples, n_features, n_targets = 10, 5, 3
>>> X = np.random.randn(n_samples, n_features)
>>> Y = np.random.randn(n_samples, n_targets)

>>> # Kernelize separately the first three columns and the last two
>>> # columns, creating two kernels of shape (n_samples, n_samples).
>>> ck = ColumnKernelizer(
...     [("kernel_1", Kernelizer(kernel="linear"), [0, 1, 2]),
...      ("kernel_2", Kernelizer(kernel="polynomial"), slice(3, 5))])

>>> # A model with precomputed kernels, as output by ColumnKernelizer
>>> model = MultipleKernelRidgeCV(kernels="precomputed")
>>> pipe = make_pipeline(ck, model)
>>> _ = pipe.fit(X, Y)

Attributes

dual_coef_array of shape (n_samples) or (n_samples, n_targets)

Representation of weight vectors in kernel space.

deltas_array of shape (n_kernels, n_targets)

Log of kernel weights.

cv_scores_array of shape (n_iter, n_targets)

Cross-validation scores, averaged over splits. By default, the scores are computed with l2_neg_loss (in ]-inf, 0]). The scoring function can be changed with solver_params[“score_func”].

X_fit_array of shape (n_samples, n_features)

Training data. If kernels == "precomputed" this is None.

n_features_in_int

Number of features (or number of samples if kernels == "precomputed") used during the fit.

dtype_str

Dtype of input data.

best_alphas_array of shape (n_targets, )

Equal to 1. / exp(self.deltas_).sum(0). For the “random_search” solver, it corresponds to the best hyperparameter alphas, assuming that each kernel weight vector sums to one (in particular, it is the case when solver_params['n_iter'] is an integer).

Methods

fit(X[, y, sample_weight])	Fit the model.
get_params([deep])	Get parameters for this estimator.
get_primal_coef(Xs_fit)	Returns the primal coefficients, assuming all kernels are linear.
predict(X[, split])	Predict using the model.
score(X, y[, split])	Return the coefficient of determination R^2 of the prediction.
set_params(**params)	Set the parameters of this estimator.