km_challenge's Introduction

Machine Learning with Kernel Methods 2021

KKML Kaggle Challenge: https://www.kaggle.com/c/machine-learning-with-kernel-methods-2021/overview

Team Name: Ker et Paix

Louis Bouvier, Roman Castagné & Julia Linhart

MVA 2020/21 - ENS Paris-Saclay

Overview:

This repository contains the Python implementation of our solution to the above stated Kaggle Challenge. Our methods make use of several kernels, either working on bags of words or on raw string data. They are briefly described in the provided report, which also indicates a justified ranking.

We have implemented from scratch the following models:

Logistic Regression
Ridge Regression
Kernel Ridge Regression
Kernel Support Vector Machines
Multiple Kernel Learning

We have dealt with the following kernels:

Gaussian kernel on bags of words
Spectrum kernel on raw sequences
Mismatch kernel on raw sequences
Substring kernel on raw sequences
Fisher kernel on raw sequences
TF-IDF extraction
Sum of kernels

Content of the Repository:

Models and kernels are implemented from scratch in:

linear_models.py containing the classes LogisticRegressor and RidgeRegressor
kernel_models.py containg the classes KernelRidgeRegressor, KernelSVM and KernelMKL
kernels.py containing all the functions nessecary to build the above stated kernels

Supplementary files:

functions.py: method-functions for these models (e.g. Newton method used for Logistic Regression)
utils.py: utility functions to load precomputed kernels, initialize a model, run the model (training and evalution on all 3 datasets), save the results in a .csv file of right format to be use as submission file

Finally, the Python Notebook KM_challenge.ipynb contains the code for example runs on our different methods.

Recommend Projects