Here is the notebook with which I made my final submission, where I ranked 16th out of the 295 competitors who submitted a solution (top 6%).

We were given two datasets (requests_train.csv, requests_test.csv), a tutorial notebook which included the custom metric competition_scorer and a baseline submission using an untuned logistic regression model and 3 features. Halfway through the competition, we were given another two datasets (individuals_train.csv and individuals_test.csv).

My approach was simple. I trained an LGBM (Light Gradient Boosting Machine) model on a subset of the train set using early stopping (with evaluations done on the remaining part of the train set), predicted on the test set, and submitted. I would normally perform cross-validation, but due to the size of the dataset (220,000 training rows) and the very limited amount of time available (2 hours), I only used a single validation set.

I'm genuinely surprised that such a simple solution did well. I attribute this to the following:

• I took great care to ensure that the model's specifications matched those of the custom evaluation metric (this was given as a hint by the organisers);
• LGBM automatically handles missing variables, categorical variables, and doesn't require feature rescaling. This allowed me to produce a good solution relatively quickly.

Had I had more time, I would have taken greater care of feature engineering, understanding the dataset, and interpreting the model's predictions. I can't understand why birth_month had a high feature importance, and suspect that it may have harmed my score on the test set. Unfortunately, I lost too much time taking care of the custom evaluation metric, and this left me little time for feature engineering / data exploration. To me, however, a reliable validation scheme is the most important thing, so I'm glad that's what I prioritised.

Constructive crticism is always welcome :)