Youth Income Prediction Challenge API

A machine learning web app and API for predicting youth income based on data from labour market surveys in South Africa

You can view the live demo of the web app here

You can interact with the API here

Table of Contents

• Overview and Objective
• Dataset
• Insights from EDA
• Models and Evaluation Metric
• Simple API Doc

Overview and Objective

Up to this moment, I have always deployed my models using Streamlit for easier interaction, testing and sharing. Moving forward, this project and subsequent ones will aim to extend beyond traditional machine learning model development in Jupyter notebooks and web apps by incorporating the development of APIs using FastAPI. Additionally, this project particularly will seek to explore various hyperparameter tuning techniques to optimize the performance of machine learning model.

Data

The dataset used in this repository is obtained from a competition on Zindi. The data comes from four rounds of a survey of youth in the South African labour market, conducted at 6-month intervals. The survey contains numerical, categorical and free-form text responses. Each person in the dataset was surveyed one year prior (the ‘baseline’ data) to the follow-up survey. In a nutshell, the objective of the challenge is to build a machine learning model that predicts whether a person is employed at the follow-up survey based on their labour market status and other characteristics during the baseline.

Insights from EDA

The importance of EDA before model building cannot be overemphasized. EDA provides a clearer picture and understanding of the distribution of the data. This include class-imbalance, outliers, correlation and so on. Below are some of the insights gained from a light EDA:

• Below is the proportion of people who have a positive outcome and otherwise.
  

• The ages of candidates with a positive outcome and those with a negative outcome seem to follow a similar distribution.
  

• People from "Urban" areas are most likely to get a positive outcome.
  

Model and Evaluation Metric

For the sake of simplicity, only one type of classification model (LightGBM Classifier) was used in the notebook. Also, the hyperparameter tunning techniques used are GridSearchCV and RandomSearchCV. In subsequent models, I hope to explore the Bayesian Optimization with Gaussian Process. The performance of the base model however and the tunned ones can be found in the notebook.

Simple API Doc

Note: All string inputs are case and whitespace sensitive