jingmouren / compose Goto Github PK

Compose is a python library for automated prediction engineering. An end user defines an outcome of interest over the data by writing a "labeling function". Compose will automatically search and extract historical training examples to train machine learning examples, balance them across time, entities and label categories to reduce biases in learning process. See the documentation for more information.

Its result is then provided to the automatic feature engineering tools Featuretools and subsequently to AutoML/ML libraries to develop a model. This automation for the very early stage of ML pipeline process allows our end user to easily define a task and solve it. The workflow of an applied machine learning engineer then becomes:

Compose can be installed by running the following command.

pip install composeml

In this example, we will generate labels on a mock dataset of transactions. For each customer, we want to label whether the total purchase amount over the next hour of transactions will exceed $300. Additionally, we want to predict one hour in advance.

With the package installed, we load in the data. To get an idea on how the transactions looks, we preview the data frame.

import composeml as cp

df = cp.demos.load_transactions()

df[df.columns[:7]].head()

To get started, we define the labeling function that will return the total purchase amount given a hour of transactions.

def total_spent(df):
    total = df['amount'].sum()
    return total

With the labeling function, we create the LabelMaker for our prediction problem. To process one hour of transactions for each customer, we set the target_entity to the customer ID and the window_size to one hour.

label_maker = cp.LabelMaker(
    target_entity="customer_id",
    time_index="transaction_time",
    labeling_function=total_spent,
    window_size="1h",
)

Next, we automatically search and extract the labels by using LabelMaker.search. For more details on how the label maker works, see Main Concepts.

labels = label_maker.search(
    df.sort_values('transaction_time'),
    num_examples_per_instance=-1,
    gap=1,
    verbose=True,
)

labels.head()

With the generated LabelTimes, we will apply specific transforms for our prediction problem. To make the labels binary, a threshold is applied for amounts exceeding $300.

labels = labels.threshold(300)

labels.head()

Additionally, the label times are shifted one hour earlier for predicting in advance.

labels = labels.apply_lead('1h')

labels.head()

After transforming the labels, we can use LabelTimes.describe to print out the distribution with the settings and transforms that were used to make these labels. This is useful as a reference for understanding how the labels were generated from raw data. Also, the label distribution is helpful for determining if we have imbalanced labels.

labels.describe()

Label Distribution
------------------
False      56
True       44
Total:    100


Settings
--------
num_examples_per_instance        -1
minimum_data                   None
window_size                  <Hour>
gap                               1


Transforms
----------
1. threshold
  - value:    300

2. apply_lead
  - value:    1h

The Feature Labs community welcomes pull requests. Instructions for testing and development are available here.

The Feature Labs open source community is happy to provide support to users of Compose. Project support can be found in four places depending on the type of question:

1. For usage questions, use Stack Overflow with the composeml tag.
2. For bugs, issues, or feature requests start a Github issue.
3. For discussion regarding development on the core library, use Slack.
4. For everything else, the core developers can be reached by email at [email protected].

Compose is built upon a newly defined part of the machine learning process - prediction engineering. If you use Compose please consider citing this paper: James Max Kanter, Gillespie, Owen, Kalyan Veeramachaneni. Label, Segment,Featurize: a cross domain framework for prediction engineering. IEEE DSAA 2016.

BibTeX entry:

@inproceedings{kanter2016label,
  title={Label, segment, featurize: a cross domain framework for prediction engineering},
  author={Kanter, James Max and Gillespie, Owen and Veeramachaneni, Kalyan},
  booktitle={2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA)},
  pages={430--439},
  year={2016},
  organization={IEEE}
}

Compose open source has been developed by Feature Labs engineering team. The open source development has been supported in part by DARPA's Data driven discovery of models program (D3M).

Compose is an open source project created by Feature Labs. We developed Compose to enable flexible definition of the machine learning task. Read more about our rationale behind automating and developing this stage of the machine learning process here.

To see the other open source projects we're working on visit Feature Labs Open Source. If building impactful data science pipelines is important to you or your business, please get in touch.