Giter Club home page Giter Club logo

klib's Introduction

klib

Flake8 & PyTest Language Last Commit Quality Gate Status Scrutinizer codecov

klib is a Python library for importing, cleaning, analyzing and preprocessing data. While the focus is on these steps, future versions will include modules and functions for model creation and optimization to provide more of an end-to-end solution. Explanations on key functionalities can be found on Medium / TowardsDataScience or in the examples section.

Installation

Use the package manager pip to install klib.

PyPI Version Downloads

pip install klib
pip install --upgrade klib

Alternatively, to install this package with conda run:

Conda Version Conda Downloads

conda install -c conda-forge klib

Usage

import klib

klib.describe # functions for visualizing datasets
- klib.cat_plot() # returns a visualization of the number and frequency of categorical features
- klib.corr_mat() # returns a color-encoded correlation matrix
- klib.corr_plot() # returns a color-encoded heatmap, ideal for correlations
- klib.dist_plot() # returns a distribution plot for every numeric feature
- klib.missingval_plot() # returns a figure containing information about missing values

klib.clean # functions for cleaning datasets
- klib.clean_column_names() # cleans and standardizes column names, also called inside data_cleaning()
- klib.convert_datatypes() # converts existing to more efficient dtypes, also called inside data_cleaning()
- klib.data_cleaning() # performs datacleaning (drop duplicates & empty rows/columns, adjust dtypes,...) on a dataset
- klib.drop_missing() # drops missing values, also called in data_cleaning()
- klib.mv_col_handling() # drops features with a high ratio of missing values based on their informational content
- klib.pool_duplicate_subsets() # pools a subset of columns based on duplicate values with minimal loss of information

klib.preprocess # functions for data preprocessing (feature selection, scaling, ...)
- klib.train_dev_test_split() # splits a dataset and a label into train, optionally dev and test sets
- klib.feature_selection_pipe() # provides common operations for feature selection
- klib.num_pipe() # provides common operations for preprocessing of numerical data
- klib.cat_pipe() # provides common operations for preprocessing of categorical data
- klib.preprocess.ColumnSelector() # selects numerical or categorical columns, ideal for a Feature Union or Pipeline
- klib.preprocess.PipeInfo() # prints out the shape of the data at the specified step of a Pipeline

Examples

Find all available examples as well as applications of the functions in klib.clean() with detailed descriptions here.

klib.missingval_plot(df) # default representation of missing values in a DataFrame, plenty of settings are available

Missingvalue Plot Example

klib.corr_plot(df, split='pos') # displaying only positive correlations, other settings include threshold, cmap...
klib.corr_plot(df, split='neg') # displaying only negative correlations

Corr Plot Example

klib.corr_plot(df, target='wine') # default representation of correlations with the feature column

Target Corr Plot Example

klib.dist_plot(df) # default representation of a distribution plot, other settings include fill_range, histogram, ...

Dist Plot Example

klib.cat_plot(data, top=4, bottom=4) # representation of the 4 most & least common values in each categorical column

Cat Plot Example

Further examples, as well as applications of the functions in klib.clean() can be found here.

Contributing

Pull requests and ideas, especially for further functions are welcome. For major changes or feedback, please open an issue first to discuss what you would like to change.

License

MIT

klib's People

Contributors

akanz1 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.