risenw / datasist Goto Github PK

• Plotting confusion Matrix

• Plotting Scatter plots with up to 5 legends

• Plotting ROC Curve

Utilizing Plotly to make a "Coin-USD" market data Candlestick plot

Required Packages

• NumPy

• Pandas

• Matplotlib

• yfinance

• Plotly

• kaleido

• Random (Python built-in)

  Default function - get_crypto_visuals
  —————————––––––––––––––––

def get_crypto_visuals(coin, period='5d', interval='15m', MA=False,days=[7,25,99], boll=False, boll_sma=25, save_fig=False, img_format='png'):



“””

  ----------------------SKIPPING PARAMETER DESCRIPTION TO END OF DOCSTRING--------------------
  



  Examples of valid use

    ----------------

  >>>  get_crypto_visuals("ETH")

  >>>  get_crypto_visuals("ETH", MA=True)

  >>>  get_crypto_visuals("ETH", period="5d", interval="15m",MA=True, days=[5, 20], save_fig=True)

  >>>  get_crypto_visuals("ETH", period="5d", interval="15m", boll=True, boll_sma=26, save_fig=True, img_format='jpeg')

”””

-------Main Code goes here-------

End Result Snippet

The to_date function in the feature_engineering module raises the above error when the row size of a dataset is less than 20. I noticed that the to_date function calls a function get_date_cols in the structdata module. This calls a private function _match_date (also in the structdata module), which returns a list of columns that matches the DateTime expression. The default sample size in the _match_date function is 20. Although, Users hardly use a data set less than 20. I was thinking it would be nice if the sample_size equal to the row size of the dataset when the row size of the dataset is less than 20 and it should be set to default if otherwise.

A feature that helps to normalize data using the log_transfromation method and also visualizes the dataset as it shows the skewness also.

A function like train_classifier in the modeling module that accepts a regression model and displays some of the most popular regression metrics. Helps in model comparison.
@Olaniyiajayi

A function that accepts a list of models, and trains and compare them according to a given metrics. Result should be displayed as a plot, for easy reading.

The y-axis in the plot of the compare_model does not display the metric being plotted.

Create a function to clean up text dataset.

Get_date_info is supposed to be a feature engineering process, I propose to move it to the feature engineering module.

i once used datasist to extract both numerical and categorical features successfully. Now, i tried to re run my codes, i only see numerical features being extracted and my categorical features is blank. Even, it extracted all the 69 features in the dataframe as numerical features while categorical features is blank. Surprisingly, it ran perfectly okay before the rerun showing both numerical and categorical features respectively, and due to this i was able to do feature engineering for my test dataset in the first place. Now that i wanted to conduct the same feature engineering for my train dataset, it only extracted numerical features leaving categorical features blank. Knowingfully well after using "dtype" that there are many object features(i.e categorical features) in both train and test datasets. In fact, there are only 25 numerical features out of 69, others are categorical.
What can be the problem? I tried running my codes both on Kaggle notebook and google colab, the same problem keeps persisting.

In regard to the vision behind Dataist, I dislike when i always have to perform EDA functions in Pandas one after the other like; df.describe, df.shape, df.dtype and so on. I will like to have it all in one line of code.

Python kernel keeps returning bug; No module named "Joblib" after installing through anaconda cmd using command "pip install datasist"

The describe function should first check if a DataFrame has categorical features before trying to describe it, to avoid error during execution.

Tests should be added for all functionality to prevent breaking in future. An CI to can be used to run PRs against to ensure its not breaking any existing functionality

----> 4 df=ds.drop_missing(data=data, percent=80)

AttributeError: module 'datasist' has no attribute 'drop_missing'

So is this not in the function

An NLP model that helps data preprocessing

This function enables visualization of the relation between numerical features and target mainly for linear regression by providing a drawdown to enable the user select the particular features.

An example Usage will be

def features_plot(df,target):

//code here

return widgets

This can be fixed by first testing for object types in the DataFrame

This function will calculate the mae of a Series and return a single value

An example usage will be

def calculate_mae(Series):
    //calculate stuffs here
   return 20

Adding pyspark module to be triggered in the event that the size of input dataset is bigger than 1mil rows.
This capability should improve the runtime generally and give improved user experience.

Add default number for n in the detect outlier feature
Raise a valueError if data and columns are not inputted