Giter Club home page Giter Club logo

yt-spam-classifier's Introduction

yt-spam-filter

Using TensorFlow to automatically filter crypto-related spam comments on YouTube. ๐ŸŽฌ See my video for more details: https://youtu.be/zSEYC3CCA1I

Repository structure

  • 00-CoreML test -> First test with Apple's CoreML.
  • 01-fetch-input-data -> Python script that fetches comments from YouTube and stores them in a CSV file
  • 02-rated-comments -> Dataset of comments from my channel, marked as spam (1) or not spam (0)
  • 03-tensorflow-training -> Jupyter notebook used to train the text classifier with TensorFlow
  • 04-inference -> Python script that runs periodically to filter latest comments on my channel

Installation

Clone this repo:

git clone https://github.com/Savjee/yt-spam-classifier.git

Install dependencies (might not be complete):

pip3 install -r requirements.txt

Create a .env file from the template, and fill in your YouTube API key and Channel ID

cp .env.dist .env

More instructions to follow.

Contribute

Feel free to fork or submit improvements but be gentle. This is my first ML and Python project (it probably shows).

yt-spam-classifier's People

Contributors

savjee avatar

Stargazers

Gopal Patwa avatar  avatar Masterlovsky avatar  avatar  avatar  avatar Satish avatar Rohan Smith avatar Xiang Fu avatar Arthur Van Loo avatar  avatar  avatar zenobit avatar Dewi Rizki Fitriani avatar Pavlo Kyrylenko avatar TechStudent10 avatar  avatar Wade avatar Kevin Heusinger avatar David Kelly avatar Ghost avatar Md. Shakil Hossain avatar  avatar Daniel Ramirez avatar Zubair avatar Vinay Chaudhari avatar Li jianying avatar K. Ali Pardhan avatar Prashanth Babu T avatar  avatar Cyrus S avatar HW avatar  avatar Muhammad Ramzan Ali Suzan avatar David Spez avatar Amr Elroumy avatar Ahsan Javed avatar Hiten avatar Thomas Spornraft avatar

Watchers

 avatar James Cloos avatar zenobit avatar  avatar Zubair avatar

yt-spam-classifier's Issues

CancelledError on M1 Macbook Air Big Sur 12.0

Hi,

I'm trying to recreate your steps and run into the following issue when running this step:
All previous steps work fine. I'm using your dataset.

early_stop = EarlyStopping(monitor='val_loss', mode='min', verbose=1, patience=10)
model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['accuracy'])

# fit the model
model.fit(X_train,
         y_train,
         epochs=40,
         batch_size=512,
         validation_data=(X_test, y_test),
         verbose=1,
         callbacks=[early_stop]
)

Which leads to the following issue:

CancelledError                            Traceback (most recent call last)
/var/folders/7t/tgrdz77j2r93smmw74pd9ny40000gn/T/ipykernel_4079/1037202941.py in <module>
      4 
      5 # fit the model
----> 6 model.fit(X_train,
      7          y_train,
      8          epochs=10,

~/miniforge3/envs/apple_tf/lib/python3.9/site-packages/tensorflow/python/keras/engine/training_v1.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_freq, max_queue_size, workers, use_multiprocessing, **kwargs)
    793 
    794     func = self._select_training_loop(x)
--> 795     return func.fit(
    796         self,
    797         x=x,

~/miniforge3/envs/apple_tf/lib/python3.9/site-packages/tensorflow/python/keras/engine/training_arrays_v1.py in fit(self, model, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_freq, **kwargs)
    642       val_x, val_y, val_sample_weights = None, None, None
    643 
--> 644     return fit_loop(
    645         model,
    646         inputs=x,

~/miniforge3/envs/apple_tf/lib/python3.9/site-packages/tensorflow/python/keras/engine/training_arrays_v1.py in model_iteration(model, inputs, targets, sample_weights, batch_size, epochs, verbose, callbacks, val_inputs, val_targets, val_sample_weights, shuffle, initial_epoch, steps_per_epoch, validation_steps, validation_freq, mode, validation_in_fit, prepared_feed_values_from_dataset, steps_name, **kwargs)
    378 
    379         # Get outputs.
--> 380         batch_outs = f(ins_batch)
    381         if not isinstance(batch_outs, list):
    382           batch_outs = [batch_outs]

~/miniforge3/envs/apple_tf/lib/python3.9/site-packages/tensorflow/python/keras/backend.py in __call__(self, inputs)
   4052       self._make_callable(feed_arrays, feed_symbols, symbol_vals, session)
   4053 
-> 4054     fetched = self._callable_fn(*array_vals,
   4055                                 run_metadata=self.run_metadata)
   4056     self._call_fetch_callbacks(fetched[-len(self._fetches):])

~/miniforge3/envs/apple_tf/lib/python3.9/site-packages/tensorflow/python/client/session.py in __call__(self, *args, **kwargs)
   1478       try:
   1479         run_metadata_ptr = tf_session.TF_NewBuffer() if run_metadata else None
-> 1480         ret = tf_session.TF_SessionRunCallable(self._session._session,
   1481                                                self._handle, args,
   1482                                                run_metadata_ptr)

CancelledError: [_Derived_]RecvAsync is cancelled.
	 [[{{node keras_layer/StatefulPartitionedCall}}]]

[KeyError]: 'nextPageToken' in CommentHelper.py

Hi Savjee,

Env : Google Colab (Python 3.7)


After setting the following things:

1) API_KEY
2) CHANNEL_ID
3) OAuth client_secret.json

Im getting this issue

image

I think that the localhost thing is breaking on the google collab server but not really sure about it. That's CommentHelper.py is kind of trickest to understand.
Can you help me to fix this issue?

Thanks.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.