Comments (3)
Hey!
Since the project code become a little bit messy and dense, I decided to split the code into the three parts:
-
construct_features.py - Code that download each URL's HTML code and tokenize all words that would be used for generating the most frequent word list for all available categories.
-
construct_models.py - After HTML codes from all websites are parsed, the next step is text normalization part. In this code the extracted word tokens of websites text are normalized by removing stop words and translating non english words with the Google Translator.
The main goal of this process is to make words tokens to be more english friendly, because the most frequent words list for each category should consist of only english words (it is possible to do that in other languages, but for this project I decided to use english). -
train_models.py - Code that trains and tests ML models.
I know that it could be a little bit confusing since README file is not up to date anymore. I'm going to update it soon.
There are some tasks that I'm going to implement it to this project when I got more free time:
- Update README file with up to date information
- Create a code that would be available to predict websites manually which could be passed as an argument (That functionality was already implemented in the previous commit versions: https://github.com/domantasm96/URL-categorization-using-machine-learning/blob/bc2a61daeab69458a6d4158120100692a0c272e1/Scripts/predict_url.py
- Improve ML models by using more advanced Machine Learning frameworks. The accuracy of prediction should also improve
Thanks for asking and if you have any more questions - feel free to ask! :)
from url-categorization-using-machine-learning.
Thanks for it and if possible I will try to improve the accuracy and update to you
from url-categorization-using-machine-learning.
That would be great!
from url-categorization-using-machine-learning.
Related Issues (12)
- Found array with 0 sample(s) (shape=(0,)) while a minimum of 1 is required. HOT 2
- RuntimeError HOT 1
- Other ML learning algorithm source code HOT 1
- I couldn't run it! HOT 1
- Dataset is too old
- Error: Executing construct_features.py HOT 4
- Running construct_data results in error message HOT 1
- Url-Categorization HOT 4
- Incorrect category for CNN HOT 2
- Additional Model Training HOT 1
- RequestsDependencyWarning HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from url-categorization-using-machine-learning.