Giter Club home page Giter Club logo

news-extractor's Introduction

news-extractor

Page that allows you to read news without ads. Simply provide website URL and click "Extract" to scrap the selected page and receive a pure list of article titles and their direct urls.

Screenshot

You can also save your extract to DB. Screenshot

Architecture

This app is built as a Docker application so it can be easily run and deployed on different environments. It is managed with docker-compose and it consists of 2 containers (services):

  • app - developed in Flask, Python framework for web application
  • mongo - Mongo database used by Flask app

Run for development

For development and debug purposes you can run Flask application locally (outside Docker).

  1. Create Python virtual env and install the requirements
    • example for Linux
    python3 -m venv venv
    source venv/bin/activate
    pip install -r backend/requirements.txt
    
  2. Run Docker service for mongo DB and DB client
    sudo docker-compose up mongo
    sudo docker-compose up mongo-express
    
  3. Create DB user
    • login to Mongo DB using initial creds
    sudo docker exec -it [container] mongo admin --username root --password secret
    
    • create database and user dedicated to the application
    db.getSiblingDB('simple-app-db').createUser({user:'user', pwd:'pass', 
    roles:[{role:'readWrite',db:'simple-app-db'}]})
    
  4. Define environment variables
    FLASK_ENV=env
    MONGO_URI=mongodb://user:pass@localhost:27017/simple-app-db
    
  5. Run application
    python manage.py runserver

Run in Docker (dev)

Database and user are created automatically while mongo DB initialization, using mongo-init.js.

  1. Simply run Docker services
    sudo docker-compose up mongo
    sudo docker-compose up mongo-express
    sudo-docker-compose up app
    

Run in Docker (prod)

  1. Update mongo-init.js

    db.auth('admin-user', 'admin-pass')
    
    db = db.getSiblingDB('simple-app')
    
    db.createUser(
            {
                user: "user",
                pwd: "pass",
                roles: [
                    {
                        role: "readWrite",
                        db: "simple-app"
                    }
                ]
            }
    );
    
  2. Update docker-compose.yml

    services:
     app:
      environment:
       MONGO_URI: valid-mongo-uri
       FLASK_ENV: prod
       
     mongo:
      environment:
        MONGO_INITDB_ROOT_USERNAME: admin-user
        MONGO_INITDB_ROOT_PASSWORD: admin-pass
        MONGO_INITDB_DATABASE: admin
    
  3. Run application

    sudo docker-compose up mongo
    sudo-docker-compose up app
    

news-extractor's People

Contributors

soniasonia avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.