Giter Club home page Giter Club logo

proxy-provider's Introduction

Proxy-Provider

is a service for downloading, checking and providing socks proxy servers.

How to install:

Unix:

To download and run this project, you need to install git

  1. Launch terminal in the folder for this project
  2. Clone project repository - git clone https://github.com/ArchiKeV/Proxy-Provider.git
  3. Go to project folder - cd Proxy-Provider
  4. Allow execution of init.sh and run.sh files - chmod + x init.sh run.sh
  5. Run init.sh for the first time - ./init.sh

Windows:

To download and run this project, you need to install git and python

Git, Python

  1. Launch terminal in the folder for this project
  2. Clone project repository - git clone https://github.com/ArchiKeV/Proxy-Provider.git
  3. Go to project folder - cd Proxy-Provider
  4. Run init.bat for the first time - init.bat

How to run:

Unix:

  1. Launch terminal in this project folder
  2. Run run.sh in a terminal every time you need to run a project - ./run.sh
  3. Use the visual text interface in the terminal and get tested proxy servers at 127.0.0.1:5000/proxy
  4. Run run.sh in a terminal every time you need to run a project - ./run.sh

Windows:

  1. Launch terminal in this project folder
  2. Run run.bat for the first time - run.bat
  3. Use the visual text interface in the terminal and get tested proxy servers at 127.0.0.1:5000/proxy
  4. The first run will create a basic config.json, use that to configure

Architecture:

Input data:

The input data for the service is described in the configuration file config.json created when the service is first started. It, among other things, describes all the used sources of proxy servers, their configuration will be discussed below.

Sources of proxy servers:

Proxy sources are provided as a Python 3 module located in the ./sources directory. An example module is described in ./sources/example_module.py. Such a module should independently obtain lists of proxy servers and return a list of dictionaries in the get_proxy() function:

[
    {
        'country_code_in': "BB", # two-letter country code or None
        'ip_in': '1.2.3.4',      # string
        'port_in': 4321,         # integer
        'type': 4                # integer - socks proxy type (4 or 5)
    },
    {
        'country_code_in': None,
        'ip_in': '4.3.2.1',
        'port_in': 1234,
        'type': 5
    }
]

Parameter country_code_in - ISO code of the country of entry into the proxy server, the parameter is optional, but desirable. This setting is used to filter out unwanted provider countries. The list of unwantedm countries is described in config.json (details below). If there is no information about the country of entry, the parameter should be filled with the value None.

Checking proxy servers:

Available proxies are checked by visiting a site with connection information through the proxy server. From this information, data on the Country of exit and the exit IP address are taken.

Provision of verified proxies:

Verified proxies are provided by the FLASK server as a REST API (currently only getting tested servers is available).

Configuration file:

The configuration file consists of three sections - db, proxy, system. Each section is responsible for its part of the settings.

DB:

db_type - db type (currently|default - sqlite, others in the future)

settings.filepath - db name (default - proxy_provider_db.sqlite)

settings.concurrent_slots - Num of concurrent db access slots (sqlite default - 1)

Proxy:

country_code_ignore_list - List of 2-letter codes of unwanted countries whose proxies will not be added to the database. Default - ["IR", ]

sources - List of proxy server source dictionaries:

"name": str - Module name and display name, DO NOT end with .py

"timer": int - Source reload timer, in seconds. It should be set based on the source update frequency and large enough so as not to be aggressive towards the source server (so as not to be banned)

checkup_timers.active_server_check_period_in_hours - Timer in hours, for working servers

checkup_timers.inactive_server_check_period_in_hours -Timer in hours, for not working servers (currently not in use)

timeouts.connection_timeout - Server connection timeout during health check (in seconds)

timeouts.read_timeout - Server read timeout during health check (in seconds)

num_of_simultaneous_checks - The number of simultaneously scanned servers, the more simultaneous scans, the faster all servers will be scanned (5 - default)

System:

tui_text_line_buffer_size - String buffer for TUI (default - 500)

debug - Debug mode (default - False). Currently affects the behavior of sqlalchemy and does not affect logs

proxy-provider's People

Contributors

archikev avatar

Stargazers

 avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.