shovradas / proxyally Goto Github PK

It ensures consistent storage of the data in a database, whereas duplicates of
HTTP(S) proxy addresses are avoided.

It is able to test the functionality of HTTP(S) proxies in regard to a certain test URL

The database stores the aggregated and processed data of the proxy lists.

1. The database stores the aggregated and processed data of the proxy lists.
2. It can also be used to store the configuration.
3. The frontend must never communicate directly with the database.

The backend aggregates and processes the data of the various proxy lists

1. The backend aggregates and processes the data of the various proxy lists.
2. It prevents the updating of the proxy lists in a too small time interval. It waits at least 10
   minutes between two requests to the same URL of a proxy list and there must not be
   more than one request per second to multiple URLs of one proxy list (for example for
   multi page lists).
3. It ensures a consistent storage of the data in a database, whereas duplicates of
   HTTP(S) proxy addresses are avoided.
4. It is able to test the functionality of HTTP(S) proxies in principle (basic functionality test).
5. It is able to test the functionality of HTTP(S) proxies in regard to a certain test URL.
6. HTTP(S) proxies are removed if they are not currently on a proxy list and the basic
   functionality test is currently negative and has not been positive within the past week.

It prevents the updating of the proxy lists in a too-small time interval. It waits at least 10 minutes between two requests to the same URL of a proxy list and there must not be more than one request per second to multiple URLs of one proxy list (for example for multi-page lists).

1. The REST based interface provides communication between the frontend and the
   database.
2. It delivers the data from the database to the frontend.
3. It can also be used to store configuration settings in the database.
4. Furthermore, it is possible to use the REST based interface to initiate an update of the
   proxy lists as well as the functionality tests of the collected HTTP(S) proxy addresses
   by the backend. Therefore, it is not necessary to implement a proactive service which
   automatically performs the update

It is able to test the functionality of HTTP(S) proxies in principle (basic functionality test)

1. The following detail information of all collected HTTP(S) proxies can be displayed:
   a) IP address
   b) port number
   c) date of the last successful basic functionality test
   d) date when the address was last found in any proxy list
   e) date when the address was first found in any proxy list
   f) for each available test URL the date of the last successful functionality test
2. The following information of all available test URLs can be displayed:
   a) test URL
   b) details of functionality test validation
3. Usability aspects are taken into account.
4. The design is appealing.

HTTP(S) proxies are removed if

• they are not currently on a proxy list and the basic
• functionality test is currently negative and has not been positive within the past week.