Giter Club home page Giter Club logo

proxypool's Introduction


proxypool

自动抓取tg频道、订阅地址、公开互联网上的ss、ssr、vmess、trojan节点信息,聚合去重测试可用性后提供节点列表

Github Actions

支持

  • 支持ss、ssr、vmess、trojan多种类型
  • Telegram频道抓取
  • 订阅地址抓取解析
  • 公开互联网页面模糊抓取
  • 定时抓取自动更新
  • 通过配置文件设置抓取源
  • 自动检测节点可用性
  • 提供clash、surge配置文件
  • 提供ss、ssr、vmess、sip002订阅

安装

以下五选一。

1. 使用Heroku

点击按钮进入部署页面,填写基本信息然后运行

其中 DOMAIN 需要填写为你需要绑定的域名,CONFIG_FILE 需要填写你的配置文件路径。

heroku app域名为appname.herokuapp.com。项目内配置文件为./config/config.yaml

配置文件模板见 config/config.yaml 文件,可选项区域均可不填。完整配置选项请查看配置文件说明

Deploy

因为爬虫程序需要持续运行,所以至少选择 $7/月 的配置 免费配置长时间无人访问会被heroku强制停止

2. 使用fly.io

注册fly.io需要绑定银行卡,支持银联借记卡。同时使用fly.io主要通过命令行工具flyctl,详情到fly.io官网了解。

下载仓库源代码,修改 fly.toml 中的app与domain。在终端使用 flyctl deploy 部署即可。

3. 从源码编译

需要 安装 Golang , 然后拉取代码,

go get -u -v github.com/ssrlive/proxypool@latest

或者,拉取代码的另一种方式

git clone https://github.com/ssrlive/proxypool.git
cd proxypool
go get
go build

then edit config/config.yaml and config/source.yaml and run it

./proxypool -c ./config/config.yaml

或者从源代码运行

go run main.go -c ./config/config.yaml

4. 下载预编译程序

从这里下载预编译好的程序 release

5. 使用docker

运行下面的命令下载 proxypool 镜像

$ docker pull lukemin/ssrlive-proxypool

下载config.yaml&source.yaml到/path/to/config

$ wget https://raw.githubusercontent.com/ssrlive/proxypool/master/config/config.yaml
wget https://raw.githubusercontent.com/ssrlive/proxypool/master/config/source.yaml

然后运行 proxypool 即可

$ docker run -d --restart=always \
  --name=proxypool \
  -p 12580:12580 \
  -v /path/to/config:/config \
  lukemin/ssrlive-proxypool \
  -c /config/config.yaml

使用 -p 参数映射配置文件里的端口
使用 -v 参数指定配置文件夹位置(配置文件要自行下载放到目录,方便修改)
使用 -c 参数指定配置文件路径,支持http链接

使用

运行该程序需要具有访问完整互联网的能力。

修改配置文件

首先修改 config.yaml 中的必要配置信息,带有默认值的字段均可不填写

source.yaml 文件中定义了抓取源,需要定期手动维护更新

完整的配置选项见配置文件说明

启动程序

使用 -c 参数指定配置文件路径,支持http链接

$ proxypool -c ./config/config.yaml

如果需要部署到VPS,更多细节请查看wiki

Clash配置文件

远程部署时Clash配置文件访问:https://domain/clash/config

本地运行时Clash配置文件访问:http://127.0.0.1:[端口]/clash/localconfig

本地检查节点可用性

此项非必须。为了提高实际可用性,可选择增加一个本地服务器,检测远程proxypool节点在本地的可用性并提供配置,见proxypoolCheck

截图

Speedtest

Fast

声明

本项目遵循 GNU General Public License v3.0 开源,在此基础上,所有使用本项目提供服务者都必须在网站首页保留指向本项目的链接

本项目仅限个人自己使用,禁止使用本项目进行营利和做其他违法事情,产生的一切后果本项目概不负责

proxypool's People

Contributors

dependabot[bot] avatar hulb avatar lukemin avatar sparkyvxcx avatar ssrlive avatar zu1k avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

proxypool's Issues

wget https://github.com/ssrlive/proxypool/releases/download/v0.6.1/proxypool-linux-amd64-v0.6.1.gz

wget https://github.com/ssrlive/proxypool/releases/download/v0.6.1/proxypool-linux-amd64-v0.6.1.gz
--2024-03-07 02:13:30-- https://github.com/ssrlive/proxypool/releases/download/v0.6.1/proxypool-linux-amd64-v0.6.1.gz
Resolving github.com (github.com)... 140.82.121.4
Connecting to github.com (github.com)|140.82.121.4|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-03-07 02:13:30 ERROR 404: Not Found.
没有找到该文件,咋办

关于定时抓取时无法读取配置文件的问题

最初是在查看docker日志时发现的,在运行编译程序时同样出现,日志片段如下:
[0101] INFO Relay Test ON
[100.0% DONE]
[0111] INFO Usablility checking done. Open * to check
[0300] ERROR cron.go: config parse error: read /proxypool-master: is a directory
[0301] INFO STATISTIC: TGChannel count=0 url=*

我也尝试过直接从源码运行,但是报错如下:
[0000] INFO Getter count: 1
Use cache to store proxies
panic: Asset assets/flags.json can't read by error: error when reading asset assets/flags.json at assets/flags.json: open assets/flags.json: no such file or directory

goroutine 1 [running]:
github.com/ssrlive/proxypool/pkg/geoIp.InitGeoIpDB()
/proxypool-0.7.6/pkg/geoIp/geoip.go:24 +0x200
main.main()
/proxypool-0.7.6/main.go:52 +0x2d8
exit status 2

同样环境下Sansui233/proxypool0.7.3版本运行正常

節點導出數量異常

proxypool-0.7.7運行結果顯示篩選出1612節點
通過/clash/prroxies導出結果僅約為670個
其中SS-350,Trojan-240,Vmess-80,無SSR節點

相同條件下Sansui233/proxypool-0.7.3篩選出1123個節點
其中SS-329,Trojan-218,Vmess-401,SSR-172

導出結果對比明顯異常,但是日誌記錄正常無報錯

[BUG]panic: time: missing Location in call to Time.In goroutine 8 [running]

  • I'm using the newest release version, and bug isn't solved
    我使用的是最新的release版本,Bug仍未解决
  • I've tested on the newest master, and bug isn't solved
    我测试的是最新master分支,Bug仍未解决
  • I've searched the issues and there's not a similar one
    我已经查看过其他issue,没有类似的情况

Bug Version出现Bug的版本:v0.7.11
Environment操作环境:linux

Description 问题描述

在我的路由器上,运行后报错

panic: time: missing Location in call to Time.In

goroutine 8 [running]:
time.Time.In(...)
	C:/Program Files/Go/src/time/time.go:1122
github.com/ssrlive/proxypool/internal/app.CrawlGo()
	C:/Users/Initsnow/Downloads/Compressed/proxypool-0.7.11/internal/app/task.go:76 +0x13c0
created by main.main
	C:/Users/Initsnow/Downloads/Compressed/proxypool-0.7.11/main.go:62 +0x468

To Reproduce 复现步骤

新路由3,padavan固件,在Windows用go编译后运行
./proxypool -c ./config/config.yaml

[BUG]Unraid部署失败

  • I'm using the newest release version, and bug isn't solved
    我使用的是最新的release版本,Bug仍未解决
  • I've tested on the newest master, and bug isn't solved
    我测试的是最新master分支,Bug仍未解决
  • I've searched the issues and there's not a similar one
    我已经查看过其他issue,没有类似的情况

Bug Version出现Bug的版本:v0.7.13
Environment操作环境:unraid(docker)

Description 问题描述

在unraid中使用webui的docker部署proxypool会出现无法部署的情况:
原因是-c /config/config.yaml-c会被unraid只识别成cpu核心的修改导致部署失败

image

To Reproduce 复现步骤

使用unraid的WebUI进行部署,100%复现

[BUG]Docker 文件不能下载。

  • [ N] I'm using the newest release version, and bug isn't solved
    我使用的是最新的release版本,Bug仍未解决
  • [ N] I've tested on the newest master, and bug isn't solved
    我测试的是最新master分支,Bug仍未解决
  • [ N] I've searched the issues and there's not a similar one
    我已经查看过其他issue,没有类似的情况

Bug Version出现Bug的版本:v0.x.x
Environment操作环境:heroku/ubuntu/mac

Description 问题描述

$ docker pull ghcr.io/ssrlive/proxypool:latest 不能下载最新docker

To Reproduce 复现步骤

QuantumultX

Description 描述

建议增加生成QuantumultX链接

[BUG]database not save

  • I'm using the newest release version, and bug isn't solved
    我使用的是最新的release版本,Bug仍未解决
  • I've tested on the newest master, and bug isn't solved
    我测试的是最新master分支,Bug仍未解决
  • I've searched the issues and there's not a similar one
    我已经查看过其他issue,没有类似的情况

Bug Version出现Bug的版本:v0.x.x
Environment操作环境:heroku/ubuntu/mac

Description 问题描述

To Reproduce 复现步骤

我使用docker部署proxypool,并使用postgresql来作为数据库存储。
docker compose文件如下
version: "3.3"

services:

proxypool:
container_name: proxypool
image: lukemin/ssrlive-proxypool
restart: always
command: -c /config/config.yaml
volumes:
- ./config:/config
ports:
- 12580:12580
depends_on:
- db
db:
image: postgres
container_name: proxypool_db
restart: always
environment:
POSTGRES_PASSWORD: JymgBuiB8nhCV8BZ5ASt
POSTGRES_USER: pooluser
volumes:
- ./pg_data:/var/lib/postgresql/data
ports:
- 15432:5432

config.yaml如下:

your domain

domain: example.com: # or example.com:9443 for reserve proxy server
port: # default 12580

source list file

source-files:

use local file

  • source.yaml

use web file

- https://example.com/config/source.yaml

======= 可选项,留空使用default值 =======

#====Postgresql database info ====#
database_url: "postgresql://pooluser:JymgBuiB8nhCV8BZ5ASt@proxypool_db:5432/pooluser"

#==== Interval between each crawling ====#
crawl-interval: # v0.5.x default 60 (minutes)
#crontime: # v0.4.x default 60 (minutes). Deprecated in the newest version
healthcheck-timeout: # default 5 (seconds)
healthcheck-connection: # default 500. The number of health check connections simultaneously

#==== Speed test ====#
speedtest: true # default false. Warning: this will consume large network resources.
speedtest-interval: # default 720 (min)
speed-timeout: # default 10 (seconds).
speed-connection: # v0.7.2 default 5. The number of speed test connections simultaneously
#connection # v0.7.1 and beblow

#==== Active proxy speed test ====#
active-interval: # default 60 (min)
active-frequency: # default 100 (requests per interval)
active-max-number: # default 100. If more than this number of active proxies, the extra will be deprecated by speed

#==== Cloudflare api ====#
cf_email: ""
cf_key: ""

开启以后,抓到数据,但是我查看数据库,只有空表proxies
认真翻了一下日志,发现第一次运行,
proxypool | [0000] INFO Running image path: /proxypool-src/proxypool
proxypool | [0000] WARN database connection info: failed to connect to host=proxypool_db user=pooluser database=pooluser: dial error (dial tcp [fc00:0:0:2::2]:5432: connect: connection refused)
proxypool | Use cache to store proxies

什么都不修改,down up,再次运行
proxypool | [0000] INFO Running image path: /proxypool-src/proxypool
proxypool | [0000] INFO database: successfully connected to: postgres
proxypool | [0001] INFO Do the first crawl...

我已经设置了
depends_on:
- db
数据库启动后才启动proxypool的。
是数据库初始化的问题么?

[BUG]go get cannot find module for path crypto/ed25519

按照源码编译步骤,当执行go get命令后报错,
root@localhost:~/proxypool/proxypool# go get
build github.com/ssrlive/proxypool: cannot find module for path crypto/ed25519

请问以上是缺少什么模块,麻烦补充并执教,谢谢!

something

hi

source from https://github.com/Sansui233/proxypool/tags

some update https://github.com/One-Piecs/proxypool

nginx reverse proxy settings

    server {
        listen 9443 ssl default_server;
        listen [::]:9443 ssl default_server;
        ssl_certificate       /fakesite_cert/chained_cert.pem;
        ssl_certificate_key   /fakesite_cert/private_key.pem;
        ssl_protocols         TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers           HIGH:!aNULL:!MD5;
        server_name           mygoodsite.org;
        index index.php index.html index.htm index.nginx-debian.html;
        root  /myfakesitefolder;
        error_page 400 = /400.html;

        location / {
            proxy_pass http://127.0.0.1:12580/; # proxypool 服务的地址
	}
    }

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.