qwj / python-proxy Goto Github PK

Hey,

Do you have a way to run pproxy asynchronous (Thread) ?
something like:
pproxy.main(asynchronous = True)

If yes can we kill this ?
something like:
pproxy.stop()

Thx

It can be "hostname:port" or "/unix_domaon_path".

Unable to get http+ssl working.
I started the server with pproxy -i http+ssl://:8080 --ssl server.crt,server.key -v,
then configured by windows LAN setting to use secured

then when i try to access the page, i get "This page isn't working" on the browser. And i see no requests on the proxy server.
I created the ssl files using the commands you provided.

is there any possibility to set rate limitation for each user ?
or set some rules inorder to prevent two devices use one account ?

Win10
Python 3.6.4
pproxy 1.7.3 and 1.8.5

Traceback (most recent call last):
File "c:\program files\python36\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "c:\program files\python36\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Program Files\Python36\Scripts\pproxy.exe_main.py", line 5, in
File "c:\program files\python36\lib\site-packages\pproxy\server.py", line 2, in
from . import proto
File "c:\program files\python36\lib\site-packages\pproxy\proto.py", line 1, in
import asyncio, socket, urllib.parse, time, re, base64, hmac, struct, hashlib, fcntl, io
ModuleNotFoundError: No module named 'fcntl'

And install form pip also has problem:

Traceback (most recent call last):
File "setup.py", line 19, in
long_description = read('README.rst'),
File "setup.py", line 6, in read
return fp.read()
UnicodeDecodeError: 'gbk' codec can't decode byte 0x94 in position 2840: illegal multibyte sequence

This is because long_description = read('README.rst') in setup.py

Hi,

See the following test. local 1080 is a socks5 server.

$ curl -L -x socks5://127.0.0.1:1080 https://gemmei.ftp.acc.umu.se/debian-cd/current/amd64/iso-dvd/debian-10.1.0-amd64-DVD-1.iso -o /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   394  100   394    0     0    160      0  0:00:02  0:00:02 --:--:--   160
  0 3720M    0  384k    0     0  43905      0 24:41:01  0:00:08 24:40:53 85166^C

$ pproxy -r socks5://:1080/ 
Using uvloop
Serving on :8080 by http,socks4,socks5 

$ curl -L -x socks5://127.0.0.1:8080 https://gemmei.ftp.acc.umu.se/debian-cd/current/amd64/iso-dvd/debian-10.1.0-amd64-DVD-1.iso -o /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to gemmei.ftp.acc.umu.se:443 

I have a simple SOCKS5 client which sends a UDP ASSOCIATE request.

I have expected python-proxy to let me establish UDP connections.
However, I get "Unknown SOCKS protocol" back. The code seems to assume that the client sends a regular TCP connection request:

Are UDP connections out of scope?

Hi,

asyncssh 1.16.0 introduced keep-alive for ssh connection. I'll draft a PR to use this functionality and change the requirements for asyncssh to enable it.

I'll probably add a function to auto re-connect if keep-alive is not working.

Cheers,

How to use properly remotes https://aa.bb.cc.dd proxies?

I have the following result:

[SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:777) from 127.0.0.1

or

Unsupported protocol from 127.0.0.1

And my python request with this proxy listening

requests.exceptions.SSLError: HTTPSConnectionPool(host='domain.com', port=443): Max retries exceeded with url: /index (Caused by SSLError(SSLError("bad handshake: SysCallError(-1, 'Unexpected EOF')",),))

I was going to try to implement #41 myself but… the code is really hard to work with. It would benefit a lot from some more newlines, and just following PEP8 in general. Even if it's just to make the linter shut up.

Would you accept a pull request if I formatted everything better?

Hi, As you know, I've been using pproxy for a while now and I just found out there's a problem with the Authentication scheme. If I Authenticate once with the proxy, my public IP address remains authenticated with the server forever. After the first request, the auth becomes optional for that IP. Is that behavior intended? Because it would pose a big security vulnerability in public networks IMO

i've tried to install your package with pip3 and pip, both installs version 1.2.2

Hi!
I'm trying to use pproxy with a remote proxy, which is a paid proxy service.
I need to authenticate the remote proxy connection with a simple http auth. Reading your docs, i understood that i need to use the username and password on the fragment part of the url, because the usual auth part on url is reserved to cyphers, right?

If i start the proxy like this: pproxy -r 'http://HOST:PORT/#USERNAME:PASSWORD' -vvv and make a call using curl --proxy "http://localhost:8080" "http://httpbin.org/ip", i receive curl: (52) Empty reply from server from the curl command.

If i do not use the pproxy and make a call using curl --proxy "http://HOST:PORT/" --proxy-user "USERNAME:PASSWORD" "http://httpbin.org/ip" works nicely.

Am i doing something wrong?

Thanks!

Possible to require auth for connect to socks/http/ss server?

The software is very good, working as expected.
But the code itself is REALLY HARD to read and follow.

Too much overuse lambda.
Difficult to expand the functionality for ordinary people like me.

windows下，现在用os模块调用，感觉有点不方便。。

The assumed default license is "all rights reserved". Most Python projects are BSD / MIT / ISC / etc...

https://docs.python-guide.org/writing/license/

Curious if you'd consider applying a license here?

Is there any examples of creating a handler that can rewrite the request or response as it passes through the proxy?

Is this something I should request on #45?

Would you be interested in adding a plugin that does network throttling ?

That is:

• limit the bandtwidth
• add delays

I am using this to mimic network conditions to test browser page loads. I've implemented it using a simple port forwarder + socks5 server ar https://github.com/tarekziade/tinap

But your proxy is much better as it implements many protocols!

I guess most of the code that does this is here: https://github.com/tarekziade/tinap/blob/master/tinap/throttler.py and should be simple to refactor as a pproxy plugin

Let me know!

Hi!

I'd like to set a proxy server, over the same IP, i.e (localhost:9090), but with different authentications to redirect the traffic over one or another proxy. Let's say

http://test1:test1@localhost:9090/ will redirect to proxy http://a.b.c.d:1234 http://test2:test2@localhost:9090/ will redirect to proxy http://e.f.g.h:1235

I'm new in this, so I don't actually know if this should be done here, or in a server (apache, nginx, etc) to redirect the traffic to one or another proxy server depending on the auth

I have read the doc, but I can't find anything like it

In case it can be done straight with python-proxy , what's the way?

Thanks in advance

It is very prone to breaking and should not be done. Consider sub-classing or making a proxy object instead. If anyone else were to do the same, this would break.

Hi,
I want a transparent proxy for http and https. At the moment i think there is a feature missing.

Setup:
Linux Laptop
2 Networks

• Home network direct internet access
• Corporate network internet access only through proxy

So i want that pproxy uses a proxy if proxy is alive otherwise fall back to direct connections.

Redirect all my local traffic to pproxy via iptables:
iptables -t nat -A OUTPUT -p tcp -m owner ! --uid-owner proxy --dport 80 -j REDIRECT --to-port 8080
iptables -t nat -A OUTPUT -p tcp -m owner ! --uid-owner proxy --dport 443 -j REDIRECT --to-port 8080

pproxy -l redir://:8080/ -r http://142.93.135.90:8080 -a 3 -vv

Start as user and group "proxy" (so that direct connections will work)
Redirect traffic to upstream proxy (corporate)
Try if alive in a 3 second interval
Verbose to check whats going on

Problem:
Tests:

curl -I http://seamex.de
curl: (8) Weird server reply
curl -I https://seamex.de
HTTP/2 200 ...

Looks like just the https traffic is redirected. Can you please fix that :)
https and http traffic needs to be redirected to the upstream proxy (i also tried auto detection like: redir+http but that dont work)

Workaround:
I installed redsocks that listens on two ports one for http and one for https to do the different connect methods (http-relay, http-connect). Redsocks upstreams to pproxy (as http started). But i dont want to use two proxies when one can handel that ;)

Something like this has already been done in Go:

https://github.com/liudanking/quic-proxy

but I think it'll need a couple extra libraries. It's exteremely fast and reliable and it's somewhat undetected under a lot of Firewalls :)

Hi, I managed to get a working statically linked binary for pproxy here. Do you think it's advisable to put that binary in the release section of the project? It works very well and doesn't depend on system's Python version. Most older Linux distros don't have python 3.6 so the binary will work just fine

I have the following use case:
1.1.1.1:1000 - a tcp service
2.2.2.2:2000 - a http proxy with authentication
On 3.3.3.3, i need to open a service which listens to port 3000, and tunnels all the requests to 1.1.1.1:1000 via the 2.2.2.2:2000 proxy.

So when i telnet 3.3.3.3:3000, i should be able to talk to 1.1.1.1:1000.

socat works just fine, but it creates a new pid for every connection and it is kind of slow and unreliable.
Can python-proxy be used for this case?

Just like the original asyncio.open_connection, it would be desirable to have these parameters exposed.

Can i load-balance between multiple rservers (which have the same protocol)?

-r http://server1:8080#user:password,http://server2:8080:user:password should forward half of the connctions to server1 and half to server2. I didn't find any support for something like this in the source.

One should not "fire and forget" futures to work as "background tasks". This is very much like launching a thread and forgetting about it, when you probably should join it back at some point. This makes the code harder to reason about. One must properly await them, at some point in the future, and they're not even saved anywhere. See https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/ for a good reasoning and their designed "nurseries" as to why this is good.

I am trying to start from the example code to achieve a dynamic reverse proxy setup but having difficulty figuring out how to do it. Server is behind NAT. I need Server to connect to CentralServer. Then, CentralServer to open a port so that browser clients can tunnel through Server behind NAT:

<-- WWW <-- ServerBehindNAT <--> CentralServer <-- BrowserClient1
                                               <-- BrowserClient2

init.py代码53行可以改为：

server_ip = writer.get_extra_info('sockname')
if server_ip != '127.0.0.1':
    wait_connect = asyncio.open_connection(host=host_name, port=port, local_addr=(server_ip, 0))
else:
    wait_connect = asyncio.open_connection(host=host_name, port=port)

不太方便使用pr，希望可以改成类似这种，可以达到访问192.168.1.100使用100代理外网，访问192.168.1.101使用101代理外网，在单主机多IP的情况下特别方便，也可以加个argparse参数控制:-)

安装python3.6
pip安装pproxy
客户端命令
➜ Python-3.6.0b2 pproxy -i http+socks://:1080 -r ss://aes-256-cfb:[email protected]:8088 -v
Serving on :1080 by socks,http
http wtfismyip.com:80 -> ss abc.net:8088
DIRECT: 0 (0.0K/s,0.0K/s) PROXY: 0 (0.0K/s,0.0K/s)

➜ ~ curl -x 127.0.0.1:1080 wtfismyip.com/json
会停在那里不反回

验证用sslocal是可以的。

您好，我想请教一个下载中断的问题。
我使用python-proxy连接了我的ss服务器后，外网访问，观看视频都是没有问题的，但是在下载文件时就会遇到连接频繁被中断的情况。一般开启下载任务后，比如下载dl.google.com域名下的东西，五秒之内就完全没有速度了。
我试过Chrome和Firefox的内置下载工具，以及proxychains+wget，结果都是五秒内就完全没速度了，进而下载中断。
我确定这个并非python-proxy的问题，因为我在自己电脑和另一台电脑都试过（包括python-proxy和ss-qt5等代理工具），都出现了这种情况。操作系统都是Ubuntu的，所以我觉得问题可能是出在Ubuntu系统的配置上。
ss服务器感觉应该是没有问题的，因为我在Windows和Android下分别通过shadowsocks连接服务器，创建的下载任务都是没有任何问题的。
请问您可否给些建议？

FYI：
SS服务器在搬瓦工搭建，使用SS和锐速

我的操作系统是Ubuntu 18.04
SS使用SOCKS5
协议为CHACHA20-IETF-POLY1305

when i try to pproxy -l http+socks5://93.11.80.25:9005/@2a00:e60:4000:100:8000:0:f5e8:2b73#user:pass -v

it gives the following error: [Errno -6] ai_family not supported from 79.127.80.11 (this is connected client IP)

it seems server gets confused, the bind ip is IPv6 but user is getting connected through IPV4:9005.

😆

Roadmap for pproxy. please keep this open

• Configuration file support (yml file would be ideal)
• Daemon mode support
• Proper logging support (with levels/files/syslog support)
• Ability to use base64: syntax for any input file (certificate etc) instead of providing the actual file
• Implement more protocols (I have nothing in mind)
• Handle SIGKILL etc. better
• Multiple user authentication
• Traffic statistics by domain, user, etc
• Logging by domain, url, etc with different levels
• Set outgoing traffic interface/IP
• Better handling of first available and proxy checking (with timeouts and test URLs)

this will be updated and checkmarked as they're done.

I'd like to use python-proxy inline in my existing python application. It seems like I could built it up myself from pproxy, but I'd be replicating a lot of the functionality in server.main. It would be great if there was a developer friendly function pproxy that took arguments and returned a coroutine that did all the proxying.

Then the server.main would look something like:

def main():
  ...
  args = parser.parse_args()
  loop = asyncio.get_event_loop()
  loop.run_until_complete(pproxy(args))
  loop.close() 

I'm not sure if this works well w/ your architecture - but it would be very handy for my use case!

Cheers!

Hello
My use case is - to configure Connection with several relay socks proxies and to use it with aiohttp - how it is possible?
thanks

What is the handling of ipv4/ipv6 outbound from server? Is it possible to specify only ipv4 or ipv6 outbound hosts if server has both ipv4 and ipv6 address?

😁

Hi, I am using pproxy as a socks5 relay server.
according to proto.py, Socks5.connect sends b'\x05\x01\x02\x01' + ... to server at first. But other socks5 clients only sends b'\x05\x02\x00\x01

Another proof is running two pproxy process, one is:

pproxy -l socks5://:8080/ -v

the other is:

pproxy -l socks5://:8081/ -v -r socks5://localhost:8080 --test http://ifconfig.co/

It results in Unsupported protocol b'1' from ::1 and Exception: Unknown remote protocol.

I am going to figure out the correct protocol then.

Thanks!

Can python-proxy be usable as a module where given a proxy and its type, tunnel the connection through python-proxy?
The use case would be proxying multiple connections at once.

Hi,

Do you think it's possible to write a parser for DAT files and add their configuration to pproxy for remote hosts (-r)? DAT files have their own standard formatting for host whitelisting and blacklisting so it would be a great integration

when running proxy with the following arguments socks gives this error: getsockaddrarg() takes exactly 2 arguments (4 given) from 95.X.X.X

pproxy -l http+socks5://93.X.X.X:9000/@00:e:0:0:0:0:edaf:80ef#user:pass -v

Hi, here's a very basic config file template designed to work with pproxy. I'm open for all sorts of feedback and improvement as this probably covers ~60 percent of what pproxy offers.

{
    "general":{
        "block":{
            "enabled": true,
            "type": "path_list/regex_list",
            "values": [
                "1.1.1.1/24",
                "2.2.2.2/24",
                "in case of path_list => /path/to/the/file1",
                "in case of path_list => /path/to/the/file2"
            ]
        },
        "check_alive":{
            "enabled": true,
            "interval": 30
        },
        "schedule":{
            "enabled":true,
            "algorithm": "fa"
        },
        "pac":{
            "enabled": true,
            "path": "asdasds"
        },
        "sys":{
            "enabled": true
        },
        "test":{
            "enabled": true,
            "path": "http://ident.me"
        }

    },
    "servers":[
            {
                "enabled": true,
                "name": "SERVER1",
                "type": "in/out",
                "addr": "0.0.0.0",
                "port": "80",
                "unix_socket": "/tmp/myproxy",
                "inverse": false,
                "http_settings":{
                    "enabled": true,
                    "disable_http_connect": true,
                    
                    "auth": {
                        "enabled": true,
                        "creds": [
                            "user1:pass1",
                            "user2:pass2"
                        ]
                    },
                    "ssl": {
                        "enabled": true,
                        "type": "base64/path/insecure",
                        "cert": "asdasdasd",
                        "key": "asdasdasd"
                    }
                },
                "socks_settings":{
                    "enabled":true,
                    "version": "4/5/4+5",
                    "auth": {
                        "enabled": true,
                        "creds": [
                            "user1:pass1",
                            "user2:pass2"
                        ]
                    },
                    "ssl": {
                        "enabled": true,
                        "type": "base64/path/insecure",
                        "cert": "asdasdasd",
                        "key": "asdasdasd"
                    }                  
                },
                "ss_settings":{
                    "enabled": true,
                    "password": "123",
                    "cipher": "aes-256-gcm",
                    "ssl": {
                        "enabled": true,
                        "type": "base64/path/insecure",
                        "cert": "asdasdasd",
                        "key": "asdasdasd"
                    }
                },
                "ssr_settings":{
                    "enabled":true,
                    "password": "123",
                    "cipher": "aes-256-gcm",
                    "plugins":{
                        "enabled": true,
                        "values":[
                            "plain",
                            "origin",
                            "http_simple",
                            "tls1.2_ticket_auth",
                            "verify_simple",
                            "verify_deflate"
                        ]
                    }
                },
                "redir_settings":{
                    "enabled": true
                },
                "tunnel_settings":{
                    "enabled": true,
                    "hosts": ["www.google.com"]
                },
                "ssh_settings":{
                    "enabled": true,
                    "remote_host": "1.2.3.4",
                    "remote_port": 22,
                    "cred_type": "password/key_path/key_base64",
                    "username": "root",
                    "cred": "whatever"
                },
                "ws_settings":{
                    "enabled": true,
                    "ssl": {
                        "enabled": true,
                        "type": "base64/path/insecure",
                        "cert": "asdasdasd",
                        "key": "asdasdasd"
                    }
                }                
            }     
        ]
}

As you know, asyncssh requires pycrypto to be installed. pycryptodome's documentation says:

One must avoid having both PyCrypto and PyCryptodome installed at the same time, as they will interfere with each other.

So we can't have both sshtunnel and C accelerations at the same time. Any workarounds?

in verbose output

I thought that if I didn't specify remote, it would forward all traffic. After starting server and connecting via browser, all pages are blank. What did I miss?

import asyncio
import pproxy

server = pproxy.Server('http+socks5://0.0.0.0')

args = dict(verbose=print)

loop = asyncio.get_event_loop()
handler = loop.run_until_complete(server.start_server(args))
try:
    loop.run_forever()
except KeyboardInterrupt:
    print('exiting on interrupt')

handler.close()
loop.run_until_complete(handler.wait_closed())
loop.run_until_complete(loop.shutdown_asyncgens())
loop.close()

比如我现在有几个网卡，ip分别为ip1, ip2

pproxy -r direct:///@IP2 这样的出口 依然是默认ip即 ip1

I had the problem with http+ssl with remote server:

SSL handshake failed
protocol: <asyncio.sslproto.SSLProtocol object at 0x7f2611d7eeb8>
transport: <_SelectorSocketTransport fd=10 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
  File "/home/proxy-server/miniconda3/lib/python3.7/asyncio/sslproto.py", line 625, in _on_handshake_complete
    raise handshake_exc
  File "/home/proxy-server/miniconda3/lib/python3.7/asyncio/sslproto.py", line 189, in feed_ssldata
    self._sslobj.do_handshake()
  File "/home/proxy-server/miniconda3/lib/python3.7/ssl.py", line 763, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1051)
SSL error in data received
protocol: <asyncio.sslproto.SSLProtocol object at 0x7f2611d7eeb8>
transport: <_SelectorSocketTransport closing fd=10 read=idle write=<idle, bufsize=0>>
Traceback (most recent call last):
  File "/home/proxy-server/miniconda3/lib/python3.7/asyncio/sslproto.py", line 526, in data_received
    ssldata, appdata = self._sslpipe.feed_ssldata(data)
  File "/home/proxy-server/miniconda3/lib/python3.7/asyncio/sslproto.py", line 189, in feed_ssldata
    self._sslobj.do_handshake()
  File "/home/proxy-server/miniconda3/lib/python3.7/ssl.py", line 763, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1051)
[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1051)

Is this common?