Giter Club home page Giter Club logo

realm's Introduction

Realm

A simple, high performance relay server written in rust.

realm realm downloads

Libs

lib doc
realm-core crates.io Released API docs
realm-io crates.io Released API docs
realm-lb crates.io Released API docs
realm-hook crates.io Released API docs
realm-syscall crates.io Released API docs

Features

  • Zero configuration. Setup and run in one command.
  • Concurrency. Bidirectional concurrent traffic leads to high performance.
  • Low resources cost.

Container

Realm can be run in a container with OCI (like Docker, Podman, Kubernetes, etc), see guides here.

Build

Install rust toolchain with rustup:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Clone this repository:

git clone https://github.com/zhboner/realm && cd realm

Build:

cargo build --release

You can also pass target_cpu=native to allow more possible optimizations:

RUSTFLAGS='-C target_cpu=native' cargo build --release

The realm binary will be available in target/release.

Build Options

  • udp: enable udp relay builtin.
  • tfo: enable tcp-fast-open deprecated.
  • trust-dns: enable trust-dns's async dns resolver builtin.
  • zero-copy: enable zero-copy on linux builtin.
  • brutal-shutdown: see realm_io/brutal-shutdown.
  • hook: see realm_hook.
  • proxy: enable proxy-protocol.
  • balance: enable load balance.
  • transport: enable ws/tls/wss.
  • multi-thread: enable tokio's multi-threaded IO scheduler.
  • mi-malloc: custom memory allocator.
  • jemalloc: custom memory allocator.
  • page-alloc: custom memory allocator.

Default: hook + proxy + balance + transport + brutal-shutdown + multi-thread.

See also: Cargo.toml.

Examples:

# simple tcp
cargo build --release --no-default-features

# enable other options
cargo build --release --features 'jemalloc'

# fully customized
cargo build --release
    --no-default-features
    --features 'transport, multi-thread, jemalloc'

Cross Compile

Please refer to https://rust-lang.github.io/rustup/cross-compilation.html. You may need to install cross-compilers or other SDKs, and specify them when building the project.

Or have a look at Cross, it makes things easier.

Usage

Realm 2.4.0 [hook][proxy][balance][brutal][transport][multi-thread]
A high efficiency relay tool

USAGE:
    realm [FLAGS] [OPTIONS]

FLAGS:
    -h, --help       show help
    -v, --version    show version
    -d, --daemon     run as a unix daemon
    -u, --udp        force enable udp forward
    -t, --ntcp       force disable tcp forward
    -f, --tfo        force enable tcp fast open -- deprecated
    -z, --splice     force enable tcp zero copy -- deprecated

OPTIONS:
    -c, --config <path>                 use config file
    -l, --listen <address>              listen address
    -r, --remote <address>              remote address
    -x, --through <address>             send through ip or address
    -i, --interface <device>            bind to interface
    -a, --listen-transport <options>    listen transport
    -b, --remote-transport <options>    remote transport

SYS OPTIONS:
    -n, --nofile <limit>          set nofile limit
    -p, --pipe-page <number>      set pipe capacity
    -j, --pre-conn-hook <path>    set pre-connect hook

LOG OPTIONS:
        --log-level <level>    override log level
        --log-output <path>    override log output

DNS OPTIONS:
        --dns-mode <mode>            override dns mode
        --dns-min-ttl <second>       override dns min ttl
        --dns-max-ttl <second>       override dns max ttl
        --dns-cache-size <number>    override dns cache size
        --dns-protocol <protocol>    override dns protocol
        --dns-servers <servers>      override dns servers

PROXY OPTIONS:
        --send-proxy                       send proxy protocol header
        --send-proxy-version <version>     send proxy protocol version
        --accept-proxy                     accept proxy protocol header
        --accept-proxy-timeout <second>    accept proxy protocol timeout

TIMEOUT OPTIONS:
        --tcp-timeout <second>    override tcp timeout
        --udp-timeout <second>    override udp timeout

SUBCOMMANDS:
    convert    convert your legacy configuration into an advanced one

Start from command line arguments:

realm -l 0.0.0.0:5000 -r 1.1.1.1:443

Start with a config file:

# use toml
realm -c config.toml

# use json
realm -c config.json

Start with environment variables:

REALM_CONF='{"endpoints":[{"local":"127.0.0.1:5000","remote":"1.1.1.1:443"}]}' realm

# or
export REALM_CONF=`cat config.json | jq -c `
realm

Convert a legacy config file:

realm convert old.json

Configuration

TOML Example

[log]
level = "warn"
output = "/var/log/realm.log"

[network]
no_tcp = false
use_udp = true

[[endpoints]]
listen = "0.0.0.0:5000"
remote = "1.1.1.1:443"

[[endpoints]]
listen = "0.0.0.0:10000"
remote = "www.google.com:443"
JSON Example

{
  "log": {
    "level": "warn",
    "output": "/var/log/realm.log"
  },
  "network": {
    "no_tcp": false,
    "use_udp": true
  },
  "endpoints": [
    {
      "listen": "0.0.0.0:5000",
      "remote": "1.1.1.1:443"
    },
    {
      "listen": "0.0.0.0:10000",
      "remote": "www.google.com:443"
    }
  ]
}

See more examples here.

Overview

├── log
│   ├── level
│   └── output
├── dns
│   ├── mode
│   ├── protocol
│   ├── nameservers
│   ├── min_ttl
│   ├── max_ttl
│   └── cache_size
├── network
│   ├── no_tcp
│   ├── use_udp
│   ├── tcp_timeout
│   ├── udp_timeout
│   ├── send_proxy
│   ├── send_proxy_version
│   ├── accept_proxy
│   └── accept_proxy_timeout
└── endpoints
    ├── listen
    ├── remote
    ├── extra_remotes
    ├── balance
    ├── through
    ├── interface
    ├── listen_transport
    ├── remote_transport
    └── network->

You should provide at least endpoint.listen and endpoint.remote, the left fields will take their default values.

Option priority: cmd override > endpoint config > global config.

endpoint

endpoint.listen: string

Local address, supported formats:

  • ipv4:port
  • ipv6:port

endpoint.remote: string

Remote address, supported formats:

  • ipv4:port
  • ipv6:port
  • example.com:port

endpoint.extra_remotes: string array

Extra remote address, same as endpoint.remote above.

endpoint.balance: string

Require balance feature.

Load balance strategy and weights of remote peers.

Format:

$strategy: $weight1, $weight2, ...

Where remote is used as default backend server, and extra_remotes are used as backups.

Available algorithms (provided by realm_lb):

  • iphash

  • roundrobin

Example:

[[endpoints]]
remote = "a:443"
extra_remotes = ["b:443", "c:443"]
balance = "roundrobin: 4, 2, 1"

The weight of [a, b, c] is [4, 2, 1] in turn.

endpoint.through: string

TCP: Bind a specific ip before opening a connection.

UDP: Bind a specific ip or address before sending packet.

Supported formats:

  • ipv4/ipv6 (tcp/udp)
  • ipv4/ipv6:port (udp)

endpoint.interface: string

Bind to a specific interface.

endpoint.listen_transport: string

Require transport feature.

See Kaminari Options.

endpoint.remote_transport: string

Require transport feature.

See Kaminari Options.

endpoint.network

The same as network, override global options.

log

log.level: string

values:

  • off
  • error
  • warn
  • info
  • debug
  • trace

default: off

log.output: string

values:

  • stdout
  • stderr
  • path (e.g. /var/log/realm.log)

default: stdout

dns

Require trust-dns feature.

dns.mode: string

Dns resolve strategy.

values:

  • ipv4_only
  • ipv6_only
  • ipv4_then_ipv6
  • ipv6_then_ipv4
  • ipv4_and_ipv6

default: ipv4_and_ipv6

dns.protocol: string

Dns transport protocol.

values:

  • tcp
  • udp
  • tcp_and_udp

default: tcp_and_udp

dns.nameservers: string array

Custom upstream servers.

format: ["server1", "server2" ...]

default:

If on unix/windows, read from the default location.(e.g. /etc/resolv.conf).

Otherwise, use google's public dns(8.8.8.8:53, 8.8.4.4:53 and 2001:4860:4860::8888:53, 2001:4860:4860::8844:53).

dns.min_ttl: unsigned int

The minimum lifetime of a positive dns cache.

default: 0

dns.max_ttl: unsigned int

The maximum lifetime of a positive dns cache.

default: 86400 (1 day)

dns.cache_size: unsigned int

The maximum count of dns cache.

default: 32

network

network.no_tcp: bool

Do not start a tcp relay.

default: false

network.use_udp: bool

Require udp feature

Start listening on a udp endpoint and forward packets to the remote peer.

It will dynamically allocate local endpoints and establish udp associations. Once timeout, the endpoints will be deallocated and the association will be terminated. See also: network.udp_timeout.

Due to the receiver side not limiting access to the association, the relay works like a full-cone NAT.

default: false

network.zero_copy: bool deprecated

Require zero-copy feature.

Use splice instead of send/recv while handing tcp connection. This will save a lot of memory copies and context switches.

default: false

network.fast_open: bool deprecated

Require fast-open feature.

It is not recommended to enable this option, see The Sad Story of TCP Fast Open.

default: false

network.tcp_timeout: unsigned int

This is connect timeout. An attempt to connect to a remote peer fails after waiting for a period of time.

To disable timeout, you need to explicitly set timeout value to 0.

default: 5

network.udp_timeout: unsigned int

Terminate udp association after timeout.

The timeout value must be properly configured in case of memory leak. Do not use a large timeout!

default: 30

network.send_proxy: bool

Require proxy feature.

Send haproxy PROXY header once the connection established. Both v1 and v2 are supported, see send_proxy_version.

You should make sure the remote peer also speaks proxy-protocol.

default: false

network.send_proxy_version: unsigned int

Require proxy feature.

This option has no effect unless send_proxy is enabled.

value:

  • 1
  • 2

default: 2

network.accept_proxy: bool

Require proxy feature.

Wait for a PROXY header once the connection established.

If the remote sender does not send a v1 or v2 header before other contents, the connection will be closed.

default: false

network.accept_timeout: unsigned int

Require proxy feature.

Wait for a PROXY header within a period of time, otherwise close the connection.

default: 5.

realm's People

Contributors

billzhong avatar noahziheng avatar sabify avatar soniccube avatar zephyrchien avatar zhboner avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

realm's Issues

realm断流问题

之前1.2版本是有断流情况,后来更新了1.3 1.4,但是原作者只说了improve code,没有提到具体的断流是否有修复

运行时候报错

系统debian9
感谢大佬分享项目,我在用编译好的文件准备调试,gnu及musl版本均出现报错

root@debian:/etc/realm# ./realm -c config.json
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { kind: InvalidInput, message: "invalid socket address" }', src/conf/dns.rs:69:54
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

config.json

{
	"dns_mode": {
		"mode": "ipv4_only",
		"protocol": "udp",
		"nameservers": ["119.29.29.29", "223.5.5.5"]
	},
	"endpoints": [
		{
			"local": "0.0.0.0:19110",
			"remote": "abc.com:11412",
               "fast_open": false,
               "zero_copy": true
		},
		{
			"local": "0.0.0.0:19117",
			"remote": "cbd.com:11412",
               "fast_open": false,
               "zero_copy": true
		},
		{
			"local": "0.0.0.0:19116",
			"remote": "def.com:11412",
               "fast_open": false,
               "zero_copy": true
		}
	]
}

有什么办法能修复下么
感谢!

TCP Fast Open 和 Zero Copy 冲突

编译时功能全开,同时开启 tfo 和 zero-copy 就会失联,关闭两者其中一个功能就正常连通。
环境时Debian 11.3 内核是保持最新的。

进程守护

想请教一下大佬进程守护systemd怎么使用 现在用的是screen

关于tfo

想请教一下作者 tfo是默认开启的吗 当前使用的是配置文件形式 如需关闭 格式是否是fast_open = false 貌似找到上次掉速原因 跟系统内核参数tcp_fastopen_blackhole_timeout_sec有关

请问是否可能支持指定出口线路

多线接入的转发机器,能否在不修改路由表/iptables的情况下支持指定出口?
如果比较麻烦,需要改动不少代码的话就当这个issue不存在吧😅

谢谢

希望支持自动移除故障节点

{
"listen": ["0.0.0.0:5000","0.0.0.0:4002","0.0.0.0:5003"],
"upstreams": ["1.1.1.1:1234","8.8.8.8:443"],
"fail_timeout": 1800
}

希望能支持类似ng的upstream方式,多端口监听,随机选择upstream,如果upstream tcp链接超时或者refuse,这个节点就禁用fail_timeout的时长

[Bug] /lib64/libc.so.6: version 'GLIBC_2.18' not found

当运行在 centos7 系统时,会有错误提示:/lib64/libc.so.6: version 'GLIBC_2.18' not found (required by ./realm)

# cat /etc/os-release

NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

todo

  • zero copy
  • tcp fast open (Linux>=4.11, Windows, BSD; Linux>=3.7 without zero-copy)
  • tcp fast open (Linux>3.7 with zero-copy)
  • bind outbound ip or address
  • bind outbound interface
  • custom dns servers
  • options to turn on/off tfo, zero-copy
  • daemonize
  • timeout
  • error handle
  • logger

code=killed, signal=ABRT

● realm.service - realm
Loaded: loaded (/etc/systemd/system/realm.service; enabled; vendor preset: enabled)
Active: activating (auto-restart) (Result: signal) since Thu 2022-04-07 17:50:09 CST; 4s ago
Process: 760 ExecStartPre=/bin/sh -c ulimit -n 51200 (code=exited, status=0/SUCCESS)
Process: 761 ExecStart=/usr/local/bin/realm -c /etc/realm/config.json (code=killed, signal=ABRT)
Main PID: 761 (code=killed, signal=ABRT)
CPU: 4ms

想问下这种情况,可能是什么问题呢,重启也没有效果

ipv6转ipv4

realm 目前似乎不能像 socat 那样监听本地 ipv6 端口,转发到远程 ipv4 端口。
测试了使用 through 绑定 ipv4 不通。但是本地 ipv6 到远程 ipv6 是没问题。

cpu占用100%

系统debian10 在启用日志后 过几分钟cpu占用100 日志文件几个g

速率下降

长时间转发后 速率会下降 kill掉后重新转发又恢复正常

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.