bfenetworks / bfe Goto Github PK

Description:

• mirror of an original request by creating background request. Responses to mirror subrequests are ignored.

Serve static content by customize rules

之前在一些介绍材料中了解到BFE除了本身组件外，还有两个核心的调度组件，一个GSLB，另外一个GTC组件，这个开源的版本中，GTC外网调度能力是否包含？

Grade A+

1. Meets the requirement of Grade A
2. TLS version >= 1.2

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

好歹给点使用文档和范例呀

changes to logid:

1. change logid to a string value in length of 32
2. generated from 16 random bytes and encoded in hex format

Add description of log fields for mod_access

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

compress response depending on the defined rules

For example this file:

https://github.com/baidu/bfe/blob/develop/conf/server_data_conf/route_rule.data

I add some new ProductRule rule, do I need to reload the bef to make the new rule take effect?

Write key logs in NSS key log format so that external
programs(eg. wireshark) can decrypt TLS connections for trouble shooting.

Log format:
https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/Key_Log_Format

Description:

• Support OpenTracing for distributed tracing.

Reference:

• https://opentracing.io/docs/overview/
• https://opentracing.io/specification/
• https://github.com/opentracing/opentracing-go

• req_cip_hash_in(patterns)
• req_header_value_hash_in(header_name, patterns, case_insensitive)
• req_cookie_value_hash_in(key, patterns, case_insensitive)
• req_query_value_hash_in(key, patterns, case_insensitive)

Protect resources from unauthorized access by adding encoded data like the hash of a specific part of the URL.

https://github.com/baidu/bfe/blob/develop/docs/zh_cn/bfe_config/bfe.conf.md => https://github.com/baidu/bfe/blob/develop/docs/zh_cn/configuration/bfe.conf.md

During response forward phase, if flushInterval <0, just flush response header after write

Description:

• Use an ACME provider (like Let's Encrypt) for automatic certificate generation.

Reference:

• https://go-acme.github.io/lego/

Set default value of cluster configuration items to Reduce configuration costs:

BackendConf

• TimeoutConnSrv: 2000 // 2 sec
• TimeoutResponseHeader: 60000 // 60 sec
• MaxIdleConnsPerHost: 0
• RetryLevel: 0

CheckConf

• Schem: "http",
• Uri: "/health_check",
• Host: "",
• StatusCode: 0,
• FailNum: 5,
• CheckInterval: 1000

GslbBasic

• CrossRetry: 0
• RetryMax: 2
• HashStrategy: 0
• SessionSticky: false

ClusterBasic

• TimeoutReadClient: 30000 // 30s
• TimeoutWriteClient: 60000 // 60s
• TimeoutReadClientAgain: 60000 // 60s
• ReqWriteBufferSize: 512 // 512B
• ReqFlushInterval: 0
• ResFlushInterval: -1
• CancelOnClientClose: false

Description：

• Add builtin RequestTemplate：COMMON

Reference:

• https://www.w3.org/Daemon/User/Config/Logging.html#common-logfile-format

expose metrics in prometheus text-based format
See: https://prometheus.io/docs/instrumenting/exposition_formats/

Add description of variables for mod_header

Remove following fields:

• req_content_len
• res_content_len
• http
• client_read_time
• cluster_time

Rename following fields:

• body_len_in -> req_body_len
• body_len_out -> res_body_len
• uri -> url

• req_header_value_contain()
• req_cookie_value_contain()
• req_query_value_contain()

Description:

• Sets cookies suitable for client identification.

Format of data config:

{
    Version: "version_num",
    Config: {
        "product1": [
             {
                 "Cond": "cond_expression",
                 "Params": {
                     "Name": "UID",
                     "Domain": "example.org",
                     "Path": "/",
                     "Expire": 31536000
                 }
             }
         ]
    }
}

Limit the amount of connections/requests a user can make in a given period of time.

log verbose information:

• vip of connection
• sni of connection
• backend ip
• duration

Add license scanning, dependency analysis & intelligent compliance
into development workflow.

Use Rlock for read operation on BalTable (Lookup/GetState, etc)

Maintain basic status about request/response forwarded

Writes information about client sessions/requests in the access log

Support HTTP/3 protocol

Because the project name is BFE, and start with the bfe string, then the custom go modules are also named bfe_xxx, it is a redundant naming rule.

Set default value of core config items to Reduce configuration costs:

[server]
httpPort = 8080
httpsPort = 8443
monitorPort = 8299
maxCpus = 0
layer4LoadBalancer = ""
tlsHandshakeTimeout = 30
clientReadTimeout = 60
clientWriteTimeout = 60
keepAliveEnabled = true
gracefulShutdownTimeout = 10
maxHeaderBytes = 1048576
maxHeaderUriBytes = 8192

hostRuleConf = server_data_conf/host_rule.data
vipRuleConf = server_data_conf/vip_rule.data
routeRuleConf = server_data_conf/route_rule.data
clusterConf = server_data_conf/cluster_conf.data
nameConf = server_data_conf/name_conf.data

clusterTableConf = cluster_conf/cluster_table.data
gslbConf = cluster_conf/gslb.data

monitorInterval = 20
debugServHttp = false
debugBfeRoute = false
debugBal = false
debugHealthCheck = false

[httpsBasic]
serverCertConf = tls_conf/server_cert_conf.data
tlsRuleConf = tls_conf/tls_rule_conf.data

cipherSuites=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256|TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
cipherSuites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256|TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
cipherSuites=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256|TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
cipherSuites=TLS_ECDHE_RSA_WITH_RC4_128_SHA
cipherSuites=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
cipherSuites=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
cipherSuites=TLS_RSA_WITH_RC4_128_SHA
cipherSuites=TLS_RSA_WITH_AES_128_CBC_SHA
cipherSuites=TLS_RSA_WITH_AES_256_CBC_SHA

curvePreferences=CurveP256
enableSslv2ClientHello = true
clientCABaseDir = tls_conf/client_ca

[sessionCache]
sessionCacheDisabled = true
servers = ""
keyPrefix = "bfe"
connectTimeout = 50
readTimeout = 50
writeTimeout = 50
maxIdle = 20
sessionExpire = 3600

[sessionTicket]
sessionTicketsDisabled = true
sessionTicketKeyFile = tls_conf/session_ticket_key.data

add config doc for mod_http_code

目前开源版本我已验证本机房多个子集群间数据分流，那么现有开源项目是否支持GSLB 跨IDC机房集群流量分流控制？

Description:

• proxy gRPC calls from clients to gRPC services.

Reference:

• https://github.com/grpc/grpc-go

Proxy http requests to a SCGI server.

Proxy http requests to a FastCGI server.

Support tls 1.3 protocol

1. determine user geolocation by IP address using MaxMind database（https://github.com/oschwald/maxminddb-golang）

2. add geolocation to request context

can you help me for give us some examples, such as some comme configure and request urls

Description:

• Validating the provided JSON Web Token (JWT) using the specified keys.

Reference:

• JSON Web Token: https://tools.ietf.org/html/rfc7519
• JSON Web Key: https://tools.ietf.org/html/rfc7517

Sets the Content-Type in a response based on the file extension.

Support the proxy protocol to be compatible with F5 BigIP/Citrix ADC etc

The proxy protocol：
https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt

Validating the user name and password using the “HTTP Basic Authentication” protocol.

See：

• https://tools.ietf.org/html/rfc7617
• https://github.com/abbot/go-http-auth

web application firewall based on ModSecurity

ModSecurity
https://modsecurity.org/rules.html

bal_gslb: use random value if hash key not found

See: https://golangci.com/r/github.com/bfenetworks/bfe

return customized response(error page/redirect) for specified error