Asynch Mode for NGINX*
- Introduction
- Licensing
- Features
- Hardware Requirements
- Software Requirements
- Additional Information
- Limitations
- Installation Instructions
- SSL Engine Framework for Configuration
- Support for Nginx Side Polling
- QATzip Module Configuration
- QAT Sample Configuration
- Known Issues
- Intended Audience
- Legal
Nginx* [engine x] is an HTTP and reverse proxy server, a mail proxy server, and a generic TCP/UDP proxy server, originally written by Igor Sysoev. This project provides an extended Nginx working with asynchronous mode OpenSSL*. With Intel® QuickAssist Technology (QAT) acceleration, Asynch Mode for NGINX* can provide significant performance improvement.
The Licensing of the files within this project is:
Asynch Mode for NGINX* - BSD License. Please
see the LICENSE file contained in the top level folder. Further details can
be found in the file headers of the relevant files.
- Asynchronous Mode in SSL/TLS processing (including http/stream/mail/proxy module)
- SSL Engine Framework for engine configuration
- Support for external polling mode and heursitic polling mode
- Release hardware resource during worker is shutting down (For more details information, please read modules/nginx_qat_module/README)
- Support OpenSSL Cipher PIPELINE feature
- Support QATzip module to accelerate GZIP compression with Intel® Quickassist Technology
- Support software fallback for asymmetric cryptography algorithms.
- Support [QAT Engine multibuffer feature][10] [10]:https://github.com/intel/QAT_Engine#intel-qat-openssl-engine-multibuffer-support
- Support FreeBSD Operating System.
Asynch Mode for NGINX* supports Crypto and Compression offload to the following acceleration devices:
- Intel® C62X Series Chipset
- Intel® Communications Chipset 8925 to 8955 Series
- Intel® Communications Chipset 8960 to 8970 Series
- Intel® C3XXX Series Chipset
This release was validated on the following:
- Asynch Mode for NGINX* has been tested with the latest Intel® QuickAssist Acceleration Driver. Please download the QAT driver from the link https://www.intel.com/content/www/us/en/developer/topic-technology/open/quick-assist-technology/overview.html
- OpenSSL-3.0.10
- QAT engine v1.3.1
- QATzip v1.1.2
-
Asynch Mode for NGINX* is developed based on Nginx-1.22.1.
-
Generate patch against official Nginx-1.22.1.
git clone https://github.com/intel/asynch_mode_nginx.git
wget http://nginx.org/download/nginx-1.22.1.tar.gz
tar -xvzf nginx-1.22.1.tar.gz
diff -Naru -x .git nginx-1.22.1 asynch_mode_nginx > async_mode_nginx_1.22.1.patch- Apply patch to official Nginx-1.22.1.
wget http://nginx.org/download/nginx-1.22.1.tar.gz
tar -xvzf nginx-1.22.1.tar.gz
patch -p0 < async_mode_nginx_1.22.1.patch- Generate patch against github official read-only mirror
git clone https://github.com/intel/asynch_mode_nginx.git
wget https://github.com/nginx/nginx/archive/release-1.22.1.tar.gz
tar -xvzf release-1.22.1.tar.gz
diff -Naru -x .git -x .hgtags nginx-release-1.22.1 asynch_mode_nginx > async_mode_nginx_1.22.1.patch- Apply patch to the github release pachage.
wget https://github.com/nginx/nginx/archive/release-1.22.1.tar.gz
tar -xvzf release-1.22.1.tar.gz
patch -p0 < async_mode_nginx_1.22.1.patch- Asynch Mode for NGINX* SSL engine framework provides new directives:
Directives
Syntax: ssl_asynch on | off;
Default: ssl_asynch off;
Context: stream, mail, http, server
Enables SSL/TLS asynchronous modeExample
file: conf/nginx.conf
http {
ssl_asynch on;
server {...}
} stream {
ssl_asynch on;
server {...}
}Directives
Syntax: proxy_ssl_asynch on | off;
Default: proxy_ssl_asynch off;
Context: stream, http, server
Enables the SSL/TLS protocol asynchronous mode for connections to a proxied server.Example
file: conf/nginx.conf
http {
server {
location /ssl {
proxy_pass https://127.0.0.1:8082/outer;
proxy_ssl_asynch on;
}
}
}Directives
Syntax: grpc_ssl_asynch on | off;
Default: grpc_ssl_asynch off;
Context: http, server
Enables the SSL/TLS protocol asynchronous mode for connections to a grpc server.Example
file: conf/nginx.conf
http {
server {
location /grpcs {
grpc_pass https://127.0.0.1:8082/outer;
grpc_ssl_asynch on;
}
}
}- Asynch Mode for NGINX* provide new option
asynchforlistendirective.
Example
file: conf/nginx.conf
http {
server{ listen 443 asynch; }
}- Support OpenSSL Cipher PIPELINE feature (Deitals information about the pipeline settings, please refer to OpenSSL Docs)
Directives
Syntax: ssl_max_pipelines size;
Default: ssl_max_pipelines 0;
Context: server
Set MAX number of pipelinesDirectives
Syntax: ssl_split_send_fragment size;
Default: ssl_split_send_fragment 0;
Context: server
Set split size of sending fragmentDirectives
Syntax: ssl_max_send_fragment size;
Default: ssl_max_send_fragment 0;
Context: server
Set max number of sending fragment-
Nginx supports
reloadoperation, when QAT hardware is involved for crypto offload, user should ensure that there are enough number of QAT instances. For example, the available qat instance number should be 2x equal or more than Nginx worker process number.For example, in Nginx configuration file (
nginx.conf) worker process number is configured asworker_processes 16;Then the instance configuration in QAT driver configuration file should be
[SHIM] NumberCyInstances = 1 NumberDcInstances = 0 NumProcesses = 32 LimitDevAccess = 0 -
When configure "worker_process auto", Asynch Mode for NGINX* will need instance number equal or larger than 2 times of CPU core number. Otherwise, Asynch Mode for NGINX* might show various issue caused by leak of instance.
-
Nginx supports QAT engine and QATzip module. By default, they use User Space DMA-able Memory (USDM) Component. The USDM component is located within the Upstream Intel® QAT Driver source code in the following subdirectory:
quickassist/utilities/libusdm_drv. We should have this component enabled and assignd enough memory before using nginx_qat_module or nginx_qatzip_module, for example:echo 2048 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages insmod ./usdm_drv.ko max_huge_pages=2048 max_huge_pages_per_process=32
-
AES-CBC-HMAC-SHA algorithm won't be offloaded to QAT_HW if Encrypt then MAC(ETM) mode is used for SSL connection(by default). Refer to QAT_Engine for more details.
-
QATzip module supports GZIP compression acceleration now, does not support user define dictionary compression yet.
-
The HTTP2 protocol does not support asynchronous functionality.
Set the following environmental variables:
NGINX_INSTALL_DIR is the directory where Nginx will be installed to
OPENSSL_LIB is the directory where the OpenSSL has been installed to
QZ_ROOT is the directory where the QATzip has been compiled to
Configure nginx for compilation:
Configure Nginx against OpenSSL 3.0
```bash
./configure \
--prefix=$NGINX_INSTALL_DIR \
--with-http_ssl_module \
--add-dynamic-module=modules/nginx_qatzip_module \
--add-dynamic-module=modules/nginx_qat_module/ \
--with-cc-opt="-DNGX_SECURE_MEM -I$OPENSSL_LIB/include -I$ICP_ROOT/quickassist/include -I$ICP_ROOT/quickassist/include/dc -I$QZ_ROOT/include -Wno-error=deprecated-declarations" \
--with-ld-opt="-Wl,-rpath=$OPENSSL_LIB/lib64 -L$OPENSSL_LIB/lib64 -L$QZ_ROOT/src -lqatzip -lz"
```
Compile and Install:
make
make installNginx supports setting worker to non-root user, for example:
Add user qat in group qat, for example run below command in your terminal:
```bash
groupadd qat
useradd -g qat qat
```
In nginx.conf, you can set worker as qat, qat is the user you added before:
```bash
user qat qat;
```
Then we need to give non-root worker enough permission to enable qat, you need to run folow
connamds in your terminal:
```bash
chgrp qat /dev/qat_*
chmod 660 /dev/qat_*
chgrp qat /dev/usdm_drv
chmod 660 /dev/usdm_drv
chgrp qat /dev/uio*
chmod 660 /dev/uio*
chgrp qat /dev/hugepages
chmod 770 /dev/hugepages
chgrp qat /usr/local/lib/libqat_s.so
chgrp qat /usr/local/lib/libusdm_drv_s.so
```
These instructions can be found on QAT engine
These instructions can be found on QATzip
These instructions can be found on Official test
These instruction can be found on 9 9: https://github.com/intel/asynch_mode_nginx/blob/master/test/performance_bkm/README.md
As QAT engine is implemented as a standard OpenSSL* engine, its behavior can be
controlled from the OpenSSL* configuration file (openssl.conf), which can be
found on QAT engine.
Note:
From v0.3.2 and later, this kind of configuration in openssl.conf will not be
effective for Nginx. Please use the following method to configure Nginx SSL
engine, such as Intel® QAT.
An SSL Engine Framework is introduced to provide a more powerful and flexible
mechanism to configure Nginx SSL engine directly in the Nginx configuration file
(nginx.conf).
A new configuration block is introduced as ssl_engine which provides two
general directives:
Sets the engine module and engine id for OpenSSL async engine. For example:
Syntax: use_engine [engine module name] [engine id];
Default: N/A
Context: ssl_engine
Description:
Specify the engine module name against engine idSets the engine module and engine id for OpenSSL async engine by only providing the engine id. The engine module should be the same as engine id in this case.
Syntax: use_engine [engine id];
Default: N/A
Context: ssl_engine
Description:
Specify the engine idSyntax: default_algorithms [ALL/RSA/EC/...];
Default: ALL
Context: ssl_engine
Description:
Specify the algorithms need to be offloaded to the engine
More details information please refer to OpenSSL engineA following configuration sub-block can be used to set engine specific
configuration. The name of the sub-block should have a prefix using the
engine name specified in use_engine, such as [engine_name]_engine.
Any 3rd party modules can be integrated into this framework. By default, a
reference module dasync_module is provided in src/engine/modules
and a QAT module nginx_qat_module is provided in modules/nginx_qat_modules.
An example configuration in the nginx.conf:
load_module modules/ngx_ssl_engine_qat_module.so;
...
ssl_engine {
use_engine qatengine;
default_algorithms RSA,EC,DH,PKEY_CRYPTO;
qat_engine {
qat_sw_fallback on;
qat_offload_mode async;
qat_notify_mode poll;
qat_poll_mode heuristic;
qat_shutting_down_release on;
}
}For more details directives of nginx_qat_module, please refer to
modules/nginx_qat_modules/README.
The qat_module provides two kinds of Nginx side polling for QAT engine,
- external polling mode
- heuristic polling mode
Please refer to the README file in the modules/nginx_qat_modules directory
to install this dynamic module.
Note: External polling and heuristic polling are unavailable in SSL proxy and stream module up to current release.
A timer-based polling is employed in each Nginx worker process to collect accelerator's response.
Directives in the qat_module
Syntax: qat_external_poll_interval time;
Default: 1
Dependency: Valid if (qat_poll_mode=external)
Context: qat_engine
Description:
External polling time interval (ms)
Valid value: 1 ~ 1000Example file: conf/nginx.conf
load_module modules/ngx_ssl_engine_qat_module.so;
...
ssl_engine {
use_engine qatengine;
default_algorithms ALL;
qat_engine {
qat_offload_mode async;
qat_notify_mode poll;
qat_poll_mode external;
qat_external_poll_interval 1;
}
}This mode can be regarded as an improvement of the timer-based external polling. It is the recommended polling mode to communicate with QAT accelerators because of its performance advantages. With the knowledge of the offload traffic, it controls the response reaping rate to match the request submission rate so as to maximize system efficiency with moderate latency, and adapt well to diverse types of network traffics.
Note:
- This mode is only available when using QAT engine v0.5.35 or later.
- External polling timer is enabled by default when heuristic polling mode is enabled.
In the heuristic polling mode, a polling operation is only triggered at a proper moment:
- Number of in-flight offload requests reaches a pre-defined threshold (in consideration of efficiency)
- All the active SSL connections have submitted their cryptographic requests and been waiting for the corresponding responses (in consideration of timeliness).
Directives in the qat_module
Syntax: qat_heuristic_poll_asym_threshold num;
Default: 48
Dependency: Valid if (qat_poll_mode=heuristic)
Context: qat_engine
Description:
Threshold of the number of in-flight requests to trigger a polling
operation when there are in-flight asymmetric crypto requests
Valid value: 1 ~ 512
Syntax: qat_heuristic_poll_sym_threshold num;
Default: 24
Dependency: Valid if (qat_poll_mode=heuristic)
Context: qat_engine
Description:
Threshold of the number of in-flight requests to trigger a polling
operation when there is no in-flight asymmetric crypto request
Valid value: 1 ~ 512Example
file: conf/nginx.conf
load_module modules/ngx_ssl_engine_qat_module.so;
...
ssl_engine {
use_engine qatengine;
default_algorithms ALL;
qat_engine {
qat_offload_mode async;
qat_notify_mode poll;
qat_poll_mode heuristic;
qat_heuristic_poll_asym_threshold 48;
qat_heuristic_poll_sym_threshold 24;
}
}This module is developed to accelerate GZIP compression with QAT accelerators through QATzip stream API released in v0.2.6. Software fallback feature of QATzip is released in v1.0.0.
Note:
- This mode is only available when using QATzip v1.0.0 or later.
- This mode relys on gzip module for SW fallback feature.
- The qatzip_sw is set to failover by default, do not load QATzip module if you do not want to enable qatzip. Or else it would be enabled and set to failover.
Directives in the qatzip_module
Syntax: qatzip_sw only/failover/no;
Default: qatzip_sw failover;
Context: http, server, location, if in location
Description:
only: qatzip is disable, using gzip;
failover: qatzip is enable, qatzip sfotware fallback feature enable.
no: qatzip is enable, qatzip sfotware fallback feature disable.
Syntax: qatzip_chunk_size size;
Default: qatzip_chunk_size 64k;
Context: http, server, location
Description:
Sets the chunk buffer size in which data will be compressed into
one deflate block. By default, the buffer size is equal to 64K.
Syntax: qatzip_stream_size size;
Default: qatzip_stream_size 256k;
Context: http, server, location
Description:
Sets the size of stream buffers in which data will be compressed into
multiple deflate blocks and only the last block has FINAL bit being set.
By default, the stream buffer size is 256K.Example
file: conf/nginx.conf
load_module modules/ngx_http_qatzip_filter_module.so;
...
gzip_http_version 1.0;
gzip_proxied any;
qatzip_sw failover;
qatzip_min_length 128;
qatzip_comp_level 1;
qatzip_buffers 16 8k;
qatzip_types text/css text/javascript text/xml text/plain text/x-component application/javascript application/json application/xml application/rss+xml font/truetype font/opentype application/vnd.ms-fontobject image/svg+xml application/octet-stream image/jpeg;
qatzip_chunk_size 64k;
qatzip_stream_size 256k;
qatzip_sw_threshold 256;For more details directives of nginx_qatzip_module, please refer to
modules/nginx_qatzip_modules/README.
file: conf/nginx.QAT-sample.conf
This is a sample configure file shows how to configure QAT in nginx.conf. This file includes:
- Enable SSL QAT engine in heretic mode.
- Support HTTPS async mode.
- Enable QATzip support.
- Select TLS-1.2 as the default ssl_protocols.
Note:
- The QAT configuration needs crypto and compression instance for the user space application.
'Orphan ring' errors in dmesg output when Nginx exit
Working with current QAT driver, Nginx workers exit with 'Orphan ring'
errors. This issue has been fixed in future QAT driver release.
Cache manager/loader process will allocate QAT instance via QAT engine
According to current QAT engine design, child process forked by master
process will initialize QAT engine automatically in QAT engine atfork
hook function. If cache manager/loader processes are employed, QAT instances
will be allocated in the same way as worker process. Cache manager/loader
processes do not perform modules' exit process method in Nginx native design
which will introduce "Orphan ring" error message in dmesg output.
Segment fault happens while sending HUP signal when QAT instances not enough
If the available qat instance number is less than 2x Nginx worker process number, segment fault
happens while sending HUP signal to Asynch Mode for NGINX*. Using qat_sw_fallback on; in qat_engine
directive as a workaround for this issue. And it needs special attention if the QAT instances
are enough when setting worker_processes auto;.
Insufficient HW resources would cause segment fault while sending HUP signal
Before running nginx, please make sure you have arranged enough HW resources for nginx, including
memory and hard disk space. Disk space exhausted or out of memory would cause core dump when
nginx receives HUP signal during handshake phase.
Performance drop under OpenSSL 3.0
- Both ECDH and PRF cause performance drop under OpenSSL 3.0.
- Due to changes in the OpenSSL 3.0 framework, TLS handshake performance is significantly degraded. See this issue for details.
The 0-RTT (early data) issue
The 0-RTT (early data) feature does not support async mode in current asynch_mode_nginx,
so it's not recommended to use async offload to QAT hardware during early data process.
CHACHA-POLY and AES-GCM throughput performance issue
With the bottleneck of memory allocation, the throughput of CHACHA-Poly and
AES-GCM can not reach the peak value when running with 4 or more QAT devices.
The target audience may be software developers, test and validation engineers, system integrators, end users and consumers for Asynch Mode for NGINX* integrated Intel® Quick Assist Technology.
Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.
This document contains information on products, services and/or processes in development. All information provided here is subject to change without notice. Contact your Intel representative to obtain the latest forecast , schedule, specifications and roadmaps.
The products and services described may contain defects or errors known as errata which may cause deviations from published specifications. Current characterized errata are available on request.
Copies of documents which have an order number and are referenced in this document may be obtained by calling 1-800-548-4725 or by visiting www.intel.com/design/literature.htm.
Intel, the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries.
*Other names and brands may be claimed as the property of others
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent