Giter Club home page Giter Club logo

taymindis / nginx-link-function Goto Github PK

View Code? Open in Web Editor NEW
231.0 24.0 41.0 439 KB

It is a NGINX module that provides dynamic linking to your application in server context and call the function of your application in location directive

Home Page: https://nginx-link-function.github.io

License: BSD 2-Clause "Simplified" License

C 94.79% Perl 5.21%
nginx nginx-cfunction ngx-http embedded golang zero-downtime dynamic-link-library dynamic-linking nginx-server test-nginx

nginx-link-function's Introduction

nginx link function Logo

nginx-link-function

The Application dynamic linking with Nginx

Table of Contents

Introduction

nginx-link-function is a nginx module which provides dynamic linking to your application in server context and call the function of your application in location directive.You could direct link your C/C++ application (or any other which accept dynamic linking) to nginx server.

How it works

Image of nginx-link-function

Usage

# nginx.conf

server {
  listen 8888;
  ...
  ngx_link_func_lib "/path/to/your/libcfuntest.so";
  ...
  ...
  location = /testCFunGreeting {
      ngx_link_func_call "my_app_simple_get_greeting"; 
  }
}

server {
  listen 8989;
  aio threads;
  ...
  ngx_link_func_lib "/path/to/your/libcfuntest.so"; # sharing data memory with server 1 if the path are same with server 1
  ...
  ...
  location = /testCFunGreeting {
      ngx_link_func_call "my_app_simple_get_greeting" 
  }
}

server {
  listen 8999;
  aio threads;
  ...
  ngx_link_func_lib "/path/to/your/libcfuntest2.so"; # another application
  ...
  ...
  location = /testPost {
      add_header Allow "GET, POST, HEAD" always;
      if ( $request_method !~ ^(POST)$ ) {
        return 405;
      }
      ngx_link_func_call "my_2nd_app_simple_get_token";
  }
}

server {
  listen 9888;
  aio threads;
  ...
  ## Download application from cloud repo e.g. ngx_link_func_download_link_lib <download_link> <dest_link_file>
  ngx_link_func_download_link_lib "http://abc.com/repos/libcfuntest.so" "/etc/nginx/libcfuntest3.so"
  ...
  ...
  location = /testPost {
      add_header Allow "GET, POST, HEAD" always;
      if ( $request_method !~ ^(POST)$ ) {
        return 405;
      }
      ngx_link_func_call "my_3rd_app_simple_get_token";
  }
}

server {
  listen 9898;
  aio threads;
  ...
  ## Download application from cloud repo with extra header e.g. ngx_link_func_download_link_lib <download_link> <headers> <dest_link_file>
  ngx_link_func_download_link_lib "https://abc.com/repos/libcfuntest.so" "Accept-Language:en_US\r\nAuthorization:Bearer KA.eyJ2ZXJzaadlasdlaldhjHJ2h3ldjklsjaklcjkljasdklcmasaskdaJxdkL3ftjM\r\n" "/etc/nginx/libcfuntest4.so"
  ...
  ...
  location = /testPost {
      add_header Allow "GET, POST, HEAD" always;
      if ( $request_method !~ ^(POST)$ ) {
        return 405;
      }
      ngx_link_func_call "my_other_app_simple_get_token";
  }
}

Installation

wget 'http://nginx.org/download/nginx-1.10.3.tar.gz'
tar -xzvf nginx-1.10.3.tar.gz
cd nginx-1.10.3/

./configure --add-module=/path/to/nginx-link-function

make -j2
sudo make install

ngx_link_func_module.h header not found when configure

When first time configure this project, I purposely do not include this header, you may need to install it to your c header file as this header file need to share with your .so application as well.

Example of installing header

install -m 644 /path/to/nginx-link-function/src/ngx_link_func_module.h /usr/local/include/

Back to TOC

Interface that expose to client application

This is the interface that you can use to get more details from nginx server, it all inside the ngx_link_func_module.h.

#define ngx_link_func_content_type_plaintext "text/plain"
#define ngx_link_func_content_type_html "text/html; charset=utf-8"
#define ngx_link_func_content_type_json "application/json"
#define ngx_link_func_content_type_jsonp "application/javascript"
#define ngx_link_func_content_type_xformencoded "application/x-www-form-urlencoded"

typedef struct {
  char *req_args; // Uri Args
  u_char *req_body; // Request Body
  size_t req_body_len; // length of body
  void *shared_mem;

  /* internal */
  void* __r__;
  void* __pl__;
  void* __log__;
} ngx_link_func_ctx_t;

extern void ngx_link_func_log_debug(ngx_link_func_ctx_t *ctx, const char* msg);
extern void ngx_link_func_log_info(ngx_link_func_ctx_t *ctx, const char* msg);
extern void ngx_link_func_log_warn(ngx_link_func_ctx_t *ctx, const char* msg);
extern void ngx_link_func_log_err(ngx_link_func_ctx_t *ctx, const char* msg);
extern u_char* ngx_link_func_get_header(ngx_link_func_ctx_t *ctx, const char*key);
extern void* ngx_link_func_get_query_param(ngx_link_func_ctx_t *ctx, const char *key);
extern void* ngx_link_func_palloc(ngx_link_func_ctx_t *ctx, size_t size);
extern void* ngx_link_func_pcalloc(ngx_link_func_ctx_t *ctx, size_t size);

extern void ngx_link_func_write_resp(
    ngx_link_func_ctx_t *ctx,
    uintptr_t status_code,
    const char* status_line,
    const char* content_type,
    const char* resp_content,
    size_t resp_len
);


// Shared Memory and Cache Scope
extern void ngx_link_func_shmtx_lock(void *shared_mem);
extern void ngx_link_func_shmtx_unlock(void *shared_mem);
extern void* ngx_link_func_shm_alloc(void *shared_mem, size_t size);
extern void ngx_link_func_shm_free(void *shared_mem, void *ptr);
extern void* ngx_link_func_cache_get(void *shared_mem, const char* key);
extern void* ngx_link_func_cache_put(void *shared_mem, const char* key, void* value);
extern void* ngx_link_func_cache_new(void *shared_mem, const char* key, size_t size);
extern void ngx_link_func_cache_remove(void *shared_mem, const char* key);

Interface break down details

malloc/calloc from nginx pool

void* ngx_link_func_palloc(ngx_link_func_ctx_t *ctx, size_t size);
void* ngx_link_func_pcalloc(ngx_link_func_ctx_t *ctx, size_t size);

get the request header parameter from

extern u_char* ngx_link_func_get_header(ngx_link_func_ctx_t *ctx, const char*key);

get the uri args

ctx->req_args;

get the query parameter

extern void* ngx_link_func_get_query_param(ngx_link_func_ctx_t *ctx, const char *key);

get the request body

ctx->req_body;

get the request body length

ctx->req_body_len;

loggin to nginx server

extern void ngx_link_func_log_debug(ngx_link_func_ctx_t *ctx, const char* msg);
extern void ngx_link_func_log_info(ngx_link_func_ctx_t *ctx, const char* msg);
extern void ngx_link_func_log_warn(ngx_link_func_ctx_t *ctx, const char* msg);
extern void ngx_link_func_log_err(ngx_link_func_ctx_t *ctx, const char* msg);

Response out

extern void ngx_link_func_write_resp(
    ngx_link_func_ctx_t *ctx,
    uintptr_t status_code, // Status code
    const char* status_line, // Status line
    const char* content_type, // Response content type
    const char* resp_content, // Response content
    size_t resp_len // Response content length
);

Sample Application Development

Feel free to clone a sample project at

Simple Project with cmake

link function google direction linkfunc-jwt-auth

#include <stdio.h>
#include <ngx_link_func_module.h>

/*** build the program as .so library and copy to the preferred place for nginx to link this library ***/
/*** gcc -shared -o libcfuntest.so -fPIC cfuntest.c ***/
/*** cp libcfuntest.so /etc/nginx/ ***/

int is_service_on = 0;

void ngx_link_func_init_cycle(ngx_link_func_cycle_t* cycle) {
    ngx_link_func_cyc_log(info, cycle, "%s", "starting application");

    is_service_on=1;
}

void my_app_simple_get_greeting(ngx_link_func_ctx_t *ctx) {
    ngx_link_func_log_info(ctx, "Calling back and log from my_app_simple_get");

    ngx_link_func_write_resp(
        ctx,
        200,
        "200 OK",
        "text/plain",
        "greeting from ngx_link_func testing",
        sizeof("greeting from ngx_link_func testing")-1
    );
}

void my_app_simple_get_args(ngx_link_func_ctx_t *ctx) {
    ngx_link_func_log_info(ctx, "Calling back and log from my_app_simple_get_args");

    if(ctx->req_args) {
        ngx_link_func_write_resp(
            ctx,
            200,
            "200 OK",
            "text/plain",
            ctx->req_args,
            strlen(ctx->req_args)
        );
    } else {
        ngx_link_func_write_resp(
            ctx,
            204,
            "",
            "text/plain",
            NULL,
            0
        );
    }
}

void my_app_simple_get_token_args(ngx_link_func_ctx_t *ctx) {
    ngx_link_func_log_info(ctx, "Calling back and log from my_app_simple_get_token_args");

    char * tokenArgs = (char*)ngx_link_func_get_query_param(ctx, "token");
    if (! tokenArgs) {
        ngx_link_func_write_resp(
            ctx,
            401,
            "401 unauthorized",
            "text/plain",
            "Token Not Found",
            sizeof("Token Not Found")-1
        );
    } else {
        ngx_link_func_write_resp(
            ctx,
            401,
            "401 unauthorized",
            "text/plain",
            tokenArgs,
            strlen(tokenArgs)
        );
    }
}

void my_app_simple_post(ngx_link_func_ctx_t *ctx) {
    ngx_link_func_log_info(ctx, "Calling back and log from my_app_simple_post");

    ngx_link_func_write_resp(
        ctx,
        202,
        "202 Accepted and Processing",
        "text/plain",
        NULL,
        0
    );
}



void my_app_simple_get_no_resp(ngx_link_func_ctx_t *ctx) {
    ngx_link_func_log_info(ctx, "Calling back and log from my_app_simple_get_no_resp");

    //  return 404 
}

void ngx_link_func_exit_cycle(ngx_link_func_cycle_t* cyc) {
    ngx_link_func_cyc_log(info, cyc, "%s\n", "Shutting down/reloading the Application");

    is_service_on = 0;
}

Noted:

The function init and exit cycle are reserved function when started the nginx, it will call init cycle function, when stop/reload nginx, it will call exit cycle function.

void ngx_link_func_init_cycle(ngx_link_func_cycle_t* cyc){}
void ngx_link_func_exit_cycle(ngx_link_func_cycle_t* cyc){}

references: The init and exit cycle hooks in application layer

Log Level

The log can be called, the logged message will be store where you config error log in nginx.conf

ngx_link_func_log_info(ctx, "This is info direct message");
ngx_link_func_log(info, ctx, "%s", "This is info with formatted message");
ngx_link_func_log_debug(ctx, "This is debug direct message");
ngx_link_func_log(debug, ctx, "%s", "This is debug with formatted message");

ngx_link_func_log_info(ctx, "%s", "This is info with formatted message"); // Wrong format
ngx_link_func_log_debug(ctx, "%s", "This is info with formatted message"); // Wrong format

provide ca-cert to download your app(.so)?? please embedded this in your nginx.conf's server context.

ngx_link_func_ca_cert "/etc/ssl/certs/ca-cert.crt"
ngx_link_func_download_link_lib "https://abc.com/repos/libcfuntest.so" "/etc/nginx/libcfuntest4.so"

provide ca-cert and extra header to download your app(.so)?? please embedded this in your nginx.conf's server context.

ngx_link_func_ca_cert "/etc/ssl/certs/ca-cert.crt"
ngx_link_func_download_link_lib "https://abc.com/repos/libcfuntest.so" "Accept-Language:en_US\r\nAuthorization:Bearer KA.eyJ2ZXJzaadlasdlaldhjHJ2h3ldjklsjaklcjkljasdklcmasaskdaJxdkL3ftjM\r\n" "/etc/nginx/libcfuntest4.so"

Test

It depends on nginx test suite libs, please refer test-nginx for installation.

cd /path/to/nginx-link-function
export PATH=/path/to/nginx-dirname:$PATH 
sudo prove -r t/

Back to TOC

Support

Please raise an issue and I will fixed it as soon as I can

Alternatively, you may email to [email protected]

Wiki

Check out wiki for more details

Back to TOC

Copyright & License

Copyright (c) 2018-2020, Taymindis [email protected]

This module is licensed under the terms of the BSD license.

All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

  1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
  2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Back to TOC

nginx-link-function's People

Contributors

cdgwoon avatar taymindis avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

nginx-link-function's Issues

How to get the value of a config file param during cycle init?

Hello,
I looked through the wiki, and had a quick look in the nginx source, but can't quite understand how to get the value of a config file parameter?

location /foo {
  root /custom/path/to/assets;
  ngx_link_func_call "foo";
}

How would I get the value of root during ngx_link_func_init_cycle?

unknown directive "ngx_link_func_lib"

The following error occurs starting the FreeBSD port of nginx 1.18.0_15,2:

# service nginx onestart
Performing sanity check on nginx configuration:
nginx: [emerg] unknown directive "ngx_link_func_lib" in /usr/local/etc/nginx/nginx.conf:42
nginx: configuration file /usr/local/etc/nginx/nginx.conf test failed
Starting nginx.
nginx: [emerg] unknown directive "ngx_link_func_lib" in /usr/local/etc/nginx/nginx.conf:42
/usr/local/etc/rc.d/nginx: WARNING: failed to start nginx

nginx has been compiled with the LINK option enabled to include the nginx-link-function module:

# pkg info www/nginx
nginx-1.18.0_15,2            
Name           : nginx      
Version        : 1.18.0_15,2
Installed on   : Wed Jun 17 14:40:05 2020 CEST
Origin         : www/nginx  
Architecture   : FreeBSD:12:amd64
Prefix         : /usr/local     
Categories     : www                                                                                                                           
Licenses       : BSD2CLAUSE      
Maintainer     : [email protected] 
WWW            : https://nginx.org/ 
Comment        : Robust and small WWW server
Options        :
[...]
        LINK           : on
[...]

The build log shows that the nginx-link-function module is indeed enabled.

To reproduce the error on FreeBSD, install nginx from ports and enable LINK:

# cd /usr/local/ports/www/nginx/
# make config
# make
# make install

Facing client request blocking issue?

Some users are having issue when proceed some heavy request in their apps layer. Due to old nginx-c-function has no threads feature, it only accept 1 request per time, it may causing slowness.

Nginx gzip filter compatibility issue

Thank you very much for making this project. I am definitely going to use it for my C++ project to integrate it with Nginx since I found it to be an excellent solution compared to all other things I considered. I found your API functions and documentation excellent and everything worked very quickly and easily.

I did however find a bug in the current version of Nginx 1.21 and this project's master branch which you may want to review.

Basically gzip doesn't work for the urls using this module and I isolated how to fix this.

In ngx_link_func_module.c, there are these 2 lines.
r->headers_out.content_type.len = resp_content_type->len;
r->headers_out.content_type.data = resp_content_type->data;

For whatever reason, ngx_http_code_module.c is checking an addition length variable, which also exists in the nginx documentation, so there is some kind of reason they have this length variable needing to be set twice. This causes this logic to evaluate as true in the gzip module which disables it: ngx_http_test_content_type(r, &conf->types) == NULL

When I add this line next to those, gzip is able to work!

r->headers_out.content_type_len = resp_content_type->len;

You can see this variable is documented here and they say it usually needs to be the same as content_type.len:
https://www.nginx.com/resources/wiki/extending/api/http/

In the process of debugging this, I learned a little bit more about how Nginx works and am less scared to try to work on it now especially since i was able to output debug messages and isolate things reasonably easy, though it did take some hunting to understand this one. It feels more like a bug in nginx to have 2 similar variable names.

Thanks again for creating a simple project that allowed me to dive into Nginx source and be productive on the first day.

nginx: [emerg] unknown directive "ngx_http_c_func_link_lib" in /home/aimso/nginx-c-function/t/servroot/conf/nginx.conf

t/sanity.t .. nginx: [emerg] unknown directive "ngx_http_c_func_link_lib" in /home/aimso/nginx-c-function/t/servroot/conf/nginx.conf:39
Bailout called. Further testing stopped: TEST 6: Set C_FUNC_TEST_GET_CALLOC_FROM_POOL - Cannot start nginx using command "nginx -p /home/aimso/nginx-c-function/t/servroot/ -c /home/aimso/nginx-c-function/t/servroot/conf/nginx.conf > /dev/null" (status code 256).
FAILED--Further testing stopped: TEST 6: Set C_FUNC_TEST_GET_CALLOC_FROM_POOL - Cannot start nginx using command "nginx -p /home/aimso/nginx-c-function/t/servroot/ -c /home/aimso/nginx-c-function/t/servroot/conf/nginx.conf > /dev/null" (status code 256).

Logging?

Your example works great and I think that I'll be able to build off of that. One question is that you give some logging examples:

ngx_http_c_func_log(info, ctx, "%s", "Starting The Application");
but it does not seem to log anywhere. Is there something special that must be done?

`ngx_link_func_init_cycle()` not found on FreeBSD

The following basic example is not accepted by the nginx-link-function module on FreeBSD 12 with nginx 1.18 :

#include <stdio.h>
#include <ngx_link_func_module.h>

int is_service_on = 0;

void ngx_link_func_init_cycle(ngx_link_func_cycle_t* cycle) {
    ngx_link_func_cyc_log(info, cycle, "%s", "starting application");

    is_service_on=1;
}

void greeting(ngx_link_func_ctx_t *ctx) {
    ngx_link_func_log_info(ctx, "Calling back and log from my_app_simple_get");

    ngx_link_func_write_resp(
        ctx,
        200,
        "200 OK",
        "text/plain",
        "greeting from ngx_link_func testing",
        sizeof("greeting from ngx_link_func testing")-1
    );
}

void ngx_link_func_exit_cycle(ngx_link_func_cycle_t* cyc) {
    ngx_link_func_cyc_log(info, cyc, "%s\n", "Shutting down/reloading the Application");

    is_service_on = 0;
}

Compilation:

$ gcc -shared -fPIC -o hello.so hello.c

The shared library is then loaded with:

    server {
        listen       80;
        server_name  localhost;
        ngx_link_func_lib "/usr/local/etc/nginx/hello.so";

        location = /greeting {
            ngx_link_func_call "greeting";
        } 
    }

Starting nginx fails with:

# service nginx start
nginx: [error] function ngx_link_func_init_cycle(ngx_link_func_cycle_t *cycle) not found in "/usr/local/etc/nginx/hello.so", at least create an empty init function block
 Undefined symbol "_nss_cache_cycle_prevention_function" in /usr/local/etc/nginx/nginx.conf:126

But the shared library contains the wanted function:

# nm -D hello.so
                 w _ITM_deregisterTMCloneTable
                 w _ITM_registerTMCloneTable
                 w __cxa_finalize
00000000000007f4 T _fini
0000000000000510 T _init
00000000000006f7 T greeting
0000000000200bb8 B is_service_on
                 U ngx_link_func_cyc_log_info
0000000000000745 T ngx_link_func_exit_cycle
0000000000000695 T ngx_link_func_init_cycle
                 U ngx_link_func_log_info
                 U ngx_link_func_write_resp
                 U snprintf

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.