marsalans / cs118-cacheing-http-proxy-server Goto Github PK
View Code? Open in Web Editor NEWThis project forked from spchuang/cs118-cacheing-http-proxy-server
This project forked from spchuang/cs118-cacheing-http-proxy-server
This is a basic implementation of a caching HTTP proxy server, based on the skeleton code from (github.com/iliamo/cs118-project1) for CS118 at UCLA. There are two main classes for this project. 1. ProxyServer - Setup The ProxyServer class represent the HTTP caching proxy server. It could be initialized by specifying the server port, maximum concurrent connections (thread number), and the timeout for the persistent connections. The server class contains one public method, "start", which setups up the proxy server. It first sets up the proxy server by creating a BSD socket to listen to a specific port where the socket is binded. Once the setup is completed without any errors, the program enters an infinite loop which accepts new HTTP connection. The server spawns a separate thread for each request and detachs the thread because we want each thread to handle an accepted socket connection until it's closed. We also setup a timeout for the socket so, once the timeout time has passed (5 seconds in this case) without any byte sent to the receiving buffer stream, the thread exits and closes the socket connection. - Connection handling (thread independent) The thread function keeps a boolean variable which keeps track of the state of the persistent connection. By default, the connection is persistent. Then the thread enters an infinite loop which reads the HTTP request and returns the appropriate HTTP response and repeat this process until there is a timeout on the thread connection or the client sent a http request with the header "conection: close", which tells the proxy server to close the connection after responding to this request. The thread first reads the string request from the client then parse it with the given HttpRequest class. If there's a parse exception thrown, the thread returns either "400 Bad Request" for invalid http request or "501 Not Implemented" for requests other than GET. Once the Http request is parsed and contains no errors, we first check if there's a header "connection:close" to modify our persistent state for the connection. After that, the thread checks whether or not the request has already been cached in the db server (see below for more details). If the request is not cached, the thread creates an instance of ProxyServerClient to handle request to the remote server. 2. ProxyServerClient The ProxyServerClient will connect to the remote server to make request and read the response. The response will then be saved in the sqlite db and returned back to the client. This wrapper abstract out the client aspect of the proxy server code and make it easier to manage the number of concurrent connections to the remote server. 3. Caching with sqlite (TODO for Benjamin) -use sqlite database to store the data. The key is host string + path string. every key corresponds to a response value and response respiration date. The proxy first accept the client request, and then look up the cache and see whether it is already in the cache, will return the response if the request hit the cache, and if not, the proxy will connect to the remote server and ask for the response. Then the proxy will store the response itself, the expiration date and its corresponding host_path key to the database for the caching purpose. [EXTRA CREDIT] Using permanent storage of a cache on the disk General ======= To learn more about ./waf, refer to http://code.google.com/p/waf/ and http://waf.googlecode.com/svn/docs/wafbook/single.html Ideally, you will need only to add your source files in wscript, which is basically a Python program. How to use skeleton =================== Build Directions (specific to UCLA seas server lnxsrv) Due to restrictions on lnxsrv, several steps necessary to compile and run (if you're using this skeleton): 1. The following lines should be added to your ~/.bashrc export PATH=/usr/local/cs/bin:$PATH export LD_LIBRARY_PATH=/u/cs/grad/yingdi/boost/lib:/usr/local/lib64/:$LD_LIBRARY_PATH 2. To configure environment ./waf configure If there are any errors, please email me: [email protected] Please, mark your message with [CS118] header 3. To build/rebuild the code ./waf 4. All compiled executables are located in build/, so you can run them as this: build/http-proxy
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.