cnv1617's People
cnv1617's Issues
add Renderer code to repository
part of #7
MSS database: the making of
Part of #24
abort raytracing when client closes the connection
Renderer (raytracer) instrumentation
Wait for #7 ?
The code of the application that performs the raytracing (called by the web server) is written in the
Java programming language and compiled into bytecode. The application is to be further instrumented
with a Java instrumentation tool (such as BIT that will be presented in the labs) in order to extract and
persistently store the dynamic performance metrics regarding the code executed. The explicit duration
(wall time) of each request handled should not be stored in the MSS.
These nodes will process the RenderFarm requests using code that was previously instrumented by the students, in order to collect relevant dynamic performance metrics regarding the application code executed (e.g. number of function calls executed, invocation stack depth, bytecodes executed, and others deemed relevant). They will allow estimating task complexity realistically, irrespective of variable delays, that could be caused by resource overcommit by the cloud provider.
The final choice of the metrics extracted, instrumentation code, and system used to store the metrics
data is free and subject to analysis and decision by the students
I think the metrics can be stored locally in the Checkpoint milestone.
E.g.
- memory
- number of instructions
Update node images
- install AWS SDK in node images (renderer/load-balancer)
cd ~
wget "http://sdk-for-java.amazonwebservices.com/latest/aws-java-sdk.zip"
unzip aws-java-sdk.zip
make sure it's version aws-java-sdk-1.11.131
-
install MSS DB access keys
~/.aws/credentials
-
make the init script download the ... jar from somewhere or include it in the image
Send the metrics results to MSS
Part of #24
Simple Load Balancer
This load balancer should be our proprietary load balancer with super awesome skillz. Our proprietary balancer must be l33t and n0t l1k3 7h3 4w5 1.
Mimic the AWS load balancer.
metrics: number of method calls
part of #10
Load-balancer image
The the same shenanigan all over again but in the init send the load-balancer parameter.
Why me tho? ๐ฅ
part of #25
project does not run on sigma anymore
It working during the checkpoint. Now ant gives the message:
run:
[java] Error occurred during initialization of VM
[java] Could not reserve enough space for object heap
[java] Picked up _JAVA_OPTIONS: -XX:-UseSplitVerifier
[java] Error: Could not create the Java Virtual Machine.
[java] Error: A fatal exception has occurred. Program will exit.
[java] Java Result: 1
Smart Load Balancer
Load balancer that uses the metrics to decide where to forward stuff.
wait for #25
metrics: number of allocations
part of #10
remove server timeout
Remove the HttpServer default timeout.
MSS
Using the amazon database with mr wrap ist we should store all of the metrics in this system where the the load-balancer should get the metrics from.
"mock" renderer request
- Receive request and parse the arguments.
- Reply parsed arguments
The URL for the renderer should have the following format:
http://<load-balancer-DNS-name>/r.html?f=<model-filename>&sc=<scene-columns>&sr=<scene-rows>&wc=<window-columns>&wr=<window-rows>&coff=<column-offset>&roff=<row-offset>
where r.html
points to the code that receives the request and the parameters express the total size of the scene in rows and columns to be considered (sc,sr), the size of the (partial) viewport window to be rendered (wc,wr), and the offsets, regarding the top left corner of the image, where the viewport window should start.
part of #4
After this is done, proceed with invoking renderer classes #5
Auto scaler
use an Amazon AWS Autoscaling group that adaptively decides how many web server nodes should be active at any given moment. It is up to the students the challenge to design the autoscaling rules that will provide the best balance between performance and cost. It should detect that the web app is overloaded and start new instances and, conversely, reduce the number of nodes when the load decreases.
The algorithms for load balancing, auto-scaling and the MSS need not be fully implemented at this stage (metrics can be stored temporarily in the computing nodes) but should be already thought out.
Renderer (Raytracer)
Make the raytracer run on the EC2 nodes.
Multi threaded WebServer
Extend the webserver class to operate multithreaded.
part of #4
Build and define classpath for renderer
Make the Render classes build. It already includes a makefile, just it's just a matter of:
- invoking this makefile (both travis CI and EC2 instances)
- consider making the EC2 instances call some script in the root of the repo. Then the script compiles and runes everything
- defining the classpath that's used to launch the WebServer.
part of #7
Write instrumentation results to file
Part of #10
WebServer implementation
Begin sending image as soon as bytes are available
Get metrics from MSS
part of #31
Render node image
The images running on the render nodes should, on startup (rc.local
) switch to the webserver
user and run the $HOME/autostart
script which:
- clones the repo:
git clone -b release --depth 1 [email protected]:Gisson/CNV1617.git CNV1617
- runs the init script in the repo like so:
cd CNV1617 && while bash init.sh render-node; do sleep 1; done
, so theautostart
script exits wheninit.sh
fails. -
rc.local
should shut-down the machine if the$HOME/autostart
script exits. This should come in handy since theinit.sh
script can not shutdown the machine (needs root). For instance:
(
sudo --login -u webserver -- /home/webserver/autostart
sleep 5
poweroff
) &
Also, make sure that:
- the github deploy key is installed
-
ant
is installed -
BIT
is already installed in $HOME/BIT for improved startup speed
Extend the init script
Make extension for the load balancer
Part of #25
Project privacy
Can I switch this project to public and add a license?
Checkpoint report
The checkpoint submission bundle must include intermediate report (2-page, double column)
describing clearly:
- a) what is already developed in the current implementation (architecture, data
structures and algorithms); - b) the specification of what remains to be implemented.
associate request to thread
The instrumentation code needs to know to what request a thread belongs to.
This will allow us to know that, for instance, request /r.html?f=test05.txt&sc=640&sr=360&wc=640&wr=360&coff=0&roff=0
needed 92069358 method calls, 29574356 allocations, etc.
A simple way to do this is for the server to store a <request, threadID> list which the instrumented code can then access before writing the metrics.
part of #10
Load balancer
The load balancer is the only entry point into the system: it receives a sequence of web requests and
selects one of the active web server cluster nodes to handle each of the requests.
In a first phase, this job can be performed by an off-the-shelf load balancer such as the ones available at Amazon AWS. Later in the project, you should design a more advanced load balancer that, using metrics data stored in the Metrics Storage System, will pick the best web server node to handle a request.
We need a load balancer for the Checkpoint, but I'm not sure whether In a first phase
means this checkpoint or not.
The algorithms for load balancing, auto-scaling and the MSS need not be fully implemented at this stage (metrics can be stored temporarily in the computing nodes) but should be already thought out.
invoke renderer classes
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.