![Logo](https://private-user-images.githubusercontent.com/109282492/338055588-cd5d981f-1928-49a1-a4d8-0a5b21b92171.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjM3Njc2MDcsIm5iZiI6MTcyMzc2NzMwNywicGF0aCI6Ii8xMDkyODI0OTIvMzM4MDU1NTg4LWNkNWQ5ODFmLTE5MjgtNDlhMS1hNGQ4LTBhNWIyMWI5MjE3MS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwODE2JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDgxNlQwMDE1MDdaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT02NWE3NmNiMTkyNTJiOGFlNWI1OTNkYmY0NTJlMGU5MmEwZjJiOGJkYjkwNzAyYWRhZmVlMjgwMWE4ZDA1NDkwJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.s6EdLAp0hnROamt8hW3iiKQi3T18Als0sZlH3Vg6ZIo)
Algocode is a DSA practice platform just like Leetcode!
Read the blog ยป
Algocode Auth Service .
Code Manager Service .
RCE Engine Service
Algocode is an online data structure and algorithm practice backend built in microservices architecture.
Algocode currently has three services: Algocode Auth Service . Code Manager Service . and RCE Engine Service
-
Algocode Auth Service Auth Service handles user management.
-
Code Manager Service Code Manager Service handles
code submission
,code exec event creation to Message Queue
, andsaving and caching code executing result
to Database. -
RCE Engine Service RCE Engine Service contains the
C++
code execution Judge. The Judge is completely isolated and it is only accessible usingevents
.
RCE Engine service forJava Judge
andPython Judge
are under development.
The C++ Judge
of RCE Engine can execute C++
codes and run test cases against the code output. It can currently handle the below events:
a. AC (Accepted)
b. WA (Wrong Answer)
c. Compilation Error
d. Time Limit Exceed
e. Memory Limit Exceed
f. Segmentation Fault
A. The C++ Judge
in RCE Engine is a pure docker implementation and no other 3rd party API or service has been used.
Please refer to the respective services to learn more about it. You will find rich and detailed documentation in the respective service's github repository, I promise!
B. Rate Limit Alert
The API endpoint to submit solution to the Algocode platfrom (API in Code Manager Service ) is rate limited to
1 request per 20 seconds
.POST https://codemanager.algocode.site/api/v1/code/submit/
You can learn more on this in the
API Guideline - Code Submission in Algocode
section.
C. Documentation
All the services have dedicated documentation page. Please visit the respective service repository to learn more.
The Algocode platform is comprised of 3 services.
Algocode Auth Service For user management.
Code Manager Service For interacting with client, data processing and event creation for RCE Engine.
RCE Engine Service For executing the code submitted by the user and produce result for Code Manager Service to consume.
The mentioned RCE Engine is responsible to execute
C++
code. The RCE Engine forJava
andPython
is under development.
-
The access to the Algocode begins from the Auth Service.
-
Algocode Auth Service is responsible for user management.
-
The user creates an account or logs in into the Algocode platform to complete the authentication/authorization process.
-
PostgreSQL
database is used for user management. The database isself-hosted
and capable ofbackup
andrestore
data periodically.
-
The Code Manager service is the true manager which is responsible to interact with users to submit the code submission request and return them the answer of their solution.
-
The user sends a
POST
request to the
POST https://codemanager.algocode.site/api/v1/code/submit/
API to submit their code solution.
-
Code Manager verifies the
authenticity of the user
,processes the submitted data
,publishes the data in a RabbbitMQ instance
and thequeue
is based on theprogramming language
the solution is written, and returns asubmission_id
to the client. -
Code Manager also listens to event produced by RCE Engine. The
RCE Engine executes the solution and publishes the result to an unified result queue
. -
Once message is consumed by Code Manager from
result queue
, Code Managercaches the result in Redis
and stores the result inMongoDB database.
-
The client can send a
GET
request to theresult check
GET https://codemanager.algocode.site/api/v1/result/check/<submission_id>/
API to get the code execution result. The API is a short polling request for 5 seconds
.
-
The
GET
request checks thecache
first for the result, thenchecks in database
and if the result is still not found, then the API process waits for5 seconds
for availability of the data. -
If the data is available within the
5 seconds
wait time, it returns the result, otherwise, the below response will be returned.
Response Key Type Value/Description status
string
pending
data
string
null
-
The RCE Engine comprises either
C++ Judge
, orJava Judge
, orPython Judge
. -
The RCE Engine consumes messages from respective
queue
. Each RCE Engine consumesdata
based on theprogramming language
queue type i.e. RCE Engine forC++ Judge
consumes data fromC++ queue
. -
The RCE Engine
processes the consumed data
,prepares the data for execution
and passes the data to theJudge
. -
The
Judge
creates asecure
andunprivileged
docker container to run the user submitted code. -
The
Judge
stores the output of the code in a file after thecode execution
is completed. -
Once the
code execution
by theJudge
is completed, the RCE Enginecompares the output with testcases
. -
After the
output comparison with testcases
, the RCE Engine formats theresult data
andpublish the result data
to an unifiedresult queue
in aRabbitMQ
instance which is consumed byCode Manager Service
. -
In case, you want to learn more about the
Judge Engine
, please check Online Judge - a light-weight solution to test the Judge locally.
- Watch from
16:30
for code execution begin and18:30
for code submission result.
- Watch from
09:30
for code execution begin and11:30
for code submission result.
Code Manager Service is deployed on AWS EC2
in Ubuntu 22.04 server.
Algocode Auth Service and RCE Engine Service are deployed on Azure VM
in Ubuntu 22.04 server.
The DNS for the platform www.algocode.site
is hosted on Cloudflare
.
1. Proxy and Monitoring:
a. Nginx as webserver.
b. Nginx Proxy Manager to manage Nginx.
c. Portainer to manage and monitor docker container in Auth Service.
2. Backend and Server:
b. Gunicorn as application server.
e. Celery to process tasks asynchronously.
f. Flower to monitor celery worker.
g. Resend for email service.
h. Django as backend.
i. Django Rest Framework for API.
3. Databases and Cache:
j. PostgreSQL for user management database.
k. MondoDB for code execution result backend.
l. Redis for celery backend and cache.
4. Containerization and Code Execution:
m. Docker for containerization.
n. Sibling Docker for secure foreign code execution.
o. RabbitMQ for asynchronous processing.
p. Rate Limit (Token Bucket Algorighm, Django Middleware) for rate limit code submission API.
5. Deployment and Testing:
q. AWS EC2 instance for deployment
r. Cloudflare for DNS hosting.
s. Postman for API testing.
As of today I have built the backend in microservices, there's no frontend for the project. I am fully focusing on the advanced backend engineering, hence, if you want to contribute or want to build a frontend for the project, please do not hesitate to email me here:
I have prepared a Notion Page for all the problems currently available in the Algocode project. Please visit this link Algocode Problem Lists to get all the available problems.
You will need to visit this page in order to submit your solution. Just read the problem statements, understand the problem, and copy the problem id
. That's all you need to submit a solution!
Please read API Guideline - Registration in Algocode
for account details and
API Guideline - Code Submission in Algocode
for detailed guide on how to submit a solution in the Algocode platform.
-
Although you can fetch all the problems and any specific problems using the below APIs.
-
To get all the Problems
GET https://codemanager.algocode.site/api/v1/problem/all/
-
The API responses with
paginated result
. -
To get a single Problem
GET https://codemanager.algocode.site/api/v1/problem/<question_uuid>/
To learn more on the Code Manager APIs and documentation, please visit the documentation of Code Manager Service
You will need postman
or insomnia
to submit a solution to the project. I know you do have postman or insomnia!
Please open postman or insomnia, and continue reading ...
The Algocode Auth Service is handling the user management.
To Know more about the
Algocode Auth Service
please visit the repository.
You have two ways to submit a solution to the Algocode
platform. You can create your own account, or you can use the already available verified account
.
Please copy the account details and the login endpoint.
POST https://auth.algocode.site/api/v1/auth/login/
Parameter | Type | Value |
---|---|---|
email |
string |
[email protected] |
password |
string |
12345678@1 |
Send a POST
request to the endpoint, and you'll receive access token
and refresh token
.
Copy the access token
and head on to the API Guideline - Code Submission in Algocode
section below.
If you want to use your own account to submit solutions, please create an account.
Creating account involves two steps:
-
submit the from
-
verify your email address
Copy the below submit endpoint along with the JSON
data format, and send a POST
request.
Please provide your authentic email address as you'll receive
a token
in your email from the Algocode Auth domainauth.algocode.site
.
POST https://auth.algocode.site/api/v1/auth/registration/
Parameter | Type | Description |
---|---|---|
username |
string |
Required Your username for the account. |
email |
string |
Required Your valid email address. |
password1 |
string |
Required Your password. |
password2 |
string |
Required Confirm your password. |
first_name |
string |
Required Your first name. |
last_name |
string |
Required Your last name. |
At this time please check your email, you will have received an email from auth.algocode.site
domain.
The email will have an email validation URL to validate your account activation. Do not click the URL
As Algocode does not have any frontend, you need to verify the account manually.
Example Token URL
:
To confirm this is correct, go to https://auth.algocode.site/api/v1/auth/registration/account-confirm-email/Mw:1sFEOr:xsqvnifUV5ppDFqbQBrmp2sIXGoRY63BmnFddsYTau4/
Copy everything after the account validation URL endpoint i.e. https://auth.algocode.site/api/v1/auth/registration/account-confirm-email/
For the above example, copy Mw:1sFEOr:xsqvnifUV5ppDFqbQBrmp2sIXGoRY63BmnFddsYTau4
.
Send a POST
request to the below endpoint to validate your account creation.
POST https://auth.algocode.site/api/v1/auth/registration/verify-email/
Parameter | Type | Value |
---|---|---|
key |
string |
Required. Your copied token from your email |
Response
Response Key | Type | Value | Description |
---|---|---|---|
detail |
string |
OK |
Registration is successful! |
detail |
string |
Any value other than OK |
Registration is unsuccessful. Check the token again. |
If you have received the response as ok
, congratulation on creating your account on Algocode!
You can now login to your account with the login endpoint
mentioned above to get access token
and
head on to the API Guideline - Code Submission in Algocode
section below.
But in case you were not able to create account, please
raise an issue
and use theready to use account
instead.
- I hope you have noted the readily available user account or you have created an account in Algocode.
If you have not done this, please check
API Guideline - Registration in Algocode
section for guideline.
Get the
acces token
using theuser credentials
as perAPI Guideline - Registration in Algocode
section.To submit a solution, you should visit the
notion page
Algocode Problem Lists for the available problems in the Algocode.Go through the problems, and copy the
problem_id
of the problem you want to submit a solution.Why notion? Did you ask?
Please check the
Problem Lists
section for information.
The Code Manager Service is handling all the activities mentioned in this section.
To Know more about the
Algocode Code Manager Service
please visit the repository.
Please follow the below steps to submit a solution in Algocode.
- The
/api/v1/code/submit/
API endpoint to submit code solutions isRate Limited
.The API endpoint is rate limited with
one request per 20 seconds
i.e.3 requests per minute
.
- Did you ask why?
Well, I am using the free
AWS
andAzure
servers with only1GB
of RAM! Hence I have no choice but toRate Limit
the API.
I have implemented the
Rate Limit
usingToken Bucket Algorithm
and I have applied the Rate Limiter class Project Wide for the quoted API usingMiddleware in Django
.Click Here to see the implementation of the
Rate Limiter Class
.
Write your solution for the problem you have chosen in C++
, as currently C++
is supported. Now, you have to convert it into JSON
format. As the Algocode does not have client till now, to share data to the backend services, you need to convert it into JSON
. Here's how you can convert the code in JSON
in python
.
import json
code = """
# paste your code here.
"""
code = json.dumps(code)
print(code)
Copy the printed code from terminal and and continue reading to learn about the request body.
As you have copied the access token
as per the API Guideline - Registration in Algocode
section, please paste the access token
in Bearer Token
option in Auth
section in Postman or Insomnia.
Now, you are ready to submit the solution to the Algocode platform. Please send a POST
request to the below API following the API reference.
POST https://codemanager.algocode.site/api/v1/code/submit/
Parameter | Type | Value/Description |
---|---|---|
problem_id |
string |
Required. The problem_id of the problem you are submitting the solution. |
lang |
string |
Required. cpp . Currently cpp is supported. java RCE Engine is under development. |
code |
string |
Required. Your solution for the problem in JSON format. |
Here's an example payload for one of a problem in Algocode Sqrt(X)
. The problem is same as this Leetcode Problem.
Example Payload
{
"problem_id": "f17f511a-8c53-41d3-b750-7673d25835af",
"lang": "cpp",
"code": "#include <iostream>\n\nint mySqrt(int x) {\n if (x == 0) return 0;\n int left = 1, right = x, result = 0;\n while (left <= right) {\n
int mid = left + (right - left) / 2;\n if (mid <= x / mid) {\n result = mid;\n left = mid + 1;\n } else {\n right = mid - 1;\n }\n
}\n return result;\n}\n\nint main() {\n int t;\n std::cin >> t;\n while (t--) {\n int x;\n std::cin >> x;\n std::cout << mySqrt(x) <<
std::endl;\n }\n return 0;\n}\n"
}
Once you have submitted your solution to the Algocode Code Manager Service
, you will receive a response with a submission_id
.
You would be able to see the result of your solution using the submission_id
.
Response Key | Type | Value/Description |
---|---|---|
result |
dict[dict]] |
A dictionary containing the request's response. |
detail |
string |
Your response has been submitted |
submission_id |
string |
The submission_id for your solution submission. |
{
"result": {
"detail": "Your response has been submitted.",
"submission_id": "5993d00f-62fa-437c-8724-ee588265175b"
}
}
Please check the API Guideline - Result Check in Algocode
section to learn how to check the code submission result.
Please make a GET
request to the below API to get the result of your solution.
GET https://codemanager.algocode.site/api/v1/result/check/<submission_id>/
Parameter | Type | Description |
---|---|---|
submission_id |
string |
Required. The submission_id of your solution as per the API Guideline - Code Submission in Algocode section. |
You would see the result of your solution.
If you want to contribute or you want to run locally, then you can fork
the development
branch on each service mentioned in the Algocode Platform.
Please follow the .envs-examples
to know the env-variables
you would need to run the project locally.
All the services are dockerized project
. You just need to cd src
, create virtual environment
, activate it, and
run make docker-up
and That's it!
This will run the project for you.
Please follow the service that you want to contribute or run locally to get detailed guideline on local development.
The project itself was a challenge for me!
Once one of my mentors told me
Do the hard things while you are learning, so that the implementation becomes easier for you.
I completely agree with this statement. I enjoy dealing with complex stuff, and bugs
give me the kick
I enjoy!
Well, enough praise of myself.
But I was not lying. When I thought to build the project 2 months ago (it took 1.5 months to wrap up the project, two weeks was idle).
I had zero knowledge whether I would be able to do it, but I had confidence that somehow
I would do it.
My thought process was simple - "I do not know exactly how
, but I would do it somehow
for sure!"
And I am writing this Readme
today that I have completed the project, and somehow
I have made it! That's my motivation.
I know something is not simple as it sounds, but I know, somehow I would manage it!
-
The initial challenge was the design. Designing a complex project like
Online Judge
inmicroservices
to build fromscratch
was not easy as it sounds. -
The communication between microservices were fun discovery. I was searching for optimal solution and I learnt
RabbitMQ
for this cause, and I ended up writing a nice blog onRabbitMQ 101
. Read RabbitMQ 101 Here. The Algocode platform is using an RabbitMQ instance from CloudAMQP platform. -
Building a Online Judge from scratch was the hardest part to accomplish. At times, I though to abandon the idea of building the online Judge from scratch and use 3rd party APIs, but my instinct did not allow it!
-
However, after countless hours of debugging and reading internals of
docker networking
,docker volume
,docker containers
,docker pyhton sdk
,docker in docker
andsibling docker
,security in docker
,linux internals
- I have explored countless number of internals and fixed hundred of bugs that taught me a lot. This resulted an Online Judge built from scratch usingdocker
. I have learnt so many new concepts while building the projectjust proper by googling!
. -
The next struggle was the
production build
for the project. I had to shift fromdevelopment
build toproduction build
and eventually to deploy the project inAWS
andAzure
servers. -
AWS closed my two new account to deploy the project, but after countless of emailing the
aws support
, they activated my account and finally I could deploy my project. -
To wrap up, the entire project was a challenge for me:
The design stage to research stage
,to development stage to production stage to deployment stage
- I had to repeat all these steps with all the 3 (currently) services behind the Algocode platform.
-
I have gained practical experience with
RabbitMQ
building this project. -
I have gained deep knowledge on
docker
,docker volumes
,docker networking
,Lunux internals
and many more. -
As the project is heavily dealing with files, I have gained valuable experience with
file handling
withpython
. -
As I have built the project from
research
,design
,dev
,production
todeployment
, I have gained invaluable knowledge on design, development, production and deploy the project incloud services
likeAWS
orAzure
. -
As the project is in
microservises architecture
, I have gained practical knowledge oncommunication
,networking
, between all other services; experience with cloud providers such aAWS
,Azure
and onveralldev to production
of aSDLC
.