Comments (4)
@stamparm, I really appreciate your thoughts. I thought the data would provide a better user experience. I will close the ticket.
If in the future you want to explore IPinfo's free data for this project or any other, please reach out to me or reopen the ticket. I will be extremely happy to help and be a part of the project. Thank you very much for reviewing the proposal.
from maltrail.
Furthermore, a large number of HTTP API calls can slow down the load time. This issue can be easily addressed by switching to the MMDB database.
<- there is no large number of HTTP API Calls. only when user hovers over the IP, the call is being made.
AFAICS, your proposition would be a huge performance hit on the server side (as all IPs would require an IP->ASN call), while the basic premise of the Maltrail's client was that all the post-processing of sensor data should be offloaded to the client (to relieve the server machine which is usually running the sensor in parallel)
from maltrail.
Thank you for reviewing the request, @stamparm. I have a very basic understanding of Maltrail and have only been using it on a single server, which serves as both a sensor and a server on the same machine. I tried to install it on another machine, but the process keeps getting killed because it is a micro VM (low ram, process getting killed
). My apologies if I did not conduct thorough research beforehand.
there is no large number of HTTP API Calls. only when user hovers over the IP, the call is being made.
I believe at the initial load, all the country-level information for the flags is being provided from https://stat.ripe.net/data/geoloc/data.json
Then, subsequent calls are made to the https://stat.ripe.net
API endpoint as well upon hover. The information returned is WHOIS data.
At least for the onload call to get the country information, we could obtainthe information from the IPinfo database, as the database provides both ASN and country information.
while the basic premise of the Maltrail's client was that all the post-processing of sensor data should be offloaded to the client (to relieve the server machine which is usually running the sensor in parallel)
If you implement the IPinfo database, you will be able to eliminate the need to get ASN and country-level information. The sensor is downloading the threat intel feeds on their side. Is it possible to download the MMDB file on the sensor end and provide the data from there instead?
My recommendation is that by using the IP to Country ASN database, you are extending country-level identification with ASN information in one single database. Maltrail will not require calls to the RIPE Stats endpoint on load to get both country and ASN level information from the local IPinfo DB.
Please let me know what you think.
from maltrail.
- those IP to country calls are made for the current page of results, that's true. nonetheless, they are quite fast (i believe that you noticed) because RIPE is providing such a service for everybody
- golden rule of engineering is if it works, don't mingle. honestly, i don't see any improvement by downloading a 3rd party DB to the maltrail server - also, it would introduce the pre-processing of log entries, which is not inline with the "fat client" story i gave in previous comment
from maltrail.
Related Issues (20)
- How to add severity to local logs? HOT 4
- [BUG]False Positive 185.199.109.133 HOT 1
- Custom image HOT 5
- Netflow or Span Port HOT 1
- [Feature Request] Show Number Of Past Entries HOT 1
- IP: 117.17.191.45 | Malware HOT 1
- [Feature Request] HOT 1
- Maltrail won't boot HOT 8
- Running a docker container built with your Dockerfile both server.py and sensor.py fail to restart. HOT 4
- [Questions and Support] ModuleNotFoundError: No module named 'thirdparty.six.moves' HOT 4
- [Questions and Support] The server.py does not raise if I define an ip in UDP_ADDRESS HOT 6
- External IP Flagged in Blocklist in Maltrail and Appears to also be affecting blocks on other sites... HOT 20
- Windows 11 Returns HOT 9
- [BUG] cruzit URL changed HOT 1
- [BUG] python six module HOT 10
- [BUG] Fortinet block page (fortinet-block-page-55.fortinet.com) listed as malicious. HOT 2
- Bad domains HOT 2
- [Questions and Support] Maltrail not listening on port 8337/udp HOT 6
- [Feature Request] Improvement for PHP-inj detection (TellYouThePass Ransomware)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from maltrail.