Comments (5)
@RuizheIsaf This repository is only for the archiver appliance configuration support. Thus, please leave your questions to EPICS tech-talk. I remembered a long time ago; someone asked the same question, and we don't have any answers on them so far. I am happy to use MariaDB.
I think, you have to implement your own codes within the archiver appliance Java application. If you are interested in delving in, please check https://github.com/slacmshankar/epicsarchiverap/tree/master/src/main/org/epics/archiverappliance/config/persistence
Thanks,
@jeonghanlee
from epicsarchiverap-env.
thanks for your reply and help @jeonghanlee:
mariadb is stable relational database, but i am not sure whether searching data will be very slow when mariadb have saved too many data...because i have really too many PVs, and there are some pv to save waveform data...so i really worry about the performance of searching history data...
so now there are three ways to choose, one is to create a new tools to manage PVs and another way is to understand the code of EPICS AA, and change the part of persistence, at last i think EPICS AA has some configures or tricks to improve the performance of searching data...i think you are expert at this domain, could you give me some suggestions or some document source in order to understand clearly EPICS AA?
thank you for your help and I look forward to your reply
from epicsarchiverap-env.
@RuizheIsaf Please run your local AA, and see what MariaDB has. They only have the latest information about each signal, not the entire archiver data.
To retrieve all archived data from the AA, MariaDB doesn't do anything (Maybe something it does). All data are the local files that you can find in STS/MTS/LTS. Reading performance depends on your disk performance, network performance, and Tomcat application performance, not on MariaDB performance.
from epicsarchiverap-env.
@jeonghanlee
really thanks for your reply and help,
i know you have rich experience in deploying epics AA, could you share some exact performance of AA? Our devices make really too much data(40Gb/s) so I want to know how many servers can meet my needs.
And whether the data will loss when a PV's data is really big and change really fast, for example some device's frequence is 1Mhz...
i think that deploying EPICS AA in muti-server can solve these questions, but i am not sure, so i wanna learn some experience from you, really look forward to your reply!
Ruizhe
from epicsarchiverap-env.
@RuizheIsaf, no, I don't have enough experience to answer your questions. From naive experience, AA isn't your tool to handle 60GB/s data. You cannot dump this kind of data to AA. Or you may do. But you have to test it.
The next question I have is how do you handle this data? How long do you want to keep it? What kind of data do you want to put them in? How do you want to use it? What is the EPICS' role in this? EPICS cannot handle this kind of speed data at all.
Especially, disk space is another point.
1 mins : 3.6 TB
1 hour: 216 TB
1 day : 5,184 TB
Do you have enough storage for this space requirement? How do you transfer this kind of data between your device and storage?
I don't have answers to these questions, and this cannot be achieved with my bits of help. You need several good engineers to design the system which handles your system requirements.
HTH,
@jeonghanlee
from epicsarchiverap-env.
Related Issues (19)
- Install problem on Rocky Linux HOT 1
- EPICSENV.local doesn't work sometimes.. HOT 1
- Typo in scripts/required_pkgs.sh HOT 1
- http 404 not found when click the button "quick chart" HOT 4
- I/O error of Tomcat... HOT 3
- build error on Debian 11 HOT 3
- Why some system needs EPICS environment variables? HOT 2
- Support for windows HOT 2
- Archived pvs not connected on reboot Debian 11 HOT 1
- Remove the ALS site-specific DB tables, and its rules
- Update the MariaDB_Connector_J
- /opt/epicsarchiverap/retrieval/logs/archappl_service.log
- No mts and lts for some PVs
- add the mariadb dependency into archappl systemd service HOT 2
- Environment doesn't build with Ubuntu 20.04 HOT 4
- Tomcat service HOT 1
- Support for Rocky Linux HOT 3
- Debian 11 Support HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from epicsarchiverap-env.