Giter Club home page Giter Club logo

eudaq's Introduction

EUDAQ version 2

Github actions Build status

EUDAQ is a Generic Multi-platform Data Acquisition Framework. Version 2 comes with more flexible cross connectivity between components, multiple data collectors, and a cleaner seperation between core functionalities and user modules.

License

This program is free software: you can redistribute it and/or modify it under the terms of the Lesser GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

Documentation and context

Please check out the online documentation at

Sources of documentation within the framework

The user's manual is provided as LaTeX source files in the repository; to generate the pdf on Linux/OSX, make sure that you have installed ImageMagick, then follow these steps:

cd build
cmake -D EUDAQ_BUILD_MANUAL=ON ..
make pdf

The manual can then be found in ./doc/manual/EUDAQUserManual.pdf.

To generate the doxygen documentation, make sure that you have installed doxygen and run

make doxygen

in the build directory after CMake. The resulting HTML files are stored in ../doc/doxygen/html.

Prerequisites

For the core Library, Executables and GUI

EUDAQ requires a C++11 compliant compiler and a Qt version 5 or higher to build GUIs. We recommend a gcc version 4.9 or later. ROOT 6 is required for the Online Monitor GUI.

User projects and modules

  • Dummy: Skeletons to add user modules - not to be changed. Required by addModule.sh, that creates your module for you
  • example: run the example without hardware, see below Execution
  • eudet: EUDET-type beam telescopes and EUDET and AIDA TLU, required: Cactus/Ipbus Software for AIDA TLU and ZestSC1+Tlufirmware+Libusb for EUDET TLU
  • aidastrip: SiStrip telescope for DESY TB24 (AIDA2020 WP15 development)
  • calice: test beam user
  • itkstrip: test beam user
  • stcontrol: USBPix/FEI4
  • tbscDESY: Slow Control System at DESY test beam
  • timepix3: Timepix3 read out, required: Spidr
  • tlu: folder for Trigger Logic Units: EUDET and AIDA TLU
  • torch: test beam user
  • MuPix8: No readme given, as only converter
  • PI Stages: Producer to control the rotation and translation stages from PI
  • experimental: developed, not tested (with hardware)
  • ITS3: producers, converters and miscellaneous sripts used in ITS3 testbeams

Compiling and installation

cmake will configure the installation and prepare the makefiles. It searches for all the required files. As a standard, it is executed in the build folder. Since the relevant CMakeLists.txt is in the main level, thus, the command is cmake ... If cmake is successful, EUDAQ can be installed. Variables set are cached in CMakeCache.txt and will again be taken into account at the next cmake run.

cmake has several options (cmake -D OPTION=ON/OFF ..) to activate or deactivate programs which will be built, here printed with their default value:

  • EUDAQ_BUILD_EXECUTABLE=ON
  • EUDAQ_BUILD_GUI=ON
  • EUDAQ_BUILD_DOXYGEN=OFF
  • EUDAQ_BUILD_MANUAL=OFF
  • EUDAQ_BUILD_PYTHON=OFF
  • EUDAQ_BUILD_STDEVENT_MONITOR=OFF
  • EUDAQ_EXTRA_BUILD_NREADER=OFF
  • EUDAQ_LIBRARY_BUILD_LCIO=OFF
  • EUDAQ_LIBRARY_BUILD_TTREE=OFF
  • USER_AIDASTRIP=OFF
  • USER_CALICE_BUILD=OFF
  • USER_EUDET_BUILD=OFF
  • USER_EXAMPLE_BUILD=ON
  • USER_EXPERIMENTAL_BUILD=OFF
  • USER_ITKSTRIP_BUILD=OFF
  • USER_STCONTROL_BUILD=OFF
  • USER_TBSCDESY=OFF
  • USER_TLU_BUILD=OFF
  • USER_TIMEPIX3_BUILD=OFF

If cmake is not successful and complains about something is missing, it is recommended to clean the build folder by rm -rf * before a new try. If problems occur during installation, please have a look in the issues, if a similiar problem already occured. If not, feel free to create a new ticket: https://github.com/eudaq/eudaq/issues

Quick installation for UNIX

Prerequisites for Ubuntu 18.04.01 LTS: sudo apt install openssh-server git cmake build-essential qt5-default xterm zlib1g-dev

Get and compile the code

git clone -b master https://github.com/eudaq/eudaq.git
mkdir -p eudaq/build
cd eudaq/build
cmake ..
make install

Notes for Windows

Visual Studio for compiling (MSVC)

The recommended windows compiler is MSVC (Microsoft Visual C++) like Visual Studio 14 2015 and later: Download Visual Studio Express Desktop (e.g. 2015 Version) here.

Compiling using cmake syntax

Start the Visual Studio "Developer Command Prompt" from the Start Menu entries for Visual Studio (Tools subfolder) which opens a cmd.exe session with the necessary environment variables already set.

If your Qt installation path has not been added to the global %PATH% variable, you need to execute the "qtenv2.bat" batch file in the Qt folder, e.g. and replace "5.1.1" with the version string of your Qt installation:

C:\Qt\Qt5.1.1\5.1.1\msvc2013\bin\qtenv2.bat

Go to the EUDAQ folder and configure, as describe above:

cd c:\[...]\eudaq\build
cmake ..

(Note: Please make sure the right compiler is found. You can select the right generator by cmake -G ... or by using the cmake-gui.)

This generates the VS project files. Installing into eudaq\bin by:

cmake --build . --target install --config Release

Notes for OS X

  • Compiler: Clang (at least version 3.1)
  • Install Qt5 or later, e.g. by using MacPorts (http://www.macports.org/): sudo port install qt5-mac-devel

Execution

In UNIX:

cd ../user/example/misc
./example_startrun.sh

The startrun script assembles the new command line syntax: Core executables are started by loading a specific module with the name option -n assigning a unique tag by the option -t:

cd bin
./euRun &
sleep 1
./euLog &
sleep 1
./euCliMonitor -n Ex0Monitor -t my_mon &
./euCliCollector -n Ex0TgDataCollector -t my_dc &
./euCliProducer -n Ex0Producer -t my_pd0 &
./euCliProducer -n Ex0Producer -t my_pd1 &

Play around with Init, Configure, Start, Stop, Re-Start or Re-Configure and Start or Reset and Re-Init and repeat -- or Terminate.

For Initialising and Configuring you have to Load to set the path to the the ini or conf file:

  • ../eudaq/user/example/misc/Ex0.ini
  • ../eudaq/user/example/misc/Ex0.conf

A description for operating the EUDET-type beam telescopes is under construction: https://telescopes.desy.de/User_manual

eudaq's People

Contributors

airqui avatar arummler avatar cburgard avatar dreyling avatar druxx avatar emlyn avatar eyiliu avatar fcrescio avatar ffeindt avatar forthommel avatar freidt avatar hperrey avatar hyeonjoongkim avatar jekroege avatar jgrosseo avatar jkvas avatar kraemeru avatar lhuth avatar mathieubenoit avatar msohailamjad avatar msuljic avatar paologb avatar rpeschke avatar rubinsky avatar simonspa avatar stephanlachnit avatar superbello avatar tbisanz avatar themperek avatar tvanat avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

eudaq's Issues

compilation fails

Ubuntu 14.04
gcc (Ubuntu 4.8.2-19ubuntu1) 4.8.2
make install failes with the following:

/home/ilcsoft/v01-17-05-trunk/v01-17-05/Eutelescope/trunk/external/eudaq/trunk/monitors/offlinemon/src/Planes.cpp:175:47: error: no matching function for call to ‘find(std::vector::iterator, std::vector::iterator, hit&)’
if (find( dummy.begin(), dummy.end(),current)== dummy.end()){
^

Altro Producer fails to build

Altro Producer cannot be built on Linux with GCC 4.7:

/home/hperrey/src/eudaq/producers/altro/include/AltroProducer.hh:4:24: fatal error: runhandler.h: No such file or directory
compilation terminated.

On Windows 7 with MSVC 12 the following error occurs:

/.../eudaq/producers/altro/include\AltroProducer.hh(2): fatal error C1083: Cannot open include file: 'pthread.h': No such file or directory [C:\Users\hperrey\Desktop\eudaq\build\producers\altro\AltroProducer.exe.vcxproj]

Make serializer code type-safe

The code in Serializer.hh takes types such as double as argument; example:

      static void write_double(Serializer & sr, const double & v){....}

This could cause problems between platforms if the width of these is defined differently and should be replaced by fixed width integer types introduced in C++11 (see http://en.cppreference.com/w/cpp/types/integer).

CMake build of Windows binaries not working

Building Windows binaries using CMake does currently fails due to several issues (CMake scripts to find libraries, code including POSIX-only headers, etc).

I've started to work on these issues on a forked branch hperrey/eudaq/tree/cmake_win32_qt5 and will refer to this issue in further commits.

TLU bitfiles and operating modes not (fully) documented

  • the bitfiles for the TLU provided through AFS are not documented at all
  • the new handshake-less operating mode is nowhere mentioned

Propose to

  • add a readme to the bitfile folder
  • add a section to the EUDAQ documentation (bitfiles/setting operation modes)
  • update the TLU EUDET note (?)

euLog Collector crashes after start

I have tried running the newest version of EUDAQ on the NI crate. the GUI version of the log collector crashed immediately with the following error

###### listenaddress=tcp://192.168.2.2:44002
       logfile=../logs/2014-02-23.log
Error: Uncaught exception: Deserialize asked for 1363231829, only have 18
  From ..\..\..\main\lib\src\BufferSerializer.cc:21
  In eudaq::BufferSerializer::Deserialize
LogThread is dying...

i tried the command line version it worked fine.

Build of Monitor.cc.o fails

Error during compilation with fresh GIT clone today:

[...]/main/lib/src/Monitor.cc: In constructor ‘eudaq::Monitor::Monitor(const std::string&, const std::string&, unsigned int, unsigned int, unsigned int, const std::string&)’:
[...]/main/lib/src/Monitor.cc:17: error: no matching function for call to ‘std::shared_ptreudaq::FileReader::shared_ptr(int)’
/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../include/c++/4.4.7/bits/shared_ptr.h:1263: note: candidates are: std::shared_ptr<_Tp>::shared_ptr(std::shared_ptr<_Tp>&&) [with _Tp = eudaq::FileReader]
/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../include/c++/4.4.7/bits/shared_ptr.h:1238: note: std::shared_ptr<_Tp>::shared_ptr() [with _Tp = eudaq::FileReader]
/usr/lib/gcc/x86_64-redhat-linux/4.4.7/../../../../include/c++/4.4.7/bits/shared_ptr.h:1236: note: std::shared_ptreudaq::FileReader::shared_ptr(const std::shared_ptreudaq::FileReader&)
make[2]: *** [main/lib/CMakeFiles/EUDAQ.dir/src/Monitor.cc.o] Error 1
make[1]: *** [main/lib/CMakeFiles/EUDAQ.dir/all] Error 2

Running Scientific Linux 6.4 (used to compile fine, tested over a month ago though)

Threading with C++11 built-in thread support broken

EUDAQ now can use threading support built-in into C++11 (via EUDAQThread class), but this does currently not work correctly (this is using branch cmake_w32_qt5 on hperrey/eudaq, where CMake has been extended to recognize C++11-aware compilers via commit hperrey/eudaq@5bc7f9a; soon to be merged):

cd build
cmake -D USE_CXX11=ON ..
[lots of output]
make install
[lots of output]
../bin/TestRunControl.exe
Uncaught exception:
Operation not permitted
Please report this to the developers.

Using pthreads this works as expected. Platform is: Ubuntu 12.10 (Kernel 3.5.0-42-generic, x64, gcc version 4.7.2)

Building with Qt5.1.1 fails under MSVC

Building with Qt5.1.1 (and MSVC 12) fails with the following error message:


"c:\Users\hanno\Documents\GitHub\eudaq\build\EUDAQ.sln" (Standardziel) (1) ->
"c:\Users\hanno\Documents\GitHub\eudaq\build\ALL_BUILD.vcxproj.metaproj" (Standardziel) (2) ->
"c:\Users\hanno\Documents\GitHub\eudaq\build\gui\euRun.exe.vcxproj.metaproj" (Standardziel) (38) ->
"c:\Users\hanno\Documents\GitHub\eudaq\build\gui\euRun.exe.vcxproj" (Standardziel) (39) ->
(ClCompile Ziel) ->
  ..\..\gui\src\euRun.cc(226): error C2039: 'getInteger' : is not a member of 'QInputDialog' [c:\Users\hanno\Documents\GitHub\eudaq\build\g
ui\euRun.exe.vcxproj]
  ..\..\gui\src\euRun.cc(226): error C3861: 'getInteger': identifier not found [c:\Users\hanno\Documents\GitHub\eudaq\build\gui\euRun.exe.v
cxproj]

incorrect reference to ZestSC1.lib

Platform: Windows
Compiler: Visual Studio 2012

the project file created by the cmake script tries to link against this file

......\extern\ZestSC1\windows\Lib\ZestSC1.lib

which does not work.

It should link against this file:

......\extern\ZestSC1\windows_7\Lib\x86\ZestSC1.lib

EUTelGeometry descriptions obsolete?

The EUTelGeometry descriptions are required by some converter plugins but only provide simple helper methods to manage hard-coded detector values and might not really be needed.
So, we should

  • remove classes altogether/absorb them into the corresponding converter plugins
  • remove any EUTelescope dependency (as there is nothing except EUTelRIO class needed)

MVD producer fails to build

On Linux with GCC 4.7 the MVD producer does not compile:

/home/hperrey/src/eudaq/producers/mvd/include/ModV550.hh:10:27: fatal error: VMEInterface.hh: No such file or directory

On Windows 7 with MSVC12 errors due to inclusion of unistd.h occur.

Missing Feature: cmake Find Rapid xml

For the cmake file for the offline monitor we would need to have a function that finds the Rapid xml files. Right now I just include the path statically.

Data Collector crashes if events are received during config stage

I noticed that if accidentaly an event is sent to the data collector during the config stage it causes a crash in the file writer thread.

The condition at line 33 of FileWriterNative goes true and then throw.

I don't know if this is a proper bug as it is triggered by an unexpected event at the wrong moment, but maybe you want to catch this possibility and not crash.

Francesco

compilation fails with BUILD_tlu=ON

Francesco,

I have tried to compile your branch https://github.com/fcrescio/eudaq/compare/aida_packet
with BUILD_tlu=ON, it files with the following

In file included from /home/telescope/eudaq/./main/include/eudaq/CommandReceiver.hh:4:0,
from /home/telescope/eudaq/./main/include/eudaq/Producer.hh:4,
from /home/telescope/eudaq/producers/tlu/src/TLUProducer.cxx:1:
/home/telescope/eudaq/./main/include/eudaq/Status.hh:41:15: error: ~@Xmap@~Y in namespace ~@Xstd@~Y does not name a type
typedef std::map<std::string, std::string> map_t;
^
/home/telescope/eudaq/./main/include/eudaq/Status.hh:44:7: error: ~@Xmap_t@~Y does not name a type
map_t m_tags; ///< Metadata tags in (name=value) pairs of strings
^
/home/telescope/eudaq/producers/tlu/src/TLUProducer.cxx: In member function ~@Xvirtual void TLUProducer::OnConfigure(const eudaq::Configuration&)@~Y:
/home/telescope/eudaq/producers/tlu/src/TLUProducer.cxx:93:41: error: invalid use of incomplete type ~@Xconst class eudaq::Configuration@~Y
std::cout << "Configuring (" << param.Name() << ")..." << std::endl;
^

Add (automated) regression testing

Since we are modifying core parts of the EUDAQ library (see e.g. #37), we need some form of regression tests, preferably automated and run (at least) nightly.

These could be set up using e.g. CTest and the scriptable interface suggested in #38 which would already allow simple dummy runs. Using something like a data re-producer we could also re-run on existing data and verify the output against a fixed reference (along the lines of data-driven black-box testing).

Tests should cover (at least) these components:

  • Configure & build (on at least one platform)
  • Start up RunControl, DataCollector and Producer(s), start run, generate data, stop run, verify output
  • collect status and output by central server, alert developers if errors occur
  • as above, but use actual data set
  • some form of high-load tests (e.g. fast generation of data)
  • Run with multiple data collectors, check output, merge using MultiFileReader, verify output
  • Run with new and old data formats (see #37)

Config.h file

Cmake creates a file with the name config.h with the contend

pragma once

// CMake uses config.cmake.h to generate config.h within the build folder.

ifndef EUDAQ_CONFIG_H

define EUDAQ_CONFIG_H

define PACKAGE_NAME "EUDAQ"

define PACKAGE_VERSION "CreateProcessError: The system cannot find the file specified.

for command: git describe --tags HEAD
in dir: D:/projects/eudaq"

define PACKAGE_STRING PACKAGE_NAME " " PACKAGE_VERSION

endif

the file gets included by euRun.cc. it prevents the project from compiling

it looks like there is some error in the cmake build process.

tool chain: vs2010

CMAKE and QT5

cmake doesnt find the QT 5 path even though it is the default path

Support for multiple DataCollectors

Add support for multiple DataCollector instances running e.g. on different machines:

  • reduces network traffic
  • reduces local disc I/O

Removal of these bottlenecks is important for running at very high track- and data-rates, e.g. the one-trigger-per-particle operation of the AIDA telescope.

(Potential) issues and ToDo:s:

  • Event count verification for all producers
  • Online monitoring
  • Data file merging for conversion

Implement data format with extended meta-information

For high-rate beam tests with asynchronous data streams between devices we need a data format that supports time-stamp meta-information; furthermore, the new data format should not rely on the "event" number (determined by trigger ID) but needs to allow trigger-ID/event counting, time-stamps, time-stamp ranges or any mixture of these.

Implication: For the online monitoring/analysis, these data streams would have to be synchronized first.

  • These changes should not require modifications to existing producers as long as the old data-taking scheme is still allowed.
  • Opening of binary files in the old format is not required (but would be nice) -- clear identification of new files and warning of the user if file format is not supported is a must however
  • The specification for the new data format can be found at https://github.com/eudaq/plans-aida

EUDAQ compiling issue

Hi all,

I took the newest version of EUDAQ and compiled and had the following error:

[morton@nafhh-atlas01]/nfs/dust/atlas/user/morton/ilcsoft/AlexanderMorton_EUTelescope_Development/v01-17-05/Eutelescope/master/external/eudaq/trunk/build% make install
[ 1%] Building CXX object main/lib/CMakeFiles/EUDAQ.dir/src/EudaqThread.cc.o
/nfs/dust/atlas/user/morton/ilcsoft/AlexanderMorton_EUTelescope_Development/v01-17-05/Eutelescope/master/external/eudaq/trunk/main/lib/src/EudaqThread.cc: In constructor ‘eudaq::eudaqThread::eudaqThread()’:
/nfs/dust/atlas/user/morton/ilcsoft/AlexanderMorton_EUTelescope_Development/v01-17-05/Eutelescope/master/external/eudaq/trunk/main/lib/src/EudaqThread.cc:17: error: ‘nullptr’ was not declared in this scope
/nfs/dust/atlas/user/morton/ilcsoft/AlexanderMorton_EUTelescope_Development/v01-17-05/Eutelescope/master/external/eudaq/trunk/main/lib/src/EudaqThread.cc: In destructor ‘eudaq::eudaqThread::~eudaqThread()’:
/nfs/dust/atlas/user/morton/ilcsoft/AlexanderMorton_EUTelescope_Development/v01-17-05/Eutelescope/master/external/eudaq/trunk/main/lib/src/EudaqThread.cc:31: error: ‘nullptr’ was not declared in this scope
/nfs/dust/atlas/user/morton/ilcsoft/AlexanderMorton_EUTelescope_Development/v01-17-05/Eutelescope/master/external/eudaq/trunk/main/lib/src/EudaqThread.cc: In member function ‘void eudaq::eudaqThread::join()’:
/nfs/dust/atlas/user/morton/ilcsoft/AlexanderMorton_EUTelescope_Development/v01-17-05/Eutelescope/master/external/eudaq/trunk/main/lib/src/EudaqThread.cc:52: error: ‘nullptr’ was not declared in this scope
/nfs/dust/atlas/user/morton/ilcsoft/AlexanderMorton_EUTelescope_Development/v01-17-05/Eutelescope/master/external/eudaq/trunk/main/lib/src/EudaqThread.cc: In member function ‘bool eudaq::eudaqThread::joinable() const’:
/nfs/dust/atlas/user/morton/ilcsoft/AlexanderMorton_EUTelescope_Development/v01-17-05/Eutelescope/master/external/eudaq/trunk/main/lib/src/EudaqThread.cc:60: error: ‘nullptr’ was not declared in this scope
make[2]: *** [main/lib/CMakeFiles/EUDAQ.dir/src/EudaqThread.cc.o] Error 1
make[1]: *** [main/lib/CMakeFiles/EUDAQ.dir/all] Error 2
make: *** [all] Error 2

I took and older version and it worked perfectly so I assume there is a bug somewhere.

thanks,
Alex.

Missing linker flag

to make the projects involving ZestSC1.lib compile with vs 2012 and beyond the following linker flag needs to be set

/SAFESEH:NO

this allows the linker to link against a lib that does not have safe exception handling (like zestSC1.lib has).

the following website explains what it does:
http://msdn.microsoft.com/en-us/library/9a89h429.aspx

Missing pieces of documentation

There are some pieces of information missing from the EUDAQ documentation (README, wiki, manual), that should be provided before release -- Please add anything else here that needs to be documented.

  • offlinemonitor: configuration, usage
  • TLU bitfiles and operating modes, see #20 (optionally part of a separate note?)
  • MultiFileReader
  • how to run regression tests using CTest

Online Monitor Crashes during run

the gui thread of the Online monitor Crashes directly after the worker thread has called this function calling the first time this function:

void OnlineMonWindow::registerTreeItem(std::string item)

call stack:
OnlineMon.exe.exe!OnlineMonWindow::registerTreeItem(std::basic_string<char,std::char_traits,std::allocator > item)
OnlineMon.exe.exe!HitmapCollection::registerPlane(const SimpleStandardPlane & p)
OnlineMon.exe.exe!HitmapCollection::fillHistograms(const SimpleStandardPlane & simpPlane)
OnlineMon.exe.exe!HitmapCollection::Fill(const SimpleStandardEvent & simpev)
OnlineMon.exe.exe!RootMonitor::OnEvent(const eudaq::StandardEvent & ev)
EUDAQ.dll!eudaq::Monitor::ProcessEvent()
EUDAQ.dll!eudaq::Monitor::OnIdle()

EUDAQ (LaTeX) Manual partially outdated and Makefile broken

The EUDAQ manual is partially outdated, e.g. it covers the RootMonitor instead of the OnlineMonitor; using make in the ./doc/manual directory results in an error message.

Before the next release, the following points need to be addressed:

  • remove Correlator.exe references (no longer existing?)
  • remove RootMonitor documentation
  • add/extend OnlineMon documentation
  • replace Makefile with CMake script
  • add section explaining how to use/setup udev rules for the TLU (instead of tluonroot utility)

Request for more detailed instructions in installing C++11 complier on Scientific Linux

Hi,

I am trying to build EUDAQ ( wanting on the new miniTLU code ) on a SLC 6.5 box. ( I want Scientific Linux since it makes is easy to install IPBus ).

I am running into trouble with getting a C++11 compatible compiler.

I followed the instructions in section 2.3 of http://eudaq.github.io/manual/EUDAQUserManual.pdf and installed the "Developer Toolset" http://linux.web.cern.ch/linux/devtoolset/ ( I installed devtoolset-2 ).

I then entered a bash shell with GCC 4.8 in the path:

scl enable devtoolset-2 bash

gcc --version then indicates 4.8

but when I "cd eudaq/build" and "cmake .." I am get "CMake Error at cmake/Platform.cmake:10 .... Please use a fully C++11 compatible compiler"

... any suggestions? I'm sure I am just using the wrong version of something, but I can't find instructions anywhere....

Online monitor still requires counted_ptr to compile

We have forgotten to check if the online monitor depends on counted_ptr. Apparently it does therefore it does not compile yet.

The workaround should be to copy this file from an older version of EUDAQ to the current one.
In Principle it should be save to completely replace it with std::shared_ptr.
I think I can do it next week

Implement scriptable interface to EUDAQ components

A scriptable interface to EUDAQ components and producers would be very helpful e.g. for automated (nightly) tests or automated measurement programs in beam tests.

Suggestion:

  • ctypes/cython/swig interface to Python

Fix stability issues: Improve Exception Handling

There are several places in the code where exceptions can be thrown but these are rarely ever handled correctly. This issue is used to track and document the progress of fixing this problem.

Documented cases of this issue:

  • log collector crashes (issue #29)

OnlineMonitor not working/tested on Windows

The OnlineMonitor has not yet been successfully been built/tested on Windows.

Need to accomplish:

  • Compile OnlineMon on Windows using CMake
  • Document installation procedure (i.e. ROOT) in README and manual
  • Test OnlineMonitor on NI crate in testbeam/lab

euRun.exe crashes 100% at startup

I have compiled the system with gcc-4.6.4 on Scientific Linux 6.4. For euRun.exe, The GUI appears and then the program crashes almost instantly (the configuration parameters for STARTRUN was merged from our working pre-git source tree installation). I get the message from euRun.exe:

DEBUG: listenaddress=tcp://daqhost.physics.carleton.ca:44000
Connect:    (waiting) (192.168.1.2:39710)
~/eudaq/main/lib/src/TransportTCP.cc:465:ProcessEvents(): Server #19, return=0, WSAError:0 (Success) Disconnected.
terminate called after throwing an instance of 'eudaq::LoggedException'
  what():  Timeout: No response in SendReceivePacket
  From ~/eudaq/main/include/eudaq/TransportBase.hh:196
  In T eudaq::TransportBase::SendReceivePacket(const string&, const eudaq::ConnectionInfo&, int) [with T = eudaq::Status, std::string = std::basic_string<char>]

The crashlog.txt file has the messages:

===============================================
Abnormal exit of EUDAQ Log Collector(1.0) at 2014-04-25 18:49:15.262
Uncaught exception:
Failed to bind socket: 44002: 
  From ~/eudaq/main/lib/src/TransportTCP.cc:337
  In eudaq::TCPServer::TCPServer(const string&)
-----------------------------------------------
===============================================
Abnormal exit of EUDAQ Data Collector(1.0) at 2014-04-25 18:49:16.276
Uncaught exception:
SocketClient Error (0): 
  From ~/eudaq/main/lib/src/TransportTCP.cc:591
  In virtual void eudaq::TCPClient::ProcessEvents(int)
-----------------------------------------------
===============================================
Abnormal exit of EUDAQ NI Producer(0.1) at 2014-04-25 18:49:17.157
Uncaught exception:
Are you sure the server is running? - Error 111 connecting to 192.168.1.2:44000: 
  From ~/eudaq/main/lib/src/TransportTCP.cc:538
  In void eudaq::TCPClient::OpenConnection()
-----------------------------------------------
===============================================
Abnormal exit of EUDAQ TLU Producer(1.0) at 2014-04-25 18:49:18.159
Uncaught exception:
Are you sure the server is running? - Error 111 connecting to 192.168.1.2:44000: 
  From ~/eudaq/main/lib/src/TransportTCP.cc:538
  In void eudaq::TCPClient::OpenConnection()
-----------------------------------------------

From the backtrace on the core file, I got the following crash information:

Core was generated by `./euRun.exe -x 0 -y 0 -w 650 -g 550 -a tcp://44000'.
Program terminated with signal 6, Aborted.
#0  0x0000003323a328a5 in raise () from /lib64/libc.so.6
#1  0x0000003323a34085 in abort () from /lib64/libc.so.6
#2  0x00007fb6894da34d in __gnu_cxx::__verbose_terminate_handler () at ~/gcc-4.6.4/libstdc++-v3/libsupc++/vterminate.cc:95
#3  0x00007fb6894d84f6 in __cxxabiv1::__terminate (handler=<value optimized out>) at ~/gcc-4.6.4/libstdc++-v3/libsupc++/eh_terminate.cc:40
#4  0x00007fb6894d8523 in std::terminate () at ~/gcc-4.6.4/libstdc++-v3/libsupc++/eh_terminate.cc:50
#5  0x00007fb6894903b1 in std::(anonymous namespace)::execute_native_thread_routine (__p=<value optimized out>)
    at ~/gcc-4.6.4/libstdc++-v3/src/thread.cc:53
#6  0x0000003dae207851 in start_thread () from /lib64/libpthread.so.0
#7  0x0000003323ae890d in clone () from /lib64/libc.so.6

Doxygen documentation unused/outdated

The doxygen documentation in ./doc/doxygen is unmaintained and the Makefile not working.

Need to:

  • Generate doxygen documentation using CMake
  • Update ./main/include/Documentation.hh (specifically supported platforms, getting the code and installation instructions)
  • add instructions to README (and manual) on how to generate doxygen files
  • Upload/publish documentation to GitHub io pages

Event mixing between MIMOSA and TLU?

Observing the behavior of eudaq over the last couple of testbeams with different TLU boxes and configurations ran with CMS pixel chips it looks like sometimes the event id reported by the tlu producer seems to be different from the one sent to the mimosa chips.

Since the CMS chips have a completely separate DAQ we have to rely on the timestamps of the events. we correlate the events by looking at the dT between to triggers recorded by eudaq and the CMS DAQ. depending on some yet unknown conditions we somehow have to compare dTs of different events to see any correlation.

This changed e.g. after changing the tlu box to one of the new revision, with one of them we couldn't figure out correlation at all.

We are really walking around somewhere in the dark when it comes to finding the source of this, i just wanted to document that here so we keep track of the fact that there seem to be issues.

Extend size of vectors handled by Serializer

The serialization code casts the size of any vector or map being serialized onto an unsigned and writes this value into the data stream; since the new data format described in #37 will reserve 64-bit to store the length of meta data and data blocks, we should extend this in the serializer accordingly -- however, this would break binary compatibility e.g. with existing files. A verified versioning of (raw) files would therefore be useful or the change needs to be thoroughly documented (e.g. along side the release of 2.0).

Furthermore, a check that we do not lose data by casting onto a smaller width variable should be implemented!

Online consistency checks and (partial) data merging of multiple data streams for monitoring

When having implemented the features

  • multiple data collectors running simultaneously (issue #25)
  • a new flexible data format with extended meta-information (issue #37)
    the data will be stored in asynchronous streams spread over different files/machines.

To verify the consistency data online and to check e.g. correlations, we need a mechanism to collect all meta-data and a specific subset of the actual data packets. This could either be implemented within the EUDAQ framework using the same communication channels and connecting through Run Control or an external (light-weight) framework. In any case, the online monitoring/verification should not impact the actual data taking and should allow for reconnects during a run.

'shift count >= width of type' warnings in Serializer.hh/StandardEvent.hh

Clang 4.2 reports the following warnings when compiling the main library:

[ 62%] Building CXX object main/lib/CMakeFiles/EUDAQ.dir/src/StandardEvent.cc.o
In file included from /Users/hanno/src/eudaq/main/lib/src/StandardEvent.cc:1:
In file included from /Users/hanno/src/eudaq/./main/include/eudaq/StandardEvent.hh:4:
In file included from /Users/hanno/src/eudaq/./main/include/eudaq/Event.hh:11:
/Users/hanno/src/eudaq/./main/include/eudaq/Serializer.hh:185:13: warning: shift count >= width of type [-Wshift-count-overflow]
          t <<= 8;
            ^   ~
/Users/hanno/src/eudaq/./main/include/eudaq/Serializer.hh:175:43: note: in instantiation of member function
      'eudaq::ReadHelper<bool>::read_int' requested here
      static reader GetFunc(...) { return read_int; }
                                          ^
/Users/hanno/src/eudaq/./main/include/eudaq/Serializer.hh:216:26: note: in instantiation of member function
      'eudaq::ReadHelper<bool>::GetFunc' requested here
      t = ReadHelper<T>::GetFunc(static_cast<T*>(0))(*this);
                         ^
/Users/hanno/src/eudaq/./main/include/eudaq/Serializer.hh:156:11: note: in instantiation of function template specialization
      'eudaq::Deserializer::read<bool>' requested here
          read(t);
          ^
/Users/hanno/src/eudaq/./main/include/eudaq/Serializer.hh:241:21: note: in instantiation of function template specialization
      'eudaq::Deserializer::read<bool>' requested here
        t.push_back(read<T>());
                    ^
/Users/hanno/src/eudaq/./main/include/eudaq/Serializer.hh:156:11: note: in instantiation of function template specialization
      'eudaq::Deserializer::read<bool>' requested here
          read(t);
          ^
/Users/hanno/src/eudaq/./main/include/eudaq/Serializer.hh:241:21: note: in instantiation of function template specialization
      'eudaq::Deserializer::read<std::vector<bool, std::allocator<bool> > >' requested here
        t.push_back(read<T>());
                    ^
/Users/hanno/src/eudaq/main/lib/src/StandardEvent.cc:27:8: note: in instantiation of function template specialization
      'eudaq::Deserializer::read<std::vector<bool, std::allocator<bool> > >' requested here
    ds.read(m_pivot);
       ^
In file included from /Users/hanno/src/eudaq/main/lib/src/StandardEvent.cc:1:
In file included from /Users/hanno/src/eudaq/./main/include/eudaq/StandardEvent.hh:4:
In file included from /Users/hanno/src/eudaq/./main/include/eudaq/Event.hh:11:
/Users/hanno/src/eudaq/./main/include/eudaq/Serializer.hh:57:13: warning: shift count >= width of type [-Wshift-count-overflow]
          t >>= 8;
            ^   ~
/Users/hanno/src/eudaq/./main/include/eudaq/Serializer.hh:47:43: note: in instantiation of member function
      'eudaq::WriteHelper<bool>::write_int' requested here
      static writer GetFunc(...) { return write_int; }
                                          ^
/Users/hanno/src/eudaq/./main/include/eudaq/Serializer.hh:83:23: note: in instantiation of member function
      'eudaq::WriteHelper<bool>::GetFunc' requested here
      WriteHelper<T>::GetFunc(static_cast<T*>(0))(*this, v);
                      ^
/Users/hanno/src/eudaq/./main/include/eudaq/Serializer.hh:103:9: note: in instantiation of function template specialization
      'eudaq::Serializer::write<bool>' requested here
        write(t[i]);
        ^
/Users/hanno/src/eudaq/./main/include/eudaq/Serializer.hh:103:9: note: in instantiation of function template specialization
      'eudaq::Serializer::write<bool>' requested here
        write(t[i]);
        ^
/Users/hanno/src/eudaq/main/lib/src/StandardEvent.cc:43:9: note: in instantiation of function template specialization
      'eudaq::Serializer::write<std::vector<bool, std::allocator<bool> > >' requested here
    ser.write(m_pivot);
        ^
2 warnings generated.

The warning is caused by the pivot pixel vector, m_pivot of type std::vectorstd::vector , being used in the write_int routine of the Serializer class where a bit shift occurs.

gcc 4.7 does not complain and I'm believe this warning does not indicate a bug -- but I'd like this to be verified by someone else as well.

EUDAQ compilation with Clang 3.5 fails due to errors in Serializer.hh

Compiling EUDAQ on Ubuntu 12.10 with Clang 3.5.0 fails due to several errors in Serializer.hh:

In file included from /home/hperrey/src/eudaq/main/lib/src/EventSynchronisationBase.cc:1:
In file included from /home/hperrey/src/eudaq/./main/include/eudaq/EventSynchronisationBase.hh:5:
In file included from /home/hperrey/src/eudaq/./main/include/eudaq/DetectorEvent.hh:5:
In file included from /home/hperrey/src/eudaq/./main/include/eudaq/TLUEvent.hh:5:
In file included from /home/hperrey/src/eudaq/./main/include/eudaq/Event.hh:11:
/home/hperrey/src/eudaq/./main/include/eudaq/Serializer.hh:94:26: error: no viable conversion from 'const eudaq::Time' to 'timeval'
      write((int)timeval(t).tv_sec);
                         ^
/usr/include/x86_64-linux-gnu/bits/time.h:31:8: note: candidate constructor (the implicit copy constructor) not viable: no known
      conversion from 'const eudaq::Time' to 'const timeval &' for 1st argument
struct timeval
       ^
/usr/include/x86_64-linux-gnu/bits/time.h:31:8: note: candidate constructor (the implicit move constructor) not viable: no known
      conversion from 'const eudaq::Time' to 'timeval &&' for 1st argument
struct timeval
       ^
/home/hperrey/src/eudaq/./main/include/eudaq/Time.hh:65:7: note: candidate function
      operator const timeval () const {
      ^
/usr/include/x86_64-linux-gnu/bits/time.h:31:8: note: passing argument to parameter here
struct timeval
       ^
In file included from /home/hperrey/src/eudaq/main/lib/src/EventSynchronisationBase.cc:1:
In file included from /home/hperrey/src/eudaq/./main/include/eudaq/EventSynchronisationBase.hh:5:
In file included from /home/hperrey/src/eudaq/./main/include/eudaq/DetectorEvent.hh:5:
In file included from /home/hperrey/src/eudaq/./main/include/eudaq/TLUEvent.hh:5:
In file included from /home/hperrey/src/eudaq/./main/include/eudaq/Event.hh:11:
/home/hperrey/src/eudaq/./main/include/eudaq/Serializer.hh:95:26: error: no viable conversion from 'const eudaq::Time' to 'timeval'
      write((int)timeval(t).tv_usec);
                         ^
/usr/include/x86_64-linux-gnu/bits/time.h:31:8: note: candidate constructor (the implicit copy constructor) not viable: no known
      conversion from 'const eudaq::Time' to 'const timeval &' for 1st argument
struct timeval
       ^
/usr/include/x86_64-linux-gnu/bits/time.h:31:8: note: candidate constructor (the implicit move constructor) not viable: no known
      conversion from 'const eudaq::Time' to 'timeval &&' for 1st argument
struct timeval
       ^
/home/hperrey/src/eudaq/./main/include/eudaq/Time.hh:65:7: note: candidate function
      operator const timeval () const {
      ^
/usr/include/x86_64-linux-gnu/bits/time.h:31:8: note: passing argument to parameter here
struct timeval
       ^
2 errors generated.

CMake doesn't resolve dependencies correctly yet

When trying to compile with EUDRB/VME support CMake doesn't correctly resolve EUDRB's dependence on the VME library. This can be seen as it attempts to build EUDRB before VME.

Also the includes are not provided, resulting in

Scanning dependencies of target EUDRBProducer.exe
[ 67%] Building CXX object producers/eudrb/CMakeFiles/EUDRBProducer.exe.dir/src/EUDRBProducer.cxx.o
In file included from ~/eudaq/producers/eudrb/src/EUDRBProducer.cxx:1:0:
~/eudaq/producers/eudrb/include/EUDRBController.hh:6:27: fatal error: VMEInterface.hh: No such file or directory
compilation terminated.
make[2]: *** [producers/eudrb/CMakeFiles/EUDRBProducer.exe.dir/src/EUDRBProducer.cxx.o] Error 1
make[1]: *** [producers/eudrb/CMakeFiles/EUDRBProducer.exe.dir/all] Error 2
make: *** [all] Error 2

Part of this might be that the producer CMake file should probably have

-INCLUDE_DIRECTORIES( include )
+INCLUDE_DIRECTORIES( include ${ext_inc_paths} )

to actually do something with that variable in case it is set.

However that's no all, and right now I'm apparently too stupid to figure out how to add the VME include dir there.

Parts of the (LaTeX) manual are outdated

The manual still has many references to the data reduction boards (EUDRB), VME and their libraries/producers required for operating the original implementation of the JRA1 EUDET telescope; this should be updated to reflect the current generation of EUDET/AIDA telescopes based on the FlexRIO/NI-crate readout.

Even though the information on the EUDRBs is technically still correct, it might cause confusion for new users and should at least be removed.

Change core components to use fixed-width integers only

Since EUDAQ runs on combinations of Windows/Unix and 32/64-bit machines, we should use only fixed width integer types -- otherwise we might run into problems when e.g. the width of "long int" is defined differently on different platforms.

I propose we make "long int" -> int32_t and "long long int" -> int64_t (same with unsigned versions). For clarity, we should also replace other integer types.

miniTLU executables fail to build

The miniTLUControl.exe executable fails to compile on Linux GCC 4.7 due to missing includes:

/home/hperrey/src/eudaq/producers/miniTLU/include/tlu/miniTLUController.hh:13:25: fatal error: uhal/uhal.hpp: No such file or directory

Creating a root dictionary with CMAKE

Hi,

I try to create a ROOT Dictionary for the ROOT producer interface.
I am using this function:

ROOT_GENERATE_DICTIONARY_OLD("${ROOTProducer_HEADERS}" "${ROOTProducer_DICTIONARY}" "${MY_INCLUDE_DIRECTORIES}")

This creates the following command line:

C:\ROOT-5.34\bin\rootcint.exe -f ROOTProducer_Dict.cxx -c -DHAVE_CONFIG_H -IC:/Users/Argg/Documents/GitHub/eudaqCmake/RootInterface/inc inc/ROOTProducer.h

Now my problem is that I want to use the precompiler therefore one should add the –p flag.

Does anyone know what I have to do to make cmake add this flag to the command?

AltroUSB producer fails to build

The AltroUSB producer fails to compile on Linux with GCC 4.7 with the following error:

/home/hperrey/src/eudaq/producers/altroUSB/include/AltroUSBProducer.hh:4:20: fatal error: ilcdaq.h: No such file or directory

Error on Windows 7 with MSVC12:

/.../eudaq/producers/altroUSB/include\AltroUSBProducer.hh(2): fatal error C1083: Cannot open include file: 'pthread.h': No such file or directory [C:\Users\hperrey\Desktop\eudaq\build\producers\altroUSB\AltroUSBProducer.exe.vcxproj]

Fix stability issues: Improve thread safety

EUDAQ has still some known stability issues, some of which are likely related to thread safety. This issue will be used to document and track the progress of fixing these problems.

Documented problems:

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.