chimeratk / deviceaccess-pythonbindings Goto Github PK

It is possible to unintentionally write data from RAM to a register, using deviceaccess.Device.write(). This occurs, if a list, which is to be written to a register, is shorter than that register. In that case all registers after the last element in the list is written with whatever is in the memory.

Example:

>>> import deviceaccess as da
>>> da.setDMapFilePath('devices.dmap')
>>> dev = da.Device('APP')
>>> dev.open()
>>> print(dev.read('BSP.FCM')[0:120])
[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
>>> dev.write('BSP.FCM', [1, 2, 3, 4, 5, 6], 10)
>>> print(dev.read('BSP.FCM')[0:120])
[  0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   1.   2.   3.   4.
   5.   6. 255.  65.   1.   2.   3.   4.   5.   6. 255. 161. 255. 255.
 144.   0. 240.  12. 255.  65.  16.   1.  46.  32. 255.  56.  12. 255.
  65.  88.   1.  46. 144. 255.   0.   8.   2.   0.   0.   0. 112. 132.
 132. 255. 255.  10.  32.  32.  16.  32.   0.  33.  48.  48.   0.   0.
  48.  11. 255.  84.  80.   1. 255. 255. 255. 120.  11. 255. 255. 152.
   1. 255. 255. 255.   0.  56.   0.   0.   0.   0. 224. 244. 244. 255.
 255.  10.  96.  32. 144.  32.   0.  33.  96.  48. 192. 144. 112.  11.
 255. 255. 144.   1.  46.  65. 255. 184.]

The data type for read_sequences uses float32 as data type. This is not sufficient to store 32 bit integer, which makes it useless for some data transfers.

Solution: Make the data type selectable (with dtype=np.xxx, like numpy arrays), (with default = float64?). In the ideal case the correct type would be picked/estimated from the catalogue information, but could be overwritten by the user, but this would be an extra ticket.

On Ubuntu 18.10, cmake does not run through as it does not find the BOOST Python library. It seems, the component used in the find_package() call is wrong (for Ubuntu 18.10). It should be there:

find_package(Boost REQUIRED COMPONENTS python3)

Maybe this also works on older BOOST resp. Ubuntu versions?

The documentation on chimeratk.github.io is essentially empty.

The default for the second parameter of "read_sequences" should be an empty string, not 'None'. Like this it is possible to use read_sequences with just one string and without specifying the registerPath variable name:

d.read_sequences('/TEST/DMA')
instead of d.read_sequences(registerPath='/TEST/DMA') or d.read_sequences('/TEST/DMA','') or d.read_sequences('','/TEST/DMA')

Looks like the naming has changed yet again on Ubuntu

cmake -DCMAKE_INSTALL_PREFIX=/home/jens/DESY/install -DCMAKE_INSTALL_LIBDIR=lib -G Ninja -DPYTHON3=TRUE -DCMAKE_BUILD_TYPE=Debug -DCMAKE_MODULE_PATH=~/DESY/install/share/cmake-3.13/Modules -DSUPPRESS_AUTO_DOC_BUILD=true -G"CodeBlocks - Ninja" -DDOOCS_DIR=~/DESY/install/lib -DCMAKE_BUILD_TYPE="Debug" /home/jens/DESY/checkout/DeviceAccess-PythonBindings
-- The C compiler identification is GNU 9.3.0
-- The CXX compiler identification is GNU 9.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Boost: /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake (found version "1.71.0") found components: system thread chrono filesystem 
-- Found ChimeraTK-DeviceAccess: /home/jens/DESY/install (found suitable version "01.11", minimum required is "01.11") 
-- Found Boost: /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake (found version "1.71.0") found components: system 
-- Found Boost: /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake (found version "1.71.0") found components: unit_test_framework 
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython3.8.so (found suitable version "3.8.2", minimum required is "3") 
CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:117 (find_package):
  Could not find a package configuration file provided by
  "boost_python3-py38" (requested version 1.71.0) with any of the following
  names:

    boost_python3-py38Config.cmake
    boost_python3-py38-config.cmake

  Add the installation prefix of "boost_python3-py38" to CMAKE_PREFIX_PATH or
  set "boost_python3-py38_DIR" to a directory containing one of the above
  files.  If "boost_python3-py38" provides a separate development package or
  SDK, be sure it has been installed.

I'm trying to debug my application using googles AddressSanitizer, which can be enabled e.g. in ControlSystemAdapter-OPC-UA-Adapter. I can not start my application because of a problem found by AddressSanitizer. Surprisingly, the problems is caused inside the python script which is executed at startup of my application. Consequently, running the python script standalone fails with the same error.

The output of AddressSanitizer looks like this:

==201008==AddressSanitizer CHECK failed: ../../../../src/libsanitizer/asan/asan_interceptors.cc:291 "((IndirectExternCall(__interception::real___cxa_throw))) != (0)" (0x0, 0x0)
    #0 0x7fe0e9489631  (/usr/lib/x86_64-linux-gnu/libasan.so.2+0xa0631)
    #1 0x7fe0e948e5e3 in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) (/usr/lib/x86_64-linux-gnu/libasan.so.2+0xa55e3)
    #2 0x7fe0e941f706 in __cxa_throw (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x36706)
    #3 0x7fe0e4d15af6  (/usr/lib/libChimeraTK-DeviceAccess.so.01.09xenial2+0x7ccaf6)
    #4 0x7fe0e4e7d859 in ChimeraTK::FixedPointConverter::vectorToCooked_impl::impl(ChimeraTK::FixedPointConverter const&, int const* const&, int const* const&, signed char*) (/usr/lib/libChimeraTK-DeviceAccess.so.01.09xenial2+0x934859)
    #5 0x7fe0e4e7905d in ChimeraTK::FixedPointConverter::reconfigure(unsigned int, int, bool) (/usr/lib/libChimeraTK-DeviceAccess.so.01.09xenial2+0x93005d)
    #6 0x7fe0e4e7d65b in ChimeraTK::FixedPointConverter::FixedPointConverter(std::__cxx11::basic_string, std::allocator >, unsigned int, int, bool) (/usr/lib/libChimeraTK-DeviceAccess.so.01.09xenial2+0x93465b)
    #7 0x7fe0e4e26de0 in ChimeraTK::NumericAddressedBackendRegisterAccessor::NumericAddressedBackendRegisterAccessor(boost::shared_ptr, ChimeraTK::RegisterPath const&, unsigned long, unsigned long, ChimeraTK::Acces
sModeFlags) (/usr/lib/libChimeraTK-DeviceAccess.so.01.09xenial2+0x8ddde0)
    #8 0x7fe0e4dc2de0 in boost::shared_ptr > ChimeraTK::NumericAddressedBackend::getRegisterAccessor_impl(ChimeraTK::RegisterPath const&, unsigned long, unsigned long, ChimeraTK::AccessModeFlags) (/usr/lib/libChimeraTK-DeviceAccess.so.01.09xen
ial2+0x879de0)
    #9 0x7fe0e4dba1eb  (/usr/lib/libChimeraTK-DeviceAccess.so.01.09xenial2+0x8711eb)
    #10 0x7fe0e56c0d3d in ChimeraTK::OneDRegisterAccessor ChimeraTK::Device::getOneDRegisterAccessor(ChimeraTK::RegisterPath const&, unsigned long, unsigned long, ChimeraTK::AccessModeFlags const&) const (/usr/lib/python3/dist-packages/mtca4udeviceaccess.so+0x33d3d)
    #11 0x7fe0e56c05c6 in mtca4upy::DeviceAccess::getRawOneDAccessor(ChimeraTK::Device const&, std::__cxx11::basic_string, std::allocator > const&, unsigned long, unsigned long) (/usr/lib/python3/dist-packages/mtca4udeviceaccess.so+0x335c6)
    #12 0x7fe0e56b6ee0 in boost::python::objects::caller_py_function_impl (*)(ChimeraTK::Device const&, std::__cxx11::basic_string, std::allocator > const&, unsigned long, unsigned 
long), boost::python::default_call_policies, boost::mpl::vector5, ChimeraTK::Device const&, std::__cxx11::basic_string, std::allocator > const&, unsigned long, unsigned long> > >::operator()(_object*, _object*) 
(/usr/lib/python3/dist-packages/mtca4udeviceaccess.so+0x29ee0)
    #13 0x7fe0e3a9c00c in boost::python::objects::function::call(_object*, _object*) const (/usr/lib/x86_64-linux-gnu/libboost_python-py35.so.1.58.0+0x2900c)
    #14 0x7fe0e3a9c207  (/usr/lib/x86_64-linux-gnu/libboost_python-py35.so.1.58.0+0x29207)
    #15 0x7fe0e3aa4052 in boost::python::handle_exception_impl(boost::function0) (/usr/lib/x86_64-linux-gnu/libboost_python-py35.so.1.58.0+0x31052)
    #16 0x7fe0e3a99408  (/usr/lib/x86_64-linux-gnu/libboost_python-py35.so.1.58.0+0x26408)
    #17 0x5c3bd6 in PyObject_Call (/usr/bin/python3.5+0x5c3bd6)
    #18 0x5354ae in PyEval_EvalFrameEx (/usr/bin/python3.5+0x5354ae)
    #19 0x539a12  (/usr/bin/python3.5+0x539a12)
    #20 0x5356a1 in PyEval_EvalFrameEx (/usr/bin/python3.5+0x5356a1)
    #21 0x539a12  (/usr/bin/python3.5+0x539a12)
    #22 0x53a6ce in PyEval_EvalCode (/usr/bin/python3.5+0x53a6ce)
    #23 0x6292c1  (/usr/bin/python3.5+0x6292c1)
    #24 0x62b769 in PyRun_FileExFlags (/usr/bin/python3.5+0x62b769)
    #25 0x62bf5b in PyRun_SimpleFileExFlags (/usr/bin/python3.5+0x62bf5b)
    #26 0x63d505 in Py_Main (/usr/bin/python3.5+0x63d505)
    #27 0x4cfd10 in main (/usr/bin/python3.5+0x4cfd10)
    #28 0x7fe0e8e2282f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
    #29 0x5d36e8 in _start (/usr/bin/python3.5+0x5d36e8)

The python does nothing fancy just reads a register via the pcie backend:

#!/usr/bin/python3
import mtca4u
import sys
deviceName = sys.argv[1]

mtca4u.set_dmap_location("devMapFile.dmap")
device = mtca4u.Device(deviceName)
status = device.read_raw("BOARD.0", "WORD_BOOT_STATUS")

The current docu is confusing because it is based on a very old scheme which we don't follow any more.

Make clear that it needs the register path as an input and returns a 2D register. Give an example how to access an individual channel (aka sequence).

When using a custom backend (e.g. by LD_PRELOADing it), which has been compiled against a different DeviceAccess version than the PythonBindings, I have observed that all registers are read as 0, even if they are set to non-zero values in the backend constructor. When compiling the PythonBindings against the same DeviceAccess version as the backend, everything works as expected.

When trying the same thing with QtHardMon (which is normally compiled against the same DeviceAccess version as the PythonBindings on my system), I get a useful error message telling me that I am mixing DeviceAccess versions. I would expect the same error message from the PythonBindings. I don't understand why it does not appear, since the error message is coming from DeviceAccess, as far as I know. Maybe the PythonBindings are circumventing the foreseen backend creation method in the factory?

Also this issue raises the question, whether the PythonBindings (and also QtHardMon) shouldn't be released as new Debian packages whenever there is a new DeviceAccess version. Otherwise the use of custom backends in Python scripts is practically impossible most of the time (without compiling the PythonBindings manually).

Let's imagine 1 dmap file having multiple device entries as such:

apple sdm://./pci:pcieunis3 apple.mapp
banana sdm://./pci:pcieunis3 banana.mapp

If we open the devices on mtca4upy and read their 'magic' registers, reading from banana returns apple's magic number

In fact you cannot access to banana registers. It depends on which order device is created. If banana device is opened first, the behavior switches.

Using latest python3-mtca4upy (01.03buster1.03-0)

I tried to read an uint32 value via a Backend. It seems like read always uses double accessors and read_raw uses int32 accessors.
So I don't know how to access an uint32.