valschmidt / kmall Goto Github PK

A python module for reading Kongsberg kmall data files.

License: Creative Commons Zero v1.0 Universal

Python 94.13% Jupyter Notebook 5.87%

kmall's Introduction

Kongsberg "KMALL" file reader

Modern Kongsberg bathymetric sonar systems, which acquire data using "Seafloor Inforation System - Version 5" produce data files in the ".kmall" data format. This module (class) and utility reads these data formats and provides several tools for reporting information about the data within the file.

Although low-level readers for many of the datagram types is in place, little other infrastucture exists. This reader remains a work in progress.

./kmall.py -h
usage: kmall.py [-h] [-f KMALL_FILENAME] [-d KMALL_DIRECTORY] [-V] [-z]
            [-l COMPRESSIONLEVEL] [-Z] [-v]

A python script (and class) for parsing Kongsberg KMALL data files.

optional arguments:
  -h, --help           show this help message and exit
  -f KMALL_FILENAME    The path and filename to parse.
  -d KMALL_DIRECTORY   A directory containing kmall data files to parse.
  -V                   Perform series of checks to verify the kmall file.
  -z                   Create a compressed (somewhat lossy) version of the
                       file. See -l
  -l COMPRESSIONLEVEL  Set the compression level (Default: 0). 0: Somewhat
                       lossy compression of soundings and imagery
                       data.(Default) 1: Somewhat lossy compression of
                       soundings with imagery omitted.
  -Z                   Decompress a file compressed with this library. Files
                       must end in .Lz, where L is an integer indicating the
                       compression level (set by -l when compresssing)
  -v                   Increasingly verbose output (e.g. -v -vv -vvv),for
                       debugging use -vvv

See the examples for details about using the module.

Coordinate systems

Vessel Coordinate System (VCS)

Origo of the VCS is the vessel reference point. The VCS is defined according to the right hand rule.

x-axis pointing forward parallel to the vessel main axis.
y-axis pointing starboard parallel to the deck plane.
z-axis pointing down parallel to the mast.

Rotation of the vessel coordinate system around an axis is defined as positive in the clockwise direction, also according to the right hand rule.

Roll - rotation around the x-axis. (positive when starboard is down)
Pitch - rotation around the y-axis. (positive when bow is up)
Heading - rotation around the z-axis. Heading as input in depth calculations is sensor data referred to true north. (positive is clockwise when looking down, like traditional heading)

Array Coordinate System (ACS)

Origo of the ACS is at the centre of the array face. The ACS is defined according to the right hand rule.

x-axis pointing forward along the array (parallel to the vessel main axis).
y-axis pointing starboard along the array plane.
z-axis pointing down orthogonal to the array plane.

Surface Coordinate System (SCS)

Origo of the SCS is the vessel reference point at the time of transmission. The SCS is defined according to the right hand rule.

x-axis pointing forward along the horizontal projection of the vessel main axis.
y-axis pointing horizontally to starboard, orthogonal to the horizontal projection of the vessel main axis.
z-axis pointing down along the g-vector.

To move SCS into the waterline, use reference point height corrected for roll and pitch at the time of transmission.

Fixed Coordinate System (FCS)

Origo of the FCS is fixed somewhere in the nominal sea surface. The FCS is defined according to the right hand rule.

x-axis pointing north.
y-axis pointing east.
z-axis pointing down along the g-vector.

Written with StackEdit.

kmall's People

Contributors

Stargazers

Watchers

Forkers

monsterkittykitty apl-ocean-engineering ericgyounkin kjerram mbadv shadoward ausseabed oceanxplorer rolker nac1050

kmall's Issues

Extract XYZ

The module needs a method to extract the XYZ data in geographic or projected (UTM) coordinates, both at a record and file level.

Missing coordinate systems definition.

The comments in kmall.py say "see Coordinate systems for definition" in many places, but I can't find that section in the comments or documents.

Can a brief summary of the coordinate system be added?

extract SPO datagram error

Hi, I have used your script to try to extract the SPO datagram so I have added the following function to extract the SPO datagrams.

this works for the majority of the kmall files logged with our SIS5 system but or few of them we get this error

Unfortunately my python / datagram decoding skills are pretty low so I was wondering if you could have a look.. I might be able to provide you the kmall file..

Parsing installation parameters

The module does not yet parse out the installation parameters from the string encased within the datagram. These need to get parsed so they are generally available within the module.

Missing the navigation when using kmall_to_ros_bag.py

Hi, I'm trying to use the kmall_to_ros_bag.py script with the aim to obtain a point cloud file from it. This is how I call the script
'python kmall_to_ros_bag.py -o myBagFile.bag myKmFile.kmall'
, which creates the 'myBagFile.bag' file ok. Then I convert it to a pcd file with this script,
"
import rosbag
import sensor_msgs.point_cloud2 as pc2
import numpy as np
import open3d as o3d

def bag_to_pcd(bag_file, topic, output_pcd_file):
accumulated_points = []

with rosbag.Bag(bag_file, 'r') as bag:
    for _, msg, _ in bag.read_messages(topics=[topic]):
        pc_data = pc2.read_points(msg, field_names=("x", "y", "z"), skip_nans=True)
        pc_np = np.array(list(pc_data))
        pc_xyz = pc_np[:, :3]
        accumulated_points.append(pc_xyz)

# Concatenate all points
if accumulated_points:
    all_points = np.concatenate(accumulated_points, axis=0)
    point_cloud = o3d.geometry.PointCloud()
    point_cloud.points = o3d.utility.Vector3dVector(all_points)
    o3d.io.write_point_cloud(output_pcd_file, point_cloud)

if name == "main":
bag_file = "myBagFile.bag"
topic = "mbes/pings"
output_pcd_file = "myPclFile.pcd"

bag_to_pcd(bag_file, topic, output_pcd_file)

"
, which also makes the 'myPclFile.pcd' file ok. However, when displaying the pcd file in a point cloud viewer, all the scans end up in the same spot, as if the vessel was not moving (which it is). I'm sure there must be a dead simple detail that I'm missing, but I'm not able to figure how. Do you have the possibility to show me how I can make sure that I get the navigation, attitudes as well as compensation for svp into the exported files?
Kind regards

kmall_tester data files

We've added a script to test the kmall module against several data files, but not all the data files have not been included in the repository, which makes the script kind of useless. We need to add those data files. (Eventually we should probably migrate the test script to a set of unit tests.)

datagram version support

Recently "Rev G" of the Kmall format was released. This introduced a new version of a few datagrams (at least the MRZ datagram). But we are not (yet) checking the datagram version number so changes we make to support this new one will break reading the old one if we're not careful. We need to add datagram version checking.