oysstu / pyxtf Goto Github PK

First, thanks for the library wich is very useful.
But i had a little problem with the XTF i generated and the same probleme happen with your fake XTF file generated in write.py
the error AttributeError: 'XTFUnknownPacket' object has no attribute 'Year' appear when i try to plot it.
Do you know the origin of it ?

Thanks,

Why do I keep getting errors calling the Qt platform when I run plot_sonar_paythy.py in examples, "The following (supported) packets are present (XTFHeaderType:count): ['sonar:10']
"And" qt.qpa.plugin: Could not load the Qt platform plugin "XCB"

Following the notebook example for writing XTF data gives the following error when modifying existing XTF packets:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[31], line 14
     12 # Write packets
     13 for packet in packets:
---> 14     f.write(packet.to_bytes())

TypeError: to_bytes() missing required argument 'length' (pos 1)

It seems the to_bytes method being used in builtins.pyi requires the following: length: SupportsIndex, byteorder: Literal["little", "big"]

this problem seems to come from my XTF files containing the following packets alongside sonar:

XTFHeaderType.navigation
42
XTFHeaderType.attitude
3
XTFHeaderType.sonar
0

I would like to modify the original data file then save a renamed copy, including these additional packets.

I'm the author of another Python XTF implementation: https://github.com/shamrin/pyxtf

Thank you for making the library, @oysstu! If you solution existed in 2013, I would be using it, instead of making my own :)

I've noticed at least one person was confused by our similarly named repos: shamrin/pyxtf#3 (comment)

That's rather unfortunate, I would like more people to know about your library. Your implementation is more up-to-date and actively supported. I'm going to add a note in my repo to let people know about your library: shamrin/pyxtf#5

Please let me know if I could do more :)

Due to the ordering of data channels, when I ran this code it broke reading my XTF file. I am only trying to access the port and starboard sonar data so changing line 228 in xtf_ctypes from

chan_info = [self.ChanInfo[i] for i in range(0, self.channel_count())]
to
chan_info = [self.ChanInfo[i] for i in range(0, 6) if self.ChanInfo[i].TypeOfChannel in [1,2]]

ended up fixing the problem.

I think this line change the makes the code more generalizable. Instead of assuming the relevant information is in the first two channels (as this code does if self.channel_count=2), it searches over all of the channels until it finds the two with port and starboard imagery.

Hi,

I have a problem when I try to read a XTF file with number 67 XTFHeaderType. In fact, my XTF file contains only R2Sonic TruePix data. Is it something that you are working on ?

Sincerly,

Vincent

Hi @oysstu ,I'm trying to use it to read aC3D side scan sonar.
pyxtf reads my datafile correctly thanks making pyxtf available!
I was now following the example notebook but I got an error because of concatenate_channel is not defined anywhere in the notebook.

Adding from pyxtf import concatenate_channel fix this issue. I was wondering if you can update the notebook or should I try making a PR? Thanks!

pyxtf is a great module and I appreciate the developer putting it up. I use it to extract track from XTFs. Recently I am getting memory errors with exceptionally large SSS-only XTFs (30 km plus, 50 m range). I think the maximum array size is being exceeded. Computer memory is wide open (55 GB of 64 GB still available). Is there any way, at this initial stage:

input_file = 'yourfile.xtf'
(file_header, packets) = pyxtf.xtf_read(input_file, verbose=True)

to decimate the input so as to only read in, say, every second trace or ping? Failure point is the second line above, so inside the pyxtf module. Thanks for your advice.

Hi

I recently start working on SSS, and got several SSS data in different formats, such as .875, .jsf. I'm wondering if there's an open-source or free software available for converting different formats to a unified XTF format. I'm considering to develop such a module to remove barriers in the way of sss data processing...

Hello,

I am trying to use pyxtf to merge port/starboard files from a Kongsberg HISAS. In trying to do this I discovered that when i try and make a file over 255 pings it becomes unreadable.

I went and tried your example script 'write_xtf.py' and tried manually changing the num_pings value to see if i could duplicate the behavior.

When creating files with 255 pings or less, there is no issue and there is no problem using pyxtf to read those files.

When creating files with 256 pings or more, attempting to read that file results in the following error:

2024-06-04 18:05:45,926 - ERROR - raise RuntimeError('Number of bytes to read exceeds the number of bytes remaining in packet.')

Given that its is this 255/256 boundary I thought maybe something like 'ping_count' or similar was 8-bit and couldn't handle a value over 255, however, I have looked through all the code base for pyxtf and so far I can't find a value in any of the file attributes that should be larger than 8-bit and isn't.

Last bit of info is that I can successfully read many other xtf files with pyxtf. It is just the ones written using pyxtf that are over 255 pings that appear to have an issue.

Any help you could provide would be most appreciated. Thanks.

The XTF file that write by your own can't be read by plot_sonar_bathy

When I read a xtf file , it caused a problem (which is shown in the error.txt). The experimental data and error are in the zip file below. How can I solve this ? Thanks a lot.
DataTest.zip

in xtf_ctypes.py

in the for loop from line 562 to 591, all instances of file_header.sonar_info[i] should instead be file_header.sonar_info[i-1]

the index is off by 1

File indexing has been disabled by default now (861a5f5), as it caused problems due to it being applied to xtf files automatically based on the file name alone. The index significantly speeds up reading parts of large files, but should have more protection in place to avoid trying to read a file with an invalid index.

Possible solutions include

• Storing a hash of the xtf file in the index (perhaps only of the file header)
• Reverting automatically to reading without an index if the magic numbers (0xFACE) cannot be found.

In the current implementation an index file is produced at the first reading of the XTF and is subsequently loaded in a way that is transparent to the user.

In general, I would not expect a "read" method to "write" something, and I would not expect a method to silently work differently between the first and the second call. (Also I find a bit dangerous to do file I/O inside a generator).

Maybe it could be better to decouple the generation/loading of the index file and leave some control to the user?