Invalid chunk 2147483647 about libpsf HOT 10 CLOSED

Could be, what do mean by bigger than determined size? How big is the PSF file? Anyways all offset references in the PSF files are 32bit numbers and I think there is a maximum file size limit of 2^31 bytes.

Try compile it with debug info and run gdb on it:
export CXXFLAGS="-O0 -g -pg"
export CFLAGS="-O0 -g -pg"
./configure --prefix=$HOME
make clean; make install

gdb python

set args yourpythoncode.py
break translate_exception
run

And hopefully gdb will break when the exception is thrown. And then to see the backtrace, write

bt

from libpsf.

Hello, thanks for the help: here is the result of gdb backtrace. I
will dive into the details as soon as I have some gap in the schedule.

Have a nice week.

Breakpoint 1, translate_exception (e=...) at psfpython.cc:91
91 void translate_exception(IncorrectChunk const& e) {
(gdb) bt
#0 translate_exception (e=...) at psfpython.cc:91
#1 0x00007ffff0bbfad5 in operator() (function_obj_ptr=,
a0=, a1=)
at /usr/include/boost/python/detail/translate_exception.hpp:59
#2 operator()<bool,
boost::python::detail::translate_exception<IncorrectChunk, void
()(IncorrectChunk const&)>,
boost::_bi::list2<boost::python::detail::exception_handler const&,
boost::function0 const&> > (
function_obj_ptr=, a0=,
a1=) at /usr/include/boost/bind/bind.hpp:375
#3 operator()<boost::python::detail::exception_handler,
boost::function0 > (function_obj_ptr=,
a0=,
a1=) at /usr/include/boost/bind/bind_template.hpp:102
#4 boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool,
boost::python::detail::translate_exception<IncorrectChunk, void
()(IncorrectChunk const&)>, boost::_bi::list3boost::arg<1,
boost::arg<2>, boost::_bi::value<void (*)(IncorrectChunk const&)> > >,
bool, boost::python::detail::exception_handler const&,
boost::function0 const&>::invoke (
function_obj_ptr=, a0=,
a1=)
at /usr/include/boost/function/function_template.hpp:131
#5 0x00007ffff0980a94 in boost::function2<bool,
boost::python::detail::exception_handler const&,
boost::function0
const&>::operator()(boost::python::detail::exception_handler const&,
boost::function0 const&) const ()
---Type to continue, or q to quit---
from /usr/lib64/libboost_python.so.5
#6 0x00007ffff097ffeb in
boost::python::handle_exception_impl(boost::function0) () from
/usr/lib64/libboost_python.so.5
#7 0x00007ffff09763cb in ?? () from /usr/lib64/libboost_python.so.5
#8 0x000000375b043ae3 in PyObject_Call () from /usr/lib64/libpython2.6.so.1.0
#9 0x000000375b0dbe90 in PyEval_EvalFrameEx ()
from /usr/lib64/libpython2.6.so.1.0
#10 0x000000375b0de561 in PyEval_EvalCodeEx ()
from /usr/lib64/libpython2.6.so.1.0
#11 0x000000375b0de672 in PyEval_EvalCode ()
from /usr/lib64/libpython2.6.so.1.0
#12 0x000000375b0f964c in ?? () from /usr/lib64/libpython2.6.so.1.0
#13 0x000000375b0f9720 in PyRun_FileExFlags ()
from /usr/lib64/libpython2.6.so.1.0
#14 0x000000375b0faafc in PyRun_SimpleFileExFlags ()
from /usr/lib64/libpython2.6.so.1.0
#15 0x000000375b10717d in Py_Main () from /usr/lib64/libpython2.6.so.1.0
#16 0x0000003dd521ec5d in __libc_start_main () from /lib64/libc.so.6
#17 0x0000000000400649 in _start ()

On Mon, Apr 4, 2011 at 6:43 AM, henjo
[email protected]
wrote:

Could be, what do mean by bigger than determined size? How big is the PSF file? Anyways all offset references in the PSF files are 32bit numbers and I think there is a maximum file size limit of 2^31 bytes.

Try compile it with debug info and run gdb on it:
export CXXFLAGS="-O0 -g -pg"
export CFLAGS="-O0 -g -pg"
./configure --prefix=$HOME
make clean; make install

gdb python

set args yourpythoncode.py
break translate_exception
run

And hopefully gdb will break when the exception is thrown. And then to see the backtrace, write

bt

Reply to this email directly or view it on GitHub:
#3 (comment)

Taimur Gibran Rabuske Kuntz
Researcher at Analog and Mixed-Signal Circuits Research Unit
INESC-ID - Instituto de Engenharia de Sistemas e Computadores -
Investigação e Desenvolvimento

Lisbon - Portugal

GPG keyID: D4D0FEB8 @ pgp.mit.edu

from libpsf.

I couldn't solve the problem, but maybe declaring the offset with uint64_t variables would solve? Maybe use the code snippet below

#define ntohll(x) ( ( (uint64_t)(ntohl( (uint32_t)((x << 32) >> 32) )) << 32) | ntohl( ((uint32_t)(x >> 32)) ) )
#define GET_INT64(buf) ntohll(*((uint64_t *)(buf)))

in place of GET_INT32()?

from libpsf.

Hi Henjo,

I played around with libpsf, but taimurgibran is right: it breaks for small to medium sized PSF files with the same bad-chunk error. It is not only files greater than 2^31 bytes (which would be pretty big 2+ GB files). It also breaks for PSF files as small as 20MB. I can send a test file if anyone wants to figure out a solution. It looks like 32-bit datatypes are pretty deeply entwined in the code, so doing something simple like changing 32 to 64 in variable declarations doesn't seem to work. I haven't spent a lot of time trying to understand the problem though.

-Paul

from libpsf.

Hi Paul,

32-bit integers are handled by the PSFInt32Scalar class which is using the PSFScalarT template. But it's hard to say what the problem is. Please send me a data file that gives the problem. I guess the file would be small enough to send over email if it's compressed.

/Henrik

from libpsf.

Hi Henrik,

Thanks for the response! I tried to send the test file by responding to your github email, but that message exceeded their email server's size limit. The file is 22MB, or about 10MB compressed into a .tar.gz. If you can send me your email address somehow, I can send you the test case.

-Paul

from libpsf.

Sure, my email is henrik[@]johome[dot]net.

from libpsf.

I finally found this problem and it was quite serious. One could not get any signals from windowed data (such as transient analysis) unless the last signal was requested :). Please try the latest version to see if the latest commit solves the problem.

from libpsf.

Seems to work now! Thanks for the help.

from libpsf.

Great, thanks for helping with tracking down this bug.

On Friday, October 10, 2014 09:35:41 paulswirhun wrote:

Seems to work now! Thanks for the help.

---

Reply to this email directly or view it on GitHub:

#3 (comment)
1038

from libpsf.