Potential REL TABLE GROUP data ingest bugs with COPY (Python) about kuzu HOT 7 CLOSED

Just wanted to follow up to mention on a single test using kuzu==0.3.2 I no longer witnessed this issue occurring, so it may be resolved. Generally, and possibly related, I also noticed that memory performance was also significantly better (lower than before). Thanks again @ray6080 for the prompt actions here!

from kuzu.

Hi, for "Missing NODE primary key ID", can you take a look at this issue (#2916)? Is it similar to what you encountered?

from kuzu.

I've reproduced the first bug “Segmentation faults during COPY operations of REL TABLE GROUP Tables." Thanks for helping with reproducing. The problem lies in our COPY rel string properties. Will update here once we have a fix.

On questions:

Is there a way to increase the documentation provided during these types of segmentation faults to help inform the user?

I don't see a practical way for us to catch segmentation faults and replace it with a meaningful error message. Possible future solutions is to provide a way to report the stack trace, but that will mostly only be useful for debugging for developers.

When re-running a COPY command which caused a segmentation fault, is there any concern about data integrity, or is this built into existing Kuzu validation procedures?

We have built-in transaction support for COPY, so that means corrupted COPY data should not be visible and data integrity should be fine.

from kuzu.

Quick update: the segmentation fault should be fixed now in #2928.

from kuzu.

Thanks @ray6080 for all the great comments and help with this!

I'm unable to reproduce the bug currently for "2. Missing NODE primary key ID's" but the issue you highlighted looks similar. I didn't attempt using different amounts of threads so it's possible that would be the fix when this occurs again. I'll give this a try the next time it occurs.

Glad to hear you were able to find a bug with the segmentation faults and that there is transaction support with Kuzu! I'd love to see the capability at some point to view the stack trace easily. This might help with issues similar to this one, where you could ask that someone provides portions of the trace alongside other troubleshooting data.

from kuzu.

Fantastic @ray6080 , awesome turnaround time! Thanks again for the updates and information here.

from kuzu.

Close as done for now.

from kuzu.