qs::qread() error - about qs HOT 6 CLOSED

Could you update to the latest version 0.19.1, install.packages("qs")?

I did some digging, but I can't see how that error can be generated. But if you could send or create a reproducible example I would very much appreciate it!

Edit: to be a bit more clear, if you search the code for where attributes are set (https://github.com/traversc/qs/search?q=Rf_setAttrib&unscoped_q=Rf_setAttrib)

-- you can see that it's always accompanied by a Rf_install which converts the char to a symbol. So I don't understand how the SET_ATTRIB function can ever see a char.

E.g. if you made a simplified Rcpp function, I can't find any inputs that would produce that error:

void test(SEXP obj, SEXP new_attrib, std::string temp_attrib_string) {
   Rf_setAttrib(obj, Rf_install(temp_attribute_string.c_str()), new_attrib);
}

One thing I noticed is that I use temp_attribute_string.data() but it should be temp_attribute_string.c_str(). I don't think this makes a difference but maybe.

Anyway, please let me know what the error is on the latest version and if you can send me the example file.

from qs.

Thanks for prompt response @traversc.

Updated package to 0.19.1. Still can't open file, but error message is generic this time:

Error in c_qread(file, use_alt_rep, strict, nthreads) : 
  Failed to open file

Unfortunately, data are confidential, so I can't share.

I'll do the batch run with new pkg version, and hope output are all read-able this time 🤞; will update here.

from qs.

"Failed to open file" happens when it can't open up the file at all, right at the beginning of the function.

from qs.

Noted; it's odd to me that it always happen to a handful (<5%) of files when running in batch (e.g. 50 output).

However, re-running the input data for those corrupted files the second time gives read-able output; I suppose it's the way of life - mutated/bad DNA every so often 😄.

On a side note, new error message on subsequent re-runs:

> qs::qread("output_data/my_file.q")
Error in c_qread(file, use_alt_rep, strict, nthreads) : 
  ReadItem: unknown type 0, perhaps written by later version of R
> qs::qread("output_data/my_file_2.q")
Error in c_qread(file, use_alt_rep, strict, nthreads) : 
  ReadItem: unknown type 88, perhaps written by later version of R

Since I can't replicate these errors manually, and can't share the data either (sorry!); I'll leave it up to you if you'd like to close issue.

from qs.

Sorry I'm not really able to help you. I use qs myself every day, and haven't run into these issues.

It's sure possible that there are bugs in qs, but to me it sounds like your files are getting corrupted some how. E.g. if you try to save to the same qs file from two different threads (e.g., using parallel processes) that might break it; if you are running a batch, make sure all the save file names are unique.

I will close, but feel free to re-open or make a new issue.

from qs.

Thanks for feedback; I do ensure file names are unique.

For the convenience qs provides, I can live with <5% lemons 😄 👍

from qs.