Export to CSV should sort like Aggregate sorts about briefcase HOT 13 CLOSED

Good question. The submission date could be used, but then the order would differ slightly from what ODK Aggregate will do. Perhaps this divergence just needs to be documented?

Submission date records the arrival of a submission on the server. Marked-as-complete date records the completion of the submission on the server. These are the same if the submission does not contain attachments but can be an arbitrary time later if the sender has connectivity problems interferring with the transmission of all media attachments or if the submission is never completely uploaded, and then much later an administrator manually marks-it-as-complete on the Form Management / Submissions Admin sub-tab.

And even later if performing pull/push copying-across ODK Aggregate instances.

from briefcase.

@yanokwa I would like to work on this enhancement to make myself familiar with the code base before GSoC17 application period.

If I understood correctly, the csv export should have records sorted by "SubmissionDate"?

from briefcase.

@rclakmal Please take it on! Note that not all forms or instances/submissions will have SubmissionDate. If it's a mix, the instances/submissions without SubmissionDate should go to the bottom.

from briefcase.

from briefcase.

Thanks for the clarification @mitchellsundt! @rclakmal, please sort by markedAsCompleteDate.

from briefcase.

@mitchellsundt and @yanokwa Thanks. Will update the PR

@mitchellsundt In the absence of this attribute, should we use a separate attribute or just leave the csv unsorted ? For an example, I'm testing with "sample" data set. It doesn't seem to have this attribute.

from briefcase.

@rclakmal I would leave it unsorted. That is, for each form export, sort all form submissions by markedAsCompleteDate, with earliest going first. If a submission does not have markedAsCompleteDate, it should go to the bottom and be unsorted. It is possible that some, all, or none of the submissions will have markedAsCompleteDate.

from briefcase.

@yanokwa @mitchellsundt I changed the implementation to use markedAsCompleteDate attribute. However, I noticed, this attribute is not getting exported to the csv even when it is present in submission.xml.

Is this the case ? Or have I done something wrong with the export?

from briefcase.

It isn't. It can be missing, too, if the submissions are pulled directly off the device.

It isn't included in the csv because the value is not durable across pulls and pushes to different ODK Aggregate instances. I.e., if you pull data from ODK Aggregate A and push to ODK Aggregate B, then B will assign different marked-as-complete dates vs. A.

This is an internal field that is central to many operations within the server and it cannot be made to be durable.

from briefcase.

@mitchellsundt Thanks for the detailed explanation.
But if we sort the csv by a field which is not visible in the export itself, wouldn't it confuse the user? The user wouldn't be able to see any direct sorting patterns.

from briefcase.

@yanokwa Could you also comment on @mitchellsundt suggestion?

from briefcase.

@rclakmal I agree that it's confusing to users to not understand how values are being sorted. I think submissionDate is a good compromise, especially since it's durable. If you are happy with how you've implemented it, I can start a review of your PR.

from briefcase.

@yanokwa Thanks. Yes it's open for review.

from briefcase.