storage-adapater-ad: list query must provide unique files only about storage-inventory HOT 2 CLOSED

Is this because multiple copies show up during the AD migration phase? (cache to permanent storage)

from storage-inventory.

First thing to fix:

The Iterator must be in order of StorageMetadata (can't believe I missed this earlier)... so the query needs an ORDER BY. I would suggest logging into sybase and looking at the query plan to see if "ORDER BY uri" or "ORDER by archive, filename" (column names?) is different and which is better. This would not effect an initial bulk load but will cause a subsequent run of tantar to fail horribly because it does a merge join of two iterators and relies on ordering.

Two possible fixes for the duplicate issue:

1. since there is an ORDER BY (!) it might be OK to do SELECT DISTINCT: Check query plan with and without to see if it changes in a meaningful way.
2. since there is an ORDER BY, a duplicate StorageMetadata will be next() to the identical one so the iterator could track the previous returned value and skip duplicates... I'd probably do this because sending extra rows from sybase to the java code is cheap and it has to work.

from storage-inventory.