MTNI-106 ⁃ Bug: download of urls runs twice in importer about memex HOT 6 CLOSED

@oliversauter I have been lost in the code base for a while now trying to wrap my head around everything. Can you recreate this error? Everything on my machine seems to be running fine.

from memex.

Actually this only seems to occur on my machine when running it with watchify. If it's just a static build no issues.

from memex.

I tried it again, also with watchify and it worked well.
Only the counter was a bit off, but that is an issue we know about. The correct number is processed, but the counter is a bit off. If it is too much work to fix that now, maybe we can fake it and make it fill up to the total number as a fallback. Otherwise it will only raise a lot of unnecessary questions at this time.

What else i found out is that the failed urls are just retried the next time when you start. So you can always just tick the boxes for retrying.
(related to something we talked about earlier)

Besides that I will keep an eye on the watchify situation and try a couple of more times to reproduce it under these conditions.

from memex.

Related to the comment above :)

Thanks a lot for getting on it so quickly.
Hope you enjoyed trip into the rabbit hole and that the intro helped you to orient yourself :)

from memex.

Nice find @oliversauter. Odd how it seems to happen depending on the type of build (I would think that should be unrelated).

@swissums:
The first thing I would investigate would be the import item list that gets built in local storage (mainly for persistency-sake) during imports prep stage (L119 imports-preparation.js). If that somehow gets called twice, duplicate items would be made, which will be fed into the import-item-processor via the PromiseBatcher instance (instantiated here, while the PromiseBatcher should fetch those import items to use as input here).

Would love some steps to reproduce if anyone manages to get it happening again. I use importer almost daily to populate data into the extension, and haven't seen this one yet. Will try with a one-time dev build later to see if that goes silly (99% of the time just using incremental builds, which may be why I haven't seen it).

from memex.

closing this for now, as not being able to reproduce.

from memex.