why is the downloading/uploading so slow about kaggle-cli HOT 9 CLOSED

@kirk86 I found time to investigate this feature request, and that's my conclusion. FYI.

from kaggle-cli.

How many MB/s should we see when uploading? There's no progress bar or ETA, so you can't tell if uploading submission crashed.

Also, some tips on making uploading submissions faster:

• zip the submission file! (I suggest a feature enhance to nag/warn you if you upload an unzipped submission file. Perhaps a kaggle competitions submit -z ... enhance?)
• use rounding. Don't write more digits than necessary. (Also makes your life easier when diff'ing or comparing submissions)

from kaggle-cli.

I would like to work on this. @kirk86 How did you compare the download/upload ? and which file were you downloading during the test ?

from kaggle-cli.

@invokerk hi, to be honest with you at the moment I don't recall exactly which file I've used but I did sth along the following lines. Just pick a kaggle competition and download the data directly from their website, then do the same using kaggle-cli. Also, do the same for the submission file, i.e. the upload. This will give you pretty much an idea of how long does it take on average for an upload/download.

from kaggle-cli.

It could delay at login, fetch download/submit page, download/submit action. The delay times depend on Kaggle's server. It could be mitigated by cache the login and page fetching, reducing requests.

from kaggle-cli.

@floydwch thanks a lot. I just have another question. How can you set up kaggle-cli if you're behind a proxy? Thanks.

from kaggle-cli.

Kaggle-cli employs MechanicalSoup as the browser, and MechanicalSoup employs Requests to handle the HTTP.

According to http://docs.python-requests.org/en/master/user/advanced/#proxies , it seems to have the chance to support proxy as a argument. However, Requests supports setting proxy by environment variable, in the meanwhile, you can just try this way.

from kaggle-cli.

@floydwch

in the meanwhile, you can just try this way.

yeah I've noticed that. In the meantime it would be really nice to have it as an option, don't you think?

from kaggle-cli.

Since we have already implemented browser caching (see 1176fd7), I'm going to close this issue, if there is still some weird deley, please feel free to reopen issue.

from kaggle-cli.