Awesome tool! I'm just curious why skipped downloads aren't put in some sort of log file? Errors.txt is probably not the right place, but having this info (so I can go through and download manually) would be super helpful.

Also, the json metadata in each game folder is something I'd love to take more advantage of with a little script, so having the option to generate these folders and json for skipped/errored files without actually downloading would be great.

I'll probably write a PR for this, but would love to hear people's thoughts.

Hey, great tool! I'm trying to download my library but it always crashes at a game that only has a dropbox link for download.

Traceback (most recent call last):
File "C:\Users*\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 192, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users*\AppData\Local\Programs\Python\Python38\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Users*\AppData\Local\Programs\Python\Python38\lib\site-packages\itchio\downloader_main.py", line 26, in
lib.download_library()
File "C:\Users*\AppData\Local\Programs\Python\Python38\lib\site-packages\itchio\library.py", line 32, in download_library
game.download(self.login)
File "C:\Users*\AppData\Local\Programs\Python\Python38\lib\site-packages\itchio\game.py", line 62, in download
itchio.utils.download(url, path, self.name +" - "+file)
File "C:\Users*\AppData\Local\Programs\Python\Python38\lib\site-packages\itchio\utils.py", line 11, in download
filename = re.search(r'filename="(.+)"', cd).group(1)
File "C:\Users*\AppData\Local\Programs\Python\Python38\lib\re.py", line 199, in search
return _compile(pattern, flags).search(string)
TypeError: expected string or bytes-like object

Since there's no way to remove games from the library, could you please add a way to ignore games with dropbox links?

The only workaround I've found is to create a .json file with the name of the game so that the downloader skips it.

Program thinks the existence of a file means it's completely downloaded. Maybe instead download to a temp file location, then rename to the final filename? Or catch a SIGTERM and delete the file in progress?

Itchio-dl crashed while downloading, I tried restarting it, but it happened again

It crashes on multiples files, so it doesn't seems related to something specific about a file.

Running the release (2.3)

Downloading Air Dasher - Dasher_1_03_003.apk
Downloaded Dasher_1_03_003.apk
Downloading Dasher_1_03_003.zip
File Already Exists! Dasher_1_03_003.zip
MD5 Mismatch! Dasher_1_03_003.zip
Moving Dasher_1_03_003.zip to old/
Downloading Air Dasher - Dasher_1_03_003.zip
Downloaded Dasher_1_03_003.zip
Downloaded Air Dasher (4 of <REDACTED>)
Traceback (most recent call last):
  File "/config/.local/bin/itch-download", line 8, in <module>
    sys.exit(main())
  File "/config/.local/lib/python3.10/site-packages/itchiodl/downloader/__main__.py", line 62, in main
    lib.download_library(args.platform)
  File "/config/.local/lib/python3.10/site-packages/itchiodl/library.py", line 108, in download_library
    for _ in r:
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 621, in result_iterator
    yield _result_or_cancel(fs.pop())
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 319, in _result_or_cancel
    return fut.result(timeout)
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 458, in result
    return self.__get_result()
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/config/.local/lib/python3.10/site-packages/itchiodl/library.py", line 99, in dl
    g.download(self.login, platform)
  File "/config/.local/lib/python3.10/site-packages/itchiodl/game.py", line 75, in download
    self.do_download(d, token)
  File "/config/.local/lib/python3.10/site-packages/itchiodl/game.py", line 110, in do_download
    md5 = utils.md5sum(str(out_file))
  File "/config/.local/lib/python3.10/site-packages/itchiodl/utils.py", line 51, in md5sum
    with path.open("rb") as f:
AttributeError: 'str' object has no attribute 'open'

there was no output except for the loading page 1.
No error.txt, Nothing.
Help

If the user deleted file (related #36), they won't always be redownloaded (even if they are missing).

A way to "force" it is to delete the .json associated to the file (or just to delete all .json files - which might be an easy fix if the script does that before starting).

I can't seem to get the downloader working, I got the bundle tool to add all my games to my library. When I start a download I get this window, it looks like it is downloading but no files or folders are made anywhere on my computer. I have tried Python 3.8 as I saw it was stable in a Reddit post and I have tried 3.10 as well. I even tried a different computer in case it was an issue with Windows 11. I have also tried using the API key as well. Is there something I am not doing?

Downloading Troika! Numinous Edition - edit
Traceback (most recent call last):
  File "c:\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "c:\Python310\lib\site-packages\itchio\downloader\__main__.py", line 26, in <module>
    lib.download_library()
  File "c:\Python310\lib\site-packages\itchio\library.py", line 32, in download_library
    game.download(self.login)
  File "c:\Python310\lib\site-packages\itchio\game.py", line 62, in download
    itchio.utils.download(url, path, self.name +" - "+file)
  File "c:\Python310\lib\site-packages\itchio\utils.py", line 11, in download
    filename = re.search(r'filename="(.+)"', cd).group(1)
  File "c:\Python310\lib\re.py", line 200, in search
    return _compile(pattern, flags).search(string)
TypeError: expected string or bytes-like object

The game in question is: https://melsonian-arts-council.itch.io/troika-numinous-edition/ purchased from the Ukraine bundle.

The type of cd is none, thus the header doesn't exist, if I understand it correctly.

The download script consistently crash on this file.

Downloading Calico - Calico.Template.zip
Traceback (most recent call last):
  File "/home/user/.local/bin/itch-download", line 8, in <module>
    sys.exit(main())
  File "/home/user/.local/lib/python3.10/site-packages/itchiodl/downloader/__main__.py", line 31, in main
    lib.download_library()
  File "/home/user/.local/lib/python3.10/site-packages/itchiodl/library.py", line 35, in download_library
    game.download(self.login)
  File "/home/user/.local/lib/python3.10/site-packages/itchiodl/game.py", line 68, in download
    itchiodl.utils.download(url, path, self.name + " - " + file)
  File "/home/user/.local/lib/python3.10/site-packages/itchiodl/utils.py", line 21, in download
    filename = re.search(r'filename="(.+)"', cd).group(1)
AttributeError: 'NoneType' object has no attribute 'group'

root@ca48796d249d:/# itch-load-bundle
Username: [email protected]
Password:
Bundle URL: http://itch.io/bundle/download/4T80AvOSA70gkpyUhMNhaaghCbEY7J0DAJjWaWZd
Traceback (most recent call last):
  File "/usr/local/bin/itch-load-bundle", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.10/site-packages/itchiodl/bundle_tool/__main__.py", line 13, in main
    b.load_games()
  File "/usr/local/lib/python3.10/site-packages/itchiodl/bundle.py", line 17, in load_games
    pages = int(s.select("span.pager_label a")[-1].text)
IndexError: list index out of range
root@ca48796d249d:/#

Did I use the Program as intended, or is this actually an issue?

When I'm downloading 1741 games for several days straight, I'd really appreciate some way to easily see how many games are still in the queue and the overall percentage of my library that's downloaded.

If a file is broken (4 corrupted zip files in my case) they don't get redownloaded.

Meaning, I have to check with another tool, and delete the broken files, so that they can be redownloaded.

It works... but it's FAR from ideal.

On Windows 11, with Python 3.9 or 3.11 and itchiodl 2.2.0, I've tried to download all the files to the "TTRPGs for Trans Rights in Florida" bundle that I bought.

If I enter "python -m itchiodl.downloader" in the command prompt without administrator privileges and enter my login information, folders get generated in my current directory with the names of items from the bundle as "downloading [bundle item]" is printed in quick succession. However, these folders remain empty.

If I enter "python -m itchiodl.bundle_tool" in the command prompt without administrator privileges and enter my login information and bundle link, it prints 'processing page [x]/[n]' several times, and I see no change. I'm not sure if this is expected.

If I enter "python -m itchiodl.downloader" in the command prompt with administrator privileges and enter my login information, it says

Error: 400 is not 200
Will print the response text (Please be careful as this may contain personal data or allow others to login to your account):
{"errors":["Incorrect username or password"]}
Traceback (most recent call last):
File "", line 198, in run_module_as_main
File "", line 88, in run_code
File "C:\Users\flipw\AppData\Local\Programs\Python\Python311\Lib\site-packages\itchiodl\downloader_main.py", line 66, in
main()
File "C:\Users\flipw\AppData\Local\Programs\Python\Python311\Lib\site-packages\itchiodl\downloader_main.py", line 48, in main
l = itchiodl.LoginAPI(user, password)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\flipw\AppData\Local\Programs\Python\Python311\Lib\site-packages\itchiodl\login.py", line 43, in LoginAPI
raise RuntimeError
RuntimeError

If I enter "python -m itchiodl.bundle_tool" in the command prompt with administrator privileges and enter the relevant information, it says

Traceback (most recent call last):
File "", line 198, in run_module_as_main
File "", line 88, in run_code
File "C:\Users\flipw\AppData\Local\Programs\Python\Python311\Lib\site-packages\itchiodl\bundle_tool_main.py", line 17, in
main()
File "C:\Users\flipw\AppData\Local\Programs\Python\Python311\Lib\site-packages\itchiodl\bundle_tool_main.py", line 13, in main
b.load_games()
File "C:\Users\flipw\AppData\Local\Programs\Python\Python311\Lib\site-packages\itchiodl\bundle.py", line 17, in load_games
pages = int(s.select("span.pager_label a")[-1].text)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^
IndexError: list index out of range

If I try to download a single item, using "python -m itchiodl.downloader --download-game "https://housedok.itch.io/on-a-black-horse"" without admin privileges and enter my information, I get

Traceback (most recent call last):
File "", line 198, in run_module_as_main
File "", line 88, in run_code
File "C:\Users\flipw\AppData\Local\Programs\Python\Python311\Lib\site-packages\itchiodl\downloader_main.py", line 66, in
main()
File "C:\Users\flipw\AppData\Local\Programs\Python\Python311\Lib\site-packages\itchiodl\downloader_main.py", line 58, in main
lib.load_game(matches.group(1), matches.group(2))
File "C:\Users\flipw\AppData\Local\Programs\Python\Python311\Lib\site-packages\itchiodl\library.py", line 56, in load_game
self.games.append(Game(k))
^^^^^^^
File "C:\Users\flipw\AppData\Local\Programs\Python\Python311\Lib\site-packages\itchiodl\game.py", line 17, in init
self.data = data["game"]
~~~~^^^^^^^^
KeyError: 'game'

If I get admin privileges and try to download the same item, it gives the 'incorrect username or password' error instead.

I have the itch.io app and am logged in with it on this computer. I'm also logged in on itch.io on Chrome. I am able to download bundle items manually through both of these. I'm on a residential IP address. If you need any more information, I will be happy to supply it.

I have two massive bundles now and this has likely saved me 24 hours of manual labour.

I bought this bundle: https://itch.io/bundle/download/fCCJhYMFZALEAiZP7yTQ6UTHjm7WHfNdoGrsnxez
and now i'm trying to download them all locally.
Installed this app per instructions. running command as indicated in readme.

It repeatedly gives this error:

Downloaded Clerics of Creation: Apocalypse and Genesis Domains (25 of 505)
Traceback (most recent call last):
File "C:\Users\oryhara\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests-2.28.1-py3.11.egg\requests\models.py", line 971, in json
return complexjson.loads(self.text, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\oryhara\AppData\Local\Programs\Python\Python311\Lib\json_init_.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\oryhara\AppData\Local\Programs\Python\Python311\Lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\oryhara\AppData\Local\Programs\Python\Python311\Lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 198, in run_module_as_main
File "", line 88, in run_code
File "C:\Users\oryhara\AppData\Local\Programs\Python\Python311\Lib\site-packages\itchiodl\downloader_main.py", line 66, in
main()
File "C:\Users\oryhara\AppData\Local\Programs\Python\Python311\Lib\site-packages\itchiodl\downloader_main.py", line 62, in main
lib.download_library(args.platform)
File "C:\Users\oryhara\AppData\Local\Programs\Python\Python311\Lib\site-packages\itchiodl\library.py", line 108, in download_library
for _ in r:
File "C:\Users\oryhara\AppData\Local\Programs\Python\Python311\Lib\concurrent\futures_base.py", line 619, in result_iterator
yield _result_or_cancel(fs.pop())
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\oryhara\AppData\Local\Programs\Python\Python311\Lib\concurrent\futures_base.py", line 317, in _result_or_cancel
return fut.result(timeout)
^^^^^^^^^^^^^^^^^^^
File "C:\Users\oryhara\AppData\Local\Programs\Python\Python311\Lib\concurrent\futures_base.py", line 456, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "C:\Users\oryhara\AppData\Local\Programs\Python\Python311\Lib\concurrent\futures_base.py", line 401, in __get_result
raise self._exception
File "C:\Users\oryhara\AppData\Local\Programs\Python\Python311\Lib\concurrent\futures\thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\oryhara\AppData\Local\Programs\Python\Python311\Lib\site-packages\itchiodl\library.py", line 99, in dl
g.download(self.login, platform)
File "C:\Users\oryhara\AppData\Local\Programs\Python\Python311\Lib\site-packages\itchiodl\game.py", line 75, in download
self.do_download(d, token)
File "C:\Users\oryhara\AppData\Local\Programs\Python\Python311\Lib\site-packages\itchiodl\game.py", line 136, in do_download
j = r.json()
^^^^^^^^
File "C:\Users\oryhara\AppData\Local\Programs\Python\Python311\Lib\site-packages\requests-2.28.1-py3.11.egg\requests\models.py", line 975, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

I'm having limited success running it repeatedly with thousands of jobs. But of the 254 creators in this bundle i've only gotten 68 downloaded so far.

I created a fresh virtualenv with python 3.10 on Ubuntu 22.04. The command appears to run but only the .json file is downloaded, neither of the two expected artifacts are downloaded. I ran pip install itchiodl and activated the virtualenv.

$ python -m itchiodl.downloader --api-key="REDACTED" --download-game "https://binary-star-games.itch.io/apocalypse-frame"
Downloading APOCALYPSE FRAME
Downloaded APOCALYPSE FRAME (1 of 1)

$ tree binary-star-games/
binary-star-games/
├── apocalypse-frame
└── apocalypse-frame.json

1 directory, 1 file

The game is owned (I can download it manually), there are no errors. There should be a pdf and an epub file.

$ pip list
Package            Version
------------------ -----------
args               0.1.0
beautifulsoup4     4.11.1
certifi            2022.12.7
charset-normalizer 3.0.1
clint              0.5.1
idna               3.4
itchiodl           2.1.2
pip                22.0.2
requests           2.28.2
setuptools         59.6.0
soupsieve          2.3.2.post1
urllib3            1.26.14
wheel              0.37.1

I tried two api keys, both had the same results. I also tried invoking with python -m and also with itch-download. No errors.txt file was generated. I also tried with -j 4 and -p all flags, same result.

Thanks!

One of the files in the game https://ladyorthetiger.itch.io/platonic-pirates is named Platonic Pirates — Light.pdf. When I run the program, either as a complete library download or downloading just the game, the file is stored on my computer as Platonic Pirates � Light.pdf, after which the program crashes with the following dialogue:

(base) C:\Daphne\itchio>python -m itchiodl.downloader -k REDACTED --download-game https://ladyorthetiger.itch.io/platonic-pirates
platonic-pirates is a purchased game.
Loading page 1
Downloading Platonic Pirates
Downloading Platonic Pirates — Light.pdf
Downloading Platonic Pirates - Platonic Pirates — Light.pdf
Downloaded Platonic Pirates â Light.pdf
Traceback (most recent call last):
File "C:\Users\flipw\anaconda3\lib\runpy.py", line 197, in _run_module_as_main
return run_code(code, main_globals, None,
File "C:\Users\flipw\anaconda3\lib\runpy.py", line 87, in run_code
exec(code, run_globals)
File "C:\Users\flipw\anaconda3\lib\site-packages\itchiodl\downloader_main.py", line 66, in
main()
File "C:\Users\flipw\anaconda3\lib\site-packages\itchiodl\downloader_main.py", line 62, in main
lib.download_library(args.platform)
File "C:\Users\flipw\anaconda3\lib\site-packages\itchiodl\library.py", line 108, in download_library
for _ in r:
File "C:\Users\flipw\anaconda3\lib\concurrent\futures_base.py", line 609, in result_iterator
yield fs.pop().result()
File "C:\Users\flipw\anaconda3\lib\concurrent\futures_base.py", line 446, in result
return self.__get_result()
File "C:\Users\flipw\anaconda3\lib\concurrent\futures_base.py", line 391, in __get_result
raise self._exception
File "C:\Users\flipw\anaconda3\lib\concurrent\futures\thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "C:\Users\flipw\anaconda3\lib\site-packages\itchiodl\library.py", line 99, in dl
g.download(self.login, platform)
File "C:\Users\flipw\anaconda3\lib\site-packages\itchiodl\game.py", line 75, in download
self.do_download(d, token)
File "C:\Users\flipw\anaconda3\lib\site-packages\itchiodl\game.py", line 187, in do_download
if utils.md5sum(out_file) != d["md5_hash"]:
File "C:\Users\flipw\anaconda3\lib\site-packages\itchiodl\utils.py", line 51, in md5sum
with path.open("rb") as f:
File "C:\Users\flipw\anaconda3\lib\pathlib.py", line 1252, in open
return io.open(self, mode, buffering, encoding, errors, newline,
File "C:\Users\flipw\anaconda3\lib\pathlib.py", line 1120, in _opener
return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: 'ladyorthetiger\platonic-pirates\Platonic Pirates — Light.pdf'

My bughunt lead me to line 95 in game.py producing an "output filename" string that doesn't match what the actual filename is in the output folder. This results in line 99 assuming a file has not been downloaded, and line 187 crashing the program by trying to determine a checksum of a file that does not exist.

This is on Windows 11, so the specific filename replacement might not be reproduced on other machines. I am capable of downloading many other files.

Currently the scripts crash on errors (eg: Temporary failure in name resolution), it would be nice if it could either retry and/or skip on errors, similarly to youtube-dl/yt-dlp --retries X and --ignore-errors

As a workaround, I currently just run the download on a while loop, but it's not ideal.

Also it "crashes" on CTRL+C

On windows, at least.

Too lazy to submit a PR, but this works:
Utils.py:

def download_url(url, output_path, desc):
    with DownloadProgressBar(unit='B', unit_scale=True,
                             miniters=1, desc=desc) as t:
        output_path = re.sub(r'[\:\*\?\"\<\>\|]', ' ', output_path)
        urllib.request.urlretrieve(url, filename=output_path, reporthook=t.update_to)

Downloading Zeph - Zeph Final Build.zip
[################################] 530139/530139 - 00:00:55
Traceback (most recent call last):
  File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/zageron/.local/lib/python3.9/site-packages/itchio/downloader/__main__.py", line 26, in <module>
    lib.download_library()
  File "/home/zageron/.local/lib/python3.9/site-packages/itchio/library.py", line 32, in download_library
    game.download(self.login)
  File "/home/zageron/.local/lib/python3.9/site-packages/itchio/game.py", line 39, in download
    self.load_downloads(token)
  File "/home/zageron/.local/lib/python3.9/site-packages/itchio/game.py", line 31, in load_downloads
    for d in j["uploads"]:
KeyError: 'uploads'

After starting again, it skipped allthe way to Zeph and then continued. It made it about about 650 downloads before this crash.

In this case, this is the erroring game: https://melsonian-arts-council.itch.io/troika-numinous-edition/

It appears to throw a 403 specifically on trying to download the Discord link.

I have gotten around this by adding super basic error handling:

def download_url(url, output_path, desc):
    with DownloadProgressBar(unit='B', unit_scale=True,
                             miniters=1, desc=desc) as t:
        try:
            urllib.request.urlretrieve(url, filename=output_path, reporthook=t.update_to)
        except urllib.error.HTTPError as err:
            print(f"\n{err.code} error while attempting to load {url}")

...but this is more a hackish workaround than a fix. I did try to fetch a redirect URL to test if a discord link, but for whatever reason, I can't do so due to the 403.

The link itself works fine in the browser. I'm not sure why urllib is having trouble with it.

C:\Users\Username\Desktop\ItchioGames>python -m itchiodl.bundle_tool
Username: removed
Password:
Bundle URL: removed
Traceback (most recent call last):
  File "C:\Users\Username\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\Username\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\Username\AppData\Local\Programs\Python\Python310\lib\site-packages\itchiodl\bundle_tool\__main__.py", line 17, in <module>
    main()
  File "C:\Users\Username\AppData\Local\Programs\Python\Python310\lib\site-packages\itchiodl\bundle_tool\__main__.py", line 13, in main
    b.load_games()
  File "C:\Users\Username\AppData\Local\Programs\Python\Python310\lib\site-packages\itchiodl\bundle.py", line 15, in load_games
    pages = int(s.select("span.pager_label a")[-1].text)
IndexError: list index out of range

Extra info: I am not using 2FA, just username and password

How can I fix this?

Maybe add the ability to input a link to a game page, like https://etra-etra-games.itch.io/vortex-an-fps-where-you-cant-damage-enemies for example, so that you can download games that aren’t able to be added to your library.

Hello,

i just discovered your tool and wanted to use it to process two bundles on itch.io. The Ukraine bundle went perfectly fine, after that i wanted to process the Bundle for Racial Justice.

Sadly it throws an error on processing the bundle (error1.txt) and once i try to restart it in a new cmd window it throws a different error (error2.txt).

let me know if you need more information and thank you for the tool!

Regards
error1.txt
error2.txt

Is there any way to differentiate between Windows, Linux and MacOS versions?
For example if I want to download all Windows versions of all games (since I don't use Linux or MacOS)?

Currently as I see it really downloads everything, even if I don't need Linux or MacOS.

please add an actual license text, just stating license = "MIT" is not enough.

we wil maybe integrate this into Debian`s game-data-packager.

this tool also exists on Fedora and is ported to Arch

It would be good to set a custom download path instead of the default one or, at least, specify where the default one is

When I run this it lists a huge list of files and creates a ton of directories. But there are no files written to any of those directories. Any idea why?

I have this exception

Traceback (most recent call last):
  File "/run/media/johndigriz/Transcend/Itchiodl/itchiodl-fix-no-hash/itchiodl/library.py", line 108, in download_library
    for _ in r:
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 621, in result_iterator
    yield _result_or_cancel(fs.pop())
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 319, in _result_or_cancel
    return fut.result(timeout)
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/run/media/johndigriz/Transcend/Itchiodl/itchiodl-fix-no-hash/itchiodl/library.py", line 99, in dl
    g.download(self.login, platform)
  File "/run/media/johndigriz/Transcend/Itchiodl/itchiodl-fix-no-hash/itchiodl/game.py", line 99, in download
    self.do_download(d, token)
  File "/run/media/johndigriz/Transcend/Itchiodl/itchiodl-fix-no-hash/itchiodl/game.py", line 232, in do_download
    if utils.md5sum(out_file) != d["md5_hash"]:
  File "/run/media/johndigriz/Transcend/Itchiodl/itchiodl-fix-no-hash/itchiodl/utils.py", line 54, in md5sum
    with path.open("rb") as f:
  File "/usr/lib/python3.10/pathlib.py", line 1119, in open
    return self._accessor.open(self, mode, buffering, encoding, errors,
FileNotFoundError: [Errno 2] No such file or directory: 'darkjaguar8/a-fairys-day-adventure/A Fairy’s Day Adventure into the Human World.pdf'

The error happens with files that contain any non-English letters, a single quote and some other symbols

When downloading purchased games individually it's necessary to first get the download_key_id. With large libraries this can be many pages, which must be repeated for each individual purchased download. One fix for this would be to download and cache the download_key_ids if it's missing, and re-use the cache when it's available.

See #63 for background.

I have hundreds of purchased games due to a few large bundles, but I don't want everything, and the itch.io official app is so bad at multiple file downloads it's unusable. Ultimately I'd like to be able to create a list of games I want to keep updated and run something like this once in a while:

itch-download -k $key --update-purchases --cache .itchiodl-cache
for $game-url in list-of-games; do
  itch-download -k $key --cache .itchiodl-cache --download-game $game-url
done

I have three bundles which I have added to my itch.io library, resulting in about 4,000 games in my library. When using python -m itchiodl.downloader with my API key to download my library, the following occurs:

• The downloader says a number of "pages" are loaded
• The downloader starts going through the list of games, saying it is downloading them
• Publisher and game folders are created for these games on my hard drive in the CWD
• JSON files are also created for games, within publisher folders
• The downloader says games have been downloaded (x of 3918)
• Nothing is actually downloaded except for the JSON files
• Eventually the downloader simply stops, with the last item being that a game was downloaded (x of 3918)

Sometimes it makes it to about 1,000 games "downloaded", sometimes 1,500 before it quits. It leaves no error message or error text file with any other information. It doesn't seem to matter if I change the number of jobs (I've tried 4 jobs and 8 jobs), run with administrator privileges (using Python through Windows command prompt), or anything else. I'm not utilizing a VPN or anything else special. I'm running Python 3.9 64-bit on Windows 10.

Currently, itchiodl will download every new file as if the older one don't exist:

• if a file with the same name exist, it will overwrite it
• if a file with a different name exist, it will just download the new file and "ignore" the existing file

This is a bit of a problem, in the sense that everytime a game update, you will more than likely keep the older version.

lgogdownloader (the main GOG.com downloader) "solves" that with --check-orphan function that will check and reports (only reports, it won't do anything to them) the files that exist that aren't available to download anymore (which is, almost all the time: older versions).

itchiodl definitely needs something like that considering how many "thing" someone can have with the bundles.

Something that can delete those older files would be even better, but a "detector" would be a good start.

PS: thanks (again) for itchiodl!

Hi,

The Bundle tool doesn't seem to be working on my end. After inspection it seems like, although the logging in request returns a HTTP code 200, the authentification isn't working properly. (Printing the scrapped HTML shows that it expects a logged in session).

I'm not quite sure why (I'm not used to work with web stuff), but it seems the session is not stored. I'm on Windows

Let me know if you need any more information

I can't seem to run this properly as it keeps crashing. It initially started with an MD5 check error which caused the crash. Now, I cannot run it again as I receive errors relating to the file already existing when I run the script again (avoiding to re-download something that's already been downloaded).

Not sure what I'm doing wrong, but I can't seem to get these working. Regular downloader always fails after page 3, and bundle downloader won't even start.

❯ python3 -m itchiodl.downloader
Username: [redacted]
Password: [typed in, tried pasting, it's really correct]
Traceback (most recent call last):
File "/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/local/Cellar/[email protected]/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.9/site-packages/itchiodl/downloader/main.py", line 20, in
l = itchiodl.LoginAPI(user, password)
File "/usr/local/lib/python3.9/site-packages/itchiodl/login.py", line 30, in LoginAPI
raise RuntimeError
RuntimeError

Traceback (most recent call last).txt
I've had a variety of issues with this code, this is the most recent.

Both scripts doesn't seem to work on Python 3.10:

itch-download:

Traceback (most recent call last):
  File "/home/user/.local/bin/itch-download", line 8, in <module>
    sys.exit(main())
  File "/home/user/.local/lib/python3.10/site-packages/itchiodl/downloader/__main__.py", line 21, in main
    l = itchiodl.LoginAPI(user, password)
  File "/home/user/.local/lib/python3.10/site-packages/itchiodl/login.py", line 29, in LoginAPI
    raise RuntimeError
RuntimeError

itch-load-bundle:

Traceback (most recent call last):
  File "/home/user/.local/bin/itch-load-bundle", line 8, in <module>
    sys.exit(main())
  File "/home/user/.local/lib/python3.10/site-packages/itchiodl/bundle_tool/__main__.py", line 12, in main
    b.load_games()
  File "/home/user/.local/lib/python3.10/site-packages/itchiodl/bundle.py", line 14, in load_games
    pages = int(s.select("span.pager_label a")[-1].text)
IndexError: list index out of range

Works fine on Python 3.9