Giter Club home page Giter Club logo

gogrepoc's People

Contributors

disi avatar eddie3 avatar h143570 avatar hamsterexastris avatar idamkini avatar jpenney avatar kalanyr avatar resonantworks avatar sarkanyi avatar sta2s avatar stevensmedia avatar trilarion avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

gogrepoc's Issues

Cyberpunk oddities

Any ideas why Cyberpunk refuses to get to the latest versions, even when you manually force full reload of the manifest of its ID? Is this GOG actively blocking full downloads for every point release? Website has 2.12 for some days now, gogrepoc can only see 2.11.

Cheers, Boris

import fails

22:01:37 | loading local manifest...
22:01:38 | collecting md5 data out of the manifest
22:01:38 | fatal...
Traceback (most recent call last):
File "/mnt/usb/gog/games - gog/downloader/gogrepoc.py", line 2777, in
main(process_argv(sys.argv))
File "/mnt/usb/gog/games - gog/downloader/gogrepoc.py", line 2558, in main
cmd_import(args.src_dir, args.dest_dir,args.os,args.lang,args.skipextras,args.skipids,args.ids,args.skipgalaxy,args.skipstandalone,args.skipshared)
File "/mnt/usb/gog/games - gog/downloader/gogrepoc.py", line 1452, in cmd_import
if game_item.md5 is not None:
UnboundLocalError: local variable 'game_item' referenced before assignment

Clean handles both pre/post Download maintenance tasks

Clean currently handles renames (which needs to be down before downloads) and orphaning files (which should be done afterwards in case the file is similiar to it's successor) need to either make these separate commands (maintain and clean?) or have flags.

Linux installers not found?

The game Master of Orion 2 has an installer for Linux (since the end of 2015, according to the GOG changelog). However, gogrepoc does not see it? Is this an issue on my side only?
This is what I tried:

python gogrepo.py update -ids 1207661633 -installers standalone 
python gogrepo.py download -ids 1207661633 -os linux -skipextras -- gog

I found no mention of a Linux installer on the manifest file and found no Linux installer found on the game directory. Am I doing something wrong?

savedir at the end of the command string fails to use path when following -os -lang or -skipfiles

All three options seem to treat the last argument, savedir as an attempt to give them an option. With -os it will complain that the specified os is not one of the valid os types. With -lang it will complain that specified language is not one of the valid languages, and finally with -skipfiles it will run but it will act on your current directory and not the supplied path, implying that the path has been entered as string to skip instead of accepted as savedir.

I don't see an option like -savedir, so I don't see a way to break the option out of itself. I've adjusted to just run the script in the folder I want the work done in order to work around this issue.

xml parsing error

Tried to update today my offline games… to meet with following errors

17:52:04 | xml parsing error occurred trying to get md5 data for patch_age_of_wonders_iii_1.802_(24203)_to_1.802_fix_(28713).exe
17:52:05 | xml parsing error occurred trying to get md5 data for patch_age_of_wonders_iii_1.801_b26272_(20409)_to_1.802_(24203).exe
17:52:06 | xml parsing error occurred trying to get md5 data for patch_age_of_wonders_iii_1.801_(19572)_to_1.801_b26272_(20409).exe
17:52:08 | (  6 / 263) fetching game details for akalabeth_world_of_doom...
17:52:10 | xml parsing error occurred trying to get md5 data for setup_akalabeth_2.0.0.1.exe
17:52:11 | xml parsing error occurred trying to get md5 data for akalabeth__world_of_doom_enUS_1_0_30902.pkg
17:52:12 | xml parsing error occurred trying to get md5 data for gog_akalabeth_world_of_doom_2.0.0.3.sh
17:52:13 | (  7 / 263) fetching game details for alders_blood_prologue...
17:52:28 | xml parsing error occurred trying to get md5 data for setup_alders_blood_prologue_1.0.20a_(64bit)_(37554).exe
17:52:28 | (  8 / 263) fetching game details for aliens_versus_predator_classic_2000...
17:52:57 | xml parsing error occurred trying to get md5 data for setup_avp_classic_20160330_(2149).exe
17:52:58 | (  9 / 263) fetching game details for amnesia_a_machine_for_pigs...
17:53:01 | xml parsing error occurred trying to get md5 data for setup_amnesia_-_a_machine_for_pigs_1.0_(20583).exe

Did gog change something on their end or is it temporary?

Download Rate Limiting & Concurrent Downloads

I was wondering if it would be possible to add arguments to control the following behaviour:

  • Download rate limit
    Something like -ratelimit x which would ensure the download rate of any individual worker does not exceed x MB/s.

  • Concurrent downloads (workers)
    Something like -workers x which would ensure x workers are spawned to do the downloads in parallel.

These two arguments would allow me to use this script like a scheduled task or service that does not consume all of my bandwidth in the background. Downloading one file at a time also helps with resuming if the script is terminated since there will be only one partial file that needs to be re-downloaded instead of 4 partial files from cancelling.

GOGRepoC cannot find the serial key(s) for the game "X4: Foundations".

Hi there,

GOGRepoC cannot find the serial key(s) for the game "X4: Foundations".

In my library, under "X4: Foundations", in the serial keys section, I have two keys:
X4: Foundations Collector's Edition Key
X4: Foundations Key

I purchased "X4: Foundations Collector's Edition" package, which is now delisted.
None of the current editions/packages "X4: Foundations", "X4: Community of Planets Edition", "X4: Community of Planets Collector's Edition" are marked as owned, but I doubt it has any relevance, as GOGRepoC only looks at the library, not the store (AFAIK).

Look into adding a GUI (via Gooey?)

(Possibly have a seperate py file that serves as the GUI frontend (using overridng/inheritance ), if can't somehow combine), probably Python 3 only ?

Lose Python 2 compatibility

Personally, I would consider dropping support for Python 2 (people who need it can continue to use the current version of gogrepo) to get a much cleaner, simpler and more modern codebase.

Then the option of using Async IO could be put on the table. There would be no need to adjust the number of concurrent workers and the system would scale better with the bandwidth that is available. But that is just my opinion.

Suggestion: Allow an executable version of the python script.

Many people don't really use Python for many things and there are programs that can be used to convert python to an executable.

Would be nice to have an option to download as an executable and would give you something to put in your releases tab. Then people could just have it setup to run off batch files when they want to refresh and update stuff.

Edit: Bonus if it can work well with a GUI like
https://github.com/Twombs/GOGRepo-GUI

Is the resume manifest really necessary?

If we store a JSON object with the list of all games at the top of the manifest, and then add one JSON object entry per game, we can always know where we left off and proceed. Say, "our collection is 54 games, I could read only 43, let me work on the missing ones".

If we want to start over, we could just delete the manifest file! I think it is a pretty clean solution.

When was skippatches removed?

I recently updated from an older version of your script, and when I tried to run it today with my typical py -2 gogrepoc.py download -skipgalaxy -skippatches -lang en -os windows mac I got the unrecognized argument message. Sure enough, searching through the python script, it has been removed. The old version (I don't remember which one, but I have the file downloaded) included it. Is there another way to accomplish the same thing? Why was skippatches removed?

Clean function tries to orphan Windows hidden files on external hard drive.

I had my GOG games stored on the root of an external hard drive I have and the clean function would always error out because it didn't have permission to delete $RecycleBin and/or System Volume Information folders (which it shouldn't do anyways). Had to move all the games into a subfolder for it to function correctly.

High memory usage when downloading

I was downloading my games using this script, but my linux box was reporting quite high memory usage. Like in the range of 10GB+. Are downloaded files being saved periodically on the disk, or keeped in the RAM? For windows files are split into 4GB chunks by GoG, but some of Linux games are over 20GB in size for single file.

Download fails if the manifest, game_item.size is None

Problem

Traceback (most recent call last):
  File "C:\Users\h1435\code\gogrepoc\gogrepoc.py", line 2777, in <module>
    main(process_argv(sys.argv))
  File "C:\Users\h1435\code\gogrepoc\gogrepoc.py", line 2539, in main
    cmd_download(args.savedir, args.skipextras, args.skipids, args.dryrun, args.ids,args.os,args.lang,args.skipgalaxy,args.skipstandalone,args.skipshared, args.skipfiles,args.downloadlimit)
  File "C:\Users\h1435\code\gogrepoc\gogrepoc.py", line 1709, in cmd_download
    work_dict[dest_file] = (game_item.href, game_item.size, 0, game_item.size-1, dest_file,downloading_file)
TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'

Using the latest build from master, deleted the manifest file and regenerated it via update -full

It looks like during a mass update, GOG servers may not always produce a file size information.
It appears that while updating the local manifest with large amount of changes, like using the -full option, the size information of some files, especially in the extras section, are not always returned. This causes the size property in the manifest to be None

This causes:

work_dict[dest_file] = (game_item.href, game_item.size, 0, game_item.size-1, dest_file,downloading_file) to fail, more precisely game_item.size-1 to fail as game_item.size is None (coming from the manifest)

Manually updating that game_item via the update command appears to be refreshing the missing information and allows further processing.

Workaround attempt Fix/workaround/robustness

Setting 'game_item.size' to 0 (or any other number) in this case allows the process to continue. While in case of download there will be a message in the logs..

1:23:30 | {tom_clancys_ghost_recon}
11:23:30 |      pass       setup_tom_clancys_ghost_recon_1.4.0.0_(19232).exe
11:23:30 |      pass       gr_manual.zip
11:23:30 |      unknown    gr_command_map.zip has no size info.  skipping
11:23:30 |      pass       gr_level_builder.zip
11:23:30 |      pass       gr_wallpaper.zip
11:23:30 |      pass       gr_vehicle_tagging_guide.zip
11:23:30 |      pass       gr_avatars.zip
11:23:30 |      pass       gr_artworks.zip
11:23:30 |      pass       gr_desktop_theme.zip
11:23:30 |      pass       gr_screensaver.zip
11:23:30 | nothing to download

When the actual download starts without chunk_tree around line 1942.

if game_item.size -> end with value of -1; which will cause a HTTP 200 response without the 'Content-Range' header and produces and exception.
if game_item.size is set to a large number it will cause a pre-allocation of a large file; and it will fail the hdr format check a few lines latter.

Suggestion

Not sure if the worker code can be updated to handle this situation. Regardless we should consider adding a log message with the suggested fix (reimport the problematic item) when the problem is encountered in line 1709.

How to manually create cookie file?

Since gogrepoc always returns:
python3 gogrepoc.py login login password 10:04:16 | attempting gog login as '[email protected]' ... 10:04:21 | cannot continue, gog is asking for a reCAPTCHA :( try again in a few minutes. 10:04:21 | exiting...

I'd like to create cookie file manually - how to do so? What format it need to have and what file name use?

Please add a "shebang" into script

Calling script in linux shell as simple executable leads to some unpredictable results, since shebang is missing

#!/usr/bin/env python3 for example.

./gogrepoc.py fails

python3 ./gogrepoc.py works

Revise the imporing of files - more trusting

If the file size and the file name are the same, and no MD5 information is provided by GOG, then perhaps I could trust that the file is the same. I would like to see this option added to make it easy to move/change my repository around.

Feature request: file blacklisting

Main reason for me - no language attribute for extras, no good way to skip downloading extras for all languages.
Proposed solution:

  1. Add command to add file(s) to blacklist, added files not count as missing on running gogrepoc.py verify etc. Count as orphaned, maybe.
  2. gogrepoc.py download -skipfiles skips blacklisted files if no filenames specified directly, or add another option, or make blacklisted files skipped by default.

Download everything?

Dumb question...Don't know if I should ask here or on gog, so doing both...

Is there a simple way to download EVERYTHING. Every language, every installer, every os, everything?

I keep trying different cmd options and all I get is the bonus files. I'm experimenting with the bash script now. But it looks like even that is somewhat limiting.

Any help?

Wishlist: Downloading ONE language by preference

So, would it be possible to have a parameter like "langpref de en" that says: Take the German version and if you can't find one the english version. This would avoid downloading two versions of a game. Or am I missing something in the docs?

Thanks for this excellent script!

Won't download games, only bonuses

I tested with the last version (master). It does not download the games but only the bonuses.

I am using linux (Manjaro 21.2.0) and python 3.9

I used these commands (without error):

gogrepoc.py login
gogrepoc.py update
gogrepoc.py download
gogrepoc.py verify

Games:

The Terraria game is missing for example, there are only bonuses:

python gogrepoc.py download -ids terraria                                                                                                                                                                                                                                 127 ✘ 
07:39:14 | loading local manifest...
07:39:14 | downloading games with id(s): {terraria}
07:39:14 | scanning manifest for renames...
07:39:14 | Cleaning up ./!downloading
07:39:14 | {terraria}
07:39:15 |      pass       terraria_soundtrack_vol_1_mp3.zip
07:39:15 |      pass       terraria_soundtrack_vol_2_mp3.zip
07:39:15 | nothing to download
07:39:15 | --
07:39:15 | total time: 0:00:00.398467
07:39:15 | exiting...

The script has nothing to download with metro 2033 redux:

python gogrepoc.py download -ids metro_2033_redux                                                                                                                                                                                                                             ✔ 
07:40:36 | loading local manifest...
07:40:36 | downloading games with id(s): {metro_2033_redux}
07:40:36 | scanning manifest for renames...
07:40:36 | Cleaning up ./!downloading
07:40:36 | {metro_2033_redux}
07:40:36 | nothing to download
07:40:36 | --
07:40:36 | total time: 0:00:00.046324
07:40:36 | exiting...

Maybe I misunderstood something ?

Empty downloads property

Hi.

Haven't seen this in the issues, and I am not willing to read 125 pages on the GOG forum, so please excuse me if this is ovious.
Just tried from a completely new folder and clean slate, still doing this.

python3 ../gogrepoc.py login
python3 ../gogrepoc.py update -full
python3 ../gogrepoc.py download -dryrun

It reports nothing but the extras to download (I have 147 games, they cannot just be 48G :D ). The manifest seems to be missing the URL to download the game itself:
'downloads': [],

Every game has this issue.

Is this something I messed up on the CLI or python dependencies ?

EDIT: Should point out I'm using the dev branch.
gogrepoc.py (version 0.4.0-a)

"no md5 data found for" Error for (nearly) all .sh files

I'm not sure if GOG broke something recently or what, but for the past few months I've been getting "request failed: 404 Client Error: Not Found for url: <...> .sh.xml" and "no md5 data found for" errors for pretty much all Linux (.sh) files without exception. Windows and Mac files seem to be fine.

Pause and resume downloads?

Hi there,

Will this let you pause and resume downloads? In some cases I can't finish downloading in one session and I would like to pause and resume.

Cheers

[bug] "GAME_STORAGE_DIR" variable is unused

Tried to implement os-specific directories, but it all came down to one issue - while cookies, manifest, !downloading and !orphaned are easy to move, GAME_STORAGE_DIR variable (which looks like something thats supposed to represent directory where game files will be moved at the end), despite being specified at the very beginning of script - in reality is never used anywhere.

Suggestion: well, just make it usefull (say, at the end of cmd_download() function)

Games that can't be downloaded / have persistent non-fatal warnings though Gogrepoc but can be downloaded though the GOG website

Please use this thread to report games / items, that have problems downloading through gogrepoc but do download through the website (please include the website link you used to verify this (ie the one with /en0game1 or /18102 at the end, not the specific download URI )
Please separate out reports of items that do not download vs those that have persistent non-fatal errors.

[Feature] OS-specific directories

Suggestion: instead of storing everything in the same directory as script, make use of OS-specific directories that serve for that purpose. For example, on linux it would be something like that:

  • "$HOME/.config/gogrepo/" for cookies and manifest
  • "$HOME/.local/share/gogrepo/" for downloaded games
  • "$HOME/.cache/gogrepo/" for downloading_dir, resume_manifest, netscape_cookies_tmp

Dirty example of how that may be achieved (covering just linux, but it can be adjusted - I just dont know the correct paths on windows and mac):
In imports:
from os.path import expanduser
In # filepath constants:
https://gist.github.com/moonburnt/275c67519d055f01afebf9a72fcc9d5f
In main:
https://gist.github.com/moonburnt/9e3b54d4290b046e83e9755b8f75cc2c

I've tested it and it works for everything but finished downloads - more about this bug there: #31

Required libraries are not mentioned in the README

The current README looks unmodified compared to the source of the fork (eddie3/gogrepo) and makes no mention of the requirement for the requests or OpenSSL libraries. While requests can be installed with the obvious (pip install requests), it it not so obvious how to install OpenSSL (pip install pyopenssl), so a mention of this in the README would be most welcome.

Problem logging in

I am trying to log in with my user and password. Always fails with cannot continue, gog is asking for a reCAPTCHA :( try again in a few minutes. message. I exported the cookies from firefox with export cookies extension into cookies.txt into the same folder and got same message. Tried also downloading from chrome with get cookies.txt extension and same message. What else can I try ?

Revise the imporing of files - less hashing

When importing files, we don't need to compute the MD5 on all files. If the size of the file does not match the size of a file on your manifest, we can be pretty sure the MD5 won't match either.

So we should filter for file size first and for MD5 second. You'll see big improvements on importing speed.

Can't download Atom RPG

Can't download Atom RPG

10:53:51 | ( 24 / 407) fetching game details for atom_rpg_postapocalyptic_indie_game...
10:53:53 | request failed: 404 Client Error: Not Found for url: https://gog-cdn-fastly.gog.com/token=nva=1716389631~dirs=6~token=0774e356615337e130b00/secure/linux_offlines/1673367037/1673367037/57186263152814002/2281/atom_rpg_post_apocalyptic_indie_game_1_190_70193.sh.xml. will not retry.
10:53:53 | no md5 data found for atom_rpg_post_apocalyptic_indie_game_1_190_70193.sh
10:53:55 | request failed: 404 Client Error: Not Found for url: https://cdn.gog.com/secure/linux_offlines/1673367037/1673367037/57186263152814002/2281/atom_rpg_post_apocalyptic_indie_game_1_190_70193.sh.xml?1223fb43868c5f174d699ac0583ffcf68b0007c0c9dfb2bcf9fd8d557a5d52608f11a35ddeb47e4c3fad684aba38351c88d553151c5f75ad31dc6126ecc536020ad7968c846c29101a52e39e5356fa803b84f2a6ae45bd5b4f21e6d9f447afb3f539d280750ae43edb4d3dfb2af06d2207266cf12654ce7544928042499ff68cb1e868ec48bcec975f377bb7a9686526f8fcbf98c8dba0e9cfe6e1c1479e942f3c359fe976d08c7099a605ab961537cc51b5fb27ec9ea1d32976f675fef004d5c4&fileExtForIe=.exe. will not retry.
10:53:55 | no md5 data found for atom_rpg_post_apocalyptic_indie_game_1_190_70193.sh
10:53:55 | Successfully fetched file info from https://www.gog.com/downloads/atom_rpg_postapocalyptic_indie_game/en3installer0 but no md5 data was available
10:53:55 | ( 25 / 407) fetching game details for atomicrops...
10:53:56 | ( 26 / 407) fetching game details for aven_colony...

Review code organization

The current code is quite long (almost 2400 lines) and not overly welcoming to newcomers in its structure or lack of comments. (I recall a function that returns nothing but alter their inputs, like filter_ functions or fetch_file_info — those really break expectations and best practices).

While a little bit more annoying to distribute, if the size or complexity of the code cannot be brought down, maybe splitting the source code in several files (e.g. one for each major command) could be an option?

Use JSON to store the manifest

This allows for other projects to read the manifest and closes the security hole that is the eval() being called on a data file. JSON is not YAML, but it also does not need to look bad.

import json
import gogrepo
manifest = gogrepo.load_manifest()
print(json.dumps(manifest[3], indent=1, sort_keys=True)

You could perhaps replace all the manifest serialization and deserialization code with less than 10 lines of code.

Orphaning dir fails because path already exists

I'm using gogrepoc for a few years now, and I am very thankful that it exists, because it enables me to download and manage my entire collection which has become quite big nowadays.
However, recently I have repeatedly stumbled upon this annoying bug, which appears to occur whenever a game on GOG is renamed (or removed) AND this exact game has already had orphaned file entries in the past.
Thus when moving the whole game directory into the orphaned folder, it fails, because a folder named like this is already there.

Subsequently the script completely stops orphaning the remaining files, because there is no proper error handling.

This error occurred with Master of Magic and some other game, and now with Wolfenstein 3D.

I always have to manually remove/rename the problematic folders, because orphaning always fails when the folder is present.

Here is the log output:

08:25:59 | orphaning dir 'wolfenstein_3d'
08:25:59 | fatal...
Traceback (most recent call last):
File "gogrepoc.py", line 2891, in
main(process_argv(sys.argv))
File "gogrepoc.py", line 2711, in main
cmd_clean(args.cleandir, args.dryrun)
File "gogrepoc.py", line 2523, in cmd_clean
shutil.move(cur_fulldir, orphan_root_dir)
File "/usr/lib/python3.8/shutil.py", line 789, in move
raise Error("Destination path '%s' already exists" % real_dst)
shutil.Error: Destination path './gog/!orphaned/wolfenstein_3d' already exists

Latest commits broke the script

I'm having errors while trying to update the manifest file. Every game gives an error similar to the following:

22:19:55 | (  1 / 901) fetching game details for 140_game...
22:20:04 |   -> title has changed "140_game" -> "140"
22:20:04 | The handled exception was:
22:20:04 | error
Traceback (most recent call last):
  File "/run/media/2844327D5A8D495D/gog/../gogrepoc/gogrepoc.py", line 313, in __getattr__
    return self[key]
           ~~~~^^^^^
KeyError: 'folder_name'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/run/media/2844327D5A8D495D/gog/../gogrepoc/gogrepoc.py", line 1715, in cmd_update
    handle_game_updates(gamesdb[item_idx], item,strict)
  File "/run/media/2844327D5A8D495D/gog/../gogrepoc/gogrepoc.py", line 657, in handle_game_updates
    if olditem.folder_name != newitem.folder_name:
                              ^^^^^^^^^^^^^^^^^^^
  File "/run/media/2844327D5A8D495D/gog/../gogrepoc/gogrepoc.py", line 315, in __getattr__
    raise AttributeError(key)
AttributeError: folder_name
22:20:04 | End exception report.

Possibility to add an option to download only games and ignore patches?

Hello,

While reviewing the code/options available I may have answered myself my question. But I was wondering if there was a possibility to skip patches of games and just download the new full release of it where there is an update or during the initial download?

I assume that can be done with simply using -skipfiles, and that it is probably the intended way. So sorry if I asked the question for nothing!

Thanks in advance for your time! And for this great project :)

Unwanted languages downloaded

On my set-up (and ever since) I have updated the manifest through python gogrepoc update -os windows -lang en.
When downloading the games via python gogrepoc download, other languages beyond english are downloaded

E.g.

  1. Master of Magic Classic: french and german were also downloaded
  2. The Witcher 3 Wild Hunt Game:
    • br, de, fr, jp, pl, ru language packs;
    • bonus packs in french, german, italian, polish, russian & spanish
  3. Project Eden: french manual
  4. Syndicate: french manual and reference card
  5. Forgotten Realms Unlimited Adventures: german manual and reference card
  6. The Witcher: italian, polish & german manual and game guides
    • other like the above examples

Is this the expected behaviour?

Games that corrupt the manifest DAT file

Version String: Python 3.11.0 (main, Oct 24 2022, 18:26:48) [MSC v.1933 64 bit (AMD64)] on win32

Hi Kalanyr,

I thought (in a similar vein to #60 and #61) I'd pass along a number of games that seem to corrupt the manifest when running py -m gogrepoc update (without a gog-manifest.dat) and the py -m download—related explicitly to null characters being present in the data file (which can be manually resolved by looking for the special regex character \0 and removing the entire game entry (e.g. from {'bg_url: to its corresponding right bracket }. The affected games are listed below the ######################### .

I have also noticed that many of the affected sections have excessively long changelog entries if that helps with diagnostics.

An excerpt of the series of outputs from running py -m gogrepoc download can be found here: https://pastebin.com/dwgKU9Jp

NOTE: In the pastebin ">>> preceded by an empty line is my redacted prompt

#########################

Cossacks Anthology: https://www.gog.com/en/game/cossacks_anthology
Cyberpunk 2077: https://www.gog.com/en/game/cyberpunk_2077
Gabriel Knight: Sins of the Fathers: https://www.gog.com/en/game/gabriel_knight_sins_of_the_fathers_20th_anniversary_edition
Geneforge 1: https://www.gog.com/en/game/geneforge_15
Planetscape - Torment: https://www.gog.com/en/game/planescape_torment_enhanced_edition
SpaceChem: https://www.gog.com/en/game/spacechem
Spelunky: https://www.gog.com/en/game/spelunky
Neverwinter Nights Diamond Edition: https://www.gog.com/en/game/neverwinter_nights_enhanced_edition_pack
Risen: https://www.gog.com/en/game/risen
Rise of the Triad: https://www.gog.com/en/game/rise_of_the_triad__dark_war and https://www.gog.com/en/game/rise_of_the_triad
Stellaris: https://www.gog.com/en/game/stellaris
Stronghold Crusader/HD: https://www.gog.com/en/game/stronghold_crusader
The Witcher: https://www.gog.com/en/game/the_witcher
The Witcher 2: Enhanced Edition (there seemed to be two entries for this game although only one was affected): https://www.gog.com/en/game/the_witcher_2
SteamWorld Dig: https://www.gog.com/en/game/steamworld_dig
Thea 2 The Shattering: https://www.gog.com/en/game/thea_2_the_shattering
Yooka Laylee: Impossible Lair: https://www.gog.com/en/game/yookalaylee_and_the_impossible_lair
Alan Wake: https://www.gog.com/en/game/alan_wake
Alan Wake: American Nightmare: https://www.gog.com/en/game/alan_wakes_american_nightmare

[Feature] Configuration file

It could be nice to have some configuration file (say, in ini format) that can be used to override some default variables automatically (e.g instead of using command-line options each time). Example options that may be tweaked this way:

  • Directory used to store games.
  • Default OS, for which game will be downloaded (win/mac/lin)
  • Should game's goodies be downloaded or not

The logic is following:

  • If config.ini doesnt exist, script makes one with default values
  • If config.ini exists, but not every necessary variable present there - for missing variables, script falls back to defaults
  • If config.ini contains incorrect variables - script throws error and sys.exit(1)
  • If config.ini exists and is correct - variables stored inside are used instead of script's defaults
  • If script has been launched with additional flags that affect the same things as config.ini - options described in flags will be used instead of options in config.ini

[QoL] Use .format() instead of %s and %d where possible

.format() is kinda common way to handle string formatting in python nowadays. Its way easier to read than many %s'es and %d'es and a bit more flexible.
For example, this:
info("not moving uncompleted download '%s', success: %s remaining bytes: %d / %d " % (downloading_path,str(succeed),sizes[path],sz))

Can be improved into something like this:
info("not moving uncompleted download {}, success: {} remaining bytes: {} / {} ".format(downloading_path,str(succeed),sizes[path],sz))

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.