Can protontricks be used with Non-Steam games? about protontricks HOT 19 CLOSED

I worked on implementing non-Steam shortcuts months back, but I couldn't figure out how Steam maps shortcuts to their corresponding Proton prefixes.

As far as I can tell, Steam uses a sequence of integers starting from 2147483651 for non-Steam Windows apps and their Proton prefixes. So, adding the first non-Steam Windows app will assign the first app with the ID 2147483651, the second with 2147483652 and so forth. This mapping can be determined (at first) by reading the user-specific shortcuts.vdf file, which has the currently existing shortcuts listed in order from the oldest to newest. This works until a shortcut is deleted, in which case the ID can no longer be determined by checking the position of the shortcut in the list.

I did some initial work on the non_proton_apps feature branch, if anyone wants to take a peek and maybe figure out how Steam assigns its shortcut-specific IDs. So far I've tried checking for Steam file changes using git and using strace to check for Steam's overall file system activity. This was months back, so Steam might have changed how this works for all I know.

from protontricks.

I worked on implementing non-Steam shortcuts months back, but I couldn't figure out how Steam maps shortcuts to their corresponding Proton prefixes.

As far as I can tell, Steam uses a sequence of integers starting from 2147483651 for non-Steam Windows apps and their Proton prefixes. So, adding the first non-Steam Windows app will assign the first app with the ID 2147483651, the second with 2147483652 and so forth. This mapping can be determined (at first) by reading the user-specific shortcuts.vdf file, which has the currently existing shortcuts listed in order from the oldest to newest. This works until a shortcut is deleted, in which case the ID can no longer be determined by checking the position of the shortcut in the list.

I did some initial work on the non_proton_apps feature branch, if anyone wants to take a peek and maybe figure out how Steam assigns its shortcut-specific IDs. So far I've tried checking for Steam file changes using git and using strace to check for Steam's overall file system activity. This was months back, so Steam might have changed how this works for all I know.

I'm in desperate need of this feature. Is it possible to try your changes by installing a specific version with pipx?

Thank you

from protontricks.

I'm in desperate need of this feature. Is it possible to try your changes by installing a specific version with pipx?

Thank you

The non_proton_apps branch still exists in the repository, but it's rather old and has the problems I mentioned. In worst case scenario, Protontricks will run winetricks on wrong Proton prefixes and leave you scratching your head as to why the commands don't seem to be working in-game.

If you still want to use it, you can try:

pipx install --force --spec git+https://github.com/Matoking/protontricks.git@non_proton_apps protontricks

I have more free time at the moment, so I'll try solving this problem again soon. No promises, though!

from protontricks.

I'm in desperate need of this feature. Is it possible to try your changes by installing a specific version with pipx?
Thank you

The non_proton_apps branch still exists in the repository, but it's rather old and has the problems I mentioned. In worst case scenario, Protontricks will run winetricks on wrong Proton prefixes and leave you scratching your head as to why the commands don't seem to be working in-game.

If you still want to use it, you can try:

pipx install --force --spec git+https://github.com/Matoking/protontricks.git@non_proton_apps protontricks

I have more free time at the moment, so I'll try solving this problem again soon. No promises, though!

That's awesome! I figured out how to do it with winetricks for now, so I'll take your advice and wait for the new version (hopefully!)

Thanks for all your hard work

from protontricks.

@ThatAnnoyedGuy
it would be awesome if you could share your findings :)

from protontricks.

I've tried tackling this issue again but I'm no closer to a solution than before. I also checked out whether Steam's D-Bus interface or Steam console (steam://open/console) would have anything helpful regarding this, but didn't find anything there.

from protontricks.

Avid gamer and hobby-dev here,
I recently made a fresh install on my Fedora 30 to clean up some stuff. While before the reinstall steam started the numbers with ...3646 my current install started with ...3650, so even that is unreliable.
What is reliable however if you monitor the compatdata folder during that first start, so you can get the number for that particular game and note it in the store-name. But while before my reinstall protontricks worked with that method, now it doesn't, says "You don't seem to have a game with that ID." even though the folder is there and filled.
Would be nice if protontricks would fallback to just calling winetricks on that folder in that case.

PS: since proton is open source, we may be able to fix at least the number association

from protontricks.

Seems like the main issue is the assignment of steam_app on line 237 of cli.py.

Given that by this point we have identified that the appid inputted has an entry in config.vdf and has a valid prefix setup in compatdata, can we not assume the application is valid at this point?

Furthermore, run_command of util.py only uses the steam_app found on the previous step to get the path of the prefix, could we not just pass the prefix directory directly?

I understand that steam_apps is iterated through to make sure the app actually exists and the prefix isn't just leftover files, but is this check needed? Maybe a flag could be set to ignore this step and pass the prefix directory path directly?

from protontricks.

Seems like the main issue is the assignment of steam_app on line 237 of cli.py.

Given that by this point we have identified that the appid inputted has an entry in config.vdf and has a valid prefix setup in compatdata, can we not assume the application is valid at this point?

Furthermore, run_command of util.py only uses the steam_app found on the previous step to get the path of the prefix, could we not just pass the prefix directory directly?

I understand that steam_apps is iterated through to make sure the app actually exists and the prefix isn't just leftover files, but is this check needed? Maybe a flag could be set to ignore this step and pass the prefix directory path directly?

The easiest workaround would be to just add "fake" Steam apps for compatdata directories that have the ID of 2147483651 or larger (it could be done in steam.py#603). This also has the benefit of working with the search functionality, though we'd have to give the apps unhelpful names like "Non-Steam shortcut: app ID 2147483651" instead of "Non-Steam shortcut: NonSteamGame.exe". And the user would still have to find out what prefix directory belongs to what shortcut.

I'm not very fond of this solution, but if there's demand this could get the issue solved in at least some way.

from protontricks.

I've added a new workaround in the branch non_steam_shortcuts_workaround. You can input the prefix directory's ID and it should work. The prefix directories are also included in the search results, although they're not very useful at this point.

You can install it with the command:

pipx install --force --spec git+https://github.com/Matoking/protontricks.git@non_steam_shortcuts_workaround protontricks

I haven't merged this into master yet, as I'd like to find a more complete solution to this issue first.

from protontricks.

I've added a new workaround in the branch non_steam_shortcuts_workaround. You can input the prefix directory's ID and it should work. The prefix directories are also included in the search results, although they're not very useful at this point.

You can install it with the command:

pipx install --force --spec git+https://github.com/Matoking/protontricks.git@non_steam_shortcuts_workaround protontricks

I haven't merged this into master yet, as I'd like to find a more complete solution to this issue first.

This solution works but will crash if there is a file present in compatdata. For example, I have a file called index so I can keep track of which prefix belongs to which non-steam game, and that gets picked up by as a compatdata_name which then gets passed into int(compatdata_name) in steam.py#597. This results in the following error:

ValueError: invalid literal for int() with base 10: 'index'

Perhaps this can be fixed with a simple

if not os.path.isdir(compatdata_path):
        continue

at the start of the loop?

from protontricks.

I've added a new workaround in the branch non_steam_shortcuts_workaround. You can input the prefix directory's ID and it should work. The prefix directories are also included in the search results, although they're not very useful at this point.
You can install it with the command:

pipx install --force --spec git+https://github.com/Matoking/protontricks.git@non_steam_shortcuts_workaround protontricks

I haven't merged this into master yet, as I'd like to find a more complete solution to this issue first.

This solution works but will crash if there is a file present in compatdata. For example, I have a file called index so I can keep track of which prefix belongs to which non-steam game, and that gets picked up by as a compatdata_name which then gets passed into int(compatdata_name) in steam.py#597. This results in the following error:

ValueError: invalid literal for int() with base 10: 'index'

Perhaps this can be fixed with a simple

if not os.path.isdir(compatdata_path):
        continue

at the start of the loop?

I fixed the branch; I was trying to catch the wrong exception (it's ValueError for strings that don't convert to integers, not TypeError), hence the error. Try installing the branch again.

from protontricks.

Works fine now, thanks.

from protontricks.

So I've done some digging and found that the list of non-steam games is stored in /steam/userdata/{userid}/config/shortcuts.vdf. This file includes the names and paths of each non-steam game, but doesn't seem to indicate whether a game is using proton or what the appid of the game is.

Checking if a game will use Proton is enough (check if the Exe prop ends with a .exe) but that still leaves the appid problem.

I found an old project that seems to understand how Steam generates appids for non-steam games here. If it's still the same then maybe we can compare the generated appids with the ones in compatdata and see which are proton vs which are not?

from protontricks.

So I've done some digging and found that the list of non-steam games is stored in /steam/userdata/{userid}/config/shortcuts.vdf. This file includes the names and paths of each non-steam game, but doesn't seem to indicate whether a game is using proton or what the appid of the game is.

Checking if a game will use Proton is enough (check if the Exe prop ends with a .exe) but that still leaves the appid problem.

I found an old project that seems to understand how Steam generates appids for non-steam games here. If it's still the same then maybe we can compare the generated appids with the ones in compatdata and see which are proton vs which are not?

Thanks for reminding me to look at that code again!

I checked that code months ago, but it generated 64-bit integers that weren't used by Steam for Wine prefixes at the time, only screenshots. That seems to have changed very recently though: the pseudo app IDs seem to be different now and don't follow the 2147483651 + N sequence anymore, and instead seem to derive the app ID from the identifier used for screenshots, which the function you linked is able to generate.

A non-Steam game I added just now got assigned the pseudo app ID/prefix name 2868032308. If I saved a screenshot for that game, it got saved into ~/.steam/steam/userdata/<steam_user_id>/760/remote/12318104966764953600/screenshots. And, most importantly, converting the 64-bit integer used for the screenshot to a 32-bit integer using a right shift seems to result in the same pseudo app ID. In other words: 12318104966764953600 >> 32 == 2868032308.

And since the code you linked allows us to calculate the 64-bit screenshot identifier (12318104966764953600), we now have a way to map the shortcuts to the Wine prefixes. I'll have to look into this tomorrow, but it looks like this issue might be solved soon enough, depending on how much effort the CRC checksum calculation takes.

from protontricks.

I've pushed a new fix to non_steam_apps. This should properly detect the non-Steam games as well as their shortcut names like so:

$ protontricks -s non-steam    
Found the following games:
Non-Steam shortcut: AM2R.exe (3392872573)
Non-Steam shortcut: Bubsy3d.exe (2182514201)
Non-Steam shortcut: Doukutsu.exe (4237170648)
Non-Steam shortcut: Dwarf Fortress.exe (2682319921)

To install it using pipx, you can run:

pipx install --force --spec git+https://github.com/Matoking/protontricks.git@non_steam_apps protontricks

I'll refine the branch tomorrow with some tests, but it would be great to hear if it is working for anyone else before doing that. :)

from protontricks.

I've pushed a new fix to non_steam_apps. This should properly detect the non-Steam games as well as their shortcut names like so:

$ protontricks -s non-steam    
Found the following games:
Non-Steam shortcut: AM2R.exe (3392872573)
Non-Steam shortcut: Bubsy3d.exe (2182514201)
Non-Steam shortcut: Doukutsu.exe (4237170648)
Non-Steam shortcut: Dwarf Fortress.exe (2682319921)

To install it using pipx, you can run:

pipx install --force --spec git+https://github.com/Matoking/protontricks.git@non_steam_apps protontricks

I'll refine the branch tomorrow with some tests, but it would be great to hear if it is working for anyone else before doing that. :)

Went ahead and tested this but got the following error:

$ protontricks -s non-steam
Traceback (most recent call last):
  File "/home/frank/.local/bin/protontricks", line 8, in <module>
    sys.exit(main())
  File "/home/frank/.local/pipx/venvs/protontricks/lib/python3.7/site-packages/protontricks/cli.py", line 148, in main
    steam_apps = get_steam_apps(steam_root, steam_lib_paths)
  File "/home/frank/.local/pipx/venvs/protontricks/lib/python3.7/site-packages/protontricks/steam.py", line 725, in get_steam_apps
    steam_apps += get_custom_windows_shortcuts(steam_root=steam_root)
  File "/home/frank/.local/pipx/venvs/protontricks/lib/python3.7/site-packages/protontricks/steam.py", line 672, in get_custom_windows_shortcuts
    target=shortcut_data["exe"], name=shortcut_data["AppName"]
KeyError: 'exe'

Looks to be a typo, so I capitalized the e in shortcut_data["exe"] in steam.py#672 and that fixed the crash 👍

from protontricks.

Looks to be a typo, so I capitalized the e in shortcut_data["exe"] in steam.py#672 and that fixed the crash +1

Thanks for the quick update.

My shortcuts.vdf file has every exe field in lowercase, but this might be the same deal as another VDF field that could be appID or appid depending on the version of Steam client that created it. Making the field names case-insensitive should fix that.

from protontricks.

I've merged the fix to master.

I'll look into releasing a new version soon enough, but you can already install the development version from master if you want to give it a try.

from protontricks.