sirwaddles / johnwickparse Goto Github PK

How do I package .json back to .uasset?

Hello. I'm trying to use the parser built from current latest sources because the release version fails to serialize some files in UE 4.25. I am trying to use it the same way as the release version, but I always keep getting File Error: The system cannot find the path specified. (os error 3). There is no stack trace, so I manually traced it to the following lines:

I would appreciate any help with this matter.

Is it in plans to support serializing blueprints?
Thanks for your work. Your parser made translating Octopath Traveler much easier :D

Hello! It looks like your parsing for FRichCurveKey is missing the "value" parameter, which causes arrays of these keys to get increasingly offset in the serializations. See, for instance, FModel's parsing of these: https://github.com/iAmAsval/FModel/blob/master/FModel/PakReader/Parsers/Objects/FRichCurveKey.cs#L26-L34

I believe this patch should do it:

diff --git a/src/assets.rs b/src/assets.rs
index 0fc6a34..23d1de3 100644
--- a/src/assets.rs
+++ b/src/assets.rs
@@ -1679,6 +1679,7 @@ struct FRichCurveKey {
     tangent_mode: u8,
     tangent_weight_mode: u8,
     time: f32,
+    value: f32,
     arrive_tangent: f32,
     arrive_tangent_weight: f32,
     leave_tangent: f32,
@@ -1692,6 +1693,7 @@ impl NewableWithNameMap for FRichCurveKey {
             tangent_mode: reader.read_u8()?,
             tangent_weight_mode: reader.read_u8()?,
             time: reader.read_f32::<LittleEndian>()?,
+            value: reader.read_f32::<LittleEndian>()?,
             arrive_tangent: reader.read_f32::<LittleEndian>()?,
             arrive_tangent_weight: reader.read_f32::<LittleEndian>()?,
             leave_tangent: reader.read_f32::<LittleEndian>()?,

(I can verify that that works for me on an older version of JWP -- I can't really test out on current JWP HEAD 'cause the more recent Fortnite changes have meant that JWP no longer works on the non-Fortnite paks that I've been working with)

Hello, is there any documentation to start the project ?

I'm trying to serialize functions from a 4.26.2 cooked uasset/uexp file pair, but am getting the following errors:

Did not read Function correctly. Current Position: 12, Bytes Remaining: 35
Did not read Function correctly. Current Position: 59, Bytes Remaining: 35
Did not read Function correctly. Current Position: 106, Bytes Remaining: 35
Did not read Function correctly. Current Position: 153, Bytes Remaining: 35
Did not read Function correctly. Current Position: 200, Bytes Remaining: 3857

No functions are serializing for any assets, though all other props and classes are serializing correctly. This issue was noted in #9 (comment) however the focus was on other, less useful properties, and was closed as a result.

I'm attempting to serialize a .uexp and .uasset pair forming a shader from Kingdom Hearts 3, but I hit this error:

Error: Property error occurred:
Export type: Material
Property Tag: MaterialAttributes (StructProperty) 36
Struct Type: MaterialAttributesInput
Property Tag: /Engine/Functions/Engine_MaterialFunctions01/Gradient/LinearGradient (/Engine/Functions/Engine_MaterialFunctions01/Gradient/LinearGradient) 0
Could not read property type: /Engine/Functions/Engine_MaterialFunctions01/Gradient/LinearGradient at pos 197

I've got the LinearGradient's .uexp and .uasset position in the directory hierarchy correct relative to the file I'm trying to serialize and even have the john-wick-parse.exe in the root directory in case it's looking relative to itself, but to no avail. I might just be using the tool wrong, but I can't find anything concerning what to do differently from searching around. I don't think it's anything specific to KH3 that's breaking since LinearGradient is a fairly standard material function, but I'm not entirely sure.

Did not read StructProperty correctly. Current Position: 12, Bytes Remaining: 24
Did not read BlueprintGeneratedClass correctly. Current Position: 164, Bytes Remaining: 116
Did not read Jason_SackHead_C correctly. Current Position: 1726, Bytes Remaining: -20
Error: Property error occurred:
Export type: CapsuleComponent
Property Tag: OnComponentBeginOverlap (MulticastDelegateProperty)
Could not read property type: MulticastDelegateProperty at pos 1936

Could somebody help me? It is uexp file

I know that #9 was closed as you didn't see purpose in the previous examples, but I have encountered issues when trying to serialize blueprints that have useful information in them that I would like to view, worth noting this is a different UE4 game.

Here are a few examples that fail to produce any serialization file:

Error: Property error occurred: Export type: BP_Base_Sword Property Tag: ItemSlotType (TextProperty) Could not read history type: 11

Error: Property error occurred: Export type: BP_Base_Pet Property Tag: ItemSubType (TextProperty) Could not read history type: 11

Error: Property error occurred: Export type: BP_Base_Pet Property Tag: StatRollData_Equipment (MapProperty) Could not read MapProperty with types: StructProperty StructProperty

And an example that doesn't display similar errors, but does manage to produce a serialization file but no substantial/meaningful data:

Did not read FloatProperty correctly. Current Position: 12, Bytes Remaining: 20 Did not read StructProperty correctly. Current Position: 44, Bytes Remaining: 24 Did not read Function correctly. Current Position: 80, Bytes Remaining: 131 Did not read BlueprintGeneratedClass correctly. Current Position: 223, Bytes Remaining: 84 Did not read Weapon_Boost correctly. Current Position: 651, Bytes Remaining: -20 Did not read FloatProperty correctly. Current Position: 643, Bytes Remaining: 20 Did not read FloatProperty correctly. Current Position: 675, Bytes Remaining: 20 Did not read FloatProperty correctly. Current Position: 707, Bytes Remaining: 20

Would it be possible to facilitate the parsing/serializing the blueprints of these?

Heyo, I know you guys develop this purely for Fortnite, but I'm trying to deserialize some tables. Some of the tables deserialize properly, but others are running into issues of Property Errors

john-wick-parse.exe serialize IconTable
Error: Property error occurred:
Property Tag: IconRarity (Int8Property) 1
Could not read property type: Int8Property at pos 159

john-wick-parse.exe serialize UserPlateBackgroundTable
Error: Property error occurred:
Property Tag: UserPlateBacgroundRarity (Int8Property) 1
Could not read property type: Int8Property at pos 144

john-wick-parse.exe serialize UnlockMusicTable
Error: Property error occurred:
Property Tag: AdaptStartTime (Int64Property) 8
Could not read property type: Int64Property at pos 107

I'll upload some example tables that aren't working, not sure if this is just unsupported data types or another issue. Would love if this was able to be supported, if not all good.

Examples: https://cyberc0re.s-ul.eu/F8dNlazm

Thanks <3

after extracting text.uexp file to .json file how to compress .json file back to text.uexp file?

While trying to extract pakchunk1000 I am met with the following error:

Error: Property error occurred:
String length too large (-884633750), likely a read error.

This is after running this command:

john-wick-parse.exe extract pakchunk1000-WindowsClient.pak ./extract

The key.txt in the following directory is correct.

Hello sir , I just downloaded your tool but I don't know how to use . I want to extract the uasset files so , I get material , static mesh and object shape.

Please help me sir as soon as possible
Thank you

I'm trying to extract files from Hi-Fi Rush. I know this program was made for Fortnite, but seeing as both games use some version of the Unreal Engine, I decided to give this a shot anyway.

Running filelist Hibiki-WindowsNoEditor.pak produces over 2 megabytes worth of file names, which indicates to me that the key I'm using is the correct one and that the .pak file is one that this program can parse.

The problems I'm experiencing are as follows:

• Running just .\john-wick-parse.exe extract .\Hibiki-WindowsNoEditor.pak predictably fails with Error: No pattern specified
• Running .\john-wick-parse.exe extract .\Hibiki-WindowsNoEditor.pak "*" does literally nothing
• Running .\john-wick-parse.exe extract .\Hibiki-WindowsNoEditor.pak "Engine/GlobalShaderCache-PCD3D_SM5.bin", with Engine/GlobalShaderCache-PCD3D_SM5.bin being a string taken verbatim from the generated file list, causes this error message:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 126, kind: Other, message: "The specified module could not be found." }', src\archives.rs:365:20
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I don't know Rust, but I'm guessing this means it can't find the oodle dll, and I'm not sure why, since that dll is right next to this program's exe:

PS C:\Program Files (x86)\Steam\steamapps\common\Hi-Fi RUSH\Hibiki\Content\Paks> dir


    Directory: C:\Program Files (x86)\Steam\steamapps\common\Hi-Fi RUSH\Hibiki\Content\Paks


Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a----          2/2/2023   6:58 PM        2144256 global.ucas
-a----          2/2/2023   6:59 PM           1465 global.utoc
-a----          2/2/2023   6:58 PM     5142665903 Hibiki-WindowsNoEditor.pak
-a----          2/4/2023   8:27 AM        2142136 Hibiki-WindowsNoEditor.pak.txt
-a----          2/2/2023   6:58 PM         314412 Hibiki-WindowsNoEditor.sig
-a----          2/2/2023   6:58 PM     3538816928 Hibiki-WindowsNoEditor.ucas
-a----          2/2/2023   6:58 PM       10342335 Hibiki-WindowsNoEditor.utoc
-a----          2/2/2023   6:58 PM      377417133 Hibiki-WindowsNoEditor_0_P.pak
-a----          2/2/2023   7:03 PM          23564 Hibiki-WindowsNoEditor_0_P.sig
-a----          2/2/2023   6:58 PM       13187104 Hibiki-WindowsNoEditor_0_P.ucas
-a----          2/2/2023   6:58 PM         607736 Hibiki-WindowsNoEditor_0_P.utoc
-a----         7/24/2020  12:00 AM        1147392 john-wick-parse.exe
-a----          2/4/2023   8:17 AM             64 key.txt
-a----         6/24/2022  12:10 PM        1007616 oo2core_8_win64.dll

I have only two games installed atm that make use of this library and I tried both versions (same file name, different file sizes), but I don't think the version's the problem.

Is what I'm trying even doable with this program?

Hello! This is from a serialization attempt on an object from Borderlands 3, not Fortnite, so feel free to chuck this in the bin. I'm pretty sure it's technically a more generic situation than just BL3, though, even if it might not be showing up in Fortnite data at the moment.

Anyway, I've got an object which fails to serialize, with the following output:

Error: Property error occurred: 
Export type: OakCharacterSoundLogicComponent
Property Tag: CharacterSoundData (MapProperty) 56
Could not read MapProperty with types: ObjectProperty ObjectProperty

OakCharacterSoundLogicComponent is clearly a BL3-specific class type, but the error's getting thrown because num_keys_to_remove isn't zero, in this block: https://github.com/SirWaddles/JohnWickParse/blob/master/src/assets.rs#L1718-L1721

As a test, I tried doing a little loop from 0..num_keys_to_remove in there, immediately after reading the value, calling this in each iteration:

read_map_value(reader, key_type, "StructProperty", name_map, import_map)

... and it allowed JWP to produce a serialization for me. Obviously this doesn't correctly process what it's being told to -- no idea what keys it's requesting be removed or whatever, but the serialization looks reasonable, and produces something at least.

Anyway, here's the files in question, if you're willing to take a peek:

https://apocalyptech.com/scratchpad/jwp/BPChar_PunkBadass.uasset
https://apocalyptech.com/scratchpad/jwp/BPChar_PunkBadass.uexp

... and the serialization once I put in my improper little hack there:

https://apocalyptech.com/scratchpad/jwp/BPChar_PunkBadass.json

Let me know if I can provide any more info! And again, totally understandable if you don't want to deal with this, of course. :) Cheers!

In my experience if index>0 it should still return import_map.get(index - 1)
My own parser works that way and your commit history also shows it was working like that in the past.. Is that an oversight? If not could you explain why the change?

Thanks.

It is possible use the serialize command for all the files inside a folder?

When running the parser, it seems to drop everything that follows consecutive underscores. e.g. I'm looking through an item database where most items show up as say ITM_078, ITM_079, &c, but others just show up as just ITM. I suspect there should a number associated with it, but it seems to be just "ITM" (not sure how I can send the uexp and uasset files to reproduce this).

Is there a way to preserve consecutive underscores and the contents that follow?

Thanks in advance! Love the work.

Hello! I hadn't updated my JWP checkout in some time, and the newer version (specifically since f498bca) doesn't seem to want to serialize for me at all. On the console it dies like so:

Error: Error: Property error occurred: 
File Error: No such file or directory (os error 2)

When run under strace, I can see that the last syscall before the abend is:

openat(AT_FDCWD, "paks/global.utoc", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

And when I add in some debugging in src/main.rs, it's definitely in this call that the process dies:

let mut dispatch = dispatch::Extractor::new("paks/global", None)?;

On the offchance that it would be okay with just a blank file, I did try creating a zero-length paks/global.utoc file, but that just made the app die in a different way. :)

Anyway, is there something else I've got to prepare, first, before running JWP? Thanks!

Reproducable asset example: CollectionBookSections

There should be several sections with the same name (as referenced in CollectionBookPages)
E.g. : 3 "weapons_Melee_Spears" sections.
Since the key is the same the key:object entry is replaced and show the latest entry only.
Perhaps a hasKey() check can be done and convert the the key:object to key:array instead and append the new object to said array.

Exporting with FModel shows all the sections since they are exported differently.

Using it via CLI in an automated manner, I'll manually export from FModel for now for these specific files.
Love the project, it's been really helpful in mining Save the world data. Thanks! :)

The file "CID_030_Athena_Commando_M_Halloween" (located in Athena\Items\Cosmetics\Characters) from Fortnite 7.4's files displays a File IO Error:

I've looked through the source code, but I'm unsure how to handle this error. It occurs when trying to parse "ParticleOptions", if that helps.