Giter Club home page Giter Club logo

cue4parse's Introduction

CUE4Parse - An Unreal Engine Archives & Packages Parsing Library in C#

NuGet Activity


Description:

CUE4Parse is a parsing library designed specifically for extracting data from archives and packages generated by Unreal Engine 4 and 5. It provides extensive support for parsing and processing native data classes such as UObject, UTexture2D, UAnimSequence, UStaticMesh, and many more. It also offers abstraction layers that enable developers to customize and type their own package formats, to allow them to work with packages that may have unique requirements or structures.

While primarily used and maintained by the FModel team for their various projects, contributions to the library are always welcome, particularly in the areas of memory efficiency, and execution time.

Quickstart:

Installation

Add the NuGet package to your project:

dotnet add package CUE4Parse

Alternatively, you can clone the repository and include it in your project as a reference:

git clone https://github.com/FabianFG/CUE4Parse.git --recursive

Example

var provider = new DefaultFileProvider(ARCHIVE_DIRECTORY_HERE, SearchOption.TopDirectoryOnly, true, new VersionContainer(EGame.GAME_UE4_27));
provider.Initialize(); // will scan the archive directory for supported file extensions

var allObjects = provider.LoadAllObjects(PACKAGE_PATH_HERE); // {GAME}/Content/Folder1/Folder2/PackageName.uasset
var fullJson = JsonConvert.SerializeObject(allExports, Formatting.Indented);

var obj = provider.LoadObject(OBJECT_PATH_HERE); // {GAME}/Content/Folder1/Folder2/PackageName.ObjectName
var objJson = JsonConvert.SerializeObject(objectExport, Formatting.Indented);

switch (obj)
{
    case UTexture2D texture:
    {
        var bitmap = texture.Decode(ETexturePlatform.DesktopMobile);
        ...
    }
    case USoundWave:
    {
        objectExport.Decode(true, out var audioFormat, out var data);
        ...
    }
    case UStaticMesh:
    case USkeletalMesh:
    case UAnimSequence:
    {
        var toSave = new Exporter(objectExport, EXPORT_OPTIONS_HERE);
        var success = toSave.TryWriteToDir(SAVE_DIRECTORY_INFO_HERE, out var label, out var savedFilePath);
        ...
    }
    default:
    {
        ...
    }
}

Further detailed documentation is available in the wiki

License:

CUE4Parse is licensed under Apache License 2.0, and licenses of third-party libraries used are listed here.

cue4parse's People

Contributors

4sval avatar adainrivers avatar amrsatrio avatar bmarquez1997 avatar entraptaa avatar eof-1141 avatar ezfndev avatar fabianfg avatar floxay avatar gmatrixgames avatar goomiiv2 avatar halfuwu avatar hypermodule avatar jacocococo avatar ka1serm avatar kein avatar klukule avatar longerwarrior avatar lukefz avatar minshug avatar narknon avatar nitrog0d avatar notofficer avatar redhaze avatar tamely avatar tectors avatar thoo0224 avatar will747 avatar xtigerhyperx avatar yretenai avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cue4parse's Issues

Publish CUE4Parse as a NuGet package

Most third-party .NET libraries are distributed as NuGet packages, which are easy for developers to discover and include in their projects. NuGet also saves consumers the trouble of configuring a special build environment in order to use one library with unusual build requirements.

Publishing CUE4Parse on NuGet would mean developers can add it to their projects with a single PackageReference line, or with a few clicks by searching for it in the package manager, without having to use submodules or install CMake into their PATH.

I've modified CUE4Parse to build NuGet packages, which are currently published on NuGet.org under an "unofficial" name. I can clean that up and submit a PR, if you'd like, but the code changes are pretty small -- I think the real work will be in numbering releases and uploading them, or writing actions to do that automatically.

Also, NuGet recommends specifying the package's license: see #39.

S.T.A.L.K.E.R. 2

Hello, you answered me earlier in the Fmodel topic, but can you tell me what kind of Mappings.usmap file from Stalker2 is? And are there any options for unpacking pak and utoc, ucas files for S.T.A.L.K.E.R. 2.

Error saying there is no game file with a certain path even though it clearly exists

I've been having this issue for days and have absolutely no clue how to fix it, i've been trying to load "FortniteGame/Content/Characters/Player/Female/Medium/Bodies/F_MED_Astronaut_01/Meshes/Parts/F_Med_Astronaut_01_Helmet.uasset" but it just keeps saying that there is no game file with path (path above). The paks with the file are loaded, the main key is submitted, I have no idea why it would be happening.

image

Discrepancy between GLTF and PSK Mesh Export

It seems that there's a difference between the vertices & edges between meshes exported as GLTF and PSK.

image
image
image
The meshes look pretty much identical in terms of where vertices are yet they differ in vertex and edge count.

ezgif com-gif-maker
It seems like there are some double vertices that aren't carrying over from gltf to psk. Is this a limitation of psk or is there some other reason for this? Would be super useful to get the psk data near identical to gltf so that morph target data would be usable on them.

Animation Curve Value Calculated Incorrectly

Hello, today I decided to build the latest dev version of FModel, which includes the last modification of the curves (c6a6e8b).

I went ahead to the ROM animation that includes tons of facial curves, I then imported all the curves into Unreal Engine to get a real showcase of the curves.

Since the animation curves had a picture online from a Epic Games Documentation page, I decided I wanted to compare photos to give you a better look of what's wrong.

image

The values are not stable, they are right for one key, then the next one is just incorrect. Basically to explain as much as I know.

For example:

  • First Key:
    Value = .4
    Time = .1

  • Next Key:
    Value = .01
    Time = .133333

I think you may be able to see the problem here, the next key is not correct, the time or value is incorrect, I presume the value is incorrect because with mocap it usually updates fast with the person's face. If you want a side by side video of their animation compared to the FModel curve animation, I am able to provide you with it.

Also to note: The implementation worked completely fine before, not in UE5 of course, but that isn't the source problem. These curves weren't working either in UE4 so we can remove the cause of the Engine version in that sense.

If you want to see the difference from before and now of the JSON files I am going to provide you with them, check below.

Comparison.zip

The values seem to differ a ton, look at them yourself. It seems like many decimals are being stripped from the value.

SubmitKeysAsync method crashing program

Hello, my program dies on this line of code in the SubmitKeysAsync method of AbstractVfsFileProvider (i dont get an exception either)

var completed = await Task.WhenAll(tasks).ConfigureAwait(false);

I have not modified any of the code in the method and I am calling it like this (all parent calls are awaited as well):

await provider.SubmitKeyAsync(new FGuid(), new FAesKey(_AES.MainKey)); //and yes im sure that _AES.MainKey isnt null

im using a StreamedFileProvider and this issue happens when running my program on Debian

my program is also on .NET Core 5

thanks

Can't read the files from pak file

I tried your example, and i searched around your code, but can't find out why it can't find the files inside the pak.
Maybe i missed some settings?
The game is built on 4.24.3, so i used GAME_UE4_24.
No encription on the files.

image

Tower of Fantasy 2.2 error

Hi!

Since I saw ToF is in EGame.cs, I thought I will put this here.
With the 2.2 patch the .pak files got changed yet again, whenever you try to open any uasset file you get this error:
Invalid uasset magic: 0x{Tag:X8} != 0x{PACKAGE_FILE_TAG:X8}, because Tag != PACKAGE_FILE_TAG && Tag != PACKAGE_FILE_TAG_SWAPPED.
Global is only using the 2.0 CN files so it currently works, but when they decide to swap to the 2.2 client it will be an issue over there as well. I tried to use the old method of getting the offset, but now it doesn't work anymore as it's not just simply xoring the numbers with an offset (at least I think it's not).
What I found weird is that umodel works just fine even with 2.2.

~PerkyJerky
image
image
image

Where can I find the code responsible for parsing .umap files?

I am looking to add .umap reading functionality into my own program, so it would be helpful for me to be able to find and study the code specifically responsible for loading the .umap files in CUE4Parse. I already have some .umap files extracted from Fortnite, but now I want to be able to read the height maps from the .umap files. Could someone here point me to which file in the source code tree contains the .umap parsing code?

Exporting Additives (how it should be)

Okay, you know exporting animations (normal) work and are perfectly fine and they are great, however something that has always been on my mind is additives and how the behavior of exporting them are.

Today, let's say you want to export the following animation from the game Fortnite:

Path: FortniteGame/Content/Animation/Game/MainPlayer/Locomotion/Medium/Male/Jog/Melee_TH/M_Jog_Melee_Twohanded_N
Additive Base Pose: FortniteGame/Content/Animation/Game/MainPlayer/Locomotion/Medium/Male/Jog/Melee_TH/M_Jog_Melee_Twohanded_N_Core

Now, when we export the additive, the bones are broken and miss-placed, for demonstration purposes:
image

And this is how the core animation looks like:
image

My theory is that the additives are exported wrong (obviously), but before telling you my theory let me explain to you what additives are in the first place (if you weren't aware).

Additives are basically a addition to a animation, to further add and possibly reuse for different purposes. Let's say you have a animation like a walking animation for the lower-body part of the game (for the legs walking part) and you have the upper-body moving along as the animation. What additives do is to take in a pose, for example the lower-body part with no upper-body movement and then have the same animation but contains the upper-body. They do this to have better performance and also being able to use the same additives across all different types of poses. They have a building tool pose animation, but have no additives and core animations? Well they just straight up use the Melee Twohanded additives.

Please tell me if you don't understand the above, you can read more about it on the internet. If you didn't get it already, a additive contains the core animation but adds to that core animation.

My theory is that you have to add the core animation to additive animation to have the final animation.

NightCorws uasset open failed

NightCrows.zip
NightCorws uasset to json failed,UE Version 5.1
UsmapParser.cs need to modify to read mappings.usmap.
var enumCount = Ar.Read();
var enums = new Dictionary<string, Dictionary<int, string>>((int) enumCount);
for (var i = 0; i < enumCount; i++)
{
var enumName = Ar.ReadName(nameLut)!;

            //var enumNamesSize =  Ar.Read<byte>();
            var enumNamesSize = Ar.Read<UInt16>();
            var enumNames = new Dictionary<int, string>(enumNamesSize);
            for (var j = 0; j < enumNamesSize; j++)
            {
                var value = Ar.ReadName(nameLut)!;
                enumNames[j] = value;
            }

            enums.Add(enumName, enumNames);
        }

maybe the failed reason is here:
Package.cs:
// We don't support loading script packages, so just return a fallback
if (outerMostImport.ObjectName.Text.StartsWith("/Script/"))
{
return new ResolvedImportObject(import, this);
}

Curves being defined Two Times

The curves are being set in the Properties object and outside of the Properties object making it have two places where it's being set.

sample:

[
  {
    "Type": "AnimSequence",
    "Name": "M_Jog_NoWep_E_Core",
    "Properties": {
      "BoneCompressionSettings": {
        "ObjectName": "AnimBoneCompressionSettings FortniteAnimBoneCompressionSettings",
        "ObjectPath": "/Game/Animation/Settings/FortniteAnimBoneCompressionSettings.2"
      },
      "CurveCompressionSettings": {
        "ObjectName": "AnimCurveCompressionSettings DefaultAnimCurveCompressionSettings",
        "ObjectPath": "/Engine/Animation/DefaultAnimCurveCompressionSettings.1"
      },
      "RetargetSource": "MPR_SK_M_MALE_Base_Skeleton",
      "AuthoredSyncMarkers": [
        {
          "MarkerName": "LeftPlant",
          "Time": 0.015393
        },
        {
          "MarkerName": "RightPlant",
          "Time": 0.336807
        }
      ],
      "TargetFrameRate": {
        "Numerator": 30
      },
      "SequenceLength": 0.6,
      "RawCurveData": {
        "FloatCurves": [
          {
            "FloatCurve": {
              "Keys": [
                {
                  "InterpMode": "RCIM_Cubic",
                  "TangentMode": "RCTM_Auto",
                  "TangentWeightMode": "RCTWM_WeightedNone",
                  "Time": 0.6,
                  "Value": 1.0,
                  "ArriveTangent": 0.0,
                  "ArriveTangentWeight": 0.33333334,
                  "LeaveTangent": 0.0,
                  "LeaveTangentWeight": 0.33333334
                }
              ],
              "DefaultValue": 3.4028235E+38,
              "PreInfinityExtrap": "ERichCurveExtrapolation::RCCE_Constant",
              "PostInfinityExtrap": "ERichCurveExtrapolation::RCCE_Constant"
            },
            "Name": {
              "DisplayName": "blendOrient1"
            },
            "CurveTypeFlags": 6
          },
          {
            "FloatCurve": {
              "Keys": [
                {
                  "InterpMode": "RCIM_Cubic",
                  "TangentMode": "RCTM_Auto",
                  "TangentWeightMode": "RCTWM_WeightedNone",
                  "Time": 0.6,
                  "Value": 1.0,
                  "ArriveTangent": 0.0,
                  "ArriveTangentWeight": 0.33333334,
                  "LeaveTangent": 0.0,
                  "LeaveTangentWeight": 0.33333334
                }
              ],
              "DefaultValue": 3.4028235E+38,
              "PreInfinityExtrap": "ERichCurveExtrapolation::RCCE_Constant",
              "PostInfinityExtrap": "ERichCurveExtrapolation::RCCE_Constant"
            },
            "Name": {
              "DisplayName": "blendParent1"
            },
            "CurveTypeFlags": 6
          },
          {
            "FloatCurve": {
              "Keys": [
                {
                  "InterpMode": "RCIM_Cubic",
                  "TangentMode": "RCTM_Auto",
                  "TangentWeightMode": "RCTWM_WeightedNone",
                  "Time": 0.6,
                  "Value": 1.0,
                  "ArriveTangent": 0.0,
                  "ArriveTangentWeight": 0.33333334,
                  "LeaveTangent": 0.0,
                  "LeaveTangentWeight": 0.33333334
                }
              ],
              "DefaultValue": 3.4028235E+38,
              "PreInfinityExtrap": "ERichCurveExtrapolation::RCCE_Constant",
              "PostInfinityExtrap": "ERichCurveExtrapolation::RCCE_Constant"
            },
            "Name": {
              "DisplayName": "blendPoint1"
            },
            "CurveTypeFlags": 6
          }
        ]
      },
      "Skeleton": {
        "ObjectName": "Skeleton Fortnite_M_Avg_Player_Skeleton",
        "ObjectPath": "/Game/Characters/Player/Male/Male_Avg_Base/Fortnite_M_Avg_Player_Skeleton.90"
      }
    },
    "SkeletonGuid": "AEBDD026-4AFC553B-E052608F-2FCAFC9C",
    "CompressedTrackToSkeletonMapTable": [
      0,
      86,
      1,
      2,
      3,
      4,
      5,
      6,
      7,
      8,
      9,
      10,
      11,
      12,
      13,
      14,
      15,
      16,
      17,
      18,
      19,
      20,
      21,
      22,
      23,
      24,
      25,
      26,
      27,
      28,
      29,
      72,
      74,
      75,
      76,
      251,
      32,
      33,
      34,
      35,
      36,
      37,
      38,
      39,
      40,
      41,
      42,
      43,
      44,
      45,
      46,
      47,
      48,
      49,
      50,
      51,
      52,
      53,
      54,
      73,
      77,
      78,
      79,
      252,
      57,
      58,
      59,
      60,
      61,
      80,
      81,
      62,
      63,
      82,
      66,
      67,
      83,
      84,
      68,
      69,
      85,
      87,
      88,
      89,
      90,
      91,
      92,
      93,
      268,
      269,
      270,
      271,
      272,
      273,
      274,
      277,
      278
    ],
    "CompressedCurveNames": [
      {
        "DisplayName": "blendOrient1"
      },
      {
        "DisplayName": "blendParent1"
      },
      {
        "DisplayName": "blendPoint1"
      }
    ],
    "CompressedCurveData": {
      "FloatCurves": [
        {
          "FloatCurve": {
            "Keys": [
              {
                "InterpMode": "RCIM_Linear",
                "TangentMode": "RCTM_Auto",
                "TangentWeightMode": "RCTWM_WeightedNone",
                "Time": 0.0,
                "Value": 1.0,
                "ArriveTangent": 0.0,
                "ArriveTangentWeight": 0.0,
                "LeaveTangent": 0.0,
                "LeaveTangentWeight": 0.0
              }
            ],
            "DefaultValue": 3.4028235E+38,
            "PreInfinityExtrap": "RCCE_Constant",
            "PostInfinityExtrap": "RCCE_Constant"
          },
          "Name": {
            "DisplayName": "blendOrient1"
          },
          "CurveTypeFlags": 4
        },
        {
          "FloatCurve": {
            "Keys": [
              {
                "InterpMode": "RCIM_Linear",
                "TangentMode": "RCTM_Auto",
                "TangentWeightMode": "RCTWM_WeightedNone",
                "Time": 0.0,
                "Value": 1.0,
                "ArriveTangent": 0.0,
                "ArriveTangentWeight": 0.0,
                "LeaveTangent": 0.0,
                "LeaveTangentWeight": 0.0
              }
            ],
            "DefaultValue": 3.4028235E+38,
            "PreInfinityExtrap": "RCCE_Constant",
            "PostInfinityExtrap": "RCCE_Constant"
          },
          "Name": {
            "DisplayName": "blendParent1"
          },
          "CurveTypeFlags": 4
        },
        {
          "FloatCurve": {
            "Keys": [
              {
                "InterpMode": "RCIM_Linear",
                "TangentMode": "RCTM_Auto",
                "TangentWeightMode": "RCTWM_WeightedNone",
                "Time": 0.0,
                "Value": 1.0,
                "ArriveTangent": 0.0,
                "ArriveTangentWeight": 0.0,
                "LeaveTangent": 0.0,
                "LeaveTangentWeight": 0.0
              }
            ],
            "DefaultValue": 3.4028235E+38,
            "PreInfinityExtrap": "RCCE_Constant",
            "PostInfinityExtrap": "RCCE_Constant"
          },
          "Name": {
            "DisplayName": "blendPoint1"
          },
          "CurveTypeFlags": 4
        }
      ]
    },
    "CompressedDataStructure": {
      "CompressedNumberOfFrames": 19
    },
    "BoneCompressionCodec": {
      "ObjectName": "AnimBoneCompressionCodec_ACL FortniteAnimBoneCompressionSettings:AnimBoneCompressionCodec_ACL_0",
      "ObjectPath": "/Game/Animation/Settings/FortniteAnimBoneCompressionSettings"
    },
    "CurveCompressionCodec": {
      "ObjectName": "AnimCurveCompressionCodec_CompressedRichCurve DefaultAnimCurveCompressionSettings:CurveCompressionCodec",
      "ObjectPath": "/Engine/Animation/DefaultAnimCurveCompressionSettings"
    },
    "CompressedRawDataSize": 17572
  }
]

As you can see from above, it's being defined two times doubling the size of the saved JSON for facial animations.

How to build

Hey, can you do a tutorial for build it please, I have a lot of cmake errors

image

Animation Blueprints Nodes (not working on latest) [UE5]

Since the release of Unreal Engine 5, all animation blueprints are random when it comes to showing the nodes, sometimes the animation blueprint shows like 2 nodes out of 500 and sometimes doesn't show any nodes.

Usually the nodes show if the animation blueprint wasn't opened by the Unreal Editor (from the creator's perspective), and would only show the nodes that weren't modified from UE4 to UE5.

UE5 changed something related to animation blueprint nodes, and we can't see anything in them. I dearly ask for one of the contributors of CUE4Parse to fix this as this would be such a great help for me and others.

Here below is a preview of the changes that happened between UE4 to UE5 related to the JSON output of CUE4Parse.


This is exported using FModel, but that won't change anything related to the JSON as it uses CUE4Parse to parse assets.

Unreal Fortnite Version
UE4_25 Fortnite Version 12.41
UE5_1 Fortnite Version 20.30
Asset
Characters/Player/Male/Medium/Bodies/M_Med_Soldier_04/Meshes/SK_M_Med_Soldier_04_Skeleton_AnimBP

Comparison_AnimBP_UE4_to_UE5.zip


Unreal Engine 4.25.0

The animation blueprint provides every single bit of animation nodes and leaves out nothing.


Unreal Engine 5.1.0

The animation blueprint leaves out almost to no nodes, only a copy pose from mesh node (which is utterly useless).


Conclusion

In conclusion, the animation blueprints change between UE4 to UE5 removing all animation nodes and anything valuable to use, causing sometimes a empty object or one node in the output JSON.

Potionomics - Unhandled exception of type 'System.ExecutionEngineException' in Unknown module.

On "Potionomics/Content/Blueprints/BP_PlayerController.uasset" FModel crashed with error "Unhandled exception of type 'System.ExecutionEngineException' in Unknown module."

Code to reproduce the error:

var exports = Provider.LoadAllObjects(fullPath); // fullPath = "Potionomics/Content/Blueprints/BP_PlayerController.uasset"

foreach (var obj in exports) // <= An error occurs in the third step.
{
    System.Diagnostics.Debug.WriteLine(obj.ToString());
}

In debugging, after an error occurs, Visual Studio does not display the call stack.
In manual debugging, the error occurs at this line (...\CUE4Parse\CUE4Parse\UE4\Assets\AbstractUePackage.cs):
public IEnumerable GetExports() => ExportsLazy.Select(x => x.Value);

FModel uses CUE4Parse version 1.0.7.0, I updated to the latest 1.0.9.0, but the error persists.
I can’t find a place on my own to localize and at least get around the cause.

Problematic asset in the attachment:
Potionomics.zip

No curves are being shown in animations on Latest Fortnite

When the current version of Fortnite dropped, the curves for animations do not work and do not show up either, however montage's curves work?

Here is before the current version of Fortnite:
image

And here is the current version of Fortnite with the same animation exported:
image

The animation I used for the photos above was: Idle_Turn_Left_Additive (FortniteGame/Content/Animation/Game/MainPlayer/Locomotion/Small/idle)

The whole "CompressedCurveData" object is gone!

Please fix this as soon as possible, thank you for reading.

Help

How can I create a export folder and Export files using Cue4parse?

Cannot decode UTexture2D

Hello, I'm trying to export a texture icon as UTexture2D, and when calling Decode() it throws this exception:
System.ArgumentNullException: Value cannot be null. (Parameter 'source') at System.Linq.ThrowHelper.ThrowArgumentNullException(ExceptionArgument argument) at System.Linq.Enumerable.TryGetFirst[TSource](IEnumerable 1 source, Func 2 predicate, Boolean& found) at System.Linq.Enumerable.FirstOrDefault[TSource](IEnumerable 1 source, Func 2 predicate)

The exception is thrown at line 113 of UTexture2D.cs

And this is how I'm loading the texture:

var texture = provider.LoadObject(@"FortniteGame/Content/UI/Foundation/Textures/Icons/Backpacks/T-Icon-Backpacks-773-Carabus-L");

var image = texture.Decode();

Thank you, I hope I was able to be descriptive on my issue, and hopefully it's just me doing something wrong.

Exporting and looping over exports causes Segmentation fault

I made a program that loops over every map and saves all the objects i want. This worked without any problems in earlier Fortnite versions. But since the 21.0 Fortnite update when loading the map fortnitegame/plugins/gamefeatures/parallelgameplay/content/leveloverlays/artemis_poi_dailybugle_002_paralleloverlay.umap and then looping over the exports using foreach throws some System.OutOfMemoryException which is then followed by a segmentation fault. The program uses about 1gb of ram and my system has more than 10gb available.

I already used the newest patches that were commited on June 5th.

My code

var provider = new DefaultFileProvider(_gameDirectory, SearchOption.TopDirectoryOnly, true, new VersionContainer(EGame.GAME_UE5_LATEST));

provider.Initialize(); 
provider.SubmitKey(new FGuid(), new FAesKey(_aesKey));
provider.LoadMappings(); 

provider.TryLoadPackage("fortnitegame/plugins/gamefeatures/parallelgameplay/content/leveloverlays/artemis_poi_dailybugle_002_paralleloverlay.umap", out var package);

var exports = package.GetExports();

foreach (var export in exports) { // here is where it crashes
}

Dotnet version: 6.0.202
OS: Ubuntu 20.04.4

Parsing fails on Asteria_Terrain.umap

since the newest season its not possible to read the map (/Athena/Asteria/Maps/Asteria_Terrain.umap) anymore because it just crashes the process with an overflow exception. i think the cause for this problem seems to be here

CUE4Parse/UE4/Assets/IoPackage.cs#104

if (uassetAr.Game != EGame.GAME_UE5_2 || bulkDataMapSize < 65535) // Fortnite moment

using ue5_1 cant parse it and using ue5_2 can parse it if i remove that line. not sure if that just needs to be removed because fortnite still is on 5.1.1. maybe there just needs to be a 5_1_1 parsing option?

Mesh Vertex Paint Data (.umap)

"Painted Vertices" data is missing, the data shows up in the UEFN (patched) editor once you copy a mesh, but in CUE4Parse the data isn't read.

An example of this can be seen in the S4 Lobby (/Game/Maps/UI/Frontend_Lobby_Season4) on latest (patched UEFN):

This mesh has vertices painted on.
image

Once we copy it, and look in notepad, we see this data:
image
image

As you can see, this is the vertex data from the mesh into UE3 text data, meaning the data is present inside of the level
Let's take a look inside FModel (that utilizes CUE4Parse):

Here is the same mesh you saw above:
image

And here is the data provided:
image

As you can see, no painted vertices data, but it does exist

Resources that may help you:
https://github.com/EpicGames/UnrealEngine/blob/a3cb3d8fdec1fc32f071ae7d22250f33f80b21c4/Engine/Source/Runtime/Engine/Classes/Components/StaticMeshComponent.h#L786
https://github.com/EpicGames/UnrealEngine/blob/a3cb3d8fdec1fc32f071ae7d22250f33f80b21c4/Engine/Source/Runtime/Engine/Private/Components/StaticMeshComponent.cpp#L1451

Locres

I think im leaving a question rn idk.

How can I parse locres files? is that a thing here

New Wiki Page - Linux

I got CUE4Parse working under Linux and thought these instructions might be useful to add to the Wiki:


Linux

These steps walk you through creating a dotnet project on Linux that can use CUE4Parse.

dotnet-sdk

Install the dotnet-sdk from Microsoft. This differs depending on your Linux distro.
With Gentoo Linux: emerge dotnet-sdk-bin
Once running dotnet --version returns something like 7.0.200 you should be all set.

Unreal Engine

CUE4Parse uses Oodle externally to decompress data from PAK files. The Unreal Engine provides this file. Some Linux distros may have an easy way to install this, but here is how to build it from source:

git clone [email protected]:EpicGames/UnrealEngine.git -b ue5-main
cd UnrealEngine
./Setup.sh
./GenerateProjectFiles.sh
make UnrealPak

detex

CUE4Parse uses detex to convert textures. Here is how to build it from source:

git clone https://github.com/hglm/detex
cd detex

# Modify Makefile.conf and change:
LIBRARY_CONFIGURATION = SHARED

# Modify detex.h the declaration for detexDecompressTextureLinear so it's loadable externally from CUE4Parse
# Replace the prefix DETEX_API with DETEX_HELPER_SHARED_EXPORT
DETEX_HELPER_SHARED_EXPORT bool detexDecompressTextureLinear(const detexTexture *texture, uint8_t *pixel_buffer,
    uint32_t pixel_format);

# Now build it:
make

CUE4Parse

Clone the CUE4Parse repo:

git clone https://github.com/FabianFG/CUE4Parse.git --recursive

Your Program

Create a new dotnet console program:

mkdir YourProgram
cd YourProgram
dotnet new console

# CUE4Parse uses SkiaSharp for Texture Bitmaps:
dotnet add package SkiaSharp.NativeAssets.Linux

Add a reference to CUE4Parse to YourProgram.csproj:

<ItemGroup>
	<ProjectReference Include="../CUE4Parse/CUE4Parse-Conversion/CUE4Parse-Conversion.csproj" />
	<ProjectReference Include="../CUE4Parse/CUE4Parse/CUE4Parse.csproj" />
</ItemGroup>

Build and run. You should see output: Hello, World!

dotnet build
dotnet run

Copy the Oodle and detex libs to a place that dotnet can find them:

cp /path/to/UnrealEngine/Engine/Source/Programs/Shared/EpicGames.Oodle/Sdk/2.9.3/linux/lib/liboo2corelinux64.so.9 ./bin/Debug/net7.0/oo2core_9_win64.dll
cp /path/to/detex/libdetex.so.0.1.2 ./bin/Debug/net7.0/Detex.dll

Update Program.cs with your program code, using CUE4Parse as you see fit.

Example code that extracts all assets from Pak files: https://gist.github.com/Sembiance/d4d65671a87c586286093d3c64beb29d

dotnet build
dotnet run --aesKey=0x0DEADBEEF0CAFEBABE0 --pakDir=/path/to/Game/UE_game/Content/Paks/ --outDir=/path/to/out/

Exported PSA Sequence data has incorrect FirstRawFrame value

Hi there, I maintain a Blender addon for importing PSK and PSA files.

Someone reported a bug (DarklightGames/io_scene_psk_psa#46) that they were unable to import PSA files that were exported out of FModel (which uses CUE4Parse).

I believe I tracked down the issue.

FirstRawFrame = sequence.NumFrames, // useless, but used in UnrealEd when importing

Here, the FirstRawFrame is being assigned the same value as the NumRawFrames. This is incorrect, as the FirstRawFrame is meant to be the frame index that the animation begins at.

It is possible to work around this by just calculating the correct starting frame after reading in all the sequence data, but it would be better to have this fixed since I believe any PSA files exported out of CUE4Parse would be incompatible with older versions of the Unreal engine.

For reference, you can see how I handle calculating the FirstRawFrame value in my addon:

https://github.com/DarklightGames/io_scene_psk_psa/blob/master/io_scene_psk_psa/psa/builder.py#L170C31-L170C31
https://github.com/DarklightGames/io_scene_psk_psa/blob/master/io_scene_psk_psa/psa/builder.py#L197

[FR] Support for deserializing 5.1+ maps with WorldPartition

UE version: 5.1.1
Example map: https://pomf2.lain.la/f/ddhmwfxw.7z

Currently it fails at IOEntry reader:

OverflowException: Array dimensions exceeded supported range.
This exception was originally thrown at this call stack:
    CUE4Parse.UE4.IO.IoStoreReader.Read(long, long) in IoStoreReader.cs
    CUE4Parse.UE4.IO.IoStoreReader.Extract(CUE4Parse.UE4.VirtualFileSystem.VfsEntry) in IoStoreReader.cs
    CUE4Parse.UE4.IO.Objects.FIoStoreEntry.Read() in FIoStoreEntry.cs
    CUE4Parse.UE4.IO.Objects.FIoStoreEntry.CreateReader() in FIoStoreEntry.cs
    ```

Parsing fails on Asteria_Terrain.umap

CUE4Parse\CUE4Parse\UE4\Assets\Exports\UObject.cs
...
if (!Flags.HasFlag(EObjectFlags.RF_ClassDefaultObject) && Ar.ReadBoolean() && Ar.Position + 16 <= validPos)
...
Ar.ReadBoolean() is causing this error:

CUE4Parse.UE4.Exceptions.ParserException: Invalid bool value (1230987625)
FAssetArchive Info: FortniteGame/Content/Athena/Asteria/Maps/Asteria_Terrain.umap Pos 7751917 Length 149233731

IO store support?

I see some references to IOstore in the code, but can't figure out if it is fully supported and if so, is there a way to read IOstore packages. Right now, filelist from DefaultFileProvider returns me only what inside the PAKs, which is half of the content.
The rest is inside IOstore packages. Can these be read to get filelist and object's exports?

[FR] Loading UMap from old ue4 4.8.3 ver

Im trying to open umap from unreal engine 4.8.3 and i got this error:

[ERR] One or more errors occurred. (Invalid FString length '1768714054')
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)

If it possible add FString support for 4.8.3

Extracting Cosmetics data

I am trying to extract a character from Fortnite files using the example, but I am having the same problem on my main project.

I leave below the output and code of the example as I modified it. Let me know if I am doing something wrong or it is a parser problem.

image
image

Animation Curves

When I try print out the JSON when using the script, it does not include any information related to animation curves.

UE4Config not found

I got System.IO.FileNotFoundException: "Could not load file or assembly 'UE4Config" error, how i can fix?

Night Crows

FileName:UsmapParser.cs
// var enumNamesSize = Ar.Read();//byte
var enumNamesSize = Ar.Read();//int16

when I debug this game,I found the enumNamesSize is 0x16c
Mappings.zip
UE5.1

PSK morph data is exported incorrectly for meshes with overlapping vertices

I had a report for this on my addon: DarklightGames/io_scene_psk_psa#49

To sum it up, the person reporting the bug has a model exported out of FModel that has overlapping vertices which also has morph data.

The code that exports the morph data (https://github.com/FabianFG/CUE4Parse/blob/b35b843a1193d225df8583b664ef66854557a613/CUE4Parse-Conversion/Meshes/MeshExporter.cs#L456C9-L470) will only ever export morph data for 1 of the overlapping vertices because of the the FindVertex call:

var index = FindVertex(vertex.Position, share.Points);

I'm not too familiar with how this code is supposed to work, but it seems like, at least for LOD0, it should be possible to forego the FindVertex call and just use the SourceIdx as the vertex index for the morph datum.

Stream overread due to incorrect `SkeletalMesh.HasRayTracingData` flag

I have been decompiling a recent Unreal Engine 4 game for the NS with FModel and noticed that most meshes were throwing NullReferenceExceptions when trying to convert the LODModels.
image

Upon further debugging, when comparing the stream offset to a hex editor, it seemed the reader was overreading when deserializing, in this specific case, the 2nd LOD model. From there I found the problem was being cause by the SkeletalMesh.HasRayTracingData flag being raised despite the mesh not having any ray tracing data.

This results in the following compatibility code interpreting bad data as, in this case, the header for a 257 length array of ray tracing data and advancing the reader beyond the start of the next LOD mesh. Removing the check seems to fix the problem entirely, leading me to believe the flag is being incorrectly raised.

if (Ar.Versions["SkeletalMesh.HasRayTracingData"])
{
var rayTracingData = Ar.ReadArray<byte>();
}

Unfortunately this is where my debugging stops.

I am aware that FModel is using an older version of CUE4Parse, so I pulled the latest changes from CUE4Parse/master and acl/develop and the issue was still present.

System.DllNotFoundException on MacOS

How to solve?

CUE4Parse 1.0.4
Fortnite 23.20

Program.cs:

using System.IO;
using System.Text.Json;
using CUE4Parse.FileProvider;
using CUE4Parse.MappingsProvider;
using CUE4Parse.UE4.Versions;

var Provider = new DefaultFileProvider("PATH_TO_PAKS", SearchOption.TopDirectoryOnly, true, new(EGame.GAME_UE5_1));
Provider.MappingsContainer = new FileUsmapTypeMappingsProvider("PATH_TO_MAPPINGS");
Provider.Initialize();
Provider.SubmitKey(new(), new("0x8D69B1A2D9A71F8BAB2A7168E688F21484D88D219A8CE896715435518FB72916"));
Provider.LoadLocalization(ELanguage.English);

File.WriteAllText("./exported.json", JsonSerializer.Serialize(Provider.LoadObjectExports(null)));

Output:

Unhandled exception. System.DllNotFoundException: Unable to load shared library 'oo2core_9_win64.dll' or one of its dependencies. In order to help diagnose loading problems, consider setting the DYLD_PRINT_LIBRARIES environment variable: 
dlopen(/Volumes/mac/.dotnet/shared/Microsoft.NETCore.App/7.0.0/oo2core_9_win64.dll.dylib, 0x0001): tried: '/Volumes/mac/.dotnet/shared/Microsoft.NETCore.App/7.0.0/oo2core_9_win64.dll.dylib' (no such file), '/usr/local/lib/oo2core_9_win64.dll.dylib' (no such file), '/usr/lib/oo2core_9_win64.dll.dylib' (no such file)
dlopen(/Volumes/mac/Source/Repos/Test/bin/Debug/net7.0/oo2core_9_win64.dll.dylib, 0x0001): tried: '/Volumes/mac/Source/Repos/Test/bin/Debug/net7.0/oo2core_9_win64.dll.dylib' (no such file), '/usr/local/lib/oo2core_9_win64.dll.dylib' (no such file), '/usr/lib/oo2core_9_win64.dll.dylib' (no such file)
dlopen(oo2core_9_win64.dll.dylib, 0x0001): tried: 'oo2core_9_win64.dll.dylib' (no such file), '/usr/local/lib/oo2core_9_win64.dll.dylib' (no such file), '/usr/lib/oo2core_9_win64.dll.dylib' (no such file), '/Volumes/mac/Source/Repos/Test/oo2core_9_win64.dll.dylib' (no such file), '/usr/local/lib/oo2core_9_win64.dll.dylib' (no such file), '/usr/lib/oo2core_9_win64.dll.dylib' (no such file)
dlopen(/Volumes/mac/.dotnet/shared/Microsoft.NETCore.App/7.0.0/liboo2core_9_win64.dll.dylib, 0x0001): tried: '/Volumes/mac/.dotnet/shared/Microsoft.NETCore.App/7.0.0/liboo2core_9_win64.dll.dylib' (no such file), '/usr/local/lib/liboo2core_9_win64.dll.dylib' (no such file), '/usr/lib/liboo2core_9_win64.dll.dylib' (no such file)
dlopen(/Volumes/mac/Source/Repos/Test/bin/Debug/net7.0/liboo2core_9_win64.dll.dylib, 0x0001): tried: '/Volumes/mac/Source/Repos/Test/bin/Debug/net7.0/liboo2core_9_win64.dll.dylib' (no such file), '/usr/local/lib/liboo2core_9_win64.dll.dylib' (no such file), '/usr/lib/liboo2core_9_win64.dll.dylib' (no such file)
dlopen(liboo2core_9_win64.dll.dylib, 0x0001): tried: 'liboo2core_9_win64.dll.dylib' (no such file), '/usr/local/lib/liboo2core_9_win64.dll.dylib' (no such file), '/usr/lib/liboo2core_9_win64.dll.dylib' (no such file), '/Volumes/mac/Source/Repos/Test/liboo2core_9_win64.dll.dylib' (no such file), '/usr/local/lib/liboo2core_9_win64.dll.dylib' (no such file), '/usr/lib/liboo2core_9_win64.dll.dylib' (no such file)
dlopen(/Volumes/mac/.dotnet/shared/Microsoft.NETCore.App/7.0.0/oo2core_9_win64.dll, 0x0001): tried: '/Volumes/mac/.dotnet/shared/Microsoft.NETCore.App/7.0.0/oo2core_9_win64.dll' (no such file), '/usr/local/lib/oo2core_9_win64.dll' (not a mach-o file), '/usr/lib/oo2core_9_win64.dll' (no such file)
dlopen(/Volumes/mac/Source/Repos/Test/bin/Debug/net7.0/oo2core_9_win64.dll, 0x0001): tried: '/Volumes/mac/Source/Repos/Test/bin/Debug/net7.0/oo2core_9_win64.dll' (no such file), '/usr/local/lib/oo2core_9_win64.dll' (not a mach-o file), '/usr/lib/oo2core_9_win64.dll' (no such file)
dlopen(oo2core_9_win64.dll, 0x0001): tried: 'oo2core_9_win64.dll' (not a mach-o file), '/usr/local/lib/oo2core_9_win64.dll' (not a mach-o file), '/usr/lib/oo2core_9_win64.dll' (no such file), '/Volumes/mac/Source/Repos/Test/oo2core_9_win64.dll' (not a mach-o file), '/usr/local/lib/oo2core_9_win64.dll' (not a mach-o file), '/usr/lib/oo2core_9_win64.dll' (no such file)
dlopen(/Volumes/mac/.dotnet/shared/Microsoft.NETCore.App/7.0.0/liboo2core_9_win64.dll, 0x0001): tried: '/Volumes/mac/.dotnet/shared/Microsoft.NETCore.App/7.0.0/liboo2core_9_win64.dll' (no such file), '/usr/local/lib/liboo2core_9_win64.dll' (no such file), '/usr/lib/liboo2core_9_win64.dll' (no such file)
dlopen(/Volumes/mac/Source/Repos/Test/bin/Debug/net7.0/liboo2core_9_win64.dll, 0x0001): tried: '/Volumes/mac/Source/Repos/Test/bin/Debug/net7.0/liboo2core_9_win64.dll' (no such file), '/usr/local/lib/liboo2core_9_win64.dll' (no such file), '/usr/lib/liboo2core_9_win64.dll' (no such file)
dlopen(liboo2core_9_win64.dll, 0x0001): tried: 'liboo2core_9_win64.dll' (no such file), '/usr/local/lib/liboo2core_9_win64.dll' (no such file), '/usr/lib/liboo2core_9_win64.dll' (no such file), '/Volumes/mac/Source/Repos/Test/liboo2core_9_win64.dll' (no such file), '/usr/local/lib/liboo2core_9_win64.dll' (no such file), '/usr/lib/liboo2core_9_win64.dll' (no such file)

   at CUE4Parse.Compression.Oodle.OodleLZ_Decompress(Byte* buffer, Int64 bufferSize, Byte* output, Int64 outputBufferSize, Int32 a, Int32 b, Int32 c, Int64 d, Int64 e, Int64 f, Int64 g, Int64 h, Int64 i, Int32 threadModule)
   at CUE4Parse.Compression.Oodle.Decompress(Byte[] compressed, Int32 compressedOffset, Int32 compressedSize, Byte[] uncompressed, Int32 uncompressedOffset, Int32 uncompressedSize, FArchive reader)
   at CUE4Parse.MappingsProvider.Usmap.UsmapParser..ctor(FArchive archive)
   at CUE4Parse.MappingsProvider.Usmap.UsmapParser..ctor(Stream data, String name)
   at CUE4Parse.MappingsProvider.Usmap.UsmapParser..ctor(String path, String name)
   at CUE4Parse.MappingsProvider.UsmapTypeMappingsProvider.Load(String path)
   at CUE4Parse.MappingsProvider.FileUsmapTypeMappingsProvider..ctor(String path)
   at Program.<Main>$(String[] args) in /Volumes/mac/Source/Repos/Test/Program.cs:line 8

Using LoadObject will freeze the program

Whenever i use the LoadObject function instead of the LoadObjectAsync function the program freezes forever. Using the LoadObjectAsync function with await seems to work but using the async function in a not async context wont work.

I figured out that the problem comes from AbstractFileProvider.cs

var uasset = await uassetTask;

I also noticed that it works when i use the debugger to step into the Task.Run(CreateReader) function thats being called above by file.CreateReaderAsync() even tho it doesnt step into the function.

Current workaround:
Using the LoadObjectAsync function in an async function seems to fix it but is quite annoying.

Unpacking a compressed .pak file with only a single asset within it will not yield a path structure

Take the following file structure of an example ModFile.pak, as seen in many mods for Hogwarts Legacy, which contain only a single file:

// ModFile.pak

Phoenix/Content/SQLiteDB/PhoenixShipData.sqlite

If this file is uncompressed, Provider.Files() will correctly list the path as "Phoenix/Content/SQLiteDB/PhoenixShipData.sqlite"

If the file is compressed however, it will be listed as just the filename "PhoenixShipData.sqlite"

When comparing mod pak structures, it can lead to issues as the output files will clash with each other as there's no other identifying path data available.

Creating a new FSimpleCurve from a FStructFallback does not properly load the keys

For some reason, when creating a new FSimpleCurve from a FStructFallback, no keys are loaded. I believe it has to do with the fact that it's possible (idk if by default or just for this game) that the keys property does not have a capital K as is assumed by Keys = data.GetOrDefault(nameof(Keys), Array.Empty<FSimpleCurveKey>())

The solution to this is to set the string comparison type to OrdinalIgnoreCase which would ignore the capital K in the search and return the correct keys.

My current work around is to just GetOrDefault keys with a lowercase k var keys = fallback.GetOrDefault("keys", Array.Empty<FSimpleCurveKey>());

It's not the best screenshot but this confirms that the fallback data does have the keys but getting from by creating a curve does not work
image

License

The repo doesn't specify any open source license for CUE4Parse. Is there going to be one?

Mostly just a question.

So I'm looking at the files and I don't see anything mentioning umap files. I'm trying to make a Blender plugin that imports Umaps (it is in python), but I don't see how Cue4Parse handles Umaps. Sorry for being a bother ^^'

Can't see any game data at all

Testing the library with FNaF:HW, but I got no result. I'm using this code to do it:
image
What is wrong with my code? Is there anything that I'm doing wrong?
EDIT: I moved the game from Steam folder, because I thought the path was too long

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.