Network replication / multiplayer issues about mediapipe4u-plugin HOT 29 CLOSED

1. Regarding the MediaPipe error, post the log file at here when an error occurs, which is usually in the saved/logs folder so that I can find out what the problem is.

2. Network, I don't think multiplayer animation has much to do with MediaPipe4U, it should be a server streaming animation data and client-side display problem, I believe UnrealEngine has the example of that. You can find UE document or market to get the sample.

3. Important note about the MediaPipeAminInstance (your animation blueprint). At the same time in the world outline you must make sure it has only one MediaPipeAminInstance , If you want to sync animation to many actors, you can use IK Reatarget to anothers. Because the landmarks data of mediapipe that made by MediaPipeHolisticComponent can be deal one time, multiple consumers (MediaPipeAminInstance ) will be get a low animation frame rate (Multiple MediaPipeAminInstance will be competition for data).

BTW: sorry for my bad English , this is a question of the structure of the program, which is difficult to explain in a short answer.

from mediapipe4u-plugin.

MediaPipe4U_Demo.log

Hey Anders, this is Sameek , and I am working with Giovanni on this project. As you requested , here is the saved log file for the error which is occuring when I am switching the number of players to more than 1 in the editor, for multiplayer testing.

I dont understand what you meant on the second point, because all the other animations are working for the player at both client and server. Only the Mediapipe Solver nodes that we have setup in the animation Blueprints, are controlling all the players together instead of just the client player.

Also , we are not looking for syncing animations. Our use case is pretty simple, the mediapipe solvers should move only the local player , that is obvious right ? Because if I am waving my hand, I only want my player to wave its hand, and not the server player. And also if I am waving my hand, the other players shall also see my hand movement, currently if the opponent players are waving their hands, I am not able to view it.

from mediapipe4u-plugin.

From the logs, it can be seen that the component was reloaded or re-initialized for some reason. when mediapipe started, you can not uninitialize it or re-initialized it.

The following invocations should not appear after you start capture.

UMediaPipeHolisticComponent::CreateObserver() 
UMediaPipeHolisticComponent::SetupObservers() 
UMediaPipeHolisticComponent::InitializeComponent()  <---------Here, why we need Initialize ? because uninitialize its owner Actor?

I think this is because you break the Actor for some reason, and moving the MediaPipe component to another Actor might solve this problem

from mediapipe4u-plugin.

What do you mean ? This is happening from your code only, we did not even have access to the source code, so we did not modify it.

from mediapipe4u-plugin.

I guess the actor is re-initialized, which causes the component attached by the actor to be re-nitialize. This is due to inappropriate usage, and it is possible that the code did not handle this error and caused the crash. But arguing about whose code it was generated did nothing to solve the problem.

from mediapipe4u-plugin.

The crash is causing as soon as I am calling the StartCamera Blueprint node from The MediaPipeHolistic component. If you dont want to solve the issues generated from your code, you can just say so that you dont want to provide support. But you are behaving as if this code is not written by you.

from mediapipe4u-plugin.

Hey I wanted to come in and just say that there is no need to argue over these things.
On our side, we are just trying to do the best we can with the available information to resolve these issues.
On Anders' side we completely understand that he is dealing with many requests and has a lot of work on his plate.

Let's keep the conversation civil please

from mediapipe4u-plugin.

From the logs, it can be seen that the component was reloaded or re-initialized for some reason. when mediapipe started, you can not uninitialize it or re-initialized it.

The following invocations should not appear after you start capture.

UMediaPipeHolisticComponent::CreateObserver() 
UMediaPipeHolisticComponent::SetupObservers() 
UMediaPipeHolisticComponent::InitializeComponent()  <---------Here, why we need Initialize ? because uninitialize its owner Actor?

I think this is because you break the Actor for some reason, and moving the MediaPipe component to another Actor might solve this problem

I'm helping you, but you've been dissatisfied, it's not the attitude of discussing problems, I've told you how to try to solve it, but you don't want to hear it at all, you're just expressing dissatisfaction.

moving the MediaPipe component to another Actor might solve this problem !!!!!

from mediapipe4u-plugin.

It is still the same, it does not matter where I am moving the MediaPipe component, as soon as I am calling the StartCamera function, it crashes for more than 1 player. For just 1 player it is fine.

from mediapipe4u-plugin.

Is there this line in every crash log file?

Error: [mediapipe_api.dll] Invalid state: pipeline running

You can give me a crash log file, after you moved MediaPipe Component to another Actor .

from mediapipe4u-plugin.

MediaPipe4U_Demo.log
Here it is.

Also please reproduce this at your end first. It is very easy to reproduce and also you will have better understanding at what is causing the issue

1. Change the Number of Players to 2 , and PlayAsListenServer
2. Call the StartCamera function of MediaPipeHolistic component in Blueprints

from mediapipe4u-plugin.

Anders, could you maybe try to reproduce this error on your end by just calling the StartCamera function while keeping the number of players to more than 1?

from mediapipe4u-plugin.

Okay, I will try to reproduce 2 players later.

Error: [mediapipe_api.dll] Invalid state: pipeline running

One thing I know from this line is that when you call StartCamera the MediapipeHolisticComponent has not finished initializing, or the level is streaming (If async load level).You call StartCamera is too early, it should be in begin play or later, I will add a check in the next release to prevent this call ...

I will try reproduce it.

from mediapipe4u-plugin.

Anyway, I'll replicate the multiplayer mode. And as a way to try it for now, inherit UMediaPipeHolisticComponent (C++, Blueprints can do the same), override its InitializeComponent function, add a feild as a flag (e.g. bInited = true) after the base UMediaPipeHolisticComponent ::InitializeComponent function call, and don't call camera when bInited not true:

YourComponent.h

UCLASS(ClassGroup="Default", meta=(BlueprintSpawnableComponent))
class UYourComponent : public UMediaPipeHolisticComponent
{
     UPROPERTY(EditAnywhere, BlueprintReadWrite)
     bool bInited = false;
     
     virtual void InitializeComponent() override;
}

YourComponent.cpp

void UYourComponent ::InitializeComponent()
{
	Super::InitializeComponent();
        bInited = true;
}

if bInited is not true, dont call StartCamera

from mediapipe4u-plugin.

when i play PlayAsListenServer, every thing is OK. But, you can't use one camera to capture two windows at the same time， But even if you do that, it won't cause a crash

from mediapipe4u-plugin.

Maybe it's a buggy for a specific version, I'm packing a new version that can be downloaded later today.

BTW: In my case, I didn't crash regardless of whether the number of players was 2 or 3, but my machine was stuck over 3

from mediapipe4u-plugin.

Download a new version,it has been tested with multi player network mode.

from mediapipe4u-plugin.

No, you just tested it with the default project as it it. But you need to test it with third person player character, without possessing the default pawn, because clearly the multiplayer here means that we are controlling the two characters.

So to reproduce the issue -

1. Remove the mannequin bp from the level
2. Add your defaultpawn class as the mannequin character in the world settings
3. Replace both the game pawn and the character reference in the mediaplayer runtime BP with "Get Player Pawn" or "Get Player Character" instead.

I am still getting the error, attached are the screenshot and the log as well

MediaPipe4U_Demo.log

from mediapipe4u-plugin.

OK~Wait for my testing

from mediapipe4u-plugin.

@sameekue5 One thing, let me know when this crash happened, from the logs it looks like stopping the game

from mediapipe4u-plugin.

Hey Anders, were you able on your end to reproduce the issue?

"So to reproduce the issue -

1. Remove the mannequin bp from the level
2. Add your defaultpawn class as the mannequin character in the world settings
3. Replace both the game pawn and the character reference in the mediaplayer runtime BP with "Get Player Pawn" or "Get Player Character" instead. "

from mediapipe4u-plugin.

I've been fixing it for 3 hours, it's a bit of a troublesome problem, It's almost done. But I'm afraid I don't have time to test this problem, when I fix it is done, please test it, and then feedback here. because In China, it's early in the morning.

from mediapipe4u-plugin.

The crash happened as soon as I press the Play button, when it calls the StartCamera function in BP.

from mediapipe4u-plugin.

I have done without full testing, just hot fix :

https://1drv.ms/u/s!AkmROUeQfSBjzgEz6gu8FCr7ibot?e=x79QpF

BTW: I'm going to bed, it's too late and too late

from mediapipe4u-plugin.

The crash is not happening, but the video tracking is coming as black, and now the hand tracking is stopped as well. Kindly check

from mediapipe4u-plugin.

Which version worked well with the video tracking on your PC？

from mediapipe4u-plugin.

Unreal 5.0. It was working fine before with single player, but after your updates it stopped displaying the video

from mediapipe4u-plugin.

Hey Anders, just wondering if you are still going to be helping us tomorrow

from mediapipe4u-plugin.

from mediapipe4u-plugin.