[ts][pixi] Adding a child Pixi object in a Slot that follows its transform about spine-runtimes HOT 23 CLOSED

The Spine class itself is a pixi container. However, it currently has no methods to interleave custom objects inside the rendering hierarchy of meshes generated for each slot.

It shouldn't be too hard to add this functionality. The basic approach would be this.

1. Add a methods ala addSlotObject(slotIndex, pixiObject) and removeSlotObject(slotIndex, pixiObject)to Spine.
2. Spine keeps an internal list of objects attached to a slot. Their rendering order is the order in which they were added.
3. Spine.getMeshForSlot() and Spine.updateGeometry() need to be modified so that slot objects and meshes generated for slot attachments are sorted correctly by setting their zIndex. Additionally, for slot objects added via addSlotObject() their transform needs to be aligned with the slot's bone.

@davidetan want to give this a try?

from spine-runtimes.

@furic This PR address should address your request :)

You can look at this example to understand its usage.

A Spine object has now the ability to attach Pixi DisplayObjects to Slots.
You can use the addSlotObject (slotRef: number | string | Slot, pixiObject: DisplayObject): void method.
As you asked, you can refer to slots by name, but you can also pass the slot index or the slot instance.

Note that you can add only one DisplayObject per slot and that its transform will be modified by the Spine object.
If you want more control over the added objects, you can add a Container where you will be free to add as many Pixi objects as you want, offset their position, angle, scale, etc.

If you want to remove a display object, just call the removeSlotObject (slotRef: number | string | Slot, pixiObject?: DisplayObject): void method.
pixiObject is optional. If you pass it, the pixi object will be removed only if it corresponds to the one in the given slot.
Note that the display object is only removed, not destroyed.

Finally, we have a getSlotObject (slotRef: number | string | Slot): DisplayObject | undefined that simply returns a pixi object given a slot, if there's any.

Let us know if you think that's ok :)

from spine-runtimes.

Thanks for the clarification. I was able to make an implementation to address the issues you pointed out, but I need to make some additional refinements.
Unfortunately, I won't be able to finish it until next week :(

from spine-runtimes.

@furic We were able to implement both your requests with this commit: 61e9be2

To enable clippinig/masking on pixi objects added using addSlotObject, we had to create a pixi mask with the same shape of the clipping attachment.

Feel free to reopen this issue, if anything does not work as expected.

from spine-runtimes.

FYI, alpha and clipping are working in the last commit. 👍

from spine-runtimes.

Hey @davidetan, I'm working with @furic on this one and we've encountered a minor issue with the slots with this latest commit.

When playing an animation with a pixiObject added with addSlotObject, the new pixiObject that was added plays its setup pose for the first frame of the animation. This causes the object to jump if the slot's setup pose doesn't match the first frame of the animation.

A workaround fix on Spine editor side is to ensure there is no discrepancy between the setup pose and animation's first frame for the slot, but it's less than ideal especially for more complicated usages.

My first assumption would be that the object updating is 1 frame or tick delayed, causing it to flash the setup pose on the first frame of the animation. Sorry I don't have an example at the moment but if you require one let me know and I'll try whip something up.

Thanks so much for your work on this! 😃

Thanks for reporting this, we'll looking into that!

from spine-runtimes.

We found the problem being not connected to this feature, but to a code reorganization in this commit e498e96. We moved the update and apply of the state and the skeleton all together. However, the first time the skeleton is rendered, the state was not yet updated. This is why the first frame was rendered in its setup pose.

With this new commit, we ensure the state update happens always before rendering.

from spine-runtimes.

Hey @davidetan, I'm working with @furic on this one and we've encountered a minor issue with the slots with this latest commit.

When playing an animation with a pixiObject added with addSlotObject, the new pixiObject that was added plays its setup pose for the first frame of the animation. This causes the object to jump if the slot's setup pose doesn't match the first frame of the animation.

A workaround fix on Spine editor side is to ensure there is no discrepancy between the setup pose and animation's first frame for the slot, but it's less than ideal especially for more complicated usages.

My first assumption would be that the object updating is 1 frame or tick delayed, causing it to flash the setup pose on the first frame of the animation. Sorry I don't have an example at the moment but if you require one let me know and I'll try whip something up.

Thanks so much for your work on this! 😃

from spine-runtimes.

I've made a minimal project to reproduce the issue. The setup in spine is simply a bone and a slot with an attachment. I use an 'empty' 1x1 transparent image for the attachment which we will replace with addSlotObject at runtime. The setup is the bone is simply at the (0,0) origin. The animation starts with a key that does not match the setup pose (the Y value is higher).

Here is the setup and animation in Spine editor.

When replacing the slot with addSlotObject, I am getting the flashing issue on the first frame where it displays the bone's setup pose. I've replaced the slot with a green rope. When I refresh the animation, you will notice very briefly it flashes at the origin and then goes up to start the animation.

It is quite fast and hard to notice, my recorder wouldn't even pick it up until I set it to 60FPS.

The project files:

{
"skeleton": { "hash": "dh5I2oLFNaw", "spine": "4.2.29", "x": -0.5, "y": -0.5, "width": 1, "height": 1 },
"bones": [
	{ "name": "root" },
	{ "name": "empty", "parent": "root" }
],
"slots": [
	{ "name": "empty", "bone": "empty", "attachment": "empty" }
],
"skins": [
	{
		"name": "default",
		"attachments": {
			"empty": {
				"empty": { "width": 1, "height": 1 }
			}
		}
	}
],
"animations": {
	"animation": {
		"slots": {
			"empty": {
				"rgba": [
					{ "color": "ffffff7d" },
					{ "time": 1.3333, "color": "ffffffff" }
				]
			}
		},
		"bones": {
			"empty": {
				"translate": [
					{ "y": 99 },
					{ "time": 1, "y": -34 },
					{ "time": 1.3333 }
				]
			}
		}
	}
}
}

slottest.png
size:16,16
filter:Linear,Linear
scale:0.3
empty
bounds:2,2,1,1

from spine-runtimes.

Hi David, we have found another issue with adding a pixiObject to a slot via the addSlotObject method. The issue is that attachment state and attachment keys are not properly being accounted for at runtime.

For example, if I have a slot with an attachment off in setup pose, I expect it to not render anything if I don't play an animation that turns the attachment on. I expect this to be true even if I add an object with addSlotObject. However, objects added with addSlotObject appear visible even if the attachment is off in setup pose (or animated with attachment keys in an animation).

Using the same example as above:

"slots": [
	{ "name": "empty", "bone": "empty", "attachment": "empty" },
]

In this scenario I have the attachment on as empty in Spine Editor.

If I remove this attachment in setup pose,

"slots": [
	{ "name": "empty", "bone": "empty" },
]

I have this, and I expect the Spine object to not render the empty slot, unless I turn on the attachment in an animation.

Perhaps it could use visible member in Pixi Display Object to render attachment states/keys?

from spine-runtimes.

After an internal discussion, we're considering to connect the visibility of the pixi object added to a slot, to the visibility of the active attachment of the same slot. However, this will be probably activated using an optional parameter.
I suppose the signature will be something like this:
addSlotObject (slotRef: number | string | Slot, pixiObject: DisplayObject, syncToAttachment?: boolean): void.

from spine-runtimes.

The custom afterUpdateWorldTransforms function works well and is simple to use! Thank you.

Cool! Glad to hear that :)
beforeUpdateWorldTransforms and afterUpdateWorldTransforms are simple but powerful because they let you modify the world based on the skeleton state and vice versa.

For instance, see in this example, how simple is to control a bone position based on some external elements.

from spine-runtimes.

The Spine class itself is a pixi container. However, it currently has no methods to interleave custom objects inside the rendering hierarchy of meshes generated for each slot.

It shouldn't be too hard to add this functionality. The basic approach would be this.

1. Add a methods ala `addSlotObject(slotIndex, pixiObject)` and `removeSlotObject(slotIndex, pixiObject)`to `Spine`.

2. `Spine` keeps an internal list of objects attached to a slot. Their rendering order is the order in which they were added.

3. `Spine.getMeshForSlot()` and `Spine.updateGeometry()` need to be modified so that slot objects and meshes generated for slot attachments are sorted correctly by setting their `zIndex`. Additionally, for slot objects added via `addSlotObject()` their transform needs to be aligned with the slot's bone.

@davidetan want to give this a try?

Yeah it would be nice to have addSlotObject in the package to achieve this, instead of doing it outside the package. I may also suggest using slotName instead of slotIndex: addSlotObject(slotName, pixiObject).

Just note that after mesh.addChild(pixiObject), the child pixiObject should be sorted with the zIndex of mesh which should reflect the draw order, but it seems not the case currently.

Otherwise, if somehow SlotMesh (extends PIXI.Container) is updating its transform and its zIndex is reflecting the draw order, then we can add the Pixi object outside the package, similar to pixi-spine's solution.

from spine-runtimes.

The pixi object will be sorted respecting the draw order.
We're working on that. It will be probably ready at the end of the day 👍

from spine-runtimes.

Released in 4.2.45.

from spine-runtimes.

@davidetan I've just finished an in-depth test. It's almost perfect but I did encounter 2 issues:

1. pixiObject doesn't follow the alpha of its parent slot
2. pixiObject doesn't get masked if its parent slot is being masked

I can provide my testing Spine, but I think it's pretty straight-forward.

from spine-runtimes.

Hello @furic and thank you for your feedback.

pixiObject doesn't follow the alpha of its parent slot

This should be straightforward to implement.

pixiObject doesn't get masked if its parent slot is being masked

With slot being masked, are you referring to the effect of clipping attachments?
In that case it would be way more harder to implement at first sight.

from spine-runtimes.

Hello @furic and thank you for your feedback.

pixiObject doesn't follow the alpha of its parent slot

This should be straightforward to implement.

pixiObject doesn't get masked if its parent slot is being masked

With slot being masked, are you referring to the effect of clipping attachments? In that case it would be way more harder to implement at first sight.

Yes I did mean clipping attachments.
We reckon 95% of our cases aren't using both slot and clipping mask at the same time. I would say it's good for now but would be nice to have it.

from spine-runtimes.

@joshsurin Right now, the pixi object is added right after the call to addSlotObject, while its transform is updated in the next render loop.
So, my guess is that the pixi object might be rendered with its transform not yet controlled by the spine object.
However, it's hard to say without a reproduction of the issue, and all my tests did not lead to it.

Can you share a minimal project to reproduce the issue?

from spine-runtimes.

I don't think that the visibility of the pixi object added using addSlotObject should be connected with the visibility of the attachment.
The pixi object is a child of the slot, not of the attachment. At most, it can be seen as a sibling of all skin placeholders and the attachment of the slot.

A possibility might be to connect the state of the pixi object with the state of the active attachment in the slot, but there might be situations in which a user doesn't want to have this connection.

It's definitely more reasonable that the pixi object in a slot has its state connected to the state of the slot. It is already like that, mostly, but visibility is probably not connected yet.
I'll definitely add this.

Let me know if this is enough. Otherwise, try to explain the use case in which you need to connect the state of the attachment to the one of the pixi object.

from spine-runtimes.

I tend to agree that the added object should be seen as more of a sibling of the slot's attachment rather than a replacement. The way we have been using the slot replacement is mainly to dynamically change sprites or text at runtime, for example for localisation, or a dynamic number. In these cases, we only have one object to replace the slot and the slot's (and attachment's) sole purpose is to act as a placeholder for this new sprite or text object. However I can definitely see why some use cases may have a slot with an attachment, as well as an additional object added, without affecting the existing attachment.

That said, there would be no way to control the visibility of the added object from the animation side except through alpha, which introduces a few complications and potential issues, for example the object would always update and calculate because we simply set the alpha to 0 even if the intended outcome is that it is no longer visible. It also adds additional requirements to set the slot's setup alpha to 0 if we don't want it visible in setup pose, which can further complicate things on the animation side.

Your suggestion of syncing the pixi object's visibility to an attachment is a great idea that allows for all these use cases I've mentioned and would certainly solve all our problems!

from spine-runtimes.

Before adding the aforementioned feature, I want to be 100% sure that you are also aware of the beforeUpdateWorldTransforms and afterUpdateWorldTransforms methods in a spine object.

The state of the spine object is updated using this internalUpdate method:

protected internalUpdate (deltaSeconds?: number): void {
	const delta = deltaSeconds ?? Ticker.shared.deltaMS / 1000;
	this.state.update(delta);
	this.state.apply(this.skeleton);
	this.beforeUpdateWorldTransforms(this);
	this.skeleton.update(delta);
	this.skeleton.updateWorldTransform(Physics.update);
	this.afterUpdateWorldTransforms(this);
}

This internal update is done right before rendering. As you can see, before and after the skeleton update, you can inject some custom logic to influence the skeleton or to query the skeleton to influence the world. By default, beforeUpdateWorldTransforms and afterUpdateWorldTransforms are two noop functions.

In your case, you can achieve the desired result by replacing afterUpdateWorldTransforms. For example, you could add these two lines of code:

const slot = spineboy.skeleton.findSlot("empty");
spineboy.afterUpdateWorldTransforms = () => pixiObject.visible = Boolean(slot.attachment);

What do you think about using one of these two functions to achieve your goal?
Let me know if you think it's too annoying to do that and still prefer having the visibility automatically synced to the slot attachment being active.

from spine-runtimes.

The custom afterUpdateWorldTransforms function works well and is simple to use! Thank you.

from spine-runtimes.