Flip Camera does not work on iPad when in a room about twilio-video-app-react HOT 10 CLOSED

Thanks for the issue @pjuchniewicz! We appreciate that you left us a lot of detail in the issue. This is very helpful.

Unfortunately, I am unable to get my hands on an iPad Air 2 to try and reproduce this issue. While I find someone at my company that has an iPad Air 2, I'm wondering if you could try something for me.

You mentioned that the publishTrack promise never completes. If you attach a catch() handler, what kind of error message do you see?

      getLocalVideoTrack(newFacingMode).then(newVideoTrack => {
        localParticipant?.publishTrack(newVideoTrack, { priority: 'low' }).catch(err => console.error('publish track error: ', err));
      });

from twilio-video-app-react.

Also, how frequently does this issue occur for you. 100% of the time?

from twilio-video-app-react.

Another idea: it may be possible that the video track hasn't actually been stopped by the time we make the call to getLocalVideoTrack. We can wait for the stopped event before calling getLocalVideoTrack. Try this:

  const toggleFacingMode = useCallback(() => {
    const localTrackPublication = localParticipant?.unpublishTrack(videoTrack!);
    // TODO: remove when SDK implements this event. See: https://issues.corp.twilio.com/browse/JSDK-2592
    localParticipant?.emit('trackUnpublished', localTrackPublication);

    videoTrack!.on('stopped', () => {
      const newFacingMode = facingMode === 'user' ? 'environment' : 'user';

      getLocalVideoTrack(newFacingMode).then(newVideoTrack => {
        localParticipant?.publishTrack(newVideoTrack, { priority: 'low' });
      });
    });

    videoTrack!.stop();
  }, [facingMode, getLocalVideoTrack, localParticipant, videoTrack]);

The fact that you got the feature to work when you separated it into two clicks (one for unpublishing, and one for switching cameras and publishing) makes me think that this ^ could do something.

from twilio-video-app-react.

@timmydoza

Also, how frequently does this issue occur for you. 100% of the time?

Yes, 100% of the time.

I tested both suggestions and, for both tests, the same thing occurred.

1. Added both a .then and a .catch to the publish() call.
2. Joined a room
3. Clicked on the Flip Camera button. The video turned off and the no video icon showed on the participant.
4. Waited a few minutes to see if anything would happen, but nothing did.
5. I ended the call and then the console.log in the .catch fired

Here's the version with the .then and .catch added. The top part is the console messages that occurred after clicking Flip Camera.

Here's the version with the .on('stopped') setup. The top part is the console messages that occurred after clicking Flip Camera.

As you can see from the console's output, neither the .then or the .catch fire. The only time they fire is when I then clicked on the end call button, shown below.

Here is the console when I add an if/else in the Flip Camera code. It will always use the environment camera since the videoTrack is unpublished every other click and I wanted to see the initial video change. Nothing is written out on the first click since it is unpublishing / stopping the video and I have no console messages setup for this process. On the second click, the video is retrieved for the environment camera and published successfully.

from twilio-video-app-react.

Thanks for the info @pjuchniewicz! I really appreciate the amount of detail that you've provided.

This looks like a legitimate bug. It seems that the publishTrack call is getting stuck on something.

I have two more ideas for you to try (if you wold be so kind):

• Add logLevel: 'debug' to the connectionOptions object in src/index.tsx. This will print verbose debugging info to the console. There may be more clues in the SDK logs.
• After you reproduce the issue again, get the Room SID (it's on window.twilioRoom.sid) and leave it in a comment here. I can use the Room SID to look up the server-side logs to see what is happening.

Thanks for all your help on this issue! In the meantime I'm going to get more eyes on the issue since it seems that the bug may even lie in the SDK itself.

from twilio-video-app-react.

Thanks for looking into this @timmydoza! Here's the info you asked for.

Room SID: RM46e37f926b177951bf5f81182ea33bd0

I left the console.log messages I had setup in there.

Here is the console after I clicked the Flip Camera button:

Unfortunately, I forgot to take a screen capture after I clicked End Call and before I turned off the device.
I rejoined the room, clicked Flip Camera, waited a little bit, and then clicked End Call. Here is console for the messages written out after End Call was clicked.

from twilio-video-app-react.

Thanks for all the extra info @pjuchniewicz! It's quite helpful.

This appears to be a known bug with the twilio-video SDK. We have an internal ticket for it.

There is a workaround. It may work if the local video track is unpublished after the new video track has been acquired. Something like this:

localVideoTrack.stop();
newLocalVideoTrack = await createLocalVideoTrack();
room.localParticipant.unpublishTrack(localVideoTrack);
room.localParticipant.publishTrack(newLocalVideoTrack);

This way, unpublishTrack() and publishTrack() both happen with one media negotiation.

I tried this, but there were track name is duplicated errors. This can be fixed by publishing the camera track with a name of camera-${Date.now()} (instead of just 'camera'). With this change, I had to adjust other parts of the app.

I have a branch up here with the whole fix (see here for the important part). I still haven't gotten my hands on an iPad Air 2 (can't get into the office with the lockdown) but I'm hoping that you can give it a try. Please let me know how it works for you!

from twilio-video-app-react.

That works @timmydoza!

It's a little slow when the switch actually happens, but it happens without any errors. I switched back and forth a few times to make sure.

from twilio-video-app-react.

Good to hear @pjuchniewicz!

We created an internal ticket here to add the workaround to the app.

from twilio-video-app-react.

@pjuchniewicz The fix has been merged into master: #185

Please let me know if you continue to see the issue. Thanks again for all your help in solving this one!

from twilio-video-app-react.