Not able to `consumeFrame` from iOS Broadcast Extension about opentok-ios-sdk-samples-swift HOT 14 CLOSED

@MathiasDum @karolkulesza I am able to screen share from the extension using iOS 2.11.3 but looks like there is an issue with 2.15. Please try 2.11.3 for now, meanwhile we will look into the issue with 2.15.

from opentok-ios-sdk-samples-swift.

@karolkulesza Is it possible to provide a sample app that demonstrate the issue ? so that I can check what is going on.

from opentok-ios-sdk-samples-swift.

@bsrao :

Hi Sridhar,

Yes, I've just shared with you (via @) a test empty app with iOS Broadcast Extension that connects with TokBox (directly from this extension) and tries to send a sample image frame (from local resources). I've used the sample code provided OpenTok, for screen sharing scenario (see TBScreenCapture class, where instead of screenshot of the app, I'm using a static image from local resources).

Before running it, please update "sessionId" and "token" values with valid ones, in class SampleHandler, in BroadcastExtension target.

After installing on iOS device, you won't see anything in the app itself, you have to invoke iOS broadcasting feature on iOS12. In order to do that, on iOS12 device please enable "Screen Recording" feature to be visible in iOS Control Center (via Settings -> Control Center -> Customize Controls) and then start broadcasting by showing Control Center, 3D tapping on broadcast/record icon and selecting "BroadcastExtension" from the list (as indicated in the screenshot attached here).

Optionally, if you'd like to debug this Broadcast Extension (as it is a separate process on iOS), you can use "Debug -> Attach to Process" feature in Xcode.

Let me know please in case of any issues in reproducing it.

Regards,
Karol

from opentok-ios-sdk-samples-swift.

I'm having the the same issue

from opentok-ios-sdk-samples-swift.

Are you able to also process the app audio from the broadcast extension?

from opentok-ios-sdk-samples-swift.

@etown :
Yes, please take a look at one of the methods declared by RPBroadcastSampleHandler protocol:

from opentok-ios-sdk-samples-swift.

@karolkulesza Thank you, I am aware replaykit provides the frames, but I’m looking to see in example of consuming the audio with opentok. Thank you!

from opentok-ios-sdk-samples-swift.

@karolkulesza could you please share the sample extension project you had before? it would be extremely helpful. thank you!

from opentok-ios-sdk-samples-swift.

@bsrao can you confirm if the issue is currently affecting 2.15? also, could you provide replaykit sample code? thank you!

from opentok-ios-sdk-samples-swift.

@etown Yes, it is effecting 2.15. I don't have any sample code that I can share but it will be straight forward, as @karolkulesza shared, you need to grab the frame from processSampleBuffer and consume it using external capturer (you can use custom video driver class in the sample repo).

from opentok-ios-sdk-samples-swift.

Hi Folks, we don't have any sample code for this yet, but we are working on fix that should be released with 2.16.0.

from opentok-ios-sdk-samples-swift.

This is an almost finished working sample for Broadcast extension using OpenTok SDK : https://github.com/opentok/opentok-ios-sdk-samples/tree/broadcast-ext-sample
Feel free to share any issues you find!

from opentok-ios-sdk-samples-swift.

I have implemented the above extension in my project but it doesn't send any frame to the other side, even i have tried it directly from this project. it doesn't appear other side of a call even though a session is being created, frames are available it doesn't even add a subscriber to the other side.

It always post error in didFailWithError, with an error "Timed out while attempting to publish."

from opentok-ios-sdk-samples-swift.

Apologies for the belated reply. The Timed out while attempting to publish error suggests that frames aren't being sent. There may be multiple reasons for this but unlikely to be related to the SDK. Can you please see if you can reproduce using the updated version: https://github.com/opentok/opentok-ios-sdk-samples/tree/main/Broadcast-Ext? Also, the broadcast sample app only works for H264.

from opentok-ios-sdk-samples-swift.