gorastudio-ceo / scnrecorder Goto Github PK

Hi, I have some trouble with installing and building a framework via Carthage.

Carthage version 0.36.0

Everything works fine on my iPad but when my iPhoneX is in landscape mode it crashes when starting the video recording (portrait works fine).

Let me know if you need any particular debug info. I'm on Ios 15.1

let sourceTextureDescriptor = self.makeSourceTextureDescriptor(basedOn: lastTexture)
guard let sourceTexture = self.device.makeTexture(
descriptor: sourceTextureDescriptor,
iosurface: surface,
plane: 0
) else {
throw Error.noSourceTexture
}

com.apple.scenekit.scnview-renderer (15): signal SIGABRT

Thank you.

ARView , Xcode 14 build error: circular reference, how to fix this error

sceneRecorder = try SceneRecorder(self)

Hello this is nice framework but i found an issue in video , its getting hanging in video with animated model which is provided in attachment so please solve this issue in updated framework , it most helpful for all developers .
Thanks .

Are you able to make the video a square? So the output of the recording is a movie file with a 1x1 aspect ratio?

Hello
Thank you for the awesome project.
I wonder about the project that has a plan to record ARView.
The performance and quality in ARSCNView is really useful and awesome.
Thank you for maintaining awesome project.

Hi there and thank you for this awesome library,
I am trying to record a video of a 3D scene with a specific duration of a specific number of seconds, but I am not able to achieve a precise timing to the order of 10 millisecond. In some cases there's difference of 10 millisecond in the output video.
My approach is to observe the duration property of the VideoRecording object and call the finishVideoRecording when the specific amount of frames has been recorded. This seems to be working for short videos (3s, 5s, 15s).
However when duration of video is set to 60 s I get an output of 60.01 s or even 60.03 s depending on the device.
Can you please help me to understand what I am doing wrong. Is there another way to achieve this kind of precision and make independent from the device?
I have pushed in my repo an example app that's using the logic I described in this issue. The repo uses the latest SCNRecorder library but you are free to use a local development version of the library by changing the Podfile and running pod install again.
This is the link to the repo. https://github.com/giomurru/SCNRecorder-SampleApp
Thank you.

Is it possible to record videos when there are two ARSCNView for left and right?

Hi,
I tried to build my project with your library as pod and it shows error in file VideoSettings.Code.
Error text is "Enum cases with associated values cannot be marked potentially unavailable with '@available'".
@available(iOS 13.0, *) case hevcWithAlpha(_ compressionProperties: HEVCCompressionProperties)

I can build project only after deleting "@available(iOS 13.0, *)" , but this is hack.

Please can you fix this bug?

When I try to build my app with Xcode 14 using 2.7 version of SCNRecorder pod, it gives this error strangely:
Non-protocol, non-class type 'Self.Audio' cannot be used within a protocol-constrained type

If I use the underline protocol names intead of the typealias, compiler builds without a problem

typealias SampleBufferAudio = AudioMediaSessionInput & SampleBufferInput

typealias SampleBufferVideo = VideoMediaSessionInput & SampleBufferInput

typealias PixelBufferVideo = VideoMediaSessionInput & BufferInput

When I use ipad to record ARKit SCNView, the picture is not normal and the size is wrong
You can see the picture here,
The first picture is the screen display
https://1drv.ms/u/s!AmpfcHWI3MLhanf1KCXPFdlZMus?e=KZ8Z5M
The second picture is the capture result
https://1drv.ms/u/s!AmpfcHWI3MLha_BwTAUGmCHfbxI?e=z17XLb

Are there any settings that need to be modified when using iPad to capture?

Thank you.

Hi again! I seemed to have found a leak. I think it takes place in the same place as my other ticket #4

Here is the leak I get back from Instruments:

15 0x2a33c4000 VM: IOSurface 00:38.873.640 • 7.98 MiB CoreVideo CVPixelBufferBacking::performStandardMemoryLayoutAndCopyIOSurfaceCreationProperties(void*, unsigned char, __CFAllocator const*, __CFDictionary const*, __CFDictionary const*, __CFDictionary const*, __CFDictionary const*, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, void**, unsigned long*, unsigned long*, unsigned long*, unsigned long*, unsigned long*, __IOSurface**, void**, __CFDictionary const**)

and

165 0x159098000 VM: IOSurface 00:37.507.010 • 1.31 MiB CoreImage CreateCachedSurface

the stack trace:
https://imgur.com/a/O9vJaLt

Both of these seem to be continuously spawning during recording. I'll keep doing some digging to see if I can find any root issues and more info. The one thing I'm not sure of is if it directly correlates with #4

Unfortunately building my project that uses SCNRecorder with the latest XCode 14 beta results in a few build errors in this module. Hopefully it is an easy fix. Thank you!

Hi there, thanks so much for building this!

I am getting an error when starting the recording (although it works):

[CAMetalLayerDrawable texture] should not be called after already presenting this drawable. Get a nextDrawable instead.

There's also a framework that has been renamed.

And one last thing, I have to handle the saved videos yes? When a recording is finished and I don't do anything with the video file, is that file temporary and will it automatically be cleaned up?

Thanks!

when I try to use SCNRecorder, it crashed

1. set the project deployment target below iOS 13
2. import RealityKit and write add code " @available(iOS 13.0, *) " in UIViewController class
3. init a Entity class or ModelEntity class ( let _ = Entity() )
4. start to run the project and it will crash in iOS 13 and iOS 14 but iOS 15 is OK, very strange (you can run my demo)
   do you know how to solve this problem , I think it is apple's bug. how do you think , hope you can reply , thank you

I know in the ReadMe it says that this package is supposed to support RealityKit:
"To support recording RealityKit, copy ARView+MetalRecordable.swift and ARView+SelfSceneRecordable.swift files to your project. Then look at ARViewController.swift for usage."
but these files mentioned here^ seem to be missing. How do I make this work with RealityKit?

When I set the minimum deployments 'iOS 10.0' and run the example ,

_cancelable = scene.subscribe( to: SceneEvents.Update.self ) { [weak sceneRecorder] (_) in sceneRecorder?.render() }
SceneEvents.Update will crash, can you help to me?

ARSessionDelegate can't run, when I use self.arView.prepareForRecording()
do you know why?

I tested the recorder with iOS 15 beta. The recorder would run like normal but the movie files could not be saved.

I can`t find how to play pcmbuffer with AVAudioPlayerNode way

let player = try AudioEngine.Player(url: url) audioEngine.player = player

Hey Vlad,
Thank you for this package!

The video recording seems to be working well, however I was not able to make the audio recording work with ARKit ARView.
I've added the custom ARView+MetalRecordable.swift and ARView+SelfSceneRecordable.swift
and set configuration.providesAudioData = true

Is this a known issue with ARView or maybe I'm missing something?

Thanks,
Alon

When the recording is resumed, it throws the error

Error Domain=AVFoundationErrorDomain Code=-11862 "Cannot append media data after ending session" UserInfo={NSUnderlyingError=0x2808ebf00 {Error Domain=NSOSStatusErrorDomain Code=-12763 "(null)"}, NSLocalizedFailureReason=The application encountered a programming error., NSDebugDescription=Cannot append media data after ending session, NSLocalizedDescription=The operation is not allowed}

The pause method of the VideoOutput finishes the session which does not allow to resume.

func pause(_ videoOutput: VideoOutput) -> Self {
    switch  self {
    case .starting:
      return .starting
    case .ready,
         .preparing:
      return .ready
    case .recording(let seconds):
      // HERE
      videoOutput.endSession(at: seconds)
      return .paused
    case .paused,
         .canceled,
         .finished,
         .failed:
      return self
    }
  }

_validateGetBytes:56: failed assertion `(origin.y + size.height)(2048) must be <= height(1536).'
crashed in MetalPixelBufferProducer.swift, line 116

Hi, I encountered some problems with the output colors while using SCNRecorder on iPhone 6 running iOS 12.5.5. iPhone X and iPad 9th gen on iOS 15.5 are fine though. The recording made with iOS 12.5.5 and iPhone 6 has dull colors that appears less saturated than normal.
I found that the issue can be solved if we do not set a custom value for the AVVideoColorPropertiesKey and we leave this key to its default value.
I created a pull-request which includes this fix.

Hello, it's time to type the new tag about
3f316b0.

Hi! So I've run into an interesting issue that I found in InternalRecorder.swift. I managed to get it to crash at this line:

https://github.com/gorastudio/SCNRecorder/blob/6ea497e347cab67460a1fcc4a4a739d5d6a79a70/Source/InternalRecorder/InternalRecorder.swift#L278

I can never capture the frame with the actual error but I was able to see it output some sort of EXC_BAD_ACCESS error. This seems to only be happening to me when recording an SCNView.

Hi,

Thanks for the good work. I am trying to record video and audio in ARKit. I setup the audio device like you mentioned. However the audio in the recorded video is not continuous and keeps interrupting. I appreciate any help.

I am running on iOS 13.7

Hi,
thanks a lot for your wonderfull job, I try to use your package and integrate it in the flutter arkit_flutter_pluggin to be able to record directly from my flutter app.
I wondering if there is a way to save the capture directly in the gallery of the Iphone ?

thanks again

The issue occurs when recording AR video from the front camera to the iPhone 11 Pro Max. Less often on Xs Max.
How to do it again: You have to try to record short videos quickly. The principle of shooting video is the same as with Instagram - press the button, video is shot. Release the button - the recording is complete.
An example code will be provided if necessary.

Is it possible to add the ability to tune the output fps of the recorded video?
I did look at the code for a parameter to change this but I wasn't able find it. Is there anything I can change to achieve this?
Thank you.

Thank you for the amazing work with SCNRecorder

I'd like to report memory leaks that occur in my case at every recording with version 2.4.0.

Here are screenshots of the memory graph and stack trace:

The profiling was made after sceneView.cancelRecording() was called followed by sceneView de-init.

Hello,

I encountered an unusual issue while working with SCNRecorder.

Videos recorded in landscape orientation on iPhone 8 specifically running iOS 14.2 are rotated sideways and almost look as if they were recorded in locked portrait mode.
Videos recorded in portrait orientation work just fine.

It is a bit difficult to properly describe this issue so please refer to the three screenshots below.

I am using version 2.3.0 of SCNRecorder.

I have failed to replicate this issue on the following devices:

• iPhone 7 running iOS 13.6.
• iPhone 11 Pro running iOS 14.2
• iPad Pro 12.9'' (2015) running iPadOS 14.2

I have also attempted to play around with the video settings (specifically the size and the scaling mode) rather than using the default video settings but that didn't seem to help.

Thank you in advance for looking into this.

Hi, I really like this package, but I'm trying to do something with it for which I can't see an easy route.

I want to catch every frame before it's added to the video, and send it elsewhere instead of save it to a video file. Is this possible?

I tried using the capturePixelBuffers method, but the handler in there doesn't get called, not sure if it's supposed to work that way though.

Any help would be greatly appreciated 🙏

When using:

let configuration = ARWorldTrackingConfiguration()
configuration.providesAudioData = true
sceneView.session.run(configuration)

On some devices (like iPad pro and iPhone 12) the volume for recording using AVFoundation's AVAudioRecorder is dramatically reduced. If anyone knows a solution for this I'd be very grateful.

When I submit to appstore connect I get an error saying that it is required to have NSCameraUsageDescription key, even though I am not using the camera because I am recording only a ScnView.
Is there a way to disable the part relative to the camera API in the SDK so that Apple does not require NSCameraUsageDescription key?

Hi, thanks so much for making this tool! I have a feature request:

I'm trying to create an application that can export SceneKit views as .mov files, as a kind of authoring tool for videos of animations. However, I would like the exported .mov files to have transparent backgrounds. Would it be easy to add the ability to record SCNViews (and other views) with transparency preserved? Currently, SCNRecorder exported .mov files always show transparent pixels as black. I cloned your repo and tried to implement transparency myself, but unfortunately my level of expertise with pixel formatting, codecs, etc is not sufficient. Perhaps it's easy if you know what you're doing, though, since it might just be a matter of proper settings.

If it's any help, the GPU debugger reported the CAMetalLayer Display Drawable in my SCNView as using bgra8Unorm_srgb, and when I examined individual pixels in the GPU debugger, alpha was 0 where the pixels were transparent. My app correctly displays the SCNView with the transparent background, but the exported .mov file does not seem to have it.

Here's how I manually create a .mov with a transparent background. When I export a video clip with a transparent background from Adobe Premiere, I choose Quicktime (.mov) and set the code to "Apple ProRes 4444 with alpha". Then, in the Mac Finder, I right click on the exported .mov file and select "Encode Selected Video Files" from the menu. Then I choose "HEVC 1080p" from Setting, and click the "Preserve Transparency" checkbox. After a bit, the Mac produces a new .mov file, which I can then bring into Xcode and play in any application. The .mov's transparent background will display correctly in the new application.

However, when I produce a .mov file from SCNRecorder, then attempt to use the Finder's encoding tool with the above steps, Finder gives me an error if I check the "Preserve Transparency" checkbox:

///////
avconvert: invalid configuration (preset name PresetHEVC1920x1080WithAlpha)
Presets compatible with file:
Preset640x480
Preset960x540
Preset1280x720
Preset1920x1080
Preset3840x2160
PresetAppleM4V480pSD
PresetAppleM4V720pHD
PresetAppleM4V1080pHD
PresetAppleM4VAppleTV
PresetAppleM4VCellular
PresetAppleM4ViPod
PresetAppleM4VWiFi
PresetAppleProRes422LPCM
PresetAppleProRes4444LPCM
PresetHEVC1920x1080
PresetHEVC3840x2160
PresetHEVCHighestQuality
PresetHighestQuality
PresetLowQuality
PresetMediumQuality
///////

Thanks again, and no worry if this feature is too much bother to build.

Yours,
Geoff

Can you help to check it ? Thank you

Hi,
This is a great framework in terms of memory consumption but the video is coming out to be choppy; not all frames are being recorded. Also I don't know how to record the audio.
Any help will be appreciated!

Hello! Is it possible to pause video recording and then resume without pausing the entire scene?

Hello,

I have been using ARSceneView and SCNRecorder was perfectly capturing video with audio before upgrading to iOS 14.0. However, on iOS 14.0.1 causes a crash with:

ARKit session error: Error Domain=com.apple.arkit.error Code=102 "Required sensor failed." UserInfo={NSLocalizedFailureReason=A sensor failed to deliver the required input., NSUnderlyingError=0x282e7c570 {Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedFailureReason=An unknown error occurred (-12780), NSLocalizedDescription=The operation could not be completed, NSUnderlyingError=0x282e66c40 {Error Domain=NSOSStatusErrorDomain Code=-12780 "(null)"}}}, NSLocalizedRecoverySuggestion=Make sure that the application has the required privacy settings., NSLocalizedDescription=Required sensor failed.}

PS: The required plist entry to access the microphone is set, the app is allowed to use the mic and everything was working just fine prior to iOS 14.

When .providesAudioData is set to false, there is no crash and the video is correctly capture (without audio). The crash happens when providesAudioData is set to true. Any idea what is happening?

Thanks

I can't seem to install this via cocoapods.

[!] Unable to find a specification for SCNRecorder (~> 1.0)

[!] Unable to find a specification for SCNRecorder

Hello, when I try to take a photo or a video using your library on iPad Pro 2020 and iOS 14, the whole app freezes. I am not able to determine what's causing it because the debugger doesn't tell you anything; simply the app completely freezes. This wasn't the case on iOS 13+, exactly the same code.

What do you think it is? Would you mind looking into it? Many thanks.

Hello

I want to save some data from ARFrame and associate the data to the video frame captured from SCNRecorder

How I can not read the time when recorder is recording

func append(pixelBuffer: CVPixelBuffer, withPresentationTime time: CMTime) throws {
  guard pixelBufferAdaptor.assetWriterInput.isReadyForMoreMediaData else { return }
  guard pixelBufferAdaptor.append(pixelBuffer, withPresentationTime: time) else {
    if assetWriter.status == .failed { throw assetWriter.error ?? Error.unknown }
    return
}

  let seconds = time.seconds
  duration += seconds - lastSeconds
  lastSeconds = seconds
}

Here I want to get the CMTime here to save with my ARFrame data.

Thanks a lot

Hello,
I use your library for recording ARSCNView and it works great. I record on the iPhone 11 and iPhone SE and it has 60 fps during recording. But when I try to record in new iPhone 12 Pro, fps drops down to 27-35 FPS. Do you know about that problem ? How can I fix it ?

Is this expected? anyway to get closer to native camera's video resolution?

Hello, Thank you for wonderful project.

I bump up the library version to 2.0.0 to 2.2.0.
After I update version, I can't take photo with crash log. And no video after video recording.

I have some devices and I test "SCNRecorder".

• iPad mini 5(iOS 13.6)
• iPhone 11 Pro Max(iOS 13.5)
• iPhone XS(iOS 14.1)
• iPad Pro 12.1 2020(iPadOS 14.1)

I test by checkout following project tag 2.0.0, 2.1.3 and 2.2.0.
And run Example Project.
In ARSCNView Example

On version 2.0.0

Nice working

• iPad mini 5(iOS 13.6)
• iPhone 11 Pro Max(iOS 13.5)
• iPhone XS(iOS 14.1) (NavigationItem bug)

Not working

• iPad Pro 2020(iPadOS 14.1) (NavigationItem bug)

StartVideoRecording Not working

On version 2.1.3

Nice working

• iPad Pro 12.1 2020(iPadOS 14.1)
• iPhone XS(iOS 14.1)

Not working

• iPad mini 5(iOS 13.6)
  &
• iPhone 11 Pro Max(iOS 13.5)

TakePhoto Not working with error.

2020-11-16 12:18:16.303563+0900 Example[529:45566] Metal API Validation Enabled
2020-11-16 12:18:18.087331+0900 Example[529:45847] [CAMetalLayerDrawable texture] should not be called after already presenting this drawable. Get a nextDrawable instead.
Fatal error: Unexpectedly found nil while unwrapping an Optional value: file SCNRecorder/ImageOutput.swift, line 60
2020-11-16 12:18:18.089814+0900 Example[529:45768] Fatal error: Unexpectedly found nil while unwrapping an Optional value: file SCNRecorder/ImageOutput.swift, line 60

StartVideoRecording Not working with black video.

On version 2.2.0

Nice working

• iPad Pro 12.1 2020(iPadOS 14.1)
• iPhone XS(iOS 14.1)

Not working

• iPad mini 5(iOS 13.6)
  &
• iPhone 11 Pro Max(iOS 13.5)

TakePhoto Not working with error.

2020-11-16 12:28:26.076353+0900 Example[555:49819] Metal API Validation Enabled
Fatal error: Unexpectedly found nil while unwrapping an Optional value: file SCNRecorder/ImageOutput.swift, line 60
2020-11-16 12:28:27.593305+0900 Example[555:50010] Fatal error: Unexpectedly found nil while unwrapping an Optional value: file SCNRecorder/ImageOutput.swift, line 60

StartVideoRecording Not working with black video.

Clue

I try to find clue
I find TakePhoto
VTCreateCGImageFromCVPixelBuffer function is not working correctly iOS 13.

I can't find any clue StartVideoRecording

I think OS version dependent on this project.
Please check.

Hope you find reason soon!

Just want to start by saying I think this library is really great, especially when compared to the few other sceneView recorders libraries. This library produces such a light weight and smooth recording experience.

But,
Every so often, the scene view will freeze when invoking the sceneView.startVideoRecording() function, causing other areas of the UI to become unresponsive resulting in a force quit by the user. I tested on iPhone 8+, 4 out 25 AR record sessions experienced this freezing issue.

Is this a know issue? Has anybody else experienced library behavior as described above?

Possibly a resurgence of #51

I'm using the latest 2.8.0 version, with clean Derived Data, and Podfile.lock

The IDE is not pinpointing the precise location of the circular reference:

Below is a shortened version of the class:

import Foundation
import SCNRecorder
import AVKit
import AVFoundation
import AssetsLibrary

@objc public class SurfaceRecorder: NSObject {
    private var audioRecorder: AVAudioRecorder? =  nil
    private var audioURL: URL? = nil
    private var methodBCaptureSession: AVCaptureSession? = nil
    private var connection : AVCaptureConnection?
    private var fileOutput: AVCaptureMovieFileOutput?
    private var fileOutputURL: URL? = nil
    
    private var IARView: IARSurfaceView? = nil
    
    private var view: SCNView {
        guard let view = IARView?.internalSceneView() else {
            fatalError("Recording methods should be called after SurfaceView.load() method")
        }
        return view
    }

    // MARK: - Private Methods
    
    (...)
    
    private func startVideoRecording() -> NSError? {
        FileLogger.shared.log(content: "Recording started...")

        view.prepareForRecording()
        
        do {
            try view.startVideoRecording()
        } catch {
            FileLogger.shared.log(content: "Failed to start recording: \(error.localizedDescription)")
            return NSError.error(from: error)
        }
        
        return nil
    }
    
    (...)
    
    @objc public func stopRecording(finalFileName: String?, completionHandler handler: @escaping (NSURL?, Error?) -> Void) {
        
        if let audioRecorder = audioRecorder {
            audioRecorder.stop()
        }
        
        guard let audioURL = audioURL else {
            handler(nil, nsError(withText: "Failed to obtain audio file URL", code: IARErrorCode.errorRecordVideo))
            return
        }
        
        view.finishVideoRecording { [weak self] (videoRecording) in
            (...)
        }
    }
}

Circular references are likely to be caused by these lines:

        view.prepareForRecording()

       // and
        
        do {
            try view.startVideoRecording()

       // and

        view.finishVideoRecording { [weak self] (videoRecording) in

any advices?