unpause-live / swiftvideo Goto Github PK

View Code? Open in Web Editor NEW

195.0 13.0 25.0 381 KB

Swift Video Framework for Linux, macOS, and iOS/iPadOS

Home Page: https://www.unpause.live

License: Apache License 2.0

Swift 95.46% C 0.25% C++ 1.34% Metal 1.56% Dockerfile 1.38%

live-streaming ffmpeg swift

swiftvideo's Introduction

SwiftVideo

Video streaming and processing framework for Linux, macOS, and iOS/iPadOS/tvOS. Swift 5.1+ because I'm just opening this up and I really don't feel like dealing with older versions of Swift.

Getting Started

For now, check the Examples directory for some hints about how to use this framework. I promise I will be creating documentation to clarify how it all fits together and how you can do useful, interesting things with this framework.

Using with SwiftPM

Add .package(url: "https://github.com/unpause-live/SwiftVideo.git", from: "0.2.0") to your package dependencies
In your target dependencies, add either SwiftVideo or SwiftVideo_Bare depending on whether or not you wish to build with FFmpeg and Freetype support.

Using with Xcode + iOS

You can use this project in Xcode for iOS as a Swift Package as of 0.2.0.

Go to File -> Swift Packages -> Add Package Dependency, or in your Project settings go to the Swift Packages tab and press +
Set the package repository URL to https://github.com/unpause-live/SwiftVideo
Select the branch or version you want to reference. Only 0.2.0 and above will be usable on iOS unless you also compile FFmpeg and Freetype for it.
Choose the SwiftVideo_Bare product when prompted. This will build SwiftVideo without FFmpeg and Freetype. If you have built those libraries for iOS and wish to use them with SwiftVideo, choose the SwiftVideo product instead.
If you are using the VideoMixer you will need to include Sources/SwiftVideo/kernels.metal in your project directly so that they are included. This will be changed when Swift 5.3 is released.

Current Features

RTMP Client and Server
"Flavor" Client and Server (toy protocol, see flavor.md)
OpenCL Support
Metal Support
Audio Decode/Encode (via FFmpeg and CoreAudio)
Video Decode/Encode (via FFmpeg and CoreVideo)
Camera capture (macOS/iOS)
Text rendering (via FreeType)
Video Mixer
Audio Mixer
Audio Resampler (via FFmpeg+SOX)

FFmpeg support is thanks to https://github.com/sunlubo/SwiftFFmpeg

swiftvideo's People

Contributors

Stargazers

Watchers

swiftvideo's Issues

Apple Audio Capture

Support CoreAudio capture on Apple devices

Audio mixer should use floating point audio

Move the audio mixer from interleaved 16-bit audio to float32 planar

Occasional crash: video mixer

Relevant stack trace

#0  0x00007fd0532179a8 in _swift_release_dealloc () at /usr/lib/swift/linux/libswiftCore.so
#1  0x00007fd05320f4fb in swift_arrayDestroy () at /usr/lib/swift/linux/libswiftCore.so
#2  0x00007fd052fecb07 in $ss18_DictionaryStorageCfd () at /usr/lib/swift/linux/libswiftCore.so
#3  0x00007fd052fecc99 in $ss18_DictionaryStorageCfD () at /usr/lib/swift/linux/libswiftCore.so
#4  0x00007fd0532179ab in _swift_release_dealloc () at /usr/lib/swift/linux/libswiftCore.so
#5  0x0000562b926feb1c in $s10SwiftVideo0B5MixerC3mix33_0B6887970F57CAB2437618B11AE13F40LL2atyAA14ClockTickEventC_tFyycfU0_6$deferL_yyF (strongSelf=...) at /root/repo/Sources/SwiftVideo/Sources/SwiftVideo/mix.video.swift:105
#6  0x0000562b926fe5a7 in $s10SwiftVideo0B5MixerC3mix33_0B6887970F57CAB2437618B11AE13F40LL2atyAA14ClockTickEventC_tFyycfU0_ ()
    at /root/repo/Sources/SwiftVideo/Sources/SwiftVideo/mix.video.swift:104
#7  0x0000562b921ba329 in $sIeg_IeyB_TR ()
#8  0x00007fd053524547 in _dispatch_call_block_and_release () at /usr/lib/swift/linux/libdispatch.so
#9  0x00007fd05353000b in _dispatch_lane_serial_drain () at /usr/lib/swift/linux/libdispatch.so
#10 0x00007fd053530cea in _dispatch_lane_invoke () at /usr/lib/swift/linux/libdispatch.so
#11 0x00007fd053538baa in _dispatch_worker_thread () at /usr/lib/swift/linux/libdispatch.so
#12 0x00007fd052c396db in start_thread (arg=0x7fceecff9700) at pthread_create.c:463
#13 0x00007fd03f9ef88f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

relevant loc:
https://github.com/unpause-live/SwiftVideo/blob/master/Sources/SwiftVideo/mix.video.swift#L105

Metal kernels: RGBA->NV12, Support additional formats (yuvs, zvuy, I420)

10-bit picture samples

Currently only 8-bit is supported, add support for 10-bit pictures.

Apple audio encode/decode

Support Audiotoolbox encode/decode for some codecs (AAC in particular)

decode
encode

Create RTMP Local server

Hello,

I'm looking for a way to create RTMP server locally on an iOS app.

It is possible to do it with this library ? Have you any exemple ?
I tried to add SwiftVideo with SPM on iOS project but it seems to be broke.

Thanks :)

Audio/video mixing
Transcoding
iOS+macOS (using SwiftUI or something?)

Occasional crash: tcp listener

Stack trace:

#0  0x0000565463f36de4 in $s10SwiftVideo10ConnectionC13channelActive7contexty3NIO21ChannelHandlerContextC_tF (context=..., self=...) at /root/repo/Sources/SwiftVideo/Sources/SwiftVideo/net.tcp.swift:101
#1  0x0000565463bb0ca2 in $s3NIO21ChannelHandlerContextC06invokeB6Active33_EEC863903996E9F191EBAFEB0FB0DFDDLLyyF (self=...)
    at /root/repo/.build/checkouts/swift-nio/Sources/NIO/ChannelPipeline.swift:1319
#2  0x0000565463bab4bd in $s3NIO21ChannelHandlerContextC04fireB10RegisteredyyFTm ()
    at .build/checkouts/swift-nio/Sources/NIO/EventLoopFuture.swift:228
#3  0x0000565463ba322f in $s3NIO22_ChannelInboundHandlerPAAE13channelActive7contextyAA0bD7ContextC_tFAA012BackPressureD0C_Tg5 (context=...) at /root/repo/.build/checkouts/swift-nio/Sources/NIO/ChannelHandler.swift:284
#4  0x0000565463bb0ca2 in $s3NIO21ChannelHandlerContextC06invokeB6Active33_EEC863903996E9F191EBAFEB0FB0DFDDLLyyF (self=...)
    at /root/repo/.build/checkouts/swift-nio/Sources/NIO/ChannelPipeline.swift:1319
#5  0x0000565463baf926 in $s3NIO15ChannelPipelineC04fireB11Registered0yyFTm ()
    at .build/checkouts/swift-nio/Sources/NIO/EventLoopFuture.swift:228
#6  0x0000565463b8759d in $s3NIO17BaseSocketChannelC13becomeActive07promiseyAA16EventLoopPromiseVyytGSg_tFAA0C0C_Tg5 (promise=..., self=...) at /root/repo/.build/checkouts/swift-nio/Sources/NIO/BaseSocketChannel.swift:122
#7  0x0000565463c26f40 in $s3NIO19ServerSocketChannelC12channelRead0yyAA6NIOAnyVFyycfU_yyKcfU_ ()
    at /root/repo/.build/checkouts/swift-nio/Sources/NIO/SocketChannel.swift:287
#8  0x0000565463c2ad4c in $s3NIO19ServerSocketChannelC12channelRead0yyAA6NIOAnyVFyycfU_yyKcfU_TA ()
    at .build/checkouts/swift-nio/Sources/NIO/EventLoopFuture.swift:228
#9  0x0000565463c26f9c in $ss5Error_pIegzo_ytytsAA_pIegnrzo_TR ()
    at .build/checkouts/swift-nio/Sources/NIO/BaseStreamSocketChannel.swift:84
#10 0x0000565463c2ad61 in $ss5Error_pIegzo_ytytsAA_pIegnrzo_TRTA ()
    at .build/checkouts/swift-nio/Sources/NIO/EventLoopFuture.swift:228
#11 0x0000565463bddb1b in $s3NIO15EventLoopFutureC7flatMap4file4line_ACyqd__Gs12StaticStringV_SuAGxctlFAA12CallbackListVycfU_yt_ytTg508$s3NIO15bcd7C15flatf30Throwing4file4line_ACyqd__Gs12ij24V_Suqd__xKctlFAGxcfU_yt_V3TG5ACyytGytyts5Error_pIegnrzo_AISuTf1ncn_n () at /root/repo/.build/checkouts/swift-nio/Sources/NIO/EventLoopFuture.swift:506
#12 0x0000565463bdfca5 in $s3NIO15EventLoopFutureC12_addCallbackyAA0F4ListVAFycFyt_Tg508$s3NIO15bcd58C7flatMap4file4line_ACyqd__Gs12StaticStringV_SuAGxctlFAA12fg9VycfU_yt_T3Tg5ACyytGytAHIegno_AA0bC7PromiseVyytGTf1cn_n0hibcd29C15flatMapThrowing4file4line_l2__mno21V_Suqd__xKctlFAGxcfU_t1_T3TG5AHytyts5Error_pIegnrzo_s0nO0VSuTf1nncn_n (callback=..., self=...)
    at /root/repo/.build/checkouts/swift-nio/Sources/NIO/EventLoopFuture.swift:663
#13 0x0000565463be31b4 in $s3NIO15EventLoopFutureC13_whenCompleteyyAA12CallbackListVycFyt_Tg508$s3NIO15bcd58C7flatMap4file4line_ACyqd__Gs12StaticStringV_SuAGxctlFAA12gh9VycfU_yt_U3Tg5ACyytGytAHIegno_AA0bC7PromiseVyytGTf1cn_n0ijbcd29C15flatMapThrowing4file4line_m2__nop21V_Suqd__xKctlFAGxcfU_u1_U3TG5AHytyts5Error_pIegnrzo_s0oP0VSuTf1nncn_n (callback=..., self=...)
    at /root/repo/.build/checkouts/swift-nio/Sources/NIO/EventLoopFuture.swift:670
#14 0x0000565463bda343 in $s3NIO15EventLoopFutureC7flatMap4file4line_ACyqd__Gs12StaticStringV_SuAGxctlFyt_ytTg508$s3NIO15bcd7C15fl---Type <return> to continue, or q <return> to quit---d 
atf30Throwing4file4line_ACyqd__Gs12ij24V_Suqd__xKctlFAGxcfU_yt_T3TG5ACyytGytyts5Error_pIegnrzo_AISuTf1nncn_nTm ()
    at /root/repo/.build/checkouts/swift-nio/Sources/NIO/EventLoopFuture.swift:605
#15 0x0000565463bd93ba in $s3NIO15EventLoopFutureC7flatMap4file4line_ACyqd__Gs12StaticStringV_SuAGxctlFyt_ytTg508$s3NIO15bcd7C15flatf30Throwing4file4line_ACyqd__Gs12ij24V_Suqd__xKctlFAGxcfU_yt_T3TG5ACyytGytyts5Error_pIegnrzo_AISuTf1nncn_n ()
    at /root/repo/.build/checkouts/swift-nio/Sources/NIO/EventLoopFuture.swift:483
#16 0x0000565463c26ea9 in $s3NIO19ServerSocketChannelC12channelRead0yyAA6NIOAnyVFyycfU_ ()
    at .build/checkouts/swift-nio/Sources/NIO/EventLoopFuture.swift:504
#17 0x0000565463c1b5e1 in $sIeg_ytIegr_TRTA () at .build/checkouts/swift-nio/Sources/NIO/EventLoopFuture.swift:737
#18 0x0000565463c17dd0 in $s3NIO19withAutoReleasePoolyxxyKXEKlFyt_Tg5 ()
    at /root/repo/.build/checkouts/swift-nio/Sources/NIO/SelectableEventLoop.swift:350
#19 0x0000565463c17dd0 in $s3NIO19SelectableEventLoopC3runyyKF (self=...)
    at /root/repo/.build/checkouts/swift-nio/Sources/NIO/SelectableEventLoop.swift:381
#20 0x0000565463bd4f05 in $s3NIO27MultiThreadedEventLoopGroupC014setupThreadAnddE033_D5D78C61B22284700B9BD1ACFBC25157LL4name11initializerAA010SelectabledE0CSS_yAA9NIOThreadCctFZyAKcfU_ ()
    at /root/repo/.build/checkouts/swift-nio/Sources/NIO/EventLoop.swift:792
#21 0x0000565463bd8a09 in $s3NIO27MultiThreadedEventLoopGroupC014setupThreadAnddE033_D5D78C61B22284700B9BD1ACFBC25157LL4name11initializerAA010SelectabledE0CSS_yAA9NIOThreadCctFZyAKcfU_TA () at .build/checkouts/swift-nio/Sources/NIO/EventLoopFuture.swift:228
#22 0x0000565463c2ed94 in $s3NIO9NIOThreadCIegg_ACytIegnr_TRTA () at .build/checkouts/swift-nio/Sources/NIO/Utilities.swift:30
#23 0x0000565463c2e4cd in $s3NIO9NIOThreadC11spawnAndRun4name12detachThread4bodyySSSg_SbyACctFZSvSgAIcfU_ ()
    at /root/repo/.build/checkouts/swift-nio/Sources/NIO/Thread.swift:118
#24 0x00007fd2434b16db in start_thread (arg=0x7fd1d37fe700) at pthread_create.c:463
#25 0x00007fd23026788f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

loc: https://github.com/unpause-live/SwiftVideo/blob/master/Sources/SwiftVideo/net.tcp.swift#L101