occ-ai / obs-backgroundremoval Goto Github PK

An OBS plugin for removing background in portrait images (video), making it easy to replace the background when recording or streaming.

License: GNU General Public License v2.0

CMake 39.67% Shell 3.71% C++ 40.46% C 4.54% Objective-C++ 0.87% JavaScript 0.46% HTML 0.18% TypeScript 2.34% Svelte 6.78% CSS 1.00%

obs obs-plugin libobs mac-osx background-segmentation obs-studio onnx onnxruntime plugin

obs-backgroundremoval's Introduction

OBS Plugin: Portrait Background Removal / Virtual Green-screen and Low-Light Enhancement

A plugin for OBS Studio that allows you to replace the background in portrait images and video, as well as enhance low-light scenes.

Usage
Download
Introduction
Building

For other AI plugins from us check out OCC AI.

Usage

OBS.Background.Removal.Plugin.Tutorial-veqNEsMqEE0.mp4

Check out the usage guide page for usage walkthrough and recommendations.

Additional tutorial videos:

Usage tutorial: https://youtu.be/veqNEsMqEE0
Depth of Field effect: https://youtu.be/jC3EKSpNjQk
Low-light enhancement: https://youtu.be/WSBLYWFrn2Q
Remove background from ANY object (not just human): https://youtu.be/N74VCDCToX8

Download

Check out the guide page for downloads and install instructions for Windows and MacOS.

Linux Installation

On Ubuntu, there are two ways to install OBS and you have to use the corresponding method to install this plugin.

If you installed OBS via the official PPA, download the deb package from the releases page and install it directly.
If you installed OBS via FlatHub, run the following command: flatpak install com.obsproject.Studio.Plugin.BackgroundRemoval

The pre-built binary of this plugin can run Ubuntu 22.04 or later.

On openSUSE, please see docs/BUILDING-OPENSUSE.md.

FlatHub

On Debian, you cannot use our deb package and we only support FlatHub installation.

On other Linux distros, use the FlatHub installation of both OBS and this plugin.

flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo
flatpak install flathub com.obsproject.Studio
flatpak install flathub com.obsproject.Studio.Plugin.BackgroundRemoval

If you install OBS in a way other than FlatHub, you have to build this plugin by yourself (see instructions for building below).

OBS Version Support and Compatibility

OBS version 29+ (download) for the latest version of this plugin, e.g. 1.1.x and above.
OBS version 28+ for this plugin's version between 0.5.x and 1.0.x.
OBS version 27+ for this plugin's version 0.4.x and below.

Introduction

This plugin is meant to make it easy to replace the background in portrait images and video. It is using a neural network to predict the mask of the portrait and remove the background pixels. It's easily composable with other OBS plugins to replace the background with e.g. an image or a transparent color.

If you like this work, which is given to you completely free of charge, please consider supporting it by sponsoring us on GitHub:

Support and Help

Reach out to us on Discord or the OBS Plugins forum for online / immediate help.

If you found a bug or want to suggest a feature or improvement please open an issue.

If you are looking for hands-on help or private consultation please select a sponsorship tier.

Technical Details

GPU support:

Currently on Windows we support DirectML, which should reduce CPU usage by 95% and effectively use the systems accelerators (GPUs if available).
On Mac we support CoreML for acceleration, which is available on M1 and M2 (not Intel, sorry).
CUDA is supported in this plugin through TensorRT, however it is supported only on Linux.
The goal of this plugin is to be available for everyone on every system, even if they don't own a GPU.

Number of CPU threads is controllable through the UI settings. A 2-thread setting works best.

The pretrained model weights used for portrait foreground segmentation are taken from:

Image enhancement (low light) models are taken from:

Some more information about how I built it: https://www.morethantechnical.com/2021/04/15/obs-plugin-for-portrait-background-removal-with-onnx-sinet-model/ and https://www.morethantechnical.com/2023/05/20/building-an-obs-background-removal-plugin-a-walkthrough/

Code Walkthrough

This video on YouTube will take you through the major parts of the code and explain them.

Building

The plugin was built and tested on Mac OSX (Intel & Apple silicon), Windows and several Linux disros (e.g. Ubuntu/Debian-ish, Fedora, and more). Help is appreciated in building on other OSs and packages.

The building pipelines in CI take care of the heavy lifting. Use them in order to build the plugin locally. We attempt to use external OpenCV, libcurl and ONNX Runtime to reduce build times.

Start by cloning this repo to a directory of your choice.

Mac OSX

Using the CI pipeline scripts, locally you would just call the zsh script. By default this builds a universal binary for both Intel and Apple Silicon. To build for a specific architecture please see .github/scripts/.build.zsh for the -arch options.

$ ./.github/scripts/build-macos -c Release

Install

The above script should succeed and the plugin files (e.g. obs-backgroundremoval.plugin) will reside in the ./release/Release folder off of the root. Copy the .plugin file to the OBS directory e.g. ~/Library/Application Support/obs-studio/plugins.

To get .pkg installer file, run for example

$ ./.github/scripts/package-macos -c Release

(Note that maybe the outputs will be in the Release folder and not the install folder like pakage-macos expects, so you will need to rename the folder from build_x86_64/Release to build_x86_64/install)

Linux

Ubuntu

Use the CI scripts again

$ ./.github/scripts/build-linux.sh

Arch Linux

The community maintains AUR packages: https://aur.archlinux.org/packages/obs-backgroundremoval

Fedora

To compile on Fedora, you need to manage the dependencies manually. See docs/BUILDING-FEDORA.md for more information.

FlatHub

The plugin is available on FlatHub: https://github.com/flathub/com.obsproject.Studio.Plugin.BackgroundRemoval

$ flatpak install com.obsproject.Studio.Plugin.BackgroundRemoval

Windows

Use the CI scripts again, for example:

> .github/scripts/Build-Windows.ps1 -Target x64 -CMakeGenerator "Visual Studio 17 2022"

The build should exist in the ./release folder off the root. You can manually install the files in the OBS directory.

obs-backgroundremoval's People

Contributors

Stargazers

Watchers

Forkers

danurrahmajati fordfairlane76 matimax insad krisezra87 morrolinux brainhub24 brunodoamaral juicedatom packysauce ibenmoore maxmax27 chirag-nahata cy99 louie-github productinfo alexliesenfeld knoxthem ryanbekabe fubar-coder dpredie shahruk10 lukasmichel streammediaworkspace peepocord alirezataleghani samcday crinai mproffitt njannasch protojimmy errcode myanmargamer rodcanada rayaanf samuelbasso hirajanwin izraifel 307677814 kelemeto76 shriramm29 podycust martin9761 tbo77 codicssoftware mocyhomes peterzhousz direcka richtong gersur lbxx02 sinsixx jorik041 stvhanna elber73 asdlei99 korc adityakhare422 chfeizy xinsuinizhuan xheydar cristiancmoises siddharthrajpal sporedubz unclestranger benitofischer leetesla ilamhm esukorion chrismarch elsaico azazar jackoniel aziizz hkk97 xcfrg raag yimilirui cadalzolc dragon-s sekwah41 kilinbox jqhxx codersun123 yichiehqq leandrodaher directrix1 anylee2021 dvonthenen geekwish umireon longji0803 itachisan sdrshnptl finejustice madmartian magicking ohki-ki eonasdan sorenisanerd

obs-backgroundremoval's Issues

a request not an issue

would it be possible to have it so we can select the area/s we want it to remove instead of it trying to do it automatically?

OBS 27.0.1-2 (64 bit) frozes and chrashes when aktivating plugin in Linux Manjaro

Specs.:

OS: Manjaro Linux x86_64
Kernel: 5.10.52-1-MANJARO
CPU: Intel i5-4460 (4) @ 3.400GHz
GPU: NVIDIA GeForce GTX 960

Issue:

If I want to use the backgroundremoval-plugin with my logitech 930e obs frozes first and then it crashes. I never see the config dialog of this plugin.
I installed with Archlinux: install the package e.g. pacman -U obs-backgroundremoval*.tar.zst

Any ideas or fixes?

makepkg install is wrong

on Arch GNU/Linux the plugin needs to be installed under the home config or if you want to do it for all users as the package is setup to do in this case, to /usr/lib/obs-plugins

Compatibility issue with the obs-studio ppa

The default instructions did not work for me on ubuntu 20.04 with obs studio ppa because obs did not detect the plugin installation. Upon investigation it turns out that the obs ppa is utilizing the /lib/obs-plugins which is not in the CMakeLists.txt. So I wrote a patch for it:

--- CMakeLists.txt      2021-09-07 14:43:11.033428381 +0200
+++ CMakeLists-patched.txt      2021-09-07 14:44:30.217512505 +0200
@@ -239,13 +239,13 @@
        file(GLOB onnxmodels data/*.onnx)
 
        install(TARGETS ${CMAKE_PROJECT_NAME}
-               LIBRARY DESTINATION "${CMAKE_INSTALL_FULL_LIBDIR}/obs-plugins")
+               LIBRARY DESTINATION "/lib/obs-plugins")
 
        install(FILES ${locale_files}
-               DESTINATION "${CMAKE_INSTALL_FULL_DATAROOTDIR}/obs/obs-plugins/${CMAKE_PROJECT_NAME}/locale")
+               DESTINATION "/lib/obs-plugins/${CMAKE_PROJECT_NAME}/locale")
 
        install(FILES ${onnxmodels}
-               DESTINATION "${CMAKE_INSTALL_FULL_DATAROOTDIR}/obs/obs-plugins/${CMAKE_PROJECT_NAME}/")
+               DESTINATION "/lib/obs-plugins/${CMAKE_PROJECT_NAME}/")
 endif()
 # --- End of section ---

And now obs studio successfully display the filter, however trying to add the filter it breaks with a segfault and this is what I see in the core dump:

StacktraceAddressSignature: /usr/bin/obs:11:/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.28+ad6b0:/usr/lib/obs-plugins/obs->
StacktraceTop:
 __memmove_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:314
 std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::operator=(std::__cxx11::basic_string<ch>
 filter_update(void*, obs_data*) () from /usr//lib/obs-plugins/obs-backgroundremoval.so
 filter_create(obs_data*, obs_source*) () from /usr//lib/obs-plugins/obs-backgroundremoval.so
 ?? () from /lib/libobs.so.0

Versions are exact match (27.0.1 atm). The core dump has a path in it which is /usr//lib/obs-plugins and that does not look right to me. Ideas are welcomed.

PS: The default debian sources libobs0 and obs-studio did not work either because the filter is not detected at all so I'm sure it's using some other plugins directory like the ppa does. Either case I need the 26+ (deb one is 25) of obs due to its virtual camera (that plugin is a pain to maintain manually and breaks frequently).

Works great live but not on a pre-recorded video

Hi,
I tried this on my Mac with a live camera and it works great.. Nice work! I was hoping it would also work on a pre-recorded video but alas, it doesn't seem to do very well. I don't supposed that's something in your queue?

thanks for the hard work!
mike

Missing dependency in docs on linux: libsimde-dev

Tested on ubuntu 20.10:
cmake .. && cmake --build . && cmake --install . -- Configuring done -- Generating done -- Build files have been written to: /home/k/Projects/Downloaded/obs-backgroundremoval/build [ 33%] Building CXX object CMakeFiles/obs-backgroundremoval.dir/src/background-filter.cpp.o In file included from /usr/include/obs/graphics/vec4.h:22, from /usr/include/obs/graphics/vec3.h:21, from /usr/include/obs/obs.h:26, from /usr/include/obs/obs-module.h:20, from /home/k/Projects/Downloaded/obs-backgroundremoval/src/background-filter.cpp:1: /usr/include/obs/graphics/../util/sse-intrin.h:24:10: fatal error: simde/x86/sse2.h: No such file or directory 24 | #include <simde/x86/sse2.h> | ^~~~~~~~~~~~~~~~~~ compilation terminated. gmake[2]: *** [CMakeFiles/obs-backgroundremoval.dir/build.make:82: CMakeFiles/obs-backgroundremoval.dir/src/background-filter.cpp.o] Error 1 gmake[1]: *** [CMakeFiles/Makefile2:95: CMakeFiles/obs-backgroundremoval.dir/all] Error 2 gmake: *** [Makefile:149: all] Error 2

This is the most relevant item:
simde/x86/sse2.h: No such file or directory

I fixed this by installing libsimde-dev

Current release crashing in Windows 10 x64

Crash log:

https://dpaste.com/BVUYJL2YW

Incomplete build description and missing dependencies

Hi Roy,

first of all, thank you so much for delving into the merits of developing a so much anticipated and much needed plugin for the awesome OBS.

However, with regards to the build instructions I would like to be so bold as to mention several issues which I experienced during my struggle to get an error free build leaving me with frustration and finally giving up on the endeavour.

First of all, I had an issue in which Homebrew would not install correctly on the version of macOS Catalina which I’m running. However, I made it to the point that I could install the mentioned prerequisites.

Secondly, and this is actually missing in you description, cmake is not part of the Xcode development environment which I’m using, but has to be brewed separately. Using the Homebrew package of cmake results in a SIGILL error which I didn’t try to track down. Instead I used the CMake Mac application which I downloaded from cmake.org.

Last, but not least, using the CMake application to generate the make files I encountered another error which pinpointed me to the necessity to have the OBS source code available to build the libobs module. However, trying to build this module again left me with errors which I didn’t spend time to investigate further upon.

All in all the prerequisites for building your background removal module seems to require refinement and also seems to be more complex for green field build environments.

Therefore I’d like to ask you if it would be possible for you to make available a binary build of the module together with the model data in a compressed file which users could download and install more easily. I’d be happy to support you in your efforts by making a donation via your preferred platform.

Kind regards, all the best, stay safe and healthy, and greetings from Switzerland

Mati

OBS Studio 26.1.2 will not open (MacOS)

I was able to successfully build the project and moved over the .so and data files to OBS, but unfortunately OBS will no longer run.

Removing obs-backgroundremoval.so from the plugins directory allows OBS to run again.

Excerpt from crash report:

Exception Type:        EXC_BAD_INSTRUCTION (SIGILL)
Exception Codes:       0x0000000000000001, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Illegal instruction: 4
Termination Reason:    Namespace SIGNAL, Code 0x4
Terminating Process:   exc handler [2894]

Application Specific Information:
detected buffer overflow

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_c.dylib             	0x00007fff2046210d __chk_fail_overflow + 16
1   libsystem_c.dylib             	0x00007fff2046274c __memcpy_chk + 18
2   libobs.0.dylib                	0x00000001076ac06b obs_register_source_s + 299
3   obs-backgroundremoval.so      	0x0000000028fa3b15 obs_module_load + 21
4   libobs.0.dylib                	0x00000001076aae0e obs_init_module + 78
5   libobs.0.dylib                	0x00000001076abc80 load_all_callback + 64
6   libobs.0.dylib                	0x00000001076abb7e obs_find_modules + 2046
7   libobs.0.dylib                	0x00000001076ab363 obs_load_all_modules + 35
8   com.obsproject.obs-studio     	0x00000001049c87ce OBSBasic::OBSInit() + 734
9   com.obsproject.obs-studio     	0x00000001049a4cd9 OBSApp::OBSInit() + 681
10  com.obsproject.obs-studio     	0x00000001049a8d28 main + 5032
11  libdyld.dylib                 	0x00007fff2052c621 start + 1

Filter don't work with browser sources

I would like to use this plugin in my OBS.Ninja setup, but this filter couldn't be applied to my browser sources.

Performance Idea - option to reprocess mask every X frames

I've had to opportunity to use this plugin now on several occasions to feed the virtual camera output into webex meetings, and it's been working well, but like many others I find that cpu usage is a bottleneck that effects performance. Tweaking and/or trying out different models could improve that. But another thing I was thinking of, and I'm not sure how feasible it is, but maybe an option could be created to reprocess the mask every X frames?

I'm guessing the work of the models reprocessing what the mask should be is the big hitter on cpu. Right now I can run about 15fps max against my cpu's (with some frame droppage), which isn't the smoothest experience as far as watching lips move on a web meeting, etc, But even rendering at 5fps the mask seems to hold up well. So I was thinking, what if for example there was an option to only reprocess the mask every X frames. Then, if you wanted the existing experience, you could set it to1, but it might be possible in my case to set it to 5 and be able to render 30fps video while in the background the models are only actually having to process 6fps. Again, I haven't reviewed the code to get an idea of how feasible it would be to implement, just throwing another idea out there on a way to maybe improve performance.

Is it possible to create a new release?

Hi! It looks like in between the latest v0.3.0-beta release and the latest commit, there have been a few features added, most importantly:

Processing mask every X frames of video (8d64b35)
Using CUDA (0f101de)

I think these features, especially the first one, are rather useful, and so, is it possible to request a new release be made along with some build artifacts, since most people might find it difficult to manually compile?

In the future, maybe we could automate this with GitHub Actions.

Thank you!

Installation on an M1 Mac w/Monterey

Managed to get a little further than #48 via a few changes.

I observed how other plugins (StreamFX & obs-ndi) had their file hierarchy on macos and changed it up to match the /Library/Application Support/obs-studio/ here's what I ended up with:

then I got busted by the code signature police:

I can probably bypass number 2 by building it myself

How install on Mac Catalina ?

but background remove filter doesnt show up,

sorry, anyone can help me?

Crash On filter effect application

Every time that I try to apply the Background Removal Effect, it crashes OBS on Manjaro:
Specs:
System: Host: Synolvia-GX1 Kernel: 5.12.19-1-MANJARO x86_64 bits: 64 compiler: gcc v: 11.1.0 Desktop: Deepin 20.2.2 Distro: Manjaro Linux base: Arch Linux Machine: Type: Desktop Mobo: Gigabyte model: H170-Gaming 3 v: x.x serial: <superuser required> UEFI-[Legacy]: American Megatrends v: F2 date: 09/01/2015 CPU: Info: Quad Core model: Intel Core i5-6600K bits: 64 type: MCP arch: Skylake-S rev: 3 cache: L2: 6 MiB flags: avx avx2 lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx bogomips: 28009 Speed: 3750 MHz min/max: 800/3900 MHz Core speeds (MHz): 1: 3750 2: 3820 3: 3697 4: 3755 Graphics: Device-1: NVIDIA TU104 [GeForce RTX 2060] vendor: eVga.com. driver: nvidia v: 470.63.01 bus-ID: 01:00.0 Device-2: NVIDIA GF108 [GeForce GT 730] vendor: eVga.com. driver: vfio-pci v: 0.2 bus-ID: 06:00.0 Device-3: Sunplus Innovation MiraBox Video Capture type: USB driver: snd-usb-audio,uvcvideo bus-ID: 1-6.4.1:9 Device-4: Logitech HD Pro Webcam C920 type: USB driver: snd-usb-audio,uvcvideo bus-ID: 1-6.4.2:11 Display: x11 server: X.Org 1.20.13 driver: loaded: nvidia resolution: 1: 1440x900~60Hz 2: 1920x1080~60Hz 3: 1440x900~60Hz OpenGL: renderer: NVIDIA GeForce RTX 2060/PCIe/SSE2 v: 4.6.0 NVIDIA 470.63.01 direct render: Yes

Add Peter Lin's RVM model support

There's a great AI network model that seems to remove background much more effectively than all currently supported models. Could you integrate it into your plugin?
https://github.com/PeterL1n/RobustVideoMatting

Doesn't work with me (I'm brown and my background is kinda brown too)

I think I'm hard to identify to this plugin, because it can't identify me and "remove" the background. If I use the default settings, it shows a black screen, and if I play a little with the settings it shows this:

Still having build issues on OSX

Good day.

Thank you for the recent updates. I'm still unable to use the plugin on OSX. So far:

With the compiled plugin: onnxruntime cannot be found. If create a symlink to 1.5 from 1.7 the app crashes.
I tried compiling on my machine again. With the updated code and instructions I'm able to run

cmake .. \
-DobsLibPath=/Applications/OBS.app/Contents/Frameworks \
-DobsIncludePath="../../obs-studio/libobs" \
-DLIBOBS_INCLUDE_DIR="../../obs-studio/libobs" \
-DLIBOBS_LIB="../../obs-studio/build/libobs/libobs.dylib"

but I DO have to specify DLIBOBS... attributes which are absent in the README. Not sure what I'm missing in my setup.

Running cmake --build . fails with :

cmake --build .
Scanning dependencies of target obs-backgroundremoval
[ 33%] Building CXX object CMakeFiles/obs-backgroundremoval.dir/src/background-filter.cpp.o
[ 66%] Building CXX object CMakeFiles/obs-backgroundremoval.dir/src/plugin-main.cpp.o
make[2]: *** No rule to make target `<path>/obs-studio/build/libobs/libobs.dylib', needed by `obs-backgroundremoval.so'.  Stop.
make[1]: *** [CMakeFiles/obs-backgroundremoval.dir/all] Error 2
make: *** [all] Error 2

I looked online and saw that other plugins are compiled using make -j4. I executed that and it worked(?):

Scanning dependencies of target obs-backgroundremoval
[ 66%] Building CXX object CMakeFiles/obs-backgroundremoval.dir/src/plugin-main.cpp.o
[ 66%] Building CXX object CMakeFiles/obs-backgroundremoval.dir/src/background-filter.cpp.o
[100%] Linking CXX shared module obs-backgroundremoval.so
[100%] Built target obs-backgroundremoval

However after I copy the plugin, OBS fails to start:

Exception Type:        EXC_BAD_INSTRUCTION (SIGILL)
Exception Codes:       0x0000000000000001, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Illegal instruction: 4
Termination Reason:    Namespace SIGNAL, Code 0x4
Terminating Process:   exc handler [19910]

Application Specific Information:
detected buffer overflow

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_c.dylib             	0x00007fff67a3dfca __chk_fail_overflow.cold.1 + 16
1   libsystem_c.dylib             	0x00007fff67a3b214 __chk_fail_overflow + 9
2   libsystem_c.dylib             	0x00007fff67a3b848 __memcpy_chk + 18
3   libobs.0.dylib                	0x0000000110daf06b obs_register_source_s + 299
4   obs-backgroundremoval.so      	0x0000000028444925 obs_module_load + 21
5   libobs.0.dylib                	0x0000000110dade0e obs_init_module + 78
6   libobs.0.dylib                	0x0000000110daec80 load_all_callback + 64
7   libobs.0.dylib                	0x0000000110daeb7e obs_find_modules + 2046
8   libobs.0.dylib                	0x0000000110dae363 obs_load_all_modules + 35
9   com.obsproject.obs-studio     	0x000000010e0d47ce OBSBasic::OBSInit() + 734
10  com.obsproject.obs-studio     	0x000000010e0b0cd9 OBSApp::OBSInit() + 681
11  com.obsproject.obs-studio     	0x000000010e0b4d28 main + 5032
12  libdyld.dylib                 	0x00007fff6796ecc9 start + 1

And that's not ideal. Please advise.

[RECOMENDATION] Do better tutorial to install, or do and .exe to install.

Hi, is hard to install this plugin. I already tried with the "tutorial" from the release page, and it's not working. I do the same that the "tutorial" say :

To install
    Windows: Copy .dlls to e.g. C:\Program Files\obs-studio\obs-plugins\64bit, and .onnx file to e.g. C:\Program Files\obs-studio\data\obs-plugins\obs-backgroundremoval

PKGBUILD archlinux minor issues

pkgver_hyphen variable definition requires a tag on the latest git commit (which is not always the case). This is easily fixed by either adding a tag for each commit, or something like:

pkgver_hyphen=$(git describe --exact-match --tags $(git rev-list --tags --max-count=1))

Also a minor issue in the build file where SIPNet_Softmax.onnx is not included in the data folder. I THINK it has worked by changing line 242 in the CmakeLists.txt to read install(FILES "data/SINet_Softmax_simple.onnx"

Black Screen

Hi,
The most recent updates by background remover and OBS has stopped the crashing, but I choose facetime camera then background remover then all the BR options are their before I turn on the BR, but when I turn on BR I see a black screen no facetime of me , but have the option to select a new color.

Mac OS High Sierra
OBS 27.1.3.
Background removal - Current

This is from the log file.

https://obsproject.com/logs/C-IaBJTW7dqIzClK

Thanks so much!

Wrong MODNet variant

The ONNX version of MODNet originally cited is optimised for still image rather than video matting.

The correct one (corresponding with this model checkpoint) will need to be re-exported to ONNX.
I have done the same, but the unoptimised/ -simplified result crashes OBS and I wasn't sure exactly what toolset was used to get to the ..._simple.onnx versions.

Installed But Not Showing

Hi Team,
I have followed instruction to install plugin but when open OBS it is not listed in the filter

s

Obs Background Removal for Ubuntu

i like they added background removal for linux (FOR ARCHLINUX)!
i am using ubuntu plz add support for ubuntu
make it .deb file
plz
Thanks

Can't apply filter to source mirror

Hi, not sure if what I was trying is even possible, but I wanted to build a blurred background thing by having the same v4l source once with and once without background removal. (For context, blur filters are, for example, available in streamfx).

Since you can't have two instances of the webcam input with different filters I used source mirrors, but I can't seem to add the background removal filter to a source mirror. The add menu opens up fine, I can name it but after pressing "OK" nothing happens (i.e. no filter appears in the menu, the effect is not applied). I get this output in the log:

info: [obs-backgroundremoval] Model input shape 3 x 320 x 320, output shape 2 x 320 x 320
info: User added filter 'Background Removal' (background_removal) to source 'Source Mirror 2'
info: [obs-backgroundremoval] Destroy scalers.

I am using the PGBUILD for Arch provided on master (currently commit ab91510) and version 27.0.0-1 of the official package on Arch.

On a side note, shouldn't the filter be an effect filter? Not really familiar with OBS but it seems that working the same way as the Chroma Key filter would make sense :)

Really happy that you built this plugin, thanks for putting in the work!

Precising Linux build instructions

I suggest expanding the build instruction with the following, which will be of interest to people who got OBS Studio from the official PPA.

The PPA does not contain a libobs or libobs-dev and one should not install the version that comes with Debian/Ubuntu as it will not be the same version.
One should get the sources for the installed version, as in the case of a MacOS build : currently git clone --single-branch -b 26.1.1 [email protected]:obsproject/obs-studio.git, then tell cmake where to find it.
Furthermore, one should instruct cmake to install to /usr, not /usr/local as it would by default, because the PPA version of OBS will look there.
So the cmake configuration line is

cmake .. -DobsIncludePath=~/Downloads/obs-studio/libobs -DCMAKE_INSTALL_PREFIX:PATH=/usr

Also, I found that a sudo ldconfig was in order after unpacking onnxruntime but that may be a peculiarity of my system.

crash after adding it to the cam

I get this:

info: [obs-backgroundremoval] Destroy scalers.
info: [obs-backgroundremoval] Model location (null)
error: [obs-backgroundremoval] Unable to get model filename from plugin.
info: User added filter 'Background Removal' (background_removal) to source 'cam1'
info: [obs-backgroundremoval] Destroy scalers.
info: [obs-backgroundremoval] Initialize scalers. Size 640 x 480
terminate called after throwing an instance of 'std::out_of_range'
  what():  vector::_M_range_check: __n (which is 3) >= this->size() (which is 0)
[1]    34840 abort (core dumped)  obs

Any ideas? seems it cant find the models? what can I do?

Error on Mac OS Big Sur OBS Studio 27

System
MacBook Pro (15-inch, 2018)
macOS Big Sur 11.6
Processor 2,2 GHz-6-Core Intel Core i7
Memory 16 GB 2400 MHz DDR4
Graphics Radeon Pro 555x 4GB Intel UHD Graphics 630 1536 MB

OBS
OBS Stduio 27.0.0 (64 bit)

no linux .zip file?

hello on obs forum says this plugin supports linux

Supported Platforms
    Windows, Mac OS X, Linux

But on the release only have two zip files one for windows and other for macOS...
https://github.com/royshil/obs-backgroundremoval/releases/tag/v0.2.4-beta

I am using Arch GNU/Linux.

Background Remover crashes OBS

Hi.
I was told by the developer that he hasn't updated it for OBS 27 yet and to bring my issue here. I was able to install it and open it in OBS 27, but every time I activate a camera either externally or internally it crashes. My main computer is a Mac OS High Sierra and this is what the OBS log report was. Thanks:
https://obsproject.com/logs/1B8vp1Uag7iHbEsg

High Computepower Usage - low FPS

I love this Plugin but it demands min. 30% of my CPU-Usage when I use the GPU-Render on my Windows System. Without Recording or Streaming. The performance drop to 18 FPS.
I am not sure if it is possible to reduce that. If not please close the Issue.

Very high CPU usage

Specs:
CPU: i7-9700K
GPU: RTX 2080 Super
OS: Windows 11
Segmentation model: Robust Video Matting
CPU Usage: over 90% (unsure, total usage goes to 100% so there may be some throttling)

I'd like to first ask others if this is an issue as well, but also more about trying to figure out if there's a way to make this a bit more efficient to reduce CPU usage. To even have it be usable while playing and streaming

I had to put it calculating the mask every 4 frames, reducing the CPU to 42-50%, so that with everything else running I get to around 90%. Not a very high quality output of course.

Selecting the GPU as the inference device still makes the CPU work at over 88-90% just for OBS, perhaps there is a way to offload more of the work in an efficient way?

Perhaps this isn't even something that's supposed to happen in the first place, happy to hear some suggestions. I'm using the Robust Video Matting because it's what works best for me. All the other ones flicker too much and aren't as accurate. I'm actually incredibly surprised that Robust Video Matting is pretty much perfect in terms of its accuracy.

CUDA mode is not working

I built a plugin with CUDA mode,
But it shows "No properties available".

I have a NVIDIA RTX 3009 installed in my pc

Can anyone help?

Masking is sometimes incomplete

OBS 27.1.3, windows x64 and plugin version 0.4.0.
Occurs with all different models (mediapipe, selfie segmentation, the new one, the old ones)
Tried it with both CPU and GPU.

I've noticed that if I make an enclosed space, then the space doesn't get masked.

I think it's the same issue when the enclosed space is bounded by an edge

Is this expected behaviour?

Not building on Archlinux

Hi,
This is what happens when I try to build on Archlinux following the README instructions:

[morro@t480s build]$ cmake .. && cmake --build . && cmake --install .
CMake Warning (dev) at /usr/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
  The package name passed to `find_package_handle_standard_args` (Libobs)
  does not match the name of the calling package (LibObs).  This can lead to
  problems in calling code that expects `find_package` result variables
  (e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
  external/FindLibObs.cmake:89 (find_package_handle_standard_args)
  CMakeLists.txt:50 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Could NOT find Libobs (missing: LIBOBS_INCLUDE_DIR) 
CMake Error at external/FindLibObs.cmake:111 (message):
  Could not find the libobs library
Call Stack (most recent call first):
  CMakeLists.txt:50 (find_package)


-- Configuring incomplete, errors occurred!
See also "/home/morro/obs-backgroundremoval/build/CMakeFiles/CMakeOutput.log".

If I edit external/FindLibObs.cmake and change find_package_handle_standard_args(Libobs ....) to find_package_handle_standard_args(LibObs ....) I still get

[morro@t480s build]$ cmake .. && cmake --build . && cmake --install .
CMake Error at /usr/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find LibObs (missing: LIBOBS_INCLUDE_DIR)
Call Stack (most recent call first):
  /usr/share/cmake-3.20/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
  external/FindLibObs.cmake:89 (find_package_handle_standard_args)
  CMakeLists.txt:50 (find_package)


-- Configuring incomplete, errors occurred!
See also "/home/morro/obs-backgroundremoval/build/CMakeFiles/CMakeOutput.log".

I do have libobs under /usr/lib/pkgconfig where libobs.pc is found but if I export LIBOBS_INCLUDE_DIR="/usr/lib/pkgconfig/" the output doesn't change so I'm probably not doing it right.
Any suggestions?

Crashed on windows when applied to camera

carsh_report.txt
A few seconds after the filter is added OBS Studio crashes.

What about an alpha channel?

Did you test out adding transparency with an alpha channel instead of setting the color?

There is an Alpha YUV format. So it seems like it wouldn't be too hard to add a parameter for transparency, convert to an AYUV, apply the mask, then convert back to the proper output format.

Thoughts? I can fork and give it a shot, but wanted to see if you had tried going direct to transparent already.

AUR inclusion

Hi,

Is there any chance of this being added to the AUR?

How to contribute?

Hello sir,

I saw both in the README and in the forum that you would like help for both Windows and Linux.

As I can help with that, I would like to know what is the process to help contributing. :)

New BG segmentation models

Checkout this and see if this gives better result https://elder.dev/posts/open-source-virtual-background/

I am not C++ developer so unfortunately cannot contribute directly over here. Hopefully this will help.

Suggestion new model

Hi @royshil .

I found one ML project that looks promising and it should be great match with this plugin.
https://github.com/PeterL1n/RobustVideoMatting

This project created by same team that create this project. https://github.com/PeterL1n/BackgroundMattingV2
I used BackgroundMattingV2 before when I made virtualcam in Ubuntu. Result is really great when I compared with ML that created for body/human recognization because this one is more simpler.

BTW RobustVideoMatting is better because it doesn't need background as initial input like BackgroundMattingV2.

I just suggest to you. Feel free to skip it. Anyways it's good to see OBS plugin like this. Thank you for your work.

Missing binary for Linux in the latest release 0.4.0

obs-backgroundremoval*.tar.zst is missing in the "Assets" of the latest release 0.4.0.

Can't install on Ubuntu

Error:

CMake Error: The source directory "/home/nick/Temp" does not appear to contain CMakeLists.txt.

Full log

nick@nickolay:~/Temp$ wget https://github.com/microsoft/onnxruntime/releases/download/v1.7.0/onnxruntime-linux-x64-1.7.0.tgz

--2021-10-19 11:16:41--  https://github.com/microsoft/onnxruntime/releases/download/v1.7.0/onnxruntime-linux-x64-1.7.0.tgz
Resolving github.com (github.com)... 140.82.121.4
Connecting to github.com (github.com)|140.82.121.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://github-releases.githubusercontent.com/156939672/ecbd9100-7b3d-11eb-9810-a4893d2e3ffc?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20211019%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211019T081641Z&X-Amz-Expires=300&X-Amz-Signature=bee7c9edf3154c107682549295d743b45f21b9b3c90f0ee5d456d97393366718&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=156939672&response-content-disposition=attachment%3B%20filename%3Donnxruntime-linux-x64-1.7.0.tgz&response-content-type=application%2Foctet-stream [following]
--2021-10-19 11:16:41--  https://github-releases.githubusercontent.com/156939672/ecbd9100-7b3d-11eb-9810-a4893d2e3ffc?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20211019%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211019T081641Z&X-Amz-Expires=300&X-Amz-Signature=bee7c9edf3154c107682549295d743b45f21b9b3c90f0ee5d456d97393366718&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=156939672&response-content-disposition=attachment%3B%20filename%3Donnxruntime-linux-x64-1.7.0.tgz&response-content-type=application%2Foctet-stream
Resolving github-releases.githubusercontent.com (github-releases.githubusercontent.com)... 185.199.110.154, 185.199.108.154, 185.199.111.154, ...
Connecting to github-releases.githubusercontent.com (github-releases.githubusercontent.com)|185.199.110.154|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4362964 (4,2M) [application/octet-stream]
Saving to: ‘onnxruntime-linux-x64-1.7.0.tgz’
onnxruntime-linux-x64-1.7 100%[==================================>]   4,16M  20,4MB/s    in 0,2s    
2021-10-19 11:16:42 (20,4 MB/s) - ‘onnxruntime-linux-x64-1.7.0.tgz’ saved [4362964/4362964]

nick@nickolay:~/Temp$ sudo tar xzvf onnxruntime-linux-x64-1.7.0.tgz --strip-components=1 -C /usr/local/ --wildcards "*/include/*" "*/lib*/"

onnxruntime-linux-x64-1.7.0/include/onnxruntime_cxx_api.h
onnxruntime-linux-x64-1.7.0/include/cuda_provider_factory.h
onnxruntime-linux-x64-1.7.0/include/onnxruntime_session_options_config_keys.h
onnxruntime-linux-x64-1.7.0/include/onnxruntime_c_api.h
onnxruntime-linux-x64-1.7.0/include/cpu_provider_factory.h
onnxruntime-linux-x64-1.7.0/include/onnxruntime_cxx_inline.h
onnxruntime-linux-x64-1.7.0/lib/
onnxruntime-linux-x64-1.7.0/lib/libonnxruntime.so.1.7.0
onnxruntime-linux-x64-1.7.0/lib/libonnxruntime.so

nick@nickolay:~/Temp$ mkdir build && cd build

nick@nickolay:~/Temp/build$ cmake .. && cmake --build . && cmake --install .

CMake Error: The source directory "/home/nick/Temp" does not appear to contain CMakeLists.txt.
Specify --help for usage, or press the help button on the CMake GUI.
nick@nickolay:~/Temp/build$

Linux (Ubuntu 21.10) GPU Instructions

I haven't seen anything in the documentation on this, and the code is geared around disabling GPU on any platform other than Windows but I've been playing around with this plugin today and was able to get this working with CUDA on Ubuntu Studio 21.10

Use this at your own risk - I'm not responsible if you break your card.

To enable this, you need to patch 3 files:

src/Model.h
src/background-filter.cp
CMakeLists.txt

Patch file

gpu-linux.diff.txt

Apply the patch file with git apply gpu-linux.diff.txt

onnxruntime

Linking needs to be with the GPU version of onnx - you can find the 1.7.0 release of this here: https://github.com/microsoft/onnxruntime/releases/download/v1.7.0/onnxruntime-linux-x64-gpu-1.7.0.tgz

Alternatively the 1.9.0 release also seems to work and can be found here: https://github.com/microsoft/onnxruntime/releases/download/v1.9.0/onnxruntime-linux-x64-gpu-1.9.0.tgz

If you're using the GPU version of onnxruntime, you can't seem to switch between CPU and GPU - I haven't looked to see if this is possible, nor am I likely to try.

Building

The build procedure is the same as already documented

mkdir build && cd build
cmake .. -DWITH_CUDA=ON && cmake --build . && sudo cmake --install .

Install (Ubuntu bundled, not PPA):

mkdir -p ~/.config/obs-studio/plugins/obs-backgroundremoval/{bin,data}
cp /usr/local/lib/obs-plugins/obs-backgroundremoval.so ~/.config/obs-studio/plugins/obs-backgroundremoval/bin
cp -r /usr/local/share/obs/obs-plugins/obs-backgroundremoval/* ~/.config/obs-studio/plugins/obs-backgroundremoval/data

Tested with NVIDIA RTX 3070 TI

The Ubuntu PPA for OBS Studio and bundled OBS Studio both seem to work fine although the bundled version takes a little more setting up.

It seems to run on GPU, I've been running it for a few hours with a video background with the following observations:

GPU warms up to about 58°C then the fan kicks in and brings it down
GPU processing is low - around 9 or 10⅜ on my card
As long as you don't use Teams for too long, it seems stable - with Teams reading a virtual camera, expect to see some lag build up over time. This doesn't happen without the plugin so I'm presuming a relationship here.
CPU version of onnxruntime-1.9.0 seemed to crash a lot when combined with StreamFX - I have not tested the GPU version of the library with StreamFX plugins loaded

2021-11-18_20-25-14.mp4

background color channels seem to be in wrong order

When i set #0000ff as the background color the picker shows it as blue, but the output color is actually red. Other way around also works (#ff0000 -> blue).
So probably somewhere the channel order needs to be reversed.

Feature request: edge padding.

Good day.

Love the plugin. Good work. I feel like a few things needed to make it usable.

Here's what I got when I tried to blur my background:

As you can see it doesn't think my hair, headset, forehead and eyebrows are part of me. And around my shoulders the transition line is very hard.

Shoulder tracking is amazing... but the issues above make this ... "interesting" to use.

Anyway, I would like to ask for a feature where instead of filling the edge at the edge of the match, if we could do so within X pixels. I.e. for my use case I don't care if I reveal 30 pixels around my shoulders, but I would really like my eyebrows (and sometimes eyes) to not be considered "background".

Like... this is funny.... but I wouldn't get on a work call like this:

If not obvious, this happens at all threshold levels. Max 'usable' threshold level is .95 which looks like this for me:

The other dials only make things worse.

Build Failure on M1 Macs

Hi! Thanks for making and supporting this :) I ran into an issue on my M1 mac and didn't see it in this tracker so I wanted to add it. It's not too surprising, but in case there's an easy fix I wanted to share.

Things go fine until I get to the build step and then I get this

❯ cmake --build . --target dist
[ 33%] Linking CXX shared module obs-backgroundremoval.so
ld: warning: ignoring file /Applications/OBS.app/Contents/Frameworks/libobs.0.dylib, building for macOS-arm64 but attempting to link with file built for macOS-x86_64
Undefined symbols for architecture arm64:
  "_bfree", referenced from:
      filter_destroy(void*) in background-filter.cpp.o
      createOrtSession(background_removal_filter*) in background-filter.cpp.o
  "_blog", referenced from:
      filter_render(void*, obs_source_frame*) in background-filter.cpp.o
      destroyScalers(background_removal_filter*) in background-filter.cpp.o
      createOrtSession(background_removal_filter*) in background-filter.cpp.o
      processImageForBackground(background_removal_filter*, cv::Mat const&, cv::Mat&) in background-filter.cpp.o
      initializeScalers(cv::Size_<int>, video_format, background_removal_filter*) in background-filter.cpp.o
      _obs_module_load in plugin-main.cpp.o
      _obs_module_unload in plugin-main.cpp.o
      ...
  "_bmalloc", referenced from:
      bzalloc(unsigned long) in background-filter.cpp.o
  "_obs_data_get_double", referenced from:
      filter_update(void*, obs_data*) in background-filter.cpp.o
  "_obs_data_get_int", referenced from:
      filter_update(void*, obs_data*) in background-filter.cpp.o
  "_obs_data_get_string", referenced from:
      filter_update(void*, obs_data*) in background-filter.cpp.o
  "_obs_data_set_default_double", referenced from:
      filter_defaults(obs_data*) in background-filter.cpp.o
  "_obs_data_set_default_int", referenced from:
      filter_defaults(obs_data*) in background-filter.cpp.o
  "_obs_data_set_default_string", referenced from:
      filter_defaults(obs_data*) in background-filter.cpp.o
  "_obs_find_module_file", referenced from:
      createOrtSession(background_removal_filter*) in background-filter.cpp.o
  "_obs_module_load_locale", referenced from:
      _obs_module_set_locale in plugin-main.cpp.o
  "_obs_properties_add_color", referenced from:
      filter_properties(void*) in background-filter.cpp.o
  "_obs_properties_add_float_slider", referenced from:
      filter_properties(void*) in background-filter.cpp.o
  "_obs_properties_add_int", referenced from:
      filter_properties(void*) in background-filter.cpp.o
  "_obs_properties_add_list", referenced from:
      filter_properties(void*) in background-filter.cpp.o
  "_obs_properties_create", referenced from:
      filter_properties(void*) in background-filter.cpp.o
  "_obs_property_list_add_string", referenced from:
      filter_properties(void*) in background-filter.cpp.o
  "_obs_register_source_s", referenced from:
      _obs_module_load in plugin-main.cpp.o
  "_text_lookup_destroy", referenced from:
      _obs_module_set_locale in plugin-main.cpp.o
      _obs_module_free_locale in plugin-main.cpp.o
  "_text_lookup_getstr", referenced from:
      _obs_module_text in plugin-main.cpp.o
      _obs_module_get_string in plugin-main.cpp.o
  "_video_scaler_create", referenced from:
      initializeScalers(cv::Size_<int>, video_format, background_removal_filter*) in background-filter.cpp.o
  "_video_scaler_destroy", referenced from:
      destroyScalers(background_removal_filter*) in background-filter.cpp.o
  "_video_scaler_scale", referenced from:
      convertFrameToBGR(obs_source_frame*, background_removal_filter*) in background-filter.cpp.o
      convertBGRToFrame(cv::Mat const&, obs_source_frame*, background_removal_filter*) in background-filter.cpp.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[3]: *** [obs-backgroundremoval.so] Error 1
make[2]: *** [CMakeFiles/obs-backgroundremoval.dir/all] Error 2
make[1]: *** [CMakeFiles/dist.dir/rule] Error 2
make: *** [dist] Error 2

Crash just after applying filter

On Linux, after applying the filter, I get vertical green stripes in the preview and shortly after, the program crashes. I'm not really familiar with OBS, openCV and the onnxruntime machine learning thing at all, so I do not have a clue on what's going on.

Some, hopefully useful details:

OS: OpenSUSE Tumbleweed 20210418 (Linux)
Compiler: gcc version 10.2.1 20210401 [revision 892024d4af83b258801ff7484bf28f0cf1a1a999] (SUSE Linux)

Webcam source I'm applying the filter on:

info: v4l2-input: Start capture from /dev/video1
info: v4l2-input: Input: 0
info: v4l2-input: Resolution: 640x480
info: v4l2-input: Pixelformat: YU12
info: v4l2-input: Linesize: 640 Bytes
info: v4l2-input: Framerate: 30,00 fps

Plugin directory structure (I hope I installed it correctly):

~/.config/obs-studio
└── plugins
    └── obs-backgroundremoval
        ├── bin
        │   └── 64bit
        │       └── obs-backgroundremoval.so
        └── data
            ├── locale
            │   └── en-US.ini
            └── SINet_Softmax.onnx

Backtrace:

Thread 15 "libobs: graphic" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffb6d59640 (LWP 2167)]
cv::hal::opt_AVX2::(anonymous namespace)::YUV422toRGB8Invoker<2, 0, 1, 3>::operator() (this=0x7fffb6d575e0, range=...)
    at /usr/src/debug/opencv-4.5.1-2.4.x86_64/modules/imgproc/src/color_yuv.simd.hpp:1711
Downloading source file /usr/src/debug/opencv-4.5.1-2.4.x86_64/modules/imgproc/src/color_yuv.simd.hpp...
1711                        v_load_deinterleave(yuv_src + i, u, vy[0], v, vy[1]);
(gdb) bt
#0  cv::hal::opt_AVX2::(anonymous namespace)::YUV422toRGB8Invoker<2, 0, 1, 3>::operator()(cv::Range const&) const
    (this=0x7fffb6d575e0, range=<optimized out>)
    at /usr/src/debug/opencv-4.5.1-2.4.x86_64/modules/imgproc/src/color_yuv.simd.hpp:1711
#1  0x00007fffa4d3ceb9 in cv::(anonymous namespace)::ParallelLoopBodyWrapper::operator() (sr=<optimized out>, this=0x7fff10001dd8)
    at /usr/src/debug/opencv-4.5.1-2.4.x86_64/modules/core/src/parallel.cpp:337
#2  cv::(anonymous namespace)::ProxyLoopBody::operator() (range=<optimized out>, this=0x7fff10001dd8)
    at /usr/src/debug/opencv-4.5.1-2.4.x86_64/modules/core/src/parallel.cpp:378
#3  tbb::interface9::internal::start_for<tbb::blocked_range<int>, cv::(anonymous namespace)::ProxyLoopBody, tbb::auto_partitioner const>::run_body (r=<optimized out>, this=0x7fff10001dc0) at /usr/include/tbb/parallel_for.h:115
#4  tbb::interface9::internal::dynamic_grainsize_mode<tbb::interface9::internal::adaptive_mode<tbb::interface9::internal::auto_partition_type> >::work_balance<tbb::interface9::internal::start_for<tbb::blocked_range<int>, cv::(anonymous namespace)::ProxyLoopBody, tbb::auto_partitioner const>, tbb::blocked_range<int> > (range=<optimized out>, start=..., this=0x7fff10001de8)
    at /usr/include/tbb/partitioner.h:438
#5  tbb::interface9::internal::partition_type_base<tbb::interface9::internal::auto_partition_type>::execute<tbb::interface9::internal::start_for<tbb::blocked_range<int>, cv::(anonymous namespace)::ProxyLoopBody, tbb::auto_partitioner const>, tbb::blocked_range<int> > (range=<optimized out>, start=..., this=0x7fff10001de8) at /usr/include/tbb/partitioner.h:256
#6  tbb::interface9::internal::start_for<tbb::blocked_range<int>, cv::(anonymous namespace)::ProxyLoopBody, tbb::auto_partitioner const>::execute() (this=0x7fff10001dc0) at /usr/include/tbb/parallel_for.h:142
#7  0x00007fffa4aeaf47 in tbb::internal::custom_scheduler<tbb::internal::IntelSchedulerTraits>::process_bypass_loop(tbb::internal::context_guard_helper<false>&, tbb::task*, long)                                                                                        
    (isolation=<optimized out>, t=0x7fff10001dc0, context_guard=<optimized out>, this=<optimized out>)
    at ../../src/tbb/custom_scheduler.h:474
#8  tbb::internal::custom_scheduler<tbb::internal::IntelSchedulerTraits>::local_wait_for_all(tbb::task&, tbb::task*)
    (this=0x7fffac024c00, parent=..., child=<optimized out>) at ../../src/tbb/custom_scheduler.h:636
#9  0x00007fffa4ae86a0 in tbb::internal::generic_scheduler::local_spawn_root_and_wait(tbb::task*, tbb::task*&)
    (this=0x7fffac024c00, first=0x7fffac0255c0, next=@0x7fffac0255b8: 0x7fffac12ff40) at ../../src/tbb/scheduler.cpp:738
#10 0x00007fffa4e2ce8c in tbb::task::spawn_root_and_wait(tbb::task&) (root=...) at /usr/include/tbb/task.h:809
#11 tbb::interface9::internal::start_for<tbb::blocked_range<int>, cv::(anonymous namespace)::ProxyLoopBody, tbb::auto_partitioner const>::run(tbb::blocked_range<int> const&, cv::(anonymous namespace)::ProxyLoopBody const&, tbb::auto_partitioner const&) [clone .constprop.0] (range=..., body=..., partitioner=...) at /usr/include/tbb/parallel_for.h:95
#12 0x00007fffa4d35761 in tbb::parallel_for<tbb::blocked_range<int>, cv::(anonymous namespace)::ProxyLoopBody>
    (body=<optimized out>, range=...) at /usr/include/tbb/parallel_for.h:201
#13 cv::(anonymous namespace)::ProxyLoopBody::operator() (this=<optimized out>)
    at /usr/src/debug/opencv-4.5.1-2.4.x86_64/modules/core/src/parallel.cpp:384
#14 tbb::interface7::internal::delegated_function<cv::(anonymous namespace)::ProxyLoopBody, void>::operator()() const
    (this=<optimized out>) at /usr/include/tbb/task_arena.h:96
#15 0x00007fffa4ae2b03 in tbb::interface7::internal::task_arena_base::internal_execute(tbb::interface7::internal::delegate_base&) const (this=this@entry=0x7fffa4f09320 <_ZN2cv12_GLOBAL__N_1L8tbbArenaE.lto_priv.0>, d=...) at ../../src/tbb/arena.cpp:1105
#16 0x00007fffa4d3c158 in tbb::interface7::task_arena::execute_impl<void, cv::(anonymous namespace)::ProxyLoopBody>
    (this=0x7fffa4f09320 <_ZN2cv12_GLOBAL__N_1L8tbbArenaE.lto_priv.0>, f=...) at /usr/include/tbb/task_arena.h:271
#17 tbb::interface7::task_arena::execute<cv::(anonymous namespace)::ProxyLoopBody>
    (f=..., this=0x7fffa4f09320 <_ZN2cv12_GLOBAL__N_1L8tbbArenaE.lto_priv.0>) at /usr/include/tbb/task_arena.h:424
#18 cv::parallel_for_impl (nstripes=-1, body=..., range=...)
    at /usr/src/debug/opencv-4.5.1-2.4.x86_64/modules/core/src/parallel.cpp:546
#19 cv::parallel_for_(cv::Range const&, cv::ParallelLoopBody const&, double) (range=..., body=..., nstripes=nstripes@entry=-1)
    at /usr/src/debug/opencv-4.5.1-2.4.x86_64/modules/core/src/parallel.cpp:512
#20 0x00007fffa5308d7b in cv::hal::opt_AVX2::(anonymous namespace)::cvtYUV422toRGB<2, 0, 1, 3>(uchar*, size_t, uchar const*, size_t, int, int)
    (dst_data=<optimized out>, dst_step=<optimized out>, src_data=<optimized out>, src_step=<optimized out>, width=<optimized out>, height=<optimized out>) at /usr/src/debug/opencv-4.5.1-2.4.x86_64/modules/imgproc/src/color_yuv.simd.hpp:1779
#21 0x00007fffa5311b29 in cv::hal::opt_AVX2::cvtOnePlaneYUVtoBGR(unsigned char const*, unsigned long, unsigned char*, unsigned long, int, int, int, bool, int, int)
    (src_data=0x7fffd01562a0 "\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020"..., src_step=1280, dst_data=0x7fffac046ac0 "", dst_step=1920, width=640, height=480, dcn=3, swapBlue=true, uIdx=0, ycn=1) at /usr/src/debug/opencv-4.5.1-2.4.x86_64/modules/imgproc/src/color_yuv.simd.hpp:1972
#22 0x00007fffa5014132 in cv::hal::cvtOnePlaneYUVtoBGR(unsigned char const*, unsigned long, unsigned char*, unsigned long, int, int, int, bool, int, int)
    (src_data=0x7fffd01562a0 "\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020\200\020"..., src_step=1280, dst_data=0x7fffac046ac0 "", dst_step=1920, width=640, height=480, dcn=3, swapBlue=true, uIdx=0, ycn=1) at /usr/src/debug/opencv-4.5.1-2.4.x86_64/modules/imgproc/src/color_yuv.dispatch.cpp:190
#23 0x00007fffa4fde674 in cv::cvtColorOnePlaneYUV2BGR(cv::_InputArray const&, cv::_OutputArray const&, int, bool, int, int)
    (ycn=1, uidx=0, swapb=true, dcn=3, _dst=..., _src=<optimized out>)
    at /usr/src/debug/opencv-4.5.1-2.4.x86_64/modules/core/include/opencv2/core/mat.inl.hpp:1198
#24 cv::cvtColor(cv::_InputArray const&, cv::_OutputArray const&, int, int)
    (_src=<optimized out>, _dst=..., code=<optimized out>, dcn=3)
    at /usr/src/debug/opencv-4.5.1-2.4.x86_64/modules/imgproc/src/color.cpp:338
#25 0x00007fffb458d6d0 in filter_render(void*, obs_source_frame*) ()
    at /home/chris/.config/obs-studio/plugins/obs-backgroundremoval/bin/64bit/obs-backgroundremoval.so
#26 0x00007ffff617667e in filter_async_video () at /usr/lib64/libobs.so.0
#27 0x00007ffff6179414 in obs_source_video_render () at /usr/lib64/libobs.so.0
#28 0x00007ffff61ac309 in  () at /usr/lib64/libobs.so.0
#29 0x00007ffff617939f in obs_source_video_render () at /usr/lib64/libobs.so.0
#30 0x00007ffff617f8bb in obs_transition_video_render () at /usr/lib64/libobs.so.0
#31 0x00007ffff617939f in obs_source_video_render () at /usr/lib64/libobs.so.0
#32 0x00007ffff61a91ab in obs_view_render () at /usr/lib64/libobs.so.0
#33 0x00007ffff61b103a in obs_graphics_thread_loop () at /usr/lib64/libobs.so.0
#34 0x00007ffff61b29c0 in obs_graphics_thread () at /usr/lib64/libobs.so.0
#35 0x00007ffff507e299 in start_thread () at /lib64/libpthread.so.0
#36 0x00007ffff4fa64a3 in clone () at /lib64/libc.so.6

UI at the moment of the crash:

If I'll find more information, I'll add it.

Crash for OBS on windows 10 when applying Background Removal filter

Added crash logs, would love some help or understanding why this does not work for me.

Thank you for this tool! :)

Crash 2021-10-19 16-53-05.txt
Crash 2021-10-19 16-53-39.txt
Crash 2021-10-19 16-56-47.txt

Crash at startup

Hello!

Latest build now crashes my OBS at startup, unfortunately.

Crash 2021-11-04 07-55-09.txt

No problem, removing the plugin solves the problem.

Thanks!

occ-ai / obs-backgroundremoval Goto Github PK

obs-backgroundremoval's Introduction

OBS Plugin: Portrait Background Removal / Virtual Green-screen and Low-Light Enhancement

Usage

Download

Linux Installation

FlatHub

OBS Version Support and Compatibility

Introduction

Support and Help

Technical Details

Code Walkthrough

Building

Mac OSX

Install

Linux

Ubuntu

Arch Linux

Fedora

FlatHub

Windows

obs-backgroundremoval's People

Contributors

Stargazers

Watchers

Forkers

obs-backgroundremoval's Issues

Patch file

onnxruntime

Building

Install (Ubuntu bundled, not PPA):

Tested with NVIDIA RTX 3070 TI

Recommend Projects

Recommend Topics

Recommend Org