Comments (3)
@Teravus Thanks a lot for trying out the code on your data and providing me with your valuable feedback.
Masks: Currently the network crops out a 512x512 region from the input frame based on the masksDL. If two frames have slight differences in masksDL prediction the aspect ratio of the crop will be different. However, the current testing code takes the output of the network, uncrop it and puts it back to the original frame resolution. Follow the image example and the same can be applied to videos to make sure the final output resolution/aspect ratio is the same as the input. You can write your own cropping function. You just need to make sure the input to the network is 512x512 centered around the human. While training I did data augmentation to allow the network some flexibility.
Reflection: Unfortunately strong reflection very close to the foreground is very hard to remove for the network. It learns to use color difference, segmentation mask, and human shape priors to predict the matte. If one of the response is very strong, the network latches on to that.
Future directions: All the constraints mentioned in the 'Capture Conditions' are certainly a drawback of the network. The main future directions are: allowing any camera movement (then it is truly green-screen), robust to auto-focus, auto-exposure, reflections, shadows.
Let me know if you have any additional questions
from background-matting.
Seems like the crop/resize happens even when the input/output sizes are the same.
I exported a 512x512 video to PNG (img and back). I also cropped the green background to 512x512 and ran it through.
The next picture is an overlay of the matting output (large version) and the deeplab mask applied to the original _img.png for the frame(the smaller version)
from background-matting.
I was able to disable the crop by commenting out lines 80-85 and adding;
back_img1 = back_img10
back_img2 = back_img20
This fixes the resize issue for me. Obviously, this would only work if your frame size matches the expected input to the network.
Thanks for your patience.
from background-matting.
Related Issues (20)
- MSCOCO HOT 2
- Can I skip the Alignment step? and How? HOT 1
- issue in JPG inputs HOT 1
- loss log HOT 1
- About the training time HOT 1
- matting performance problem HOT 1
- Real time results with constant background HOT 1
- SAD in the paper
- Evaluation on Synthetic-Composite Adobe Dataset
- [error] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR HOT 1
- Background Image HOT 3
- How to test after training on unlabeled real videos
- Camera Model For Fix-Camera dataset
- 这个对抗损失的意义是什么?怎么理解? HOT 1
- The download link for pre-trained model is failed! HOT 1
- Possible Memory Leak with AdobeDataAffineHR
- 是否可以训练他来分割别的东西,比如说超市商品?
- Could he be trained to split other things, such as supermarket goods?
- Using sample data does not achieve the effect in the paper
- Colab notebook is no longer works HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from background-matting.