Faster Depth-Map Creation Using Depth Propagation Tool
The “Juneau” case study offers an example application of YUVsoft’s DP tool for 2D-to-stereo-video conversion with minimized manual work. The input data is a complex non-static shot (see the source video). Using DP to process background and foreground objects decreases the required number of roto masks by more than 6 times without noticeably sacrificing quality. Here’s a look at how it was done.
Input data for depth propagation:
|Image 1: Source frame #63.|
Depth maps must be created separately for the background and foreground:
- Background: Key depth frames were hand painted for the entire image (see Images 2–5). Background objects can be painted without any rotoscoping, but to create four key depth frames using roto, 184 masks are required.
- Foreground: Key depth images (see Image 6) were hand painted for the 10 most important objects.
Four key depth frames for the background:
|Image 2: Key depth frame #00.||Image 3: Key depth frame #33.|
|Image 4: Key depth frame #67.||Image 5: Key depth frame #97.|
|Image 6: Key image example for an object. Contrast is increased for detail visibility. Input for the DP tool included four such key depth images for this object.|
Roto masks prevent boundary leakage from the background, but the number of necessary roto masks is lower: practically, just a single mask per object. All together, this example involved creation of 10 isolating roto masks (see Images 7–8) per frame to merge the propagated background and foreground depth.
|Image 7: Isolating roto masks for several objects.||Image 8: Isolating roto mask for a single object, frame #63.|
The traditional approach requires rotoscoping of 78 masks for each frame to achieve a depth map of the same quality, totaling 7,800 roto masks for 100 source frames.
Note that propagation preserves depth details, and it also takes into account depth changes over time (see Images 9–10)
|Picture 9: Propagated depth for a single object that is cut out using the respective isolating mask, frame #63. Contrast is increased for detail visibility||Picture 10: Propagated background depth, frame #63.|
The table below compares the amount of work needed for each approach, where each manually drawn object contour for a given frame is termed a “mask”:
|Layer||Traditional Approach||DP-Tool Approach|
|FG||32 * 100 = 3,200 masks||10 * 100 = 1,000 isolating masks;
also, 31 masks to make key frames
|BG||46 * 100 = 4,600 masks||46 * 4 = 184 masks to make key frames|
|Total||7,800 masks||1,215 masks|
As a result, the DP tool greatly reduces the rotoscoping workload by 6.4 times, even for nontrivial shots, saving important resources: time and money.