miun.sePublikationer
Ändra sökning
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Depth Map Upscaling for Three-Dimensional Television: The Edge-Weighted Optimization Concept
Mittuniversitetet, Fakulteten för naturvetenskap, teknik och medier, Institutionen för informationsteknologi och medier. (Realistic3D)ORCID-id: 0000-0002-2578-7896
2012 (Engelska)Licentiatavhandling, sammanläggning (Övrigt vetenskapligt)
Abstract [en]

With the recent comeback of three-dimensional (3D) movies to the cinemas, there have been increasing efforts to spread the commercial success of 3D to new markets. The possibility of a 3D experience at home, such as three-dimensional television (3DTV), has generated a great deal of interest within the research and standardization community.

A central issue for 3DTV is the creation and representation of 3D content. Scene depth information plays a crucial role in all parts of the distribution chain from content capture via transmission to the actual 3D display. This depth information is transmitted in the form of depth maps and is accompanied by corresponding video frames, i.e. for Depth Image Based Rendering (DIBR) view synthesis. Nonetheless, scenarios do exist for which the original spatial resolutions of depth maps and video frames do not match, e.g. sensor driven depth capture or asymmetric 3D video coding. This resolution discrepancy is a problem, since DIBR requires accordance between the video frame and depth map. A considerable amount of research has been conducted into ways to match low-resolution depth maps to high resolution video frames. Many proposed solutions utilize corresponding texture information in the upscaling process, however they mostly fail to review this information for validity.

In the strive for better 3DTV quality, this thesis presents the Edge-Weighted Optimization Concept (EWOC), a novel texture-guided depth upscaling application that addresses the lack of information validation. EWOC uses edge information from video frames as guidance in the depth upscaling process and, additionally, confirms this information based on the original low resolution depth. Over the course of four publications, EWOC is applied in 3D content creation and distribution. Various guidance sources, such as different color spaces or texture pre-processing, are investigated. An alternative depth compression scheme, based on depth map upscaling, is proposed and extensions for increased visual quality and computational performance are presented in this thesis. EWOC was evaluated and compared with competing approaches, with the main focus was consistently on the visual quality of rendered 3D views. The results show an increase in both objective and subjective visual quality to state-of-the-art depth map upscaling methods. This quality gain motivates the choice of EWOC in applications affected by low resolution depth.

In the end, EWOC can improve 3D content generation and distribution, enhancing the 3D experience to boost the commercial success of 3DTV.

Ort, förlag, år, upplaga, sidor
Sundsvall, Sweden: Mittuniversitetet , 2012. , s. 57
Serie
Mid Sweden University licentiate thesis, ISSN 1652-8948 ; 92
Nyckelord [en]
3d video, 3DTV, video coding, capture, distribution, EWOC, depth map upscaling, time-of-flight
Nationell ämneskategori
Signalbehandling
Identifikatorer
URN: urn:nbn:se:miun:diva-17048ISBN: 978-91-87103-41-4 (tryckt)OAI: oai:DiVA.org:miun-17048DiVA, id: diva2:561904
Presentation
2012-11-22, O111, Mittuniversitetet - Holmgatan 10, Sundsvall, 09:00 (Engelska)
Opponent
Handledare
Tillgänglig från: 2012-10-22 Skapad: 2012-09-24 Senast uppdaterad: 2017-08-22Bibliografiskt granskad
Delarbeten
1. Depth Map Upscaling Through Edge Weighted Optimization
Öppna denna publikation i ny flik eller fönster >>Depth Map Upscaling Through Edge Weighted Optimization
2012 (Engelska)Ingår i: Proceedings of SPIE - The International Society for Optical Engineering / [ed] Atilla M. Baskurt, Robert Sitnik, SPIE - International Society for Optical Engineering, 2012, s. Art. no. 829008-Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

Accurate depth maps are a pre-requisite in three-dimensional television, e.g. for high quality view synthesis, but this information is not always easily obtained. Depth information gained by correspondence matching from two or more views suffers from disocclusions and low-texturized regions, leading to erroneous depth maps. These errors can be avoided by using depth from dedicated range sensors, e.g. time-of-flight sensors. Because these sensors only have restricted resolution, the resulting depth data need to be adjusted to the resolution of the appropriate texture frame. Standard upscaling methods provide only limited quality results. This paper proposes a solution for upscaling low resolution depth data to match high resolution texture data. We introduce We introduce the Edge Weighted Optimization Concept (EWOC) for fusing low resolution depth maps with corresponding high resolution video frames by solving an overdetermined linear equation system. Similar to other approaches, we take information from the high resolution texture, but additionally validate this information with the low resolution depth to accentuate correlated data. Objective tests show an improvement in depth map quality in comparison to other upscaling approaches. This improvement is subjectively confirmed in the resulting view synthesis.

Ort, förlag, år, upplaga, sidor
SPIE - International Society for Optical Engineering, 2012
Nyckelord
3DTV, depth map, upscaling, time-of-flight, view synthesis, optimization, edge detection
Nationell ämneskategori
Elektroteknik och elektronik
Identifikatorer
urn:nbn:se:miun:diva-15805 (URN)10.1117/12.903921 (DOI)000304302300007 ()2-s2.0-84861935064 (Scopus ID)STC (Lokalt ID)978-081948937-1 (ISBN)STC (Arkivnummer)STC (OAI)
Konferens
3-Dimensional Image Processing (3DIP) and Applications II;Burlingame, CA;24 January 2012through26 January 2012;Code90039
Tillgänglig från: 2012-02-16 Skapad: 2012-01-31 Senast uppdaterad: 2017-08-22Bibliografiskt granskad
2. Adaptive depth filtering for HEVC 3D video coding
Öppna denna publikation i ny flik eller fönster >>Adaptive depth filtering for HEVC 3D video coding
2012 (Engelska)Ingår i: 2012 Picture Coding Symposium, PCS 2012, Proceedings, IEEE conference proceedings, 2012, s. 49-52Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

Consumer interest in 3D television (3DTV) is growing steadily, but current available 3D displays still need additional eye-wear and suffer from the limitation of a single stereo view pair. So it can be assumed that auto-stereoscopic multiview displays are the next step in 3D-at-home entertainment, since these displays can utilize the Multiview Video plus Depth (MVD) format to synthesize numerous viewing angles from only a small set of given input views. This motivates efficient MVD compression as an important keystone for commercial success of 3DTV. In this paper we concentrate on the compression of depth information in an MVD scenario. There have been several publications suggesting depth down- and upsampling to increase coding efficiency. We follow this path, using our recently introduced Edge Weighted Optimization Concept (EWOC) for depth upscaling. EWOC uses edge information from the video frame in the upscaling process and allows the use of sparse, non-uniformly distributed depth values. We exploit this fact to expand the depth down-/upsampling idea with an adaptive low-pass filter, reducing high energy parts in the original depth map prior to subsampling and compression. Objective results show the viability of our approach for depth map compression with up-to-date High-Efficiency Video Coding (HEVC). For the same Y-PSNR in synthesized views we achieve up to 18.5% bit rate decrease compared to full-scale depth and around 10% compared to competing depth down-/upsampling solutions. These results were confirmed by a subjective quality assessment, showing a statistical significant preference for 87.5% of the test cases.

Ort, förlag, år, upplaga, sidor
IEEE conference proceedings, 2012
Nyckelord
3-D displays; 3-D television; 3D video coding; Auto stereoscopic; Bit rates; Coding efficiency; Consumer interests; Depth information; Depth Map; Depth value; Edge information; High energy; Multiview displays; Multiview video; Stereo view; Subjective quality assessments; Test case; Upsampling; Upscaling; Video frame; Viewing angle
Nationell ämneskategori
Signalbehandling
Identifikatorer
urn:nbn:se:miun:diva-16211 (URN)10.1109/PCS.2012.6213283 (DOI)000306962400013 ()2-s2.0-84864026988 (Scopus ID)STC (Lokalt ID)978-1-4577-2048-2 (ISBN)STC (Arkivnummer)STC (OAI)
Konferens
29th Picture Coding Symposium, PCS 2012;Krakow;7 May 2012through9 May 2012;Code91163
Projekt
Realistic3D
Tillgänglig från: 2012-09-14 Skapad: 2012-05-14 Senast uppdaterad: 2017-08-22Bibliografiskt granskad
3. Improved edge detection for EWOC depth upscaling
Öppna denna publikation i ny flik eller fönster >>Improved edge detection for EWOC depth upscaling
2012 (Engelska)Ingår i: 2012 19th International Conference on Systems, Signals and Image Processing, IWSSIP 2012, IEEE conference proceedings, 2012, s. 1-4Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

The need for accurate depth information in three-dimen-sional television (3DTV) encourages the use of range sensors,i.e. time-of-flight (ToF) cameras. Since these sensors provideonly limited spatial resolution compared to modern high res-olution image sensors, upscaling methods are much needed.Typical depth upscaling algorithms fuse low resolution depthinformation with appropriate high resolution texture frames,taking advantage of the additional texture information in theupscaling process. We recently introduced a promising up-scaling method, utilizing edge information from the textureframe to upscale low resolution depthmaps. This paper exam-ines how a more thorough edge detection can be achieved byinvestigating different edge detection sources, such as inten-sity, color spaces and difference signals. Our findings showthat a combination of sources based on the perceptual quali-ties of the human visual system (HVS) leads to slightly im-proved results. On the other hand these improvements implya more complex edge detection.

Ort, förlag, år, upplaga, sidor
IEEE conference proceedings, 2012
Serie
Systems, Signals and Image Processing (IWSSIP), ISSN 2157-8672 ; 19
Nyckelord
3DTV, EWOC, depth map, ToF, upscaling, perceptual edge detection, HVS, CIE2000
Nationell ämneskategori
Signalbehandling
Identifikatorer
urn:nbn:se:miun:diva-16210 (URN)2-s2.0-84863949324 (Scopus ID)STC (Lokalt ID)978-1-4577-2191-5 (ISBN)STC (Arkivnummer)STC (OAI)
Konferens
2012 19th International Conference on Systems, Signals and Image Processing, IWSSIP 2012;Vienna;11 April 2012through13 April 2012;Category numberCFP1255E-ART;Code91138
Tillgänglig från: 2012-09-14 Skapad: 2012-05-14 Senast uppdaterad: 2017-08-22Bibliografiskt granskad
4. Incremental depth upscaling using an edge weighted optimization concept
Öppna denna publikation i ny flik eller fönster >>Incremental depth upscaling using an edge weighted optimization concept
2012 (Engelska)Ingår i: 3DTV-Conference, 2012, s. Art. no. 6365429-Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

Precise scene depth information is a pre-requisite in three-dimen-sional television (3DTV), e.g. for high quality view synthesis inautostereoscopic multiview displays. Unfortunately, this informa-tion is not easily obtained and often of limited quality. Dedicatedrangesensors, suchastime-of-flight(ToF)cameras, candeliverre-liable depth information where (stereo-)matching fails. Nonethe-less, since these sensors provide only restricted spatial resolution,sophisticated upscaling methods are sought-after, to match depthinformation to corresponding texture frames. Where traditionalupscaling fails, novel approaches have been proposed, utilizingadditional information from the texture for the depth upscalingprocess. We recently proposed the Edge Weighted OptimizationConcept (EWOC) for ToF upscaling, using texture edges for ac-curate depth boundaries. In this paper we propose an importantupdate to EWOC, dividing it into smaller incremental upscalingsteps. We predict two major improvements from this. Firstly, pro-cessing time should be decreased by dividing one big calculationinto several smaller steps. Secondly, we assume an increase inquality for the upscaled depth map, due to a more coherent edgedetection on the video frame. In our evaluations we can showthe desired effect on processing time, cutting down the calculationtime more than in half. We can also show an increase in visualquality, based on objective quality metrics, compared to the origi-nal implementation as well as competing proposals.

Nyckelord
3DTV, EWOC, DIBR, time-of-flight, depth map, upscaling, edge detection, incremental, optimization, view synthesis
Nationell ämneskategori
Signalbehandling
Identifikatorer
urn:nbn:se:miun:diva-17023 (URN)10.1109/3DTV.2012.6365429 (DOI)2-s2.0-84872059517 (Scopus ID)STC (Lokalt ID)978-146734905-5 (ISBN)STC (Arkivnummer)STC (OAI)
Konferens
2012 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video, 3DTV-CON 2012;Zurich;15 October 2012through17 October 2012;Category numberCFP1255B-ART;Code94817
Projekt
3D video: Capture and Compression for Distribution
Tillgänglig från: 2012-09-25 Skapad: 2012-09-19 Senast uppdaterad: 2017-08-22Bibliografiskt granskad

Open Access i DiVA

fulltext(5980 kB)1444 nedladdningar
Filinformation
Filnamn FULLTEXT02.pdfFilstorlek 5980 kBChecksumma SHA-512
83361a2ba8d948b83cdebb0451d890e6202eeffc54dea1184d3e5076a4661d8d33d124c7254feebab29ed56b3be2c580785062bfa8c18afe8075d3f876fa144e
Typ fulltextMimetyp application/pdf

Personposter BETA

Schwarz, Sebastian

Sök vidare i DiVA

Av författaren/redaktören
Schwarz, Sebastian
Av organisationen
Institutionen för informationsteknologi och medier
Signalbehandling

Sök vidare utanför DiVA

GoogleGoogle Scholar
Totalt: 1444 nedladdningar
Antalet nedladdningar är summan av nedladdningar för alla fulltexter. Det kan inkludera t.ex tidigare versioner som nu inte längre är tillgängliga.

isbn
urn-nbn

Altmetricpoäng

isbn
urn-nbn
Totalt: 1424 träffar
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf