miun.sePublikationer
Ändra sökning
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Statistical quality of experience analysis: on planning the sample size and statistical significance testing
RISE AB Acreo. (Realistic3D)ORCID-id: 0000-0001-5060-9402
Deggendorf Institute of Technology (DIT), University of Applied Sciences, Deggendorf.
2018 (Engelska)Ingår i: Journal of Electronic Imaging (JEI), ISSN 1017-9909, E-ISSN 1560-229X, Vol. 27, nr 5, s. 053013-1-053013-11, artikel-id 053013Artikel i tidskrift (Refereegranskat) Published
Abstract [en]

This paper analyzes how an experimenter can balance errors in subjective video quality tests betweenthe statistical power of finding an effect if it is there and not claiming that an effect is there if the effect is not there,i.e., balancing Type I and Type II errors. The risk of committing Type I errors increases with the number ofcomparisons that are performed in statistical tests. We will show that when controlling for this and at thesame time keeping the power of the experiment at a reasonably high level, it is unlikely that the number oftest subjects that are normally used and recommended by the International Telecommunication Union (ITU),i.e., 15 is sufficient but the number used by the Video Quality Experts Group (VQEG), i.e., 24 is more likelyto be sufficient. Examples will also be given for the influence of Type I error on the statistical significance ofcomparing objective metrics by correlation. We also present a comparison between parametric and nonparametricstatistics. The comparison targets the question whether we would reach different conclusions on the statisticaldifference between the video quality ratings of different video clips in a subjective test, based on thecomparison between the student T-test and the Mann–Whitney U-test. We found that there was hardly a differencewhen few comparisons are compensated for, i.e., then almost the same conclusions are reached. Whenthe number of comparisons is increased, then larger and larger differences between the two methods arerevealed. In these cases, the parametric T-test gives clearly more significant cases, than the nonparametrictest, which makes it more important to investigate whether the assumptions are met for performing a certaintest.

Ort, förlag, år, upplaga, sidor
IS&T - the Society for Imaging Science and Technology , 2018. Vol. 27, nr 5, s. 053013-1-053013-11, artikel-id 053013
Nyckelord [en]
Type-I error, video quality, statistical significance, quality of experience, Student T-test, Bonferroni, Mann–Whitney U-test, parametric versus nonparametric test
Nationell ämneskategori
Elektroteknik och elektronik Mediateknik Signalbehandling Kommunikationssystem
Identifikatorer
URN: urn:nbn:se:miun:diva-34508DOI: 10.1117/1.JEI.27.5.053013ISI: 000449229800015Scopus ID: 2-s2.0-85054069504OAI: oai:DiVA.org:miun-34508DiVA, id: diva2:1251082
Forskningsfinansiär
KK-stiftelsen, 20160194
Anmärkning

Copyright (2018) Society of Photo-Optical Instrumentation Engineers. One print or electronic copy may be made for personal use only. Systematic reproduction and distribution, duplication of any material in this paper for a fee or for commercial purposes, or modification of the content of the paper are prohibited.

Tillgänglig från: 2018-09-26 Skapad: 2018-09-26 Senast uppdaterad: 2019-10-11Bibliografiskt granskad

Open Access i DiVA

fulltext(1742 kB)415 nedladdningar
Filinformation
Filnamn FULLTEXT01.pdfFilstorlek 1742 kBChecksumma SHA-512
2047984830a72ea9c89ec856e15f55080d966c8867df51d3271480cb47de5ce50d5fe39370e4e97f26b331d9b9fc5ada3cdd80fdef625d3a5d398c761744c026
Typ fulltextMimetyp application/pdf

Övriga länkar

Förlagets fulltextScopus

Personposter BETA

Brunnström, Kjell

Sök vidare i DiVA

Av författaren/redaktören
Brunnström, Kjell
I samma tidskrift
Journal of Electronic Imaging (JEI)
Elektroteknik och elektronikMediateknikSignalbehandlingKommunikationssystem

Sök vidare utanför DiVA

GoogleGoogle Scholar
Totalt: 415 nedladdningar
Antalet nedladdningar är summan av nedladdningar för alla fulltexter. Det kan inkludera t.ex tidigare versioner som nu inte längre är tillgängliga.

doi
urn-nbn

Altmetricpoäng

doi
urn-nbn
Totalt: 416 träffar
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf