Design effect information

In survey methodology, the design effect (generally denoted as $D_{\mathrm {eff} }$ or $D_{\mathrm {eft} }^{2}$ ) is a measure of the expected impact of a sampling design on the variance of an estimator for some parameter. It is calculated as the ratio of the variance of an estimator based on a sample from an (often) complex sampling design, to the variance of an alternative estimator based on a simple random sample (SRS) of the same number of elements.^[1]^: 258 The $D_{\mathrm {eff} }$ (be it estimated, or known a priori) can be used to adjust the variance of an estimator in cases where the sample is not drawn using simple random sampling. It may also be useful in sample size calculations and for quantifying the representativeness of a sample. The term "design effect" was coined by Leslie Kish in 1965.

The design effect is a positive real number that indicates an inflation ( $D_{\mathrm {eff} }>1$ ), or deflation ( $D_{\mathrm {eff} }<1$ ) in the variance of an estimator for some parameter, that is due to the study not using SRS (with $D_{\mathrm {eff} }=1$ , when the variances are identical).^[2]^: 53, 54

Some potential complex sampling that could introduce $D_{\mathrm {eff} }$ that is different than 1 include: cluster sampling (such as when there is correlation between observations), stratified sampling, cluster randomized controlled trial, disproportional (unequal probability) sample, non-coverage, non-response, statistical adjustments of the data, etc..

$D_{\mathrm {eff} }$ can be used in sample size calculations, quantifying the representative of a sample (to a target population), as well as for adjusting (often inflating) the variance of some estimator (in cases when we can calculate that estimator's variance assuming SRS).^[3]

The term "Design effect" was coined by Leslie Kish in 1965.^[1]^{: 88, 258} Ever since, many calculations (and estimators) have been proposed, in the literature, for describing the effect of known sampling design on the increase/decrease in the variance of estimators of interest. In general, the design effect varies between statistics of interests, such as the total or ratio mean; it also matters if the design (e.g.: selection probabilities) are correlated with the outcome of interest. And lastly, it is influenced by the distribution of the outcome itself. All of these should be considered when estimating and using design effect in practice.^[4]^: 13

^ ^a ^b Kish, Leslie (1965). Survey Sampling. New York: John Wiley & Sons, Inc. ISBN 0-471-10949-5.
^ Cite error: The named reference sarndal1992 was invoked but never defined (see the help page).
^ Heo, Moonseong; Kim, Yongman; Xue, Xiaonan; Kim, Mimi Y. (2010). "Sample size requirement to detect an intervention effect at the end of follow-up in a longitudinal cluster randomized trial". Statistics in Medicine. 29 (3): 382–390. doi:10.1002/sim.3806. PMID 20014353. S2CID 30001378. Archived from the original on 5 January 2013.
^ Cite error: The named reference park2006 was invoked but never defined (see the help page).

[Kish1965-1] Kish, Leslie (1965). Survey Sampling. New York: John Wiley & Sons, Inc. ISBN 0-471-10949-5.

[sarndal1992-2] Cite error: The named reference sarndal1992 was invoked but never defined (see the help page).

[3] Heo, Moonseong; Kim, Yongman; Xue, Xiaonan; Kim, Mimi Y. (2010). "Sample size requirement to detect an intervention effect at the end of follow-up in a longitudinal cluster randomized trial". Statistics in Medicine. 29 (3): 382–390. doi:10.1002/sim.3806. PMID 20014353. S2CID 30001378. Archived from the original on 5 January 2013.

[park2006-4] Cite error: The named reference park2006 was invoked but never defined (see the help page).

Design effect information

and 25 Related for: Design effect information

Design effect

Effect

Analysis of variance

Effect size

Factorial experiment

Design of experiments

Regression discontinuity design

Cluster sampling

Overview effect

Curb cut effect

Wald test

Linear regression

Droste effect

Statistical significance

Confounding

List of statistics articles

Statistics

Taguchi methods

Monte Carlo method

Analysis of covariance

Ishikawa diagram

Mass Effect

Optimal experimental design

Rehbinder effect

Repeated measures design