Skip to contents

ODA power analysis via simulation

Source: R/oda_power.R
oda_power.Rd

Estimates planning power for unit-weighted binary 2×2 ODA-equivalent designs. The implemented design assumes fixed group sizes n1/n2 and binomial outcome probabilities p1/p2, then evaluates whether the resulting 2×2 table is significant by Fisher's exact test at the (optionally Sidak-adjusted) alpha.

Usage

oda_power(
  n1,
  n2 = n1,
  p1 = NULL,
  p2 = NULL,
  ess = NULL,
  alpha = 0.05,
  comp = 1L,
  nsim = 10000L,
  mc_seed = NULL
)

Arguments

n1

Integer (or integer vector) giving the per-group sample size for class 0. When a vector is supplied, power is estimated at each element.

n2

Integer (or integer vector) giving the per-group sample size for class 1. Defaults to n1 (balanced design). If scalar and n1 is a vector, n2 is recycled.

p1

Probability of the event in class 0. Ignored when ess is supplied.

p2

Probability of the event in class 1. Ignored when ess is supplied.

ess

Effect Strength for Sensitivity (percent, \(0 < \text{ESS} < 100\)) under the symmetric balanced convention. Mutually exclusive with p1/p2.

alpha

Nominal significance level. Default 0.05. May be a vector to evaluate power at multiple alpha levels simultaneously.

comp

Number of comparisons for Sidak multiple-comparison correction. Default 1 (no correction). Must be a single positive integer.

nsim

Number of Monte Carlo replications per cell. Default 10000.

mc_seed

Integer seed passed to set.seed() once before all simulations, or NULL to use the current RNG state.

Value

An object of class "oda_power", a list with elements:

power

Numeric matrix (rows = n1, cols = alpha_adj) of estimated power. Simplified to a named vector if one dimension is scalar, or to a scalar if both are.

n1, n2

Per-group sample sizes.

p1, p2

Per-group event rates used.

ess_input

ESS supplied, or NA if p1/p2 used.

alpha, alpha_adj

Input and Sidak-adjusted alpha.

comp, nsim, mc_seed

Input parameters.

Details

This is the binary lowest-measurement planning case discussed by Rhodes (2020). See also Yarnold and Soltysik (2005) for the underlying ODA/Fisher isomorphism. Scope: unit-weighted, binary class, binary (2-level) attribute only. This is not a general CTA, LORT, SDA, weighted, or multiclass power method.

Method: For each Monte Carlo replicate, binomial draws are generated under (p1, p2) with fixed group sizes (n1, n2). The resulting 2×2 table is tested by Fisher's exact test; power is the proportion of replicates in which the null is rejected. The prospective sampling treats group sizes as fixed and outcomes as binomial within each group; the Fisher test is then applied to the generated table with its realized marginals. This is the standard simulation-based power approach for 2×2 contingency analyses.

Effect-size input: Specify the effect either as per-group proportions p1 and p2 directly, or as ess (Effect Strength for Sensitivity, percent) under the symmetric balanced convention: \(p_2 = \text{ESS}/200 + 0.5\), \(p_1 = 1 - p_2\).

Sidak correction: When comp > 1, the working \(\alpha\) is Sidak-adjusted: \(\alpha_{\text{adj}} = 1 - (1 - \alpha)^{1/\text{comp}}\).

References

Rhodes, N. J. (2020). Statistical power analysis in ODA, CTA and Novometrics. Optimal Data Analysis, 9. https://odajournal.files.wordpress.com/2020/02/v9a5.pdf

Yarnold PR, Soltysik RC (2005). Optimal Data Analysis: A Guidebook with Software for Windows. Washington, DC: APA Books.

Examples

# Power for ESS = 48%, n = 50 per group (CRAN-safe nsim; use 10000L for publication)
oda_power(n1 = 50, ess = 48, nsim = 500L, mc_seed = 42L)
#> ODA Power Analysis
#>   Effect:  p1 = 0.2600, p2 = 0.7400  (ESS = 48.00%)
#>   alpha = 0.0500
#>   nsim  = 500
#> 
#> Power = 0.996

# Power curve across a range of n
oda_power(n1 = c(30, 50, 80), ess = 48, nsim = 500L, mc_seed = 42L)
#> ODA Power Analysis
#>   Effect:  p1 = 0.2600, p2 = 0.7400  (ESS = 48.00%)
#>   alpha = 0.0500
#>   nsim  = 500
#> 
#> Power estimates:
#>  n=30  n=50  n=80 
#> 0.972 0.998 1.000 

# Direct proportions (p1 = 0.26, p2 = 0.74)
oda_power(n1 = 50, p1 = 0.26, p2 = 0.74, nsim = 500L, mc_seed = 42L)
#> ODA Power Analysis
#>   Effect:  p1 = 0.2600, p2 = 0.7400
#>   alpha = 0.0500
#>   nsim  = 500
#> 
#> Power = 0.996

# Sidak correction for 3 comparisons
oda_power(n1 = 80, ess = 48, comp = 3L, nsim = 500L, mc_seed = 42L)
#> ODA Power Analysis
#>   Effect:  p1 = 0.2600, p2 = 0.7400  (ESS = 48.00%)
#>   alpha = 0.0500  (Sidak-adjusted for 3 comparisons: 0.016952)
#>   nsim  = 500
#> 
#> Power = 1.000