Minimum sample size for survival‐outcome prediction models

Compute the minimum sample size required to develop a prediction model with a time-to-event (survival) outcome. As with the other wrappers, this uses a simulation-based learning curve, fitted by default to every simulated replicate (method = "curve"; method = "mlpwr" uses a Gaussian-process surrogate instead), to locate the smallest $n$ meeting the chosen performance criterion.

Usage

simulate_survival(
  signal_parameters,
  noise_parameters = 0,
  complexity = 1,
  data_control = NULL,
  maximum_achievable_cindex,
  baseline_hazard = 1,
  censoring_rate,
  model = c("coxph", "lasso", "ridge", "rf", "xgboost"),
  metric = "calibration_slope",
  target_performance,
  n_reps_total = 1000,
  mean_or_assurance = "assurance",
  auto_n_reps_total = TRUE,
  ...
)

Arguments

signal_parameters

Integer. Number of candidate predictors associated with the outcome (i.e., true signal features).

noise_parameters

Integer. Number of candidate predictors not associated with the outcome (noise features). Default is 0.

complexity

Integer in 1:4 selecting the data-generating signal structure (see Data control). Default 1. Complexity 4 requires signal_parameters = 5.

data_control

Optional named list controlling the predictors (see Data control). Default NULL (generator defaults).

maximum_achievable_cindex

Numeric in (0, 1). Maximum achievable C-index with effectively unlimited data. This is used to calibrate the data-generating mechanism and is not the minimum acceptable threshold.

baseline_hazard

Numeric greater than 0. Baseline hazard level used by the data-generating mechanism (e.g., the constant hazard in an exponential baseline). Larger values imply shorter event times, all else equal.

censoring_rate

Numeric in [0, 1). Proportion of individuals expected to be censored in the simulated datasets (administrative or random censoring). Higher values imply fewer observed events for a fixed $n$.

model

Character string specifying the modelling algorithm. One of "coxph" (Cox proportional hazards), "lasso", "ridge", "rf" (random survival forest), or "xgboost" (gradient boosting with a Cox objective).

metric

Character string naming the performance metric used to assess the sample size; defaults to "calibration_slope". Metric identifiers use one canonical form throughout the package: "calibration_slope", "auc", "r2", "cindex" and "csse". Metrics where smaller is better, such as "calibration_in_the_large", cannot be used as search targets.

"calibration_slope" is the slope from regressing the observed outcome on the model's linear predictor in held-out data; 1 indicates perfect calibration, and values below 1 indicate overfitting. Note that for the machine-learning models ("lasso", "ridge", "rf", "xgboost") this is converted internally to the calibration slope squared error for optimisation and translated back before results are returned; you don't need to do anything, and target_performance is still given on the calibration slope scale. Results derived this way carry a footnote marker in the printed output.

"csse" is the calibration slope squared error, $-(1 - s)^2$ for a calibration slope $s$, so that larger is better and 0 is perfect calibration. It can be requested directly, which is mainly useful for advanced use and for comparison against the internal conversion described above. When requesting it directly you are responsible for supplying target_performance on the CSSE scale: a calibration slope target of 0.9 corresponds to a CSSE target of -0.01. No adjustment is applied on your behalf, and results are reported on the CSSE scale.

target_performance

Numeric. Minimum acceptable value of the selected performance metric $M^*$; the algorithm searches for the smallest $n$ meeting the chosen criterion with respect to this threshold.

n_reps_total

Integer. Total number of simulation replications used by the engine across the search.

mean_or_assurance

Character string, either "mean" or "assurance". Controls whether the minimum $n$ is defined by the mean-based criterion or the assurance-based criterion (with the assurance level $\delta$ controlled by the engine's defaults or additional arguments in ...).

auto_n_reps_total

Logical. When TRUE (default) and method = "mlpwr", n_reps_total is raised to 2000 for designs with at most 10 signal parameters fitted with "lm", "glm" or "coxph", where the Gaussian-process search is limited by simulation budget rather than by the design. It has no effect with the default method = "curve". The rule only ever increases the budget, and reports the change with a message. Set FALSE to use n_reps_total exactly as supplied.

...

Additional options passed to simulate_custom() (e.g., assurance level $\delta$, per-iteration settings).

Value

An object of class "pmsims" containing the estimated minimum sample size and simulation diagnostics (inputs, fitted GP curve, intermediate evaluations, and summary metrics).

Criteria

Two formulations are supported.

Mean-based: find the smallest $n$ such that the expected model performance exceeds the target $M^*$, i.e. $$\min_n \; \mathbb{E}_{D_n}\{ M \mid D_n \} \ge M^*.$$
Assurance-based: find the smallest $n$ such that the probability the performance exceeds $M^*$ is at least $\delta$ (e.g. 0.80), i.e. $$\min_n \; \mathbb{P}_{D_n}\!\left( M \mid D_n \ge M^* \right) \ge \delta.$$

Here, $M$ is the chosen performance metric and the probability/expectation is over repeated samples of training data of size $n$. The assurance criterion explicitly accounts for variability across training sets; models with higher variance typically require larger $n$ to satisfy it.

Data control

complexity selects the signal structure of the data-generating mechanism: 1 purely linear, 2 linear + quadratic, 3 linear + quadratic + interaction, 4 the Friedman function. The Friedman function uses five predictors, so complexity 4 requires exactly five signal predictors (signal_parameters = 5) and stops with an error otherwise; any number of noise predictors can be added. data_control is an optional list fine-tuning the predictors:

nonlinear_strength: Numeric in [0, 1). Fraction of the signal variance carried by the nonlinear, linearly-inaccessible component. Applies to complexity 2 and 3 only; ignored (with a warning) for 1 and 4. If omitted, the generator's per-complexity default is used.
correlation: Numeric in $[-1, 1]$. Pairwise correlation among the candidate predictors, induced through a Gaussian copula. Default 0.3. This is the correlation of the underlying normal variables: for "normal" predictors it is also the observed correlation, but for other distributions the observed correlation is lower. For example, binary predictors with prevalence 0.3 correlate at about 0.18 when correlation = 0.3.
predictor_distribution: One of "normal", "uniform", "binary", "exponential", "lognormal", "t", "laplace". "binary" selects 0/1 predictors and requires binary_predictor_prevalence; any other value selects continuous predictors from that family. Default "normal".
binary_predictor_prevalence: Numeric in (0, 1). Prevalence of the binary predictors; required when predictor_distribution = "binary", ignored (with a warning) otherwise. Note: binary predictors are incompatible with complexity 2/3 because squaring a 0/1 variable returns itself.

Examples

# \donttest{
set.seed(123)
est <- simulate_survival(
  signal_parameters = 1,
  noise_parameters = 0,
  complexity = 1,
  data_control = list(correlation = 0),
  maximum_achievable_cindex = 0.70,
  baseline_hazard = 0.01,
  censoring_rate = 0.30,
  model = "coxph",
  metric = "calibration_slope",
  target_performance = 0.9,
  mean_or_assurance = "assurance",
  min_sample_size = 25,
  max_sample_size = 500,
  n_reps_total = 1000,
  auto_n_reps_total = FALSE,
  test_n = 30000,
  progress = FALSE
)
#> ℹ Using user-specified min_sample_size and max_sample_size. Adaptive starting values will not be used.
#> ℹ Estimating the learning curve... (pilot)
#> ℹ Refining the learning curve near the target...
#> ! The target is close to the best performance this model can reach: doubling the sample size from the answer gains less than the Monte Carlo error, so the sample size is poorly determined.
est
#>                     ┌────────────────────────────────────────┐
#>                     │ pmsims: Sample size simulation summary │
#>                     └────────────────────────────────────────┘
#> 
#> ──────────────────────────────────── Inputs ────────────────────────────────────
#> 
#> Data-generating scenario
#>   Outcome                  Time-to-event
#>   Baseline hazard          0.01
#>   Censoring rate           0.30
#>   Predictors               1 signal
#>   Predictor distribution   Normal
#>   Predictor correlation    0.00
#>   Signal form              Linear
#> 
#> Model and performance
#>   Model                    Cox proportional hazards
#>   Large-sample C-index     0.700
#>   Sample-size criterion    Calibration slope ≥ 0.900
#> 
#> Simulation
#>   Mode                     Assurance
#>   Replications             1,000
#> 
#> ──────────────────────────────────── Results ───────────────────────────────────
#> 
#>   Minimum sample size      170
#> 
#>   Performance at N = 170
#>     Calibration slope      0.900    (target ≥ 0.900)
#>     C-index                0.702
#> 
#>   Running time             22 seconds
#> 
#> ────────────────────────────────────────────────────────────────────────────────
#> Assurance mode selects N so that the target is achieved with high probability
#> across repeated datasets.
est$min_n
#> [1] 170
plot(est)

# }