Compare model engines from backtest scores

Summarises and ranks engines across horizons based on forecast accuracy scores.

Usage

compare_models(scores, by = "engine")

Arguments

scores

Output of score_forecasts().

by

Grouping variable(s). Default "engine".

Value

A tibble with average scores per group, sorted by MAE.

Examples

# \donttest{
sim <- simulate_dynamics(n_lineages = 3,
  advantages = c("A" = 1.2, "B" = 0.8),
  n_timepoints = 20, seed = 1)
bt <- backtest(sim, engines = "mlr",
  horizons = c(7, 14), min_train = 42)
sc <- score_forecasts(bt)
compare_models(sc)
#> # A tibble: 1 × 9
#>   engine     mae    rmse coverage     wis    crps log_score   dss calibration
#>   <chr>    <dbl>   <dbl>    <dbl>   <dbl>   <dbl>     <dbl> <dbl>       <dbl>
#> 1 mlr    0.00701 0.00967        1 0.00214 0.00645     -3.10 -8.04      0.0762
# }