mELO Ratings

This function implements the multidimensional ELO (mELO) rating system for performing pairwise comparisons. The mELO rating system has the desirable properties of being able to handle cyclic interactions and is better behaved in the presence of redundant copies of players/agents or tasks.

mELO(
  match_data,
  init_data = NULL,
  init_c_mat = NULL,
  init_rating = 2200,
  k = ifelse(!is.null(init_c_mat), ncol(as.matrix(init_c_mat)), 1),
  eta_1 = 27,
  eta_2 = 1,
  p1_advantage = 0,
  save_history = TRUE,
  sort = TRUE
)

Arguments

match_data	A data frame containing four columns: (1) a numeric vector denoting the time period in which the game took place (2) a numeric or character identifier for player one (3) a numeric or character identifier for player two and (4) the result of the game expressed as a number, typically equal to one for a player one win, zero for a player two win and one half for a draw.
init_data	Initialise a rating model with ratings component of the returned list of the output from a previous mELO() call, or a data.frame with columns Player and Rating.
init_c_mat	Initialise a rating model with c_mat component of the returned list of the output from a previous mELO() call, or a matrix with with dimensions (#agents, k). Recall the C matrix encodes the non-transitive interactions of the agents. This is initialised randomly if not provided.
init_rating	The initial rating for players not appearing in init_data.
k	Integer defining the complexity of non-transitive interactions to model.
eta_1	Learning rate for the ratings vector.
eta_2	Learning rate for elements of the C matrix.
p1_advantage	Player 1 advtange parameter. Either a single value or a vector equal to the number of rows in match_data.
save_history	If TRUE return the rating history for each player.
sort	If TRUE, sort the output ratings from highest to lowest.

Value

A list object of class "mELO_rating" with the following components:

ratings

A data frame of the results at the end of the final time period. The variables are self explanatory except for Lag, which represents the number of time periods since the player last played a game. This is equal to zero for players who played in the latest time period, and is also zero for players who have not yet played any games.

history

An array containing the ratings history for all players.

c_mat

An estimate of the C matrix.

c_mat_history

An array containing the history of the C matrix.

p1_advantage

A single value or a vector values for the advantage Player 1 had.

k

Integer defining the complexity of non-transitive interactions to model.

eta_1

Learning rate for the ratings vector.

eta_2

Learning rate for elements of the C matrix.

type

The type of model. In this case, "mELO".

preds

The player 1 success probabilities predicted prior to adjusting to the outcome of the match.

outcomes

The outcome for player 1 for each match.

preds_logloss

The mean logloss error for all predictions.

References

Balduzzi, David, et al. "Re-evaluating Evaluation." arXiv preprint arXiv:1806.02643 (2018).

Examples

# Rock paper scissors
head(rps_df)
#> # A tibble: 6 x 4
#>   time_index throw_1  throw_2  outcome
#>        <int> <chr>    <chr>      <dbl>
#> 1          1 PAPER    ROCK           1
#> 2          2 ROCK     SCISSORS       1
#> 3          3 SCISSORS PAPER          1
#> 4          4 ROCK     PAPER          0
#> 5          5 SCISSORS ROCK           0
#> 6          6 PAPER    SCISSORS       0

# Note that ELO doesn't perform well
rps_ELO <- ELO(rps_df)
rps_ELO
#> 
#> ELO Ratings For 3 Players Playing 120 Games
#> 
#>     Player Rating Games Win Draw Loss Lag
#> 1 SCISSORS 2204.6    80  40    0   40   0
#> 2     ROCK 2204.6    80  40    0   40   1
#> 3    PAPER 2190.7    80  40    0   40   0
#> 
ELO_preds <- predict(
    rps_ELO,
    head(rps_df)
)
cbind(
    head(rps_df),
    ELO_preds
)
#>   time_index  throw_1  throw_2 outcome ELO_preds
#> 1          1    PAPER     ROCK       1 0.4799619
#> 2          2     ROCK SCISSORS       1 0.4999898
#> 3          3 SCISSORS    PAPER       1 0.5200482
#> 4          4     ROCK    PAPER       0 0.5200381
#> 5          5 SCISSORS     ROCK       0 0.5000102
#> 6          6    PAPER SCISSORS       0 0.4799518
    # Predictions are all ~0.5

# Fit a mELO model that can handle these types of interactions.
rps_mELO <- mELO(rps_df, k=1)
rps_mELO
#> 
#> mELO Ratings For 3 Players Playing 120 Games
#> 
#> k = 1.
#> 
#>     Player Rating Games Win Draw Loss Lag
#> 1    PAPER 2203.1    80  40    0   40   0
#> 2     ROCK 2198.7    80  40    0   40   1
#> 3 SCISSORS 2198.2    80  40    0   40   0
#> 
# Inspect advantage matrix
get_adv_mat(rps_mELO)
#>              PAPER      ROCK  SCISSORS
#> PAPER        0.000  1130.431 -1140.229
#> ROCK     -1130.431     0.000  1134.765
#> SCISSORS  1140.229 -1134.765     0.000
# Get predictions
mELO_preds <- predict(
    rps_mELO,
    head(rps_df)
)
cbind(
    head(rps_df),
    mELO_preds
)
#>   time_index  throw_1  throw_2 outcome  mELO_preds
#> 1          1    PAPER     ROCK       1 0.998546265
#> 2          2     ROCK SCISSORS       1 0.998550921
#> 3          3 SCISSORS    PAPER       1 0.998551335
#> 4          4     ROCK    PAPER       0 0.001453735
#> 5          5 SCISSORS     ROCK       0 0.001449079
#> 6          6    PAPER SCISSORS       0 0.001448665
    # Much better predictions!