Bots & AI Player Performance: PR, Elo and Luck Explained

Four numbers. Four different stories

After every match, GamesGrid shows you four pieces of information: Level of Play, PR, Luck and your Career Elo Rating. They are easy to confuse, but they describe different things, and the one most players care about, their rating, does not tell the whole story of how they played.

This page explains what each one means, how we calibrated our own players against them, and why we publish the method instead of just the result.

PR: how well you actually played

Performance Rating measures the quality of your decisions, independently of whether you won. The arithmetic is straightforward:

Add up the equity you gave away across the match. Divide by the number of decisions you made. Multiply by 500.

Lower is better. Zero means no errors according to the analysis engine.

"Equity given away" is the difference between the move you played and the move the engine preferred, measured by their expected value. A move that looks ugly can cost nothing. A quiet, natural-looking play can be the most expensive decision of the match. PR does not care how it looked.

Two things follow from that definition, and both matter.

It does not care whether you won. You can win while playing badly and lose while playing the best backgammon of your life. PR helps you separate the result from the decisions you made.

It is an average per decision, so a short match says very little. Ten easy decisions and almost anyone can score well. A match full of difficult positions gives you more chances to go wrong. One match is a data point, not a verdict. A stable picture of your performance needs a run of matches behind it, which is why a career of them tells you more than a single session.

What the numbers on the scale mean

PR is more useful when you know where you stand on it. This is the ladder our own players are built on, with each character assigned an individual target within its level.

These are the characters' target bands and starting ratings, not guaranteed results for an individual match.

For orientation outside our own club, a world-class human plays somewhere around PR 3 to 5 across a tournament. That's worth keeping in mind before judging your own first results too harshly.

There is another useful comparison in the human data used to build the cast. A strong player, around PR 2.3, finds the engine's best move about 85% of the time. A weaker player, around PR 11, does so about 69% of the time.

The difference is sixteen percentage points. Backgammon is not a game you master by never being wrong. It is a game you improve at by being wrong less often, and less expensively.

Career Elo Rating: who has been winning

Your Career Elo Rating is the Elo-style number that moves with your results in rated play. It rises when you win and falls when you lose, with the size of the change depending on your opponent's rating.

It answers a different question from PR: where do you stand in the competitive field, and who are you trying to catch?

But it is a record of outcomes, and outcomes in backgammon include luck. A season gives you a broader picture than an evening. Neither should be confused with a direct assessment of every decision you made.

On GamesGrid, each character starts at the Career Elo Rating assigned to its level, from 1100 for a Beginner to 2300 at the top. From there, it changes with results, just like yours.

Luck: the number that ends the argument

Beside your PR, the post-match screen shows Luck. For every roll, it measures the difference between what you received and what an average roll in that position would have been worth, then adds those differences across the match.

It is there for one reason. Once luck is a number on the screen, it stops being an argument. You might have received a run of excellent rolls and still played poorly. You might have been rolled over while making good decisions. Both happen constantly. Separating the quality of your play from the rolls you received is where getting better starts.

Luck is shown after every match, including unrated ones, because it describes the dice rather than your place on the leaderboard.

The four measures, side by side

These distinctions are part of how the platform works, not just how the results screen is arranged. Career Elo Rating records outcomes. PR measures decision quality. Level of Play identifies the opponent's level. Luck describes what the dice contributed.

They are kept separate, and nothing you can buy improves any of them.

How we calibrated the cast against these numbers

A PR band on a page is a claim. Here is what was done to support it.

We measured people first. About 200,000 decisions made by around 100 real players were analysed, including all our test matches, spanning PR 1.7 to PR 15.9, each one compared with the engine's verdict. That is where the shape of the playing model comes from: where humans make mistakes, how often, how costly those mistakes are, and where they rarely go wrong at all.

In that human sample, races and bear-offs accounted for about 15% of moves but less than 1% of the total equity lost. That pattern shaped how our members handle those positions too.

Checker play and cube handling were fitted separately. They are different skills, and a player can be stronger at one than the other. What places a character in its level is the combined cost of its checker and cube errors.

Then every level was tested by measurement, not just by feel. Each was run through 200 games with fresh dice, assessed by the analyst at full strength, before it shipped. Only after that were the opponents played by hand, match after match, and the levels adjusted again based on what those sessions revealed. A number can be right while the opponent still feels wrong.

Those two steps work together. Numbers alone can produce an opponent that meets a statistical target but doesn't feel convincing. Judgement alone produces a claim nobody else can examine.

The claim we make is specific: the cast is measured against its intended levels. We do not claim that a character will produce exactly its target PR in your next match. The same variation that affects your performance affects theirs.

How the model works, including imperfect evaluation, cube temperaments and the absence of an error budget, is explained on the Bots & AI page.

The judge never changes

None of these comparisons would mean much if the analyst changed its standard from one opponent to another.

Whoever you played, the review afterwards uses the same settings: two-ply evaluation, no added noise, and the same move filter, every time.

The analyst is never weakened for a Beginner or strengthened for GG Guru. Both are assessed by the same standard, and so are you.

That keeps comparisons within GamesGrid consistent and lets us check our own calibration. If measured performance moves outside a level's target band, the analysis gives us a way to identify it.

Check the numbers yourself

Every match can be exported as a standard .mat file. Load it into GNU Backgammon, eXtreme Gammon, or another compatible analysis program, and review the decisions at your preferred settings.

We would rather you did. An analysis you cannot examine independently is an analysis you have to take entirely on trust. GamesGrid gives you the match record so you can look for yourself.

A note on the original GamesGrid bots

The bots that ran on GamesGrid from 1996 to 2008, Forever, Raccoon, Beaver, Otter, Weasel, Chipmunk and Tutor, were graded differently. A single full-strength engine used induced errors at a controlled frequency to create weaker opponents.

That produced a clear ladder from one underlying model. The new cast takes a different approach, using the measured patterns of human mistakes to shape each opponent's decisions.

The original bots' documented ratings and the archive-based account of their development are on The GG Family page.