Hiding Information from Players

As a game designer, you should never hide information that the player already has from the player.  The reasoning: the player has this information already, so all you are doing by not tracking it within the game is encouraging the player to track it manually out-of-game.  This is a big waste of time for the player, and it never feels good to have to choose between two bad options: either spend a bunch of time tracking it manually or playing worse on purpose by not tracking whatever information.

An obvious example of what I’m talking about is with things like in-game maps. It’s not a requirement that a roguelike includes a map the player can browse at-will; the game could hide the map from the player and still function properly. Probably no one would play that roguelike, of course, since it’s unbelievably annoying to have to keep track of everything manually.

The game that has me writing about this is actually Dominion, however. If you haven’t played it (or heard of it), the only information you really need to know is that it’s a card game, and it has a discard pile where you put cards you’ve already played. The relevant thing to this post is that Dominion actually includes a rule saying you cannot look through your discard pile.

To me, this is absolutely insane. It’s completely impossible for a card in Dominion to enter your discard pile without you knowing that it did so, so this is a case where the player already knows all the information. It’s really easy to check, and honestly takes almost no time at all to do so. The ostensible justification for the rule, I think, is to speed up gameplay … but the problem is that probably it does just the opposite.

If you’re just playing casually then probably you don’t care too much to track which cards in your deck you have already seen, so checking your discard pile does not happen to begin with and the rule is irrelevant. However, if you are taking the game seriously, then it’s pretty important in a lot of situations to know what cards remain in your deck and what cards you’ve already played. So the “best” way to account for this, as a player, is to spend a lot of time making absolutely sure that you know what cards you’ve played, since you are not allowed to check this afterward.

The rules also say you can’t track this stuff on paper, so what you want to do is figure out some other way of keeping track. Probably this comes at the cost of doing things like actually playing your cards quickly, since after every card play you need to make sure you add it to your mental tracker. In practice perhaps this ends up being too complicated, so you just have a vague, inexact idea about how many of each card you’ve played. To me, that just feels really unsatisfying and hurts my enjoyment. Thankfully it’s not too much of a problem in real life since the appropriate thing to do here is just house rule away the problem by letting people look through their discard pile.

It’s a pretty serious problem in online Dominion, though. The no-notetaking rule is completely unenforcable. It’s trivial to keep a text file open and use it to track which cards you’ve played. It’s technically cheating, but you can’t ever stop people from doing it, and the information is available to the player anyway. Removing the restriction on looking through your discard pile strongly reduces the incentive to take notes in this fashion, so the rule should just go away. And as I said above, it doesn’t actually help anyone to begin with: if you don’t care about tracking your deck, then you don’t care whether you can look through your discard pile.

I should note here I only mean looking through your own discard pile. Opponents’ discard piles are not actually public information, and that’s fine. Additionally, while it is helpful to track your opponents’ cards in the same way you’d like to track your own, the no-notetaking rule is fine because notetaking actually does slow down play, so there is a benefit to the rule existing. (It’s still a problem online, of course, but there’s a good argument for the online version of Dominion sticking to the rulebook in this case.)

Also, lest it sound like I’m ragging on Dominion too much here, I should note that the game is amazing. It’s just this rule that’s bad.

A great way to handle this in online play is the way Star Realms does it. In the Star Realms app you can click on any discard pile to view the contents of that discard pile, and you can also click on either player’s deck to view the cards remaining in the deck. This is all public information anyway, so letting the player access it so easily is a great thing.

Advertisements

Estimating Trevor Story

Trevor Story, of course, became the first player to homer in each of his first four MLB games.  Obviously this is pretty improbable, so let’s estimate how improbable.

First we need the chance for Story to hit a home run in a given plate appearance.  I’ll use the FanGraphs depth chart projections, so that would be 15 HR in 490 PA, or about 3% per PA.

Story actually got 6, 4, 4, and 5 PAs in his first four games, so I’ll just look at games with 4-6 PAs.

For 4 PAs in one game, there’s an 11.7% chance that Story hits at least 1 HR.  For 5 PAs, 14.4%, and for 6 PAs a 17.0% chance.

For the number of PAs Story actually got, this gives a 1 in about 2984 chance to homer in all of his first four games.  This is somewhat sensitive to the actual PA distribution, but it seems the chance is somewhere in the neighborhood of 1 in 2500 to 1 in 3000 or so, if you get that total number of PAs.  Obviously, very improbable … for any individual player.

It’s a little less astonishing that this happened when you realize that there have been somewhere around 18000 major league baseball players in history.  Using the FanGraphs leaderboards, I get a total of 7417 non-pitchers who have recorded at least 20 PAs since 1900.

It’s … a lot of work to figure out the actual probability of some major league player homering in his first four games of his career.  So I don’t have a good actual estimate for the chance of this happening.  But I think that it’s not terrifically unlikely that some player would have, by this time, homered in his first four major league games. It’s probably reasonably likely after accounting for what I’ve seen called the Wyatt Earp effect; in short, large populations are almost certain to have a few large outliers just by chance.

Still, this is certainly a remarkable start to any individual player’s career.  Congrats to Trevor Story for a resoundingly successful major-league debut; he’s been the fifth-best position player in the majors so far this year, by FanGraphs.

A few League of Legends notes

This was originally going to be a post about something different, but then I found League of Graphs so no need for me to look at side advantage on my own like I was going to (blue side has a small advantage in professional play).

 

So in replacement, a few thoughts with unfortunately no current statistics to back them up:

It seems likely to me that instead of KDA what you probably really want to look at is kill differential, or some other similar stat (where you are adding numbers instead of multiplying).  My intuition here comes largely from baseball, where K/BB is actually quite bad compared to K%-BB%, even if the latter isn’t exactly mathematically sound.  In baseball actually often you go one step further and just use FIP or xFIP.  My guess is a similar metric would be better for actually evaluating League of Legends players, since KDA mainly increases by not dying, just as K/BB mainly increases by not walking batters. Like not walking batters in baseball, not dying is a good thing, but it’s not as much of a good thing as KDA would claim.

Of course this isn’t actually used by Riot because KDA is better for giving to your playerbase; it will never be negative (by definition), and it’s usually a number noticeably larger than 1, so it probably makes more players feel good when they see it in their in-game profile.

Since we’re nearing playoff time, this is also a good time to bring up a post I wrote up on LiquidLegends last year about regressing observed winrates.  Immortals is a very good team, but they don’t have a true-talent 94% winrate against LCS competition.  Estimating with this method gives about an 80% true-talent winrate.  So, they’re still definite favorites against anyone they play, but certainly not unbeatable.

A 2-1 result in a best-of-3 is rarely more likely than a 2-0

This is just a minor thing that bothers me when hearing predictions.  It comes up a lot with League of Legends casters; I hear a fair number of 2-1 predictions for a best of 3 that the casters feel is pretty evenly matched.

In the case of the teams being exactly evenly matched and there being nothing like home-field advantage (or side advantage in League), you get a 2-0 result 50% of the time and a 2-1 result 50% of the time.  Making the teams uneven of course makes a 2-0 result more common.

If there is a home-field/side advantage, such that the team that is more likely to win game 1 is less likely to win game 2, then it is true that with two very closely matched teams a 2-1 is actually very slightly more likely than a 2-0.

Of course what the casters really mean is they just think the teams are close to even.  But you should not be surprised when teams that are evenly matched go 2-0 in a series, unless you have some reason to believe that winning the first game makes you less likely to win the second game.

 

Playoff projections in sports

I follow baseball pretty regularly, and with the 2016 season yet to start this is of course time for everyone to talk about team projections for the upcoming season.  Most of this discussion focuses on the projected win totals, and there’s lots of talk, both good and bad, about the win totals on various sites.  (If you’re interested, I think Phil Birnbaum’s blog has some of the best posts, though I’m too lazy to find the particular posts I’m thinking of right now.)

Finding a real flaw in the mean projected win totals is pretty hard and takes tons of data, so instead of talking about the actual projected win totals let’s think about projecting playoff odds instead.  FanGraphs has projected playoff odds to go with their mean win totals in the standings.  In other sports you have things like FiveThirtyEight’s basketball playoff odds, or Football Outsiders’ DVOA playoff odds (which they don’t have up right now, of course).

I’m not sure of the exact method used to generate these playoff odds, but I can say I’m pretty confident that most of them are at least somewhat wrong.  The main problem is that we can’t be certain what a particular team’s actual true talent level is.  While projections, at least in baseball, are pretty good at getting the mean right, there’s still certainly some deviation between a team’s actual true talent level and the projections’ estimate of that team’s true talent level.  It turns out that even if you are actually correct on the average, you still get playoff odds wrong by simulating seasons using a fixed true talent level for each team.

There’s a simple illustration of this that I think is convincing, though it’s not perfect.  Let’s look at the NL West, and assume that the projection for the Dodgers to win 94 games is exactly correct, and additionally let’s assume that the Diamondbacks, Padres, and Rockies always win fewer than 94 games.  So we’re left with just the Giants to consider as contenders to the Dodgers in winning the division.

Given a set true talent level and some ass, it’s possible to analytically solve for the probability of a team winning at least a given number–in this case 94–games in a season.  Steve Staude at The Hardball Times created a spreadsheet that does just that, along with simulating playoff series.  If we assume that the Giants’ true talent winrate is exactly .540, as the Fangraphs projections have, then they win 94 or more games 13.4% of the time (and I’ll assume this means they always win the division).  Fangraphs has the Giants’ division odds at 23.5%, so this estimate doesn’t seem horrifically wrong given that I’m assuming the Dodgers always win exactly 94 games.  In reality, of course, there are a lot of times the Dodgers win fewer than 94 games if the projections are correct, and since decreasing variance favors the favorite we should expect our quick estimate to be low.

Ok, now on to the reason most simulated projections are wrong: let’s add in variation in the Giants’ true talent level.  I’m not sure what the actual standard deviation for true talent compared to projections is in baseball, and I’m not aware of anyone looking at that particular question, so I’m just going to make a quick assumption and say that it’s 3-ish games.  In fact, since this is just going to be a quick estimate, let’s assume that the Giants have a true talent winrate of .540 1/3 of the time, a true talent winrate of .555 1/3 of the time, and a true talent winrate of .525 1/3 of the time.  Importantly, this means our projections are still correct on average.

With a true talent of .555, the Giants win 94 games 23.5% of the time.  With a true talent of .540, as I said, they win 94 games 13.4% of the time, and with a true talent of .525 they win 94 games 6.82% of the time.  So this new situation gives the Giants a combined 14.6% chance of winning the division, instead of our original 13.4% estimate.  As expected, variance here favors the underdog.

Running the same exercise with the Giants fixed at 88 wins and looking at the Dodgers, we get am 80.7% chance for at least 88 wins with a fixed .579 true talent, or a 79.2% chance with the same varying true talent.  So it works both ways as we expect.

The problem is that it is absolutely impossible to simulate variable true talent across an entire season using all 30 teams.  Using a fixed true talent, to simulate an entire season once means using 2430 random numbers, essentially.  Or, working with just orders of magnitude, 103 numbers. Doing this 10,000 times brings us to 108. This is entirely doable on a daily basis.

Adding in the simplest reasonable true-talent variation is problematic, to say the least. Giving each team three possible true talent levels means you need to do 330 as many simulations, at a minimum (one for each possible combination of team true talent levels). 330 is the same order of magnitude as 1014. Now we’re up to 1022 random numbers, which you have to then compare to another number to determine which team wins. There are only 105 seconds in a day, so you need to simulate each game in less than 10-17 seconds to run this simulation daily, as FanGraphs does for their playoff odds. That’s not feasible on a home computer.  You can reduce the number of times you simulate each true-talent combination season from 10,000 down to something lower, but even if you go to 1 I still think it’s not something you can reasonably run daily on a home computer (I might check this at some point).

Since this is the simplest reasonable case, and it’s already too complicated to actually simulate, it’s safe to say that this form of true-talent variation is most likely not what FanGraphs does for their simulations.  So, unless they compensate for the increased variance in some other way in their playoff odds, their playoff odds are wrong.  I think they’re pretty good–the effect here is maybe a percentage point or so in magnitude–but I certainly wouldn’t trust the decimal points.