Why Esports isn’t the Promise Land for AIs

I have often written about the power data and AI has in esports. In my early, naïve days I was shocked by what I saw inside of Tier-1 teams. They had all that data and yet sophisticated applications of data were a rarity. I would find PowerPoints with a few win-rate tables, the occasional heatmap or even a print-screen for a Solo Queue analyser like U.GG or OP.GG.

Why, I asked myself, was no one taking this to the next level? How come all I see is rudimentary spreadsheets and the occasional graphic? It took 18-months of work before I began to answer this question.

Before we dive deeper, let me be clear and say that I fundamentally still agree that there is so much more to be done and this isn’t written as a way of putting people off the idea. I instead wish to present my honest view on the subject, instead of an overly simplistic and optimistic view that distorts the truth. It may also help if some of you are reading this wondering why professional teams seem to be doing so little in the space.

To begin, what gave me my initial confidence? What arguments did I hold in my head that drove me to quit my job in consulting and go full-time on the pursuit of AI in esports?

The foundation of this argument is simple: it’s already been done. Football, Baseball, Basketball or F1, wherever you look in traditional sports you will story after story of analytics being used to deliver significant edges. Obviously, Moneyball being the most famous of these, yet there are plenty of others and there are more every season.

If it’s been done, it can be done again. In fact, we can build on this argument by saying that it can be done, if anything, better. Esports is gaming and gaming is virtual. Data, likewise, is virtual. In other words, there’s no middle-step required to turn the sport into 1s and 0s. The sport is 1s and 0s. This makes generating data far easier.

The next obvious benefit esports brings is the existence of solo queue. Millions of games played every day, some of those even played by the very professionals who compete on stage on the weekends. This provides a dataset that would be impossible to generate in any other sport.

Finally, the game is played in a programmatic system, and therefore can be measured exactly. Given a rather small set of factors, I can tell you exactly how much damage a player will do with a specific combination (ignoring critical strikes). If you repeated it again, you’ll do exactly the same. Imagine being able to measure exactly how fast a Footballer could kick a ball and where it will land based on some rudimentary factors like who they are, how fast they were travelling and what boots they wore. You could get a range, sure, but never exact.

It sounds like an idyllic world to apply the ever-growing field of data and AI, particularly given the surge in open-source models and computation costs hurtling towards zero. What exactly then, are the drawbacks? Again, take each these with a mindset of “I wonder how I can overcome that”, as opposed to “OK so data is useless then”.

The first I’d like to talk about is one that has played on my mind more and more throughout my time working in the field. In Football there is a statistic known as Expected Goals. It essentially tries to remove aspects of luck out of the game and give the “average” score-line if the game had been repeated exactly as it was over and over again.

To help explain it, let us imagine a penalty kick where there is, let’s say, a 75% chance of scoring (credit goes to Football Hackers for this example). That means if the team was awarded a penalty then their Expected Goals increases by 0.75, or 75% of a goal. If, instead, a player took a wild shot from the half-way line, they may have a 1% chance of scoring and so their Expected Goals only goes up by 0.01. The probability can be refined by looking at things like: who the player is, how many players are near them, who the goalkeeper is and so on. You can then sum up all these chances and get the metric Expected Goals.

Esports, particularly MOBAs like League of Legends, have the hardest time replicating a metric like this. Every game-state is vastly different to the next. A single team fight has a thousand factors to consider before you can get close to measuring the probability of success. This then only worsens as the game goes on, as the result of the previous engagement will significantly affect the remaining ones.

It is also incredibly difficult to put a measure on “success”. If you lose 2 players but steal the Dragon, is it a winning or losing trade-off? It depends. Who got the kills? What dragon was it? What was the game-state beforehand? Where were the lanes?

Compare this to Football, or most other traditional sports. A score is a score. If you concede a point there isn’t anything to be gained. When the game restarts the probabilities of both sides scoring again are almost exactly the same as before, the only variance being some difficult-to-measure and likely low impact factors like how the goal may impact the players mindset.

In other words, in Football it’s fairly simple to calculate probabilities of success (chance to score a goal), and success is specific and measurable (+1 goal). In League of Legends there is a thousand factors to think through when looking at any number of plays, and the success of that play is inherently difficult to measure. You could even secure the objective and lose nothing for a seemingly obvious win, then realise you’ve lost all momentum and 30 seconds later you find yourself losing something greater than you gained.

This is why I have stated a number of times that I feel the hardest challenge for AI in esports is providing help in macro decision making. It’s why I focused so much on draft. It’s at least more static and replicable than anything done in live play.

The next issue is something I’ve spoken about a number of times before. Patches can, and often will, significantly change the game. Traditional sports have (roughly) the same rules and although the game still evolves, it’s a natural change as conditions, coaching, data and professionalism improves. The rules, balls, nets and pitches don’t get tweaked every other week.

This issue and the previous then compound. You need to have a huge amount of data since the sheer number of variances throughout the games make it very difficult to find an adequate number of samples to say “These 1,000 plays are very similar and they succeeded X times”. Then, every subsequent patch your previous data becomes slightly more irrelevant. You may have 1,000 similar plays from last season but how relevant are their success rates compared to this season? Is success even measured the same? A dragon today isn’t a dragon last year.

A final point to consider is the variance between Solo Queue and Professional play. One is not exactly like the other, in fact many argue that they are barely comparable at all. The requirement to build massive datasets each patch requires Solo Queue, yet you need to identify and factor in the many differences between the two in order to properly apply it. When I was looking into competitive Draft I found there were a number of examples where in Solo Queue Champion A hard-countered Champion B, then in Professional Play it was vice-versa. Likewise, the various scenarios play out entirely differently given the inclusion of verbal communication and the try-hard nature of professional play.

To end on a more positive note, let me make it clear that esports is far from the efficient frontier of data and AI usage. There is low-hanging fruit as far as the eye can see and all we need is smart people to break into the scene to begin picking at them. However, most of these sit in the realm of data-understanding, graphical representation, simple statistical context and the likes - not in the deep and complex world of Neural Networks and Reinforcement Learning. We need to build on top of solid foundations if we want to build up to the clouds.