Mikel Arteta received a lot of attention in December for his comments about analytics and win probability, most of it focused on opinions that it was either a manager of a struggling team looking for excuses or one who didn’t understand analytics. Neither of those seemed to me like valid criticisms, but the statement still raised red flags. Now with Arsenal’s fortunes looking up, I think it’s a more appropriate time to look at the true issues with his explanation..
Arteta’s Statement and Immediate Implications
“Last year we won the game against Everton with a 25% chance of winning, you win 3-2. Last weekend, it was a 67% chance of winning, any Premier League game in history, and a 9% chance of losing, and you lose — 3% against Burnley and you lose, 7% against Spurs and you lose. There is something else apart from that. It is not just the performance on the pitch, it is something else that needs to go our way and at the moment it doesn’t.” - Mikel Arteta after his team’s loss to Everton
To start, there’s not necessarily anything incorrect about the statistics Arteta stated - he seems to just be using win probabilities provided to him based on xG, which are in turn based on where shots are created from.
While that may not be incorrect, it’s certainly missing important context - xG and win probabilities derivative of them cannot measure the game by strategy and how that impacts the chances. In the case of Burnley, that’s a team that waits for you to create chances and then be clinical with their limited opportunities; I don’t necessarily expect a public model to be able to account for that, but would hope a team’s analytics department would.
The Tottenham case is an even simpler explanation - Tottenham went up two goals early on, and then had no reason to push for more. Tottenham had limited opportunities, but they came early in the game and drove the strategy later on; that isn’t a matter of luck, it’s a matter of Arsenal defenders not stepping up to Son and allowing their team to fall behind. This is something that, in my opinion, hockey analytics do much better at this moment, at least in the case of public models - score effects are crucial to truly understanding the flow of the game and who had control.
As I emphasize all the time, analytics without appropriate context are completely lacking in usefulness. These cases are perfect examples of this - Arteta uses information provided to him by his team’s analytics staff to try to portray his team as just being unlucky, even though the appropriate context would show that in many cases they were being outplayed and allowing the opponents to then sit back against a toothless attack.
I can’t reach a conclusion that Arteta really has much blame in that situation - one of a manager’s key duties is to keep his squad motivated and focused, and that could be all he was looking to do. What I am concerned about is that a Premier League football club is using analytics that appear to be no better than those available publicly to everyone. Maybe they are doing more behind the scenes, but The Athletic confirmed with members of the club that the statistics quoted are based on chances and xG. For a club that portrays itself as being on the cutting edge of the use of analytics, the larger implications of this are extremely concerning.
The Bigger Picture
In August, Arsenal drew headlines by making significant cuts to its scouting staff, including their head of international recruitment. Arsenal’s Technical Director, Edu, explained it at the time as wanting to work with fewer people and increase the use of StatDNA. Clearly most of the red flags at the time focused on this appearing to instead be another move to decrease spending; with the new context of Arteta’s comments, I’m instead concerned about the focus on StatDNA.
StatDNA was purchased by Arsenal in 2012, symbolizing a move to become a leading force in soccer analytics. The company was run by Jaeson Rosenfeld, who is widely respected both within Arsenal and the analytics community; the company came to be involved in almost every aspect of Arsenal’s processes. However, Rosenfeld left in 2020 to join Arsene Wenger at FIFA.
Obviously Rosenfeld built the company to be more than just himself, but his departure does raise even more red flags. That’s especially the case because it was reported at the time that the decision was driven by Arsenal reducing Rosenfeld's impact.
As I mentioned in my analysis of Arsenal’s tactics previously, this again screams of a club that can’t decide on a consistent direction. If Arsenal is truly making a move towards more of a focus towards analytics based decisions, then they certainly need to be doing so with more revolutionary and useful methods than what appeared to back up Arteta’s comments. If not, there’s no advantage to be gained by basing decisions on the same data being used by everyone else.
It may be even more concerning if they aren’t truly making a move that way - in that case this continues to be a club seeming to look to decrease costs at every turn without anyone to put the club in one consistent direction. Either way, a clear improvement is needed to take Arsenal back to where it should be.