Bots won’t buy you any fan engagement – or will they?

Fan Engagement in the MLS

We’re about three months into the MLS season – time for another edition of the highly in-official Hype-O-Meter. With two important changes:

  1. How “real” are your followers: When Atlanta United entered the scene, there was much debate on whether how much they had bought themselves into their “from-hero-to-zero” status on Twitter. An expansion team suddenly having the most followers in the league – smells like fake. And it seems to be exactly that. Inspired by the press coverage of Trump’s Twitter exploits, I used “Twitter Audit” to analyze a sample of Twitter followers for each team. Although this is not a perfect representation for the exact makeup of each team’s followers (e.g., I did not pay for a pro account and therefore had to rely on data from earlier analyses, only sub-samples were analyzed), it can serve as a decent proxy.

“Each audit takes a sample of up to 5000 (or more, if you subscribe to Pro) Twitter followers for a user and calculates a score for each follower. This score is based on number of tweets, date of the last tweet, and ratio of followers to friends. We use these scores to determine whether any given user is real or fake. Of course, this scoring method is not perfect but it is a good way to tell if someone with lots of followers is likely to have increased their follower count by inorganic, fraudulent, or dishonest means.”

What I found was quite interesting. As some commentators had suspected earlier: Atlanta seems to have added quite a bit their follower count. In fact: Only 48% of their 517,134 followers are deemed real. Interestingly, though, that does not seem to hurt them. Usually, simple bots or fake accounts won’t buy you any fan engagement (as measured in likes and retweets), but Atlanta still comes in a solid 3rd. However, they are the exception. There are several other teams in the MLS that seem to have added to their follower count — and as a result, find themselves mostly towards the bottom of the engagement chart.

Here is the complete list of “fakeness” in the MLS:

Team Followers % of “Real” Followers
Vancouver 308005 26
Toronto 290872 42
Atlanta 517134 48
Houston 343049 51
Montral 284534 61
Orlando 387925 63
San Jose 203446 63
Washington 117203 64
Seattle 407927 66
Portland 285669 73
Los Angeles 391280 74
NY Red Bulls 184798 76
Dallas 134959 77
Colorado 84481 77
Kansas City 297709 78
Chicago 139334 80
Philadelphia 109161 81
NYCFC 332828 83
New England 89855 84
Columbus 143143 85
Salt Lake 127778 87
Minnesota 60619 88

No surprise: all professional / celebrity accounts will draw some noise and attract the occasional bot follower. However, the result for Vancouver was quite shocking. Of their more than 300k followers, only 26% (or about 80k) are active enough to be counted as “real”. I almost hope that this is some form of a slip-up, but the engagement numbers would match the trend. For the 2nd time in a row, the Whitecaps are among the bottom three of the league, generating only .86 favorites and .6 retweets per 10000 followers. They were also voted as least appealing Twitter Account by Howler Magazine. Houston and Montreal, the only two teams with less fan engagement, also rank in the first quarter of the “fake followers” analysis. On the other side of the spectrum: Minnesota United deserves a big shout out. The expansion franchise seemingly chose the slow(er) route of organic growth on social media and now tops the Engagement Ranking for the 2nd time in a row — with a whopping margin.

2. Bye, bye – Facebook. I decided to leave Facebook out of the analysis. Both platforms are very different and lumping them together in one analysis is likely to confound the results. Instead, I decided to focus on Twitter — which became even richer from a data perspective given the addition of “Twitter Audit” and some planned further analyses.

Some interesting findings / thoughts.

  • Overall engagement and tweet frequency are up from the pre-season analysis. Which makes perfect sense given that game days are expected to a) see more tweets, and b) get fans much more involved.
  • Chicago Fire and L.A. Galaxy upped their game. Not sure if it is the “Schweinsteiger-Effekt” for Chicago or getting quite a bit of TV time for Los Angeles, but both teams jump significantly in the ranking.  They might have also simply upped their social media efforts for the season: L.A. was just voted as having the best memes in the MLS.
  • Can’t buy me love – or can I? Atlanta is somewhat of a conundrum. I just complained about their (presumably) artificially bolstered follower count and how that should diminish their fan engagement scores, and yet they rank among the top three for the 2nd time in a row. How can that be? We can’t be sure, but there are some possible explanations:
    • 1) Even without the suspected bots, United would still sport almost 250k followers – the 4th most in the MLS (when all teams are adjusted to their true follower count based on Twitter Audit data). So: There is quite some buzz surrounding the team — and maybe making the follower count look nice early on kick-started overall engagement. When we only look at this “core” group of followers and calculate engagement based on them, Atlanta comes in first. By far.
    • 2) I don’t want to suggest anything here – I really like how Atlanta has kick-started their campaign on the pitch as well as online – but one could also suspect that they invested in smart bots that could automatically like and share content instead of “dead” fake accounts. In the end, though, any brand engaging in such behavior would shoot itself in the foot. No matter how well a bot is programmed, unless you also train in to buy your merchandise and sit in the stands, the ROI simply won’t be there. Instead, you’ll have to explain why you seem to have a gazillion fans — that never buy anything.
  • Love thy fans! There is quite a variance in the amount of interactivity among teams and their fans – at least when taking replies (and retweets) on Twitter as a proxy. Seattle leads the reply charts: more than 28% of all original tweets were in reply to another Twitter account — compared to 4% for Orlando and Montreal. Looking at retweets, Salt Lake is king. Almost 39% of all tweets are re-tweets. On the other end of the spectrum: Philadelphia posts the most original content with “only” 9% of RTs.

The Language of Engagement

Figure 1. Average number of favorites and retweets across Twitter accounts
Figure 1. Average number of favorites and retweets across FCB Twitter accounts

Following my analysis of the languages spoken by #Copa100 fans on Twitter, somebody asked me: Does it even make sense to have language destinations if most people flock to the major account anyways? In other words: My resources are limited – so why put effort into crafting language-specific content when the majority of fans does not seem to care?

Good question.

The answer is: yes, language destinations make sense. A lot of sense.

And here is why: Although we don’t reach as many people with the additional accounts (the average “foreign language” account has about 63% fewer followers), the ones that we reach are usually more committed. And greater commitment means more engagement with our content — and ultimately a stronger bond with our brand. At least that’s the theory.

Are “international” fans really more engaged?

Take Bayern Munich, for example. Their main Twitter account (@FCBayern) has about 2,85 million followers. However, given the popularity and social significance of Bayern Munich in German society (games and player signings often serve as token for conversation), many followers are likely to be less committed (read: average sports fans that just want to stay up-to-date) and therefore consume information rather passively. For many followers, Bayern Munich might only be their 2nd or 3rd favorite club that they revert to when the club plays internationally. Following (the entertaining) @FCBayernUS, on the other hand, requires more commitment to soccer in general and Bayern in particular, as the sport and club are not “mainstream-topics” in the US. As a result, a more active audience should be expected. Similarly, fans of Chicharito Hernandez following the Spanish-language account of Bayer Leverkusen (@bayer04_es) should be more inclined to interact with content that is specifically tailored towards their interests.

Figure 2. FC Barcelona provides 9 language-specific Twitter accounts
Figure 2. FC Barcelona provides 9 language-specific Twitter accounts

But is the really the case? Testing my hypothesis, I compared a total of 14 language destinations — including those of two leagues (Bundesliga, MLS) and three clubs (Bayern Munich, Bayer Leverkusen, FC Barcelona). This is by no means a representative sample, but rather a purposive one. I chose Bayern mainly because of the “unusual” way they run @FCBayernUS. To engage fans in the US, the account features more entertaining content (informal language, GIFs, emojis, retweeting of user generated content) than most “traditional” team accounts. In theory, this should result in greater engagement. Similarly, the Spanish Leverkusen account (started in 2015 after signing Chicharito) provides content tailored to his fans. Furthermore, I chose the official Bundesliga accounts (German and English), to assess how the expanded international TV deals (especially in the US) affect engagement. Similarly, I was interested in potential differences between the English and Spanish accounts of the @MLS. Finally, I added three @FCBarcelona accounts — just because the club is probably the most extreme example of creating language destinations (see Figure 2). Also: The club’s main account is in English rather than Spanish (all other clubs and leagues in the sample use their “native” language for the main account). And: In contrast to most other entities, all Barcelona accounts tweet the exact same content (with very few exceptions). In other words: They do not tailor content towards specific audience segments, which might reduce the benefit of language destinations. Here is what I found:

Language destinations show more engagement

This slideshow requires JavaScript.

  • Teams get more engagement than leagues. Fans identify with their favorite club – not necessarily the league the club plays in.
  • Language destinations out-perform the “original” account. For all entities in the sample, the language-specific accounts received more favorites and retweets per 10,000 followers. The most impressive numbers come from @FCBayernUS (7 x more favorites; 10 x more retweets than @FCBayern) and Leverkusen’s international destinations.
  • It is easier to like than to share: All accounts received more favorites than retweets. This yields support for the argument that a retweet/share should be valued higher than a favorite/like when evaluating social media metrics. Favoriting a tweet involves lesser commitment and effort than retweeting and thereby endorsing a tweet and might be done for a different reason (e.g., archiving function, social token).
  • Content matters: Language-specific channels yield the biggest benefits when their content is specifically tailored towards the targeted audience segment. In other words, simply translating the “original” content is not enough. Language destinations designed around a specific purpose (e.g., a player, cultural engagement) tend to generate the most engagement.

Method: Some detail on the analysis

Data Collection: I accessed the Twitter API using the userTimeline function of the twitteR package in “R” to call up the timelines of the selected accounts. Using this method, Twitter limits the search to a relatively short period of time (usually between 1 – 3 weeks. However, I was able to go back until November 2015 for @Bayer04_es). Other methods (such as Pablo Barbera’s getTimeline function) allow downloading up to 3200 tweets, but showed inconsistencies for key variables during data collection. Therefore, I chose data-quality over sample size and defer the larger-scale analysis until later. Overall, I collected 6556 tweets across 14 accounts. The number of tweets per account ranged from a low of 88 (@MLS) to a high of 1639 (@FCBarcelona).

Analysis: Twitter provides two metrics that are commonly used as a proxy for user engagement by both industry and academia: favorites and retweets. Despite questions about the validity of these measures (e.g., does a favorite on Twitter really mean somebody engaged with your tweet – or is it a social currency acknowledging your relationship?) and uncertainties about their value (how much is a favorite worth – and how much more value should be attached to a retweet that actually increases your audience?), they a) still seem to be accepted as the industry standard, and b) are the ones I can easily measure automatically. To allow for direct comparison of all analyzed accounts, I normalized both engagement measures as averages per 10,000 followers. By doing so, @Bayer_EN (18k followers) and @FCBarcelona (17,8m followers) have a level playing field to compete on.

You can find some descriptive statistics here.