Home / Trends In E-Marketing & E-Business / 5 a/b test ideas for mobile gaming apps

5 a/b test ideas for mobile gaming apps

We talk a lot about a/b testing here on MarketingExperiments. What we don’t usually talk about is a/b testing for the mobile web…especially testing within mobile apps.

I thought we should change that. As I was scouring the web looking for mobile a/b tests, I found this 2-year old video by Amazon.

Apparently, Amazon Web Services (AWS) at one point had an a/b testing feature that is now closed
.
When they had testing, however, one app developer used it extensively and shared their experiences in a promotional video for the feature on Amazon. The developers were behind the game Air Patriots. Russell Caroll was the Senior Producer for the game and Julio Gorge was the Game Development Engineer. The game is a kind of aerial take on the classic tower defense game genre.

air patriots screenshot

Air Patriots

Now granted, this was a promotional video, but the content still speaks for itself. These guys had (and still have by the looks of it) a fairly successful mobile app and they ran some successful tests. It’s a great starting place for what you can test in your mobile app.

By the way, while Amazon has shut down its a/b testing feature, there are a lot of other tools for testing mobile apps that will accomplish the same thing the developers talk about in the video.

Test #1: What is the impact of ads on customer experience? (1:34)

The first thing the team tested was the impact of ads are on their customers. They wanted to make sure the ads did not harm the customer experience. So they tested a single ad in the main menu near the bottom of the screen.

air patriots menu ad

They found that the ads didn’t affect customer retention. This meant that they could insert ads and generate more revenue without hurting their customers.

Test #2: Will in-game ad placement affect customer retention? (2:56)

In the second test, the team put ads in the game screen.

air patriots in-game ad

In both the first and second tests, the ads had a little “X” that the customers could tap to hypothetically dismiss the ads. When they tapped, a pop up came up that told customers they could eliminate ads with any purchase in the game’s store.

In this test, there was again, no impact on customer retention, but there was a statistically significant increase in revenue.

Test #3: Simple game-circle icon test (4:20)

In this test, the team wanted to know whether an icon to the game-circle (Amazon’s game stats and leaderboards portal) would improve performance.

air patriots game-circle

It’s not clear which icon won, or even why this particular test was useful for the team, but they did get a favorable result, and the lesson they wanted to drive home was that simple changes like icons can make a difference. We’ve, of course found that to be the case in a large number of our tests on MarketingExperiments.

Test #4: Does game difficulty affect revenue? (4:58)

In this 4th test, Caroll made a mistake. He accidentally changed the game difficulty to make it about 10% harder. As a result, every metric that was important to them tanked.

air patriots metrics

The team of course fixed it as fast as possible, but it gave them an idea.

What would happen to revenue if they made the game easier?

So they ran a test that had 5 treatments: The control and then 4 difficulty levels that were easier than the control.

It turned out that the easiest difficulty performed the best. By making it easier, they players playing 20% longer and revenue went up 20%.

Test #5: When is the best time to have push-notifications for re-engagement in inactive players? (7:43)

The team then tested a push-notification that offered inactive players and incentive for picking the game back up.

air patriots push

They wanted to know when the best time to send the notification would be. So they tested a few different variables and found that the best time was 3 days after the last game play.

They also found that sending the notification 7 days after the last game play negatively impacted their performance metrics.

With these 5 tests and probably a few more that have been happening off the record, the team was able to develop a great app for their customers and steadily increase their revenue. At the end of the video Carol gives a few key takeaways for marketers who are a/b testing their apps.

You might also like:

Email Research Chart: Email opens trends on mobile devices in 2015

Mobile Marketing Chart: Amount of revenue from the mobile channel, by merchant type

3 A/B Testing Case Studies from Smart Brand-Side Marketers

Click Here For Original Source Of The Article

Check Also

Digital Marketing News: ROI Acronyms, Google Ranking Factors and Twitter’s New Look

Digital Marketing News: ROI Acronyms, Google Ranking Factors and Twitter’s New Look

The Hipster’s Guide to ROI [Infographic]
Marketing lingo has expanded and with all of the acronyms, it’s hard to decipher and differentiate combinations of letters. This infographic will show you the most common acronyms and esoteric language related to marketing ROI, giving you an explanation of what they are and why they matter. (LinkedIn Marketing Solutions Blog)

SEMrush Ranking Factors Study 2017
Google ranking factors constantly updates with every major algorithm change. In this report, the 12 most substantial and controversial factors (including website visits, pages per session and content) were chosen to show what impacts search results and to identify consistent patterns in the ranking mechanism that could be helpful to the SEO community. (SEMrush)

Check Out Our New Look!
Twitter has listened to the feedback from its users and have made some updates to the design. Some of the new features include: Typography has been refined to be more consistent with bolder headlines and rounded profile photos, Tweets are now updated instantly on the mobile app with replies, retweets and like counts so you can see real-time conversations and links to articles and websites now open in Safari’s viewer in iOS so you can easily access accounts on websites you’re already signed into. (Twitter Blog)

LinkedIn Adds Images in Comments, New Opportunities for Job Listings
There have been many small yet impactful new updates to LinkedIn recently, due to audience demand. One new feature is you can now add images into comments on posts within the LinkedIn platform. Another boost for LinkedIn is Google’s new tool which helps people find jobs directly through Google search, which sorts through various listings, including LinkedIn. (Social Media Today)

Instagram Stories Now Has 250 Million Daily Active Users, Heating Up Its Rivalry With Snapchat
Instagram Stories is the section of disappearing posts, which recently pulled ahead of Snapchat with an increase of 50 million users in just two months. Instagram also announced that users are now allowed to replay live video instead of it immediately disappearing. (AdWeek)

Google’s Job Listings Search is Now Open to All Job Search Sites & Developers
Google is now offering a formal path for outsiders to add job listings in Google search. Although it doesn’t have an official name, it’s part of the Google for Jobs initiative. You can also track how well your job listings are doing in Google search with a new filter in the Search Analytics report in the Google Search Console. (Search Engine Land)

Oh, How Pinteresting!
Pinterest rolled out a fresh new look for Lens, and instead of only being able to recreate your favorite restaurant dishes at home, Lens can now recognize and recommend outfit ideas including shoes, shirts, hats and other styles. The new interface and built-in tools make it easy to Lens the world around you. (Pinterest Blog)

The Most Important Skills for B2B Tech Marketers
B2B technology marketers rely on many skills for their niche market. The most important skills among Millennials, Generation Xers and Baby Boomers were soft skills, including communication and people management and writing skills. Others included digital media marketing and content marketing. (MarketingProfs)

What were your top digital marketing news stories this week?

We'll be back next week with more top digital marketing news stories. Craving more news in the meantime? Check out TopRank Marketing on Twitter @toprank!

The post Digital Marketing News: ROI Acronyms, Google Ranking Factors and Twitter’s New Look appeared first on Online Marketing Blog - TopRank®.

Leave a Reply

Your email address will not be published. Required fields are marked *

css.php