Analytics Blog: Optimize

Better A/B Testing with Firebase

November 7, 2017

Earlier this year, the Google Optimize and Firebase teams worked together to bring A/B testing functionality to Firebase. Last week, at the Firebase Dev Summit, we announced that A/B testing is now available in beta to all app developers.

This post originally appeared on The Firebase Blog.

Announcing Better A/B Testing with Firebase

If you're like most app developers, you know that small changes can often make a big difference in the long term success of your app. Whether it's the wording that goes into your "Purchase" button, the order in which dialogs appear in your sign-up flow, or how difficult you've made a particular level of a game, that attention to detail can often make the difference between an app that hits the top charts, or one that languishes.

But how do you know you've made the right changes? You can certainly make some educated guesses, ask friends, or run focus groups. But often, the best way to find out how your users will react to changes within your app is to simply try out those changes and see for yourself. And that's the idea behind A/B testing; it lets you release two (or more!) versions of your app simultaneously among randomly selected users to find out which version truly is more successful at getting the results you want.

And while Firebase Remote Config did allow you to perform some simple A/B testing through it's "random percentile" condition, we've gone ahead and added an entirely new experiment layer in Firebase that works with Remote Config and notifications to make it quick and easy to set up and measure sophisticated A/B tests. Let's take a quick tour of how it works!

Getting to Know the New A/B Testing Feature

With the new A/B testing feature, you can create an A/B test that will allow you to play with any combination of values that you can control through Remote Config. Setting up an A/B test allows you to define how the experiment will behave in a number of different ways, including determining how many of your users are involved with the experiment at first…

…how many variants you want to run, and how your app might behave differently for each variant…

...and what the goal of the experiment is.

Different experiments might have different desired goals, and A/B testing supports a number of common outcomes, like increasing overall revenue or retention in your app, reducing the number of crashes, or increasing the occurrence of any event you're measuring in Google Analytics for Firebase, such as finishing your in-app tutorial.

Once you've defined your A/B test, Firebase takes over by delivering these different variations of your app to randomly-selected members of your audience. Firebase will then measure your users' behavior over time, and let you know when an experiment appears to be performing better, based on those goals you've defined earlier. Firebase A/B testing measures these results for you with the same Bayesian statistical models that power Google Optimize, Google's free testing and personalization product for websites.

Using A/B Tests for Better Onboarding: A Case Study

Fabulous, a motivational app for building better habits, recently made improvements to their app's onboarding by using Firebase A/B testing. When the user first starts an app, Fabulous shows them how to complete a habit, presents them with a letter about forming better habits, and then asks them to commit to a simple routine. The team suspected that if they removed a few steps from this onboarding process, more people might complete it.

Some of the screens a typical user encounters when first using Fabulous.

So they ran an A/B test where some users didn't see the letter, others didn't see the request to commit to a simple routine, and others skipped both of those steps. The Fabulous team found that by removing both of these steps from the onboarding process, there was a 7% improvement in the rate of users completing the onboarding flow. More importantly, they confirmed that this shorter onboarding experience didn't have any impact on their app's retention.

Test Your Notifications, Too!

You also have the ability to A/B test your app notification messages through the Firebase Notifications console. You can try out different versions of your notification message and see which ones lead to more users opening up your app from that notification, or which messages lead to users performing some intended goal within your app, like making a purchase.

Getting Started

A/B testing is available in beta to all Firebase developers starting today. If you're excited to get started, you should make sure that your app is wired up to use Remote Config and/or Firebase Cloud Messaging, and that you've updated these libraries to the latest and greatest versions. You can always find out more about A/B testing in our documentation, or check out the A/B Test Like a Pro video series we've been building.

Then, head on over to the Firebase Console and start making your app better — one experiment at a time!

Posted by Todd Kerpelman, Google Firebase team

Google Optimize now offers more precision and control for marketers

October 26, 2017

Savvy businesses review every step of the customer journey to ensure they are delivering the best experience and to find ways to offer more value. Today, we’re releasing two new features that will make it easier for you to improve each of those steps with the help of Google Optimize and Optimize 360.

AdWords integration: Find the best landing page

Marketers spend a lot of time optimizing their Search Ads to find the right message that brings the most customers to their site. But that's just half the equation: Sales also depend on what happens once people reach the site.

The Optimize and AdWords integration we announced in May gives marketers an easy way to change and test the landing pages related to their AdWords ads. This integration is now available in beta for anyone to try. If you’re already an Optimize user, just enable Google Optimize account linking in your AdWords account. (See the instructions in step 2 of our Help Center article.) Then you can create your first landing page test in minutes.

Suppose you want to improve your flower shop's sales for the keyword “holiday bouquets.” You might use the Optimize visual editor to create two different options for the hero spot on your landing page: a photo of a holiday dinner table centerpiece versus a banner reading "Save 20% on holiday bouquets." And then you can use Optimize to target your experiment to only show to users who visit your site after searching for “holiday bouquets.”

If the version with the photo performs better, you can test it with other AdWords keywords and campaigns, or try an alternate photo of guests arriving with a bouquet of flowers.

Objectives: More flexibility and control

Since we released Optimize and Optimize 360, users have been asking us for a way to set more Google Analytics metrics as experiment objectives. Previously,
Optimize users could only select the default experiment objectives built into Optimize (like page views, session duration, or bounces), or select a goal they had already created in Analytics.

With today's launch, Optimize users no longer need to pre-create a goal in Analytics, they can create the experiment objective right in Optimize:

Build the right objective for your experiment directly in the Optimize UI.

When users build their own objective directly in Optimize, we’ll automatically help them check to see if what they’ve set up is correct.

Plus, users can also set their Optimize experiment to track against things like Event Category or Page URL.

Learn more about Optimize experiment objectives here.

Why do these things matter?

It's always good to put more options and control into the hands of our users. A recent study showed that marketing leaders – those who significantly exceeded their top business goal in 2016 – are 1.5X as likely to say that their organizations currently have a clear understanding of their customers' journeys across channels and devices.¹ Testing and experimenting is one way to better understand and improve customer journeys, and that's what Optimize can help you do best.

>>> Check out these new features in Optimize now<<<

¹Econsultancy and Google, "The Customer Experience is Written in Data", May 2017, U.S.

Posted by Rotimi Iziduh and Mary Pishny, Product Managers, Google Optimize

Sigma Sport spins up 28% higher revenue with Google Optimize 360

August 9, 2017

If you’re a road cyclist or triathlete, chances are you know Sigma Sport. This global retailer sells bikes, clothing, energy bars, anti-chafe balm and everything else you need to power your way through your next big event — or just enjoy your next friendly ride in the country.

Recently Sigma Sport set out to address a vital need: to find more customers with high potential lifetime value. “Growth with high-value customers is key to our success,” says Nik Hill, the company’s Head of Digital. “We knew we needed to change our website experience to better engage these customers.”

To reach its goal, Sigma Sport turned to its agency, the digital conversion specialists Merkle | Periscopix. And together they turned to Google Optimize 360, part of the Google Analytics 360 Suite.

Using Optimize 360, Merkle | Periscopix created an experiment where they replaced Sigma Sport’s homepage carousel with brand-specific images of the site’s three top-performing brands: Castelli, Specialized, and Assos. Then they targeted the experiment to the audiences they had already built in Analytics.

This allowed Merkle | Periscopix to serve personalized experiences to fans of each brand. “We used the Analytics audience targeting feature in Optimize 360 to serve bespoke experiences to subsets of users,” says Shahina Meru, Associate Analytics Lead at Merkle | Periscopix. “We created three distinct Analytics audiences who had earlier bought or interacted with the top three brands, then used these as targeting rules in Optimize 360. Anyone who had looked at or bought a Specialized bike in the past, for instance, now saw Specialized products in their carousel.”

When Sigma Sport tested its new personalized home page, they saw right away that it was a hit with users. The experiment drove a 28% rise in revenue and a 32% increase in e-commerce conversion rate during the experiment. In fact, Sigma Sport saw uplift across the entire customer journey with a 90%+ probability to beat the baseline.

The bottom line: Personalization worked, both for bike-shopping customers and for Sigma Sport. Now Merkle | Periscopix is looking for more ways to enhance user experience with personalization from Analytics and Optimize 360.

>>Read the full case study here<<

Posted by Tiffany Siu, Product Marketing Manager, Google Optimize 360

Now Optimize users can innovate in 37 new languages

August 8, 2017

It just got a whole lot easier to share Google Optimize with your teams around the world.

Optimize is now available in 37 new languages. Got a team in Thailand? No trouble. Cross-functional partner in Croatia? You're covered. You'll find the full list of supported languages here.

We're always glad to bring our products to more of the world. But in this case, we're extra excited about the way this will help teams collaborate and innovate not just across the office but across the globe.

In this data-rich world, everyone in your company needs to be part of building a culture of growth: a culture that embraces testing and analytics as the best way to learn what customers like most and to improve their experience day by day. Optimize opens the door for innovators at every level to explore how even tiny UI changes can improve results.

Often those innovators take the form of a small "X-team" — maybe an analyst, a designer, and an engineer working together and totally focused on testing and optimization. With Optimize, a group like that can create changes in minutes instead of days, and they can more easily share that growth mindset and inspire others across their organization.

Now with 37 more languages in play, Optimize makes it possible for many more local teams to take on the role of optimizers, innovators, and culture-changers.

If you have team members who have selected one of the 37 new languages in their Google Account preferences, they'll see Optimize in that language whenever they next sign in. (If you’d like to select a language preference just for Optimize, you can do so in your Optimize user settings at any time.) And if you're happy with your current Optimize language, you're fine: No action is needed.

To learn more about your global language options, visit our help center. Happy optimizing!

Posted by Rotimi Iziduh, Product Manager, Google Optimize

Google Optimize and Surveys 360 Join Forces with AdWords

May 24, 2017

Meet two new ways to understand and better serve your customers

Here's good news for marketers: as you heard yesterday at Google Marketing Next both Optimize and Surveys 360 will soon be integrating with AdWords. The Surveys 360 integration is now live in the U.S. and Canada; the integration with Optimize will be available in the coming weeks.

Optimize is an A/B testing and personalization tool that makes it easy to see which changes to your web pages work best for your users and your business. Surveys 360 is a market research tool that helps enterprises gather fast, reliable insights from real people online and on mobile.

Why the new integrations? To make it easier than ever to understand and better serve your potential customers. Here's some detail on both.

Better landing pages, better results

Advertisers naturally spend a lot of time thinking about their ads. What gets people to click? Will the words "free shipping" sell more than "10% off"? AdWords has always made it easy to create many different ad campaigns to see which performs best. But the ad is only part of the experience.

The new integration between Optimize and AdWords makes it easier than ever to take the next step: to improve and personalize the landing pages those ads lead to. The integration gives marketers a fast way to create and test custom landing pages based on the keyword, ad group, or campaign associated with an ad – with no need to deal with destination URLs or messy query parameters.

It's worth it. 90% of organizations that invest in personalized consumer experiences agree that they contribute significantly to more business profitability. ¹

Suppose a hotel wants to improve its landing page for the keyword family friendly hotels. Using Optimize, the hotel can create and test a new variation of the landing page, one that features an image of a family enjoying themselves at the hotel pool, instead of a generic image of the hotel exterior. If the new page leads to more reservations, they've got a win. Then it's easy to keep testing headlines and images that might also do well.

Target any combination of your AdWords account, campaign, ad group, and keywords directly in Optimize.

The AdWords integration will be available for both Optimize and Optimize 360 and will be available to start using in the coming weeks. If you haven’t tried Optimize, you can get started for free here.

Why not ask your customers?

We all need faster insights these days. That's one reason we added Surveys 360 to the Google Analytics 360 Suite last year. Surveys 360 lets you ask questions directly to a pool of 15 million real people as they browse the web or use their smartphone. The results arrive in days, or sometimes in just hours.

Now, what if you could combine that kind of speedy real-world feedback with the wealth of data that you already have in AdWords? Then you could understand both what users do and why they do it.

That's what we're announcing today: remarketing lists published in AdWords are now available in Surveys 360 for surveys targeting. That means you can survey the users on your remarketing lists to find out what worked best for them (or didn't).

Want to know why shoppers abandoned their shopping carts? Ask them! Curious about how many customers converted due to your new free shipping offer? Ask them!

Easily target your survey to remarketing audiences published in AdWords.

Then change your marketing message on the spot to match what you learn. If your survey shows that the words "family friendly" are what brought customers to your hotel, you can build new ad groups to take advantage of this information. (You might even use Optimize to test new landing pages with that phrase!)

Here's an early report from the online shopping site Jet:

"Google Product Listing Ads (PLAs) have been an effective way for Jet.com to drive website traffic, but we needed to optimize for conversions. Surveys 360 connected us directly to our customers through remarketing audiences to determine which factors influence their purchase decisions most. The results were clear: customers care most about free, fast shipping and our free returns. We used this insight to revise our messaging in PLAs and across Jet.com which quickly improved performance."
–Ben Babcock, Director of UX Research at Jet.com

Getting started is easy: Just log into your Surveys 360 account with the same credentials used for your AdWords account. When you select "remarketing audience" for survey targeting, your AdWords remarketing lists will be automatically pulled into Surveys 360 and ready for use. Learn more.

All together now

These new integrations are one more way for Optimize and Surveys 360 users to make the most of their AdWords investments. We hope you'll find them a fast and simple way to understand what works for your customers and give them more of what they want.
¹Econsultancy and Google, Marketing and Measurement Survey, March 2017

Posted by Michael Cumberbatch, Product Manager, Surveys 360 and Rotimi Iziduh, Product Manager, Optimize

Making marketing easier with new tools for productivity

May 23, 2017

This post originally appeared on the Inside AdWords blog.

Mobile continues to raise the bar for marketers - as consumers increasingly reach for their smartphones to get things done, they expect fast and frictionless assistance. And to help them in these moments with more useful and personalized ads, you need scalable, easy-to-use tools that allow you to manage your marketing in smarter ways.

This morning at Google Marketing Next, we shared the next generation of AdWords, DoubleClick and Google Analytics tools, each of them shaped by your feedback and built to help you work as productively and efficiently as possible.

Below is a round-up!

The new AdWords experience will be available to all advertisers by the end of the year

The new AdWords experience marks the most powerful change we’ve made to how advertisers visualize and manage their campaigns in over 15 years. The redesign makes AdWords significantly easier to use to help you reach your unique marketing goals and get things done.

Earlier this year, we previewed a few of the new tools and features in the new AdWords experience. This includes the new Overview page, which automatically surfaces relevant insights about your performance so you can take action more quickly.

Here are some other new things to check out:

Campaign creation

Build campaigns designed around your marketing goals

Performance estimates

View performance estimates to help plan your display campaigns

Landing page report

See how your landing pages are affecting campaign performance

Since rolling out the new AdWords experience, we’ve seen some inspiring results. We see that pages load much faster -- on average, 20% faster. Customers like iProspect North also reported that the new design and layout saves them 30% more time when completing everyday tasks.

Starting today, we’re rolling out the new experience to millions of additional AdWords accounts and it will be available to all advertisers by the end of the year. Get started by reviewing our Best Practice guide.

Better together: Google Optimize and Google Surveys 360 integrate with AdWords

Your marketing workflows should be seamless and integrated, from managing media to optimizing your site and collecting performance insights. This allows you to easily apply learnings from one step to another and take action faster. With new integrations for Google Optimize and Google Surveys 360 with AdWords, you'll be able to test landing pages and gain insights about your ads more efficiently.

Available globally to all advertisers at no cost, Optimize helps marketers test and deliver custom site experiences to improve performance. With the Optimize and AdWords integration, you can quickly and easily create new versions of your landing pages and then apply them to any combination of AdWords campaigns, ad groups, and keywords – no coding or webmaster required. For example, on a summer beachwear campaign, test whether a landing page that features an image of beachgoers performs better than one that showcases your latest swimsuit styles. Optimize will figure out which variation works best for your business and help you get more value from your AdWords campaigns.

Surveys 360 makes it easy for anyone to create a survey, find a specific audience sample across the web, and generate results in just a few days. With the Surveys 360 and AdWords integration, you can easily target surveys to people in your remarketing audiences to help you understand the “why” behind how consumers react to your marketing. For example, ask customers what they thought was the most compelling part of your ad. Or, if you want to know what customers thought of the landing pages you tested on your summer beachwear campaign, quickly launch a survey and ask site visitors what stood out to them on the page.

Smarter planning in DoubleClick Bid Manager

The programmatic revolution has focused on combining data, machine learning and automation to drive better performance. However, few of these benefits have carried over to the world of media planning. To help save you time and improve performance, we’re bringing the power of machine learning to campaign planning with a new workflow in DoubleClick Bid Manager.

Bid Manager uses machine learning to analyze your past campaigns and understand what's performed well for you before. With these insights, it will recommend the right inventory and targeting to reach your audience based on your unique marketing goals. The result is an optimized media plan tailored to your business, which you can then tweak, approve and activate with one click.

“Planning in DoubleClick Bid Manager helps us find the right inventory and strategies in a matter of seconds. Now we can invest even more time in understanding our audience rather than how we should reach them.”- Louise Lévesque, Vice-President, Marketing & Sponsorship, National Bank of Canada

Unique Reach arrives for AdWords Display campaigns and DoubleClick

Once your ads are up and running, you want to know how many people are seeing them. Reach and frequency are hallmark metrics to measure the impact of display and video campaigns today. At the same time, you need to be thoughtful about how often you connect with a single user to avoid overloading them. 30% of people today use 5 or more devices¹ - that means if you reach someone three times on each screen, you could be showing them fifteen ads!

We launched Unique Reach for YouTube in AdWords earlier this year and are soon expanding availability to Display campaigns in AdWords. We’re also bringing Unique Reach to DoubleClick for all your video and display ads, no matter where they run. Unique Reach helps you measure the number of unique users and average impressions-per-user you get from video and display ads. Metrics are de-duplicated across devices, campaigns, inventory and formats so you can see how many people you reached and how often you reach them.

We can’t wait to get these new tools into your hands to hear what you think and help you work faster and smarter. To get the full replay of new innovations at Google Marketing Next, watch the keynote here. Until next year!

Posted by Paul Muret, Vice President of Display, Video and Analytics

1. Google / TNS Connected Consumer Survey, US population 18+, n=1000, Q1 2016

This is not a test: Google Optimize now free — for everyone

March 30, 2017

Businesses often have one big question for us: How can they better understand their website visitors and deliver more relevant, engaging experiences?

To help businesses test and take action, last spring we launched our enterprise-class A/B testing and personalization product, Google Optimize 360. We saw great demand, so we made it more accessible with a free beta version last fall — and that response also exceeded our expectations, with over 250,000 users requesting an Optimize account.

Today we're very excited to announce that both Optimize and Optimize 360 are now out of beta. And Optimize is now immediately available to everyone — for free. This is not a test: You can start using it today.

Easy to implement

A recent survey showed 45% of small and medium businesses don’t optimize their websites through A/B testing.¹ The two most common reasons given were a "lack of employee resources" and "lack of knowledge to get started."

If you're part of that 45%, Optimize is a great choice for you. Optimize has many of the same features as Optimize 360. It's just right for small and medium-sized businesses who need powerful testing, but don't have the budget or team resources for an enterprise-level solution. Optimize is easy for anyone to set up. Early users of Optimize have been happy with how easy it is to use. In fact, it's built right on top of Analytics, so if you're already an Analytics user you'll add just a single line of code to get Optimize up and running. With just a few clicks more, you can start using your Analytics data to design experiments and improve the online experience for your users.

Easy to use

Worried about having to hire someone to run A/B tests on your site, or frustrated about not knowing how to do it yourself? Don't be. The Optimize visual editor allows for WYSIWYG (what-you-see-is-what-you-get) editing so you can change just about anything on your site with a drag and a drop. And more advanced users will enjoy the ability to edit raw HTML or add JavaScript or CSS rules directly in the editor.

Powerful targeting capabilities within Optimize allow you to serve the right experiences to just the right set of users. And you have flexible URL targeting capabilities to create simple or complex rules for the pages where you want your experiment to run. To find out if a targeting rule you've set will apply to a specific URL on your site, use the new Optimize URL tester. Just enter a URL and the tester will immediately tell you if that page is a match for your targeting rule.

Easy to understand

Optimize calculates results based on your existing Analytics metrics and objectives using advanced Bayesian methods, so the reporting shows you exactly what you need to know to make better and faster decisions.

We’ve also upgraded the improvement overview (see image above) to help you quickly see how an experiment affects the metrics you care about most, whether that means purchases, pageviews, session lengths, or whatever else you’re tracking in Analytics.

Easy to try

Leading businesses are building a culture of growth that embraces the use of data and testing to improve the customer experience every day. We’re delighted to offer Optimize to everyone to help deliver better user experiences across the board.

As of today, Optimize is available in over 180 countries. (A special note for our European users: We’ve added a new data processing amendment to the Google Optimize Terms of Service that you may review in the UI and accept if you wish.) And we're not done yet: Keep an eye out for more improvements and announcements in the future.

What are you waiting for? Try it right now!

Happy Optimizing!

¹Google Surveys, "Website Optimization Challenges for SMBs," Base: 506 Small/Medium Business Owners and Managers, Google Surveys Audience Panel, U.S., March 2017

Posted by Rotimi Iziduh and Jon Mesh, Product Managers, Google Optimize

Lessons Learned: Testing and Optimization Tales from the Field

March 14, 2017

Max van der Heijden is a user experience and conversion specialist at Google who works with companies across Europe, the Middle East, and Africa. Max shares his thoughts about how companies can build a culture of growth and experimentation.

How many times have you launched new features or page designs on your website without testing first?

In an ideal world, companies should test everything before rolling out site changes. But some websites have too little traffic to generate credible results from experiments, and some bugs should just be fixed if they prevent users from achieving their goal. At the very least, analyze your analytics data and use qualitative methods such as user tests and surveys to validate any improvement ideas you have before implementing. If you have the traffic volume: Test!

I’m part of a team at Google that works with advertisers to identify opportunities for improving website user experiences through experiments and testing roadmaps. When our team of UX specialists begins consulting with a new company, the first three things I tell them are:

The possibilities for improvement are enormous. Even if an experiment increases your conversion rate by “just 5%,” you can calculate the positive effect on your revenue.
What works for one may not work for all. No matter how many times we have seen recommendations or “best practices” work on other — maybe even similar — websites, that does not mean it will work for your users or your business.
Expect failures — and learn from them. Testing takes time, and it's hard to know which tests will pay off. Embrace failures and the lessons learned from them.

Making the switch from “get-it-live” mode to a test-and-learn mindset takes time and effort. Leading companies are building a culture of growth: one where people focus on using data and testing to optimize the customer experience day by day. Below are some of the key lessons learned as we work with teams embracing this growth mindset.

Get top-level support

When we first talk with customers, we insist a decision-maker attend our meetings. If there's no support from the top, all of our testing ideas could end up on the shelf collecting dust. Obviously, the marketing executive or CEO won’t have an a-ha moment if you frame testing as a way to improve conversions. The trick is to show how testing impacts a business goal, such as revenue or, better yet, profit. Then the decision-maker will have an ohhh moment: As in, “Ohhh, I knew this was important, but I didn’t think about how a small change could have such a big impact on our bottom line.”

Top-level support will help you get the resources you need and unlock the potential of people already working on experiments. The trend we see is typically one or two persons who start doing the optimizations. They are usually mid-level designers or data analysts who have an affinity for conversion rate optimization, but are often working in a silo.

On the other end of the spectrum, we see companies that have fully bought into the power of experimentation. Multiple customers even have a group of product managers who work on projects with a group of specialists, including a data scientist, copywriter, designer, and even a design psychologist.

Tip: Look for these three types of people to jumpstart a culture of growth in your organization.

Prioritize, prioritize, prioritize

You can't test every idea at once. And prioritization should not be a guessing game.

When we surveyed a group of our EMEA advertisers at a conversion rate optimization event, 38% of the respondents said they use their gut or instinct to prioritize, while 14% allow the HiPPO (highest paid person’s opinion) to call the shots.¹ Instead, try using a framework that takes into account past lessons learned and resource requirements.

Map test ideas in a speed-versus-impact grid, and prioritize experiments that are quick to launch and likely to have the biggest impact. Keeping track of all prior test results is another way to ensure past learnings come into play when faced with a HiPPO.

Tip: Start with ideas that will be simple to test and look like they could have the biggest potential impact.

Turn fairweather fans into engaged experimenters

Over time, as you share testing insights and achieve a few wins, more people will jump on board and you’ll need to train people on a repeatable testing framework.

Testing is part of a cycle: What does the data tell you? Did the experiment succeed or fail for every user, or just for a specific segment of users? Analyze your test results, especially from failed experiments, and use those insights to improve the customer experience across your touchpoints. And then conduct another test.

Just as important: How do you keep people excited and engaged in the process? Try using a shared document to invite everyone to submit their improvement suggestions for your website or app. You can even add gamification to this by keeping score of the most impactful ideas. Or, have people guess which test variation will win before you run the test. When you share the results, recognize or reward people who correctly predicted the winner.

Tip: Three ways to get your team engaged with testing and optimization

Feel good about failures

By its very nature, experimentation involves a lot of failure. A typical website might have 10 or 100 or even 1,000 things to test, but it might be that only a small percentage of those tests lead to significant, positive results. Of course, if that one winner leads to a 5% or 10% improvement in conversions, the impact on revenue can be enormous.

When we surveyed EMEA advertisers at one of our events, we found that companies running one to two tests a month had a 51% success rate. But for respondents who said they ran more than 21 tests a month, the success rate decreased to 17%.²

In the beginning, it’s easier to spot areas for improvement and “low-hanging fruit.” The more experiments you run, the more you’ll be focusing on smaller and smaller things. Then, the more you test, the less “successful” you will be. "Our test success rate is about 10%," says Jesse Nichols, Head of Growth at Nest. "But we learn something from all our tests."

Download the guide How to Build a Culture of Growth to learn more about best practices for testing and optimization.

^1-2 Source: Conversions@Google 2016 - State of CRO event attendee survey, 145 respondents, EMEA, September 2016.

Posted by Max van der Heijden, user experience and conversion specialist at Google, EMEA

What does a good website test look like? The essential elements of testing

February 9, 2017

"Test! Test! Test!" We've all heard this advice for building a better website. Testing is the heart of creating a culture of growth ― a culture where everyone on your team is ready to gather and act on data to make the customer experience better day by day.

But how do you run a good test? Is it just a matter of finding something you're not sure about and switching it around, like changing a blue "Buy now" button for a red one? It depends: Did you decide to test that button based on analytics, or was it a wild guess?

Assuming the former, a good test also means that even if it fails, you’ve still learned something. A bad test may make your website performance worse than before, but it’s even worse if you don’t take those learnings into account in the future.

The key to running good tests is to establish a testing framework that fits your company.

Watch our on-demand webinar to hear Krista Seiden, Google Analytics Advocate, and Jesse Nichols, Head of Growth at Nest, share a six-step framework for testing and building better websites.

Frameworks vary from business to business, but most include three key ideas:

Start with an insight and a hypothesis.
A random "I wonder what would happen if …" is not a great start for a successful test. A better way to start is by reviewing your data. Look for things that stand out: things that are working unusually well or unusually badly.

Once you have an insight in hand, develop a hypothesis about it: Why is that element performing so well (or so badly)? What is the experience of users as they encounter it? If it's good, how might you replicate it elsewhere? If it's bad, how might you improve it? This hypothesis is the starting point of your test.

For example, if you notice that your mobile conversion rate was less than on desktop, you might run tests to help you improve the mobile shopping or checkout experience. The team at The Motley Fool found that email campaigns were successfully driving visitors to the newsletter order page, but they weren’t seeing the conversions. That led them to experiment on how to streamline the user experience.

Come up with a lot of small ideas.
Think about all the ways you could test your hypothesis. Be small-c creative: You don't have to re-invent the call-to-action button, for instance, but you should be willing to test some new ideas that are bold or unusual. Switching your call-to-action text from "Sign up now" to "Sign up today" may be worth testing, but experimenting with "Give us a try" may give you a broader perspective.

When in doubt, keep it simple. It's better to start with lots of small incremental tests, not a few massive changes. You'll be surprised how much difference one small tweak can make. (Get inspiration for your experiments here.)

Go for simple and powerful.
You can't test every idea at once. So start with the hypotheses that will be easy to test and make the biggest potential impact. It may take less time and fewer resources to start by testing one CTA button to show incremental improvement in conversion rates. Or, you may consider taking more time to test a new page design.

It may help to think in terms of a speed-versus-impact grid like this. You don't want quiet turtles; the items you're looking for are those potential noisy rabbits.

The best place to begin a rabbit hunt is close to the end of your user flow. "Start testing near the conversion point if you can," says Jesse Nichols, Head of Growth at Nest. “The further you go from the conversion point, the harder it gets to have a test that really rocks — where the ripple effect can carry all the way through to impact the conversion rate,” says Jesse.

Stick with it
A final key: Test in a regular and repeatable way. Establish an approach and use it every time, so you can make apples-to-apples comparisons of results and learn as you go.

A clear and sturdy framework like this will go a long way toward making your team comfortable with testing — and keeping them on the right track as they do.

Download the eBook How to Build a Culture of Growth to learn more about best practices for testing and optimization.

Posted by Tiffany Siu, Product Marketing Manager, Google Optimize 360

Why Building a Culture of Optimization Improves the Customer Experience

November 8, 2016

How can we be more useful to our customers today?

That's the simple question that drives any marketing organization focused on testing, improvement, and growth.

But answering the question is not always so simple in our data-rich world. The old challenge of gathering enough data has been replaced by a new one: gleaning insights from the mountains of data we’ve collected — and taking action.

In response to this flood of data, many of today's most successful businesses have turned to a new approach: building what's called a culture of growth and optimization.

This growth-minded culture is one where everyone is ready to:

Test everything
Value data over opinion
Keep testing and learning, even from failures

Most companies have a few people who are optimizers by nature, interest, or experience. Some may even have a “growth team.” But what really moves the dial is when everyone in the company is on board and embraces the importance of testing, measuring, and improving the customer experience across all touchpoints.

"We refuse to believe that our customers’ experiences should be limited by our resources." - Andrew Duffle, Director of Analytics, APMEX

Why should marketers care?
Because they'll be leading the revolution. 86% of CMOs and senior marketing executives believe they will own the end-to-end customer experience by 2020, according to a recent survey from the Economist Intelligence Unit.¹ And a culture of growth and optimization offers an excellent path to major gains in those experiences.

As testing and optimization proves itself, it tends to generate higher-level investments of support, talent, and resources. The payoff arrives in the form of more visitors, more sales, happier customers and a healthier bottom line.

If you're curious about building a culture of optimization in your marketing organization, register for our Nov. 10 webinar, Get Better Every Day: Build a Marketing Culture of Testing and Optimization.

This webinar will cover:

The critical elements of a culture of optimization
Tips for building that culture in your own company
A case study discussion with Andrew Duffle, Director of Analytics at APMEX, a retailer that boosted revenues with continuous testing and optimization

This kind of culture doesn't happen by command, but it’s also simple to start building.

We look forward to sharing tips on how you can get started. Happy optimizing!

The Economist Intelligence Unit, "The Path to 2020: Marketers Seize the Customer Experience." Survey and a series of in-depth interviews with senior executives. Survey base: 499 CMOs and senior marketing executives, global, 2016.

Posted by Jon Mesh, Product Manager, Google Optimize and Google Optimize 360

How to Apply Holiday Shopping Insights to Your Analytics Strategies

October 24, 2016

The Year of the Supershopper
We all have that friend — the one who somehow knows the latest brands, the season’s must-have products, and where to find the best deals at the snap of a finger. In years past, this friend was an enigma, making us wonder how does he or she do it?

Today, we can all be that friend. With the ability to instantly discover, research, and purchase, shoppers around the world are more informed and more efficient than ever before - they’ve transformed into supershoppers seemingly overnight.

But what defines supershoppers? And what does this mean for retailers trying to win them over this holiday season? Let’s find out.

They Keep Their Options Open
Last year, more than 50% of holiday shoppers said they were open to purchasing from new retailers¹. This is especially true online. More than three-quarters of smartphone shoppers who usually go to the same physical stores when they shop for products are very open to new retailers and brands online². Why? Mobile makes it easy to explore all of your options no matter when or where you’re shopping. In fact, after searching on Google, 76% of mobile shoppers have changed their mind about which retailer or brand to purchase³.

Mobile is Their Muse
It used to be that shoppers would thumb through catalogues or stare longingly at the holiday window displays, but mobile is now the super shopper’s go-to source for inspiration. Sixty-four percent of smartphone shoppers turn to mobile search for ideas about what to buy before heading into a store⁴. And 1 in 4 mobile video viewers in the U.S. have visited YouTube for help with a purchase decision while they were at a store or visiting a store's website⁵.

But shoppers aren’t only making purchase decisions, they’re discovering new brands and products along the way: more than half of smartphone users have discovered a new company or product when conducting a search on their phones⁶.

They Want the Best - Not the Cheapest
In July we learned that shoppers are on the quest for the best — and this still rings true more than ever today. Last holiday, mobile searches related to “best gift” grew 70% year over year while mobile searches related to cheap or inexpensive gifts grew about 35%⁷. They’re also willing to do the research to the make the best decision: on YouTube, mobile watch time for product review videos has grown 60% year over year⁸.

But supershoppers don’t only want the best - they want personalized, unique, cool gifts. Mobile searches related to “unique gifts” grew more than 65% while mobile searches related to “cool gifts” grew a whopping 80%⁹.

Mobile is Their Door to the Store
Although more and more people are willing to buy on mobile, we know that mobile is still used predominantly as a door to the store. In fact, 76% of people who search for something nearby on their smartphone visit a related business within a day, and 28% of those searches result in a purchase¹⁰.

Once they’re inside your store, they expect the experience to be a seamless one: more than 40% of smartphone shoppers want retailers to automatically surface relevant information such as the location of the item in the store, a special deal or related products¹¹.

They Shop ‘Til they Drop
Supershoppers live up to their name as the holiday season progresses. From November through mid-December we see online conversion rates increase across devices. Last year, on mobile alone, they jumped 30% on Black Friday and 50% on Cyber Monday when compared to November 1¹².

Tips to Using Analytics to Reach Today’s Shopper
Analytics is a critical marketing tool all year long. But that importance is amplified during the holiday season. Here are five ideas on how you can use analytics to get the most out of the holiday shopping season.

1. Understand which days drive the most sales — be there. In order to develop a successful holiday retail strategy, it’s important to first understand the days that drive the most sales for your business. Once you understand this, you can craft strategy — across mobile, desktop, and tablet — that optimizes your media and promotion not only for these days, but for the entire holiday season.
For example, adjust bids for auction-based media and use the lift in transaction rates over the average transaction rate as your bid multiplier. Use Smart Bidding in AdWords or DoubleClick Search to maximize your performance.

2. Get personal when engaging with window shoppers. The great thing about the holidays is that you can start shopping at any time — and then wait for discounts. Many consumers may already be browsing your site looking for gift ideas. This is an audience that may convert at a higher rate. Use Google Analytics to create remarketing lists including these shoppers, and then customize your marketing campaigns to better suit their needs.

But don’t stop there: Remember, supershoppers crave personalization. Test different variations of your website with this same audience to offer more customized experiences. Try Google Optimize, our free site testing and personalization tool, to get started.

3. Act fast: Create a culture of optimization. Your holiday marketing plans are set long before the holiday season starts. It’s very important to monitor your activities to ensure they’re driving business growth. And if they’re not you need to take action by immediately identifying areas of opportunity and improvement.

Google Analytics offers a number of ways to help simplify reporting, sift through data, and spotlight insights for you. Custom reports, dashboards and shortcuts all let you customize reporting so you spend less time looking for data and more time analyzing important information. Or better yet, try Google Data Studio and create a holiday shopping dashboard for different teams in your organization.

4. Analyze your performance against your competitors. Benchmarking your business against your competitors helps you see the big picture. If you’re a Google Analytics user, you can tailor this analysis and approach, using your own data and the data available in our benchmarking reports. The benefit of using those reports is that they will provide you a comparison between your data and your competitors, using characteristics such as website traffic, country and detailed vertical information.

5. Start developing your New Year marketing strategy. With so many new shoppers entering your stores, sites, and apps, data and analytics are critical to helping you convert first-time shoppers into long-term, loyal customers. Use your website data to create lists of first-time customers during the holidays, and deliver personalized communication across channels to build relationships. With Google Analytics you can create remarketing lists and easily connect with this audience.

Happy analyzing!

Posted by Julie Krueger, Retail Managing Director, Google and Casey Carey, Director of Analytics Marketing, Google

Google/ Ipsos, Post Holiday Shopping Intentions Study - Total Shoppers Report, Jan 2016, Base: US Holiday Shoppers, n=1,500

Google/Euromonitor International, Micro-Moments Survey, US, July 2016, Smartphone shoppers = 1000, Same store shoppers = 801

Google/Euromonitor International, Micro-Moments Survey, US, July 2016, Smartphone shoppers = 1000

Google-commissioned Ipsos Brand Building on Mobile Survey, U.S., December 2015 n=1000, 18-54 year olds

Google/Ipsos, "Consumers in the Micro-Moment," Wave 3, U.S., n=1291 online smartphone users 18+, August 2015

Source: Google Search Data, Apparel, Home & garden, Beauty & personal care, Computers & electronics, Gifts, Toys & games, Photo & video, Nov-Dec 2014 vs Nov-Dec 2015, United States

YouTube data, U.S., Classification as a “Product Review" video was based on public data such as headlines and tags, and may not account for every such video available on YouTube, Nov - Dec 2014 and Nov - Dec 2015

Google Search Data, Apparel, Home & garden, Beauty & personal care, Computers & electronics, Gifts, Toys & games, Photo & video, Nov-Dec 2014 vs Nov-Dec 2015, United States

Google/Purchased Digital Diary: How Consumers Solve Their Needs in the Moment, May 2016, Representative sample of US Smartphone users = 1000. Local searchers = 634, Purchases = 1,140

Google/Euromonitor International, Micro-Moments Survey, US, July 2016, Smartphone shoppers = 1000

Google Analytics Shopping category data, Nov 1, 2015–December 14, 2015, United States

An Easier Way for Everyone to Improve the Customer Experience: Meet Google Optimize

September 29, 2016

Nearly 9 in 10 business leaders in a recent survey said that improving the customer experience is fundamental to their future success and brand reputation.¹

Yet less than half of business leaders say they have actually taken action around their customer experience activities.²

Today, consumers have higher expectations for businesses to deliver more personalized site experiences based on the information they’ve shared. Businesses need to be able to quickly discover useful insights from their data and act on what they’ve learned.

To help those leaders (and everyone else) take action to deliver more personalized site experiences, we're happy to introduce Google Optimize. It's a free version of our popular enterprise-class testing and personalization product, Google Optimize 360, which was announced earlier this year.

Google Optimize will start rolling out globally next month. If you’d like to be one of the first to use it, visit our signup page. We'll send you an invitation by email as soon as it is available.

Making it easy …

Google Optimize helps any business test and deliver better website experiences. And to help guide you through the process, we’ve made it easy every step of the way:

It’s easy to implement. Test new and engaging web experiences across your entire site in minutes. Google Optimize is built on top of Google Analytics and Google Tag Manager so you can either add a single line of code to your existing Google Analytics implementation, or serve the Google Optimize tag directly using Google Tag Manager.
It's easy to use. Use the Google Optimize WYSIWYG (What-You-See-Is-What-You-Get) editor, to change just about anything on your site – text, images, layouts, and more – just click to edit! There is no need to recode your site each time you want to test a new experience.
It's easy to understand and act on results. Google Optimize calculates results based on your existing Google Analytics metrics and business objectives. It’s clear to see what you should do next to drive even more impact for your business.

… And powerful

With Google Optimize you can create and deploy a variety of experiment types: Choose from A/B, multivariate, or redirect. For any experiment, Google Optimize offers powerful capabilities:

Native integration with Google Analytics. Test what matters to your business. Because Google Optimize is built on Google Analytics, you can more quickly and easily identify areas of your site that can be improved upon. Then in Google Optimize you can use your existing Google Analytics goals and metrics as your experiment objective.
Advanced statistical modeling. Google Optimize uses Bayesian statistical methods to model the real-world performance of your experiments. We’ll show you how much more effective one variant is over the others, leading to more accurate results that you can trust.
Sophisticated targeting tools. Google Optimize will help you deliver the right experiences to the right customers at the right moments.

Ready to deliver better site experiences?

With Google Optimize, you’ll have all the basic capabilities you need to get started with site testing – making it perfect for any business. Google Optimize is free for anyone to use, so why not give it a try? Visit our signup page today.

And if you are part of a larger enterprise or business with more sophisticated testing and support needs, check out Google Optimize 360, part of the Google Analytics 360 Suite. With Google Optimize 360, you’ll be able to take your site experiments even further by delivering custom experiences to your Google Analytics audiences. You’ll also be able to run more experiments across your site at the same time and add additional experiment objectives — even after an experiment has started. Talk to a sales representative to learn more.

Happy testing!

¹"Two Years' Warning: The Customer Centricity Crisis." The Storytellers, March 15, 2016. Quoted on eMarketer.com.
²"Data Elevates the Customer Experience." Forbes Insights / SAS, May 2, 2016. Quoted on eMarketer.com.

Posted by Jon Mesh and Dan Cary, Product Managers, Google Optimize

Announcing: Smart, Fast-Acting Analytics — For Everyone

September 28, 2016

This past March, we launched the Google Analytics 360 Suite, a set of integrated data and marketing analytics products designed for the needs of enterprise-class marketers in today’s multi-screen world.

Today we have another exciting announcement. Next month we’ll start rolling out a new product: Google Optimize — a free version of our enterprise-class testing and personalization product, Google Optimize 360. Google Optimize will be globally available at that time. Additionally, we’re including enhancements to many of our existing free products. Read on for details.

Businesses of all sizes need tools to measure and succeed

First of all, why this free product now? Because companies big and small face the same challenge today: Consumer expectations are higher than ever in a mobile-first world. How can brands be more useful to people in their moments of need? With ever more data flooding in, marketers need access to the right data so they can uncover useful insights — and act on them quickly.

The companies that are already making this leap from data to action are transforming their businesses by becoming more useful to consumers throughout these micro-moments. How? Analytics and testing are the key. In fact, according to an Econsultancy survey of 4,000 marketers, the top two digital trends in 2020 are going to be a focus on customer experience (24%) and personalization (23%).

That’s what has led to today's announcement. We've made several advancements to the free versions of our analytics products to bring modern measurement to everyone, and help them create happier customers by providing more relevant experiences. Let’s take a closer look.

How can every business deliver personalized customer experiences?
Google Optimize (beta). This free web and mobile-web testing and personalization tool helps businesses improve their customer experiences and business metrics. Because it’s built on top of Google Analytics, businesses can use their existing information to experiment and personalize site experiences with minimal setup requirements. Request an invite.

"With Google Optimize, we’ve been able to provide more engaging content to The Next Web's readers – converting them from new readers to loyal customers."

Martijn Scheijbeler

Director of Marketing - The Next Web

How can everyone access and share powerful data?
Google Data Studio (beta), our free reporting and data visualization product, is now available globally in 21 new countries.* It enables businesses to easily spot and share insights, and collaborate to drive better decisions. Additionally, next month we’re launching report templates, making it even easier to get started. As with templates in Google Docs, Slides, and Sheets, businesses can create, edit, and use a library of templates to get up and running in a matter of minutes. Get started. The enterprise and free versions of Google Data Studio launched earlier this year in the U.S. These updates are also available in Data Studio 360, part of the Google Analytics 360 Suite.

How can everyone get useful insights, not just more data?
Google Analytics, our free customer analytics product and measurement centerpiece, analyzes customer data from all touchpoints — and soon it’s becoming even smarter. Following the launch of our automated insight stream in the Google Analytics mobile app, we’re excited to announce Session Quality Score: A new metric that incorporates machine learning to predict the likelihood of a visitor making a transaction (purchase) on your site or app. Businesses can use session quality score to provide better customer experiences and / or remarket to their most engaged website visitors. This feature is coming soon in beta and will also be available in Analytics 360, part of the Google Analytics 360 Suite.

How can everyone gain full visibility into customer journeys?
Google Tag Manager empowers marketers to move faster and make decisions with confidence. It offers a simplified way to gather site information (all those tiny bits of code), and powerful APIs to increase data accuracy and streamline workflows. And today we’re greatly expanding our openness by integrating an additional 20 tags from Quantcast, Twitter, Microsoft Bing, Nielsen, and many more. See the full list of supported tags here. These updates will roll out over the next couple weeks and are also available in Tag Manager 360, part of the Google Analytics 360 Suite.

Modern measurement for everyone

Put simply, that's our goal with these new free products. As today’s businesses are shifting to compete on customer experience and personalized marketing, we want to give all businesses the tools and access to compete — and ultimately, drive better online consumer experiences.

If you’re interested in learning more, please watch our blog this week and next. We’ll dive deeper into each product announcement and show you how they work. In the meantime, check out the resources below to learn more.

Optimize: Request an invite
Data Studio: Get started for free
Session Quality: Help Center update coming soon!
Tag Manager: Visit the Help Center

As always, you can visit our website to sign-up for our free analytics products.

*Canada, Brazil, Germany, Netherlands, Spain, France, United Kingdom, Italy, Indonesia, India, Japan, South Korea, Australia, Hong Kong, Malaysia, New Zealand, Philippines, Singapore, Thailand, Taiwan, Vietnam

Posted by Babak Pahlavan, Senior Director of Product Management, Analytics Solutions and Measurement, Google

Google Analytics 360 + Salesforce.

Better A/B Testing with Firebase

Google Optimize now offers more precision and control for marketers

Sigma Sport spins up 28% higher revenue with Google Optimize 360

Now Optimize users can innovate in 37 new languages

Google Optimize and Surveys 360 Join Forces with AdWords

Making marketing easier with new tools for productivity

This is not a test: Google Optimize now free — for everyone

Lessons Learned: Testing and Optimization Tales from the Field

What does a good website test look like? The essential elements of testing

Why Building a Culture of Optimization Improves the Customer Experience

How to Apply Holiday Shopping Insights to Your Analytics Strategies

An Easier Way for Everyone to Improve the Customer Experience: Meet Google Optimize

Announcing: Smart, Fast-Acting Analytics — For Everyone

Labels

Archive

Feed

Learning Center

Company-wide

Products

Developers