New SEO Experiments: A/B Split Testing Google’s UGC Attribute
Posted by Cyrus-Shepard
When Craig Bradford of Distilled reached out and asked if we'd like to run some SEO experiments on Moz using DistilledODN, our reply was an immediate "Yes please!"
If you're not familiar with DistilledODN, it's a sophisticated platform that allows you to do a number of cool things in the SEO space:
- Make almost any change to your website through the ODN dashboard. Since the ODN is a cloud platform that sits in front of your website (like a CDN) it doesn't matter how your website is built or what CMS it uses. You can change a single page — or more likely — entire sections.
- The ODN allows you to A/B split test these changes and both measure and predict their impact on organic traffic. They also have a feature called full-funnel testing allowing you to measure impact on both SEO and CRO at the same time.
When you find something that works, you see a positive result like this:
SEO experimentation is great, but almost nobody does it right because it's impossible to control for other factors. Yes, you updated your title tags, but did Google roll out an update today? Sure, you sped up your site, but did a bunch of spam just link to you?
A/B split testing solves this problem by applying your changes to only a portion of your pages — typically 50% — and measuring the difference between the two groups. Fortunately, the ODN can deploy these changes near-instantly, up to thousands of pages at a time.
It then crunches the numbers and tells you what's working, or not.
Testing Google's UGC link attribute
For our first test, we decided to tackle something simple and fast. Craig suggested looking at Google's new link attributes, and we were off!
To summarize: Google recently introduced new link attributes for webmasters/SEOs to label links. Those attributes are:
- rel="sponsored" - For paid and sponsored links
- rel="ugc" - For links in user-generated content (UGC)
- rel="nofollow" - Remains a catch-all for all followed links
On the Moz blog, all comments links are currently marked "nofollow" — following years of SEO best practices. Google has stated that using the new attributes won't give you a rankings boost. That said, we wanted to test for ourselves if changing these links to "ugc" would impact the rankings/traffic of our blog pages.
To be clear: We are not testing if the pages we link to change rankings, but instead the source page that hosts the link — in this case, the blog pages with comments.
Here's an example of a comment the ODN modified.
After we set the test running, 50% of blog posts had comments with "ugc" links, while 50% kept their original "nofollow" attributes.
Experiment results
We expected a "null" test — meaning we wouldn't see a significant impact.
In fact, that's exactly what happened.
If we detected a significant change, the probability cone at the bottom right would have pointed more dramatically up or down.
In fact, at a 95% confidence interval, the test predicted traffic would either fall 26,000 visits/month or gain 9,300 visits/month.
Hence, a null result.
This validates Google's statements that using the "ugc" attribute won't give you a ranking boost.
What should Moz test next?
While "null" tests aren't as fun as a positive result, we have a lot of cool A/B SEO testing ahead of us.
The great thing is we can now test out changes with the ODN, and when we find one that works, pass that to our developers to make the changes permanently. This cuts down on needless development work and stops the guessing game.
We have a Trello board set up for test ideas, and we'd love to add some community ideas to the mix. The ODN is currently running on the Moz Blog and Q&A, so anything in these site sections is fair game.
We're also looking at experiments where we use Moz data to inform these decisions. For example, a Moz Pro crawl identified that the Moz Blog titles currently use H2 tags instead of H1. Google recently indicated this likely shouldn't impact rankings, but wouldn't it be good to test?
What wild/clever/ridiculous/obvious SEO things should we test? With each good test, we'll publish the results. Leave your ideas in the comments below.
Big thanks to the Distilled Team, including Will Critchlow and Tom Anthony, for embarking on this journey with us.
And if you'd like to learn more about DistilledODN and SEO split testing in general, this post is highly recommended.
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
Categories
- 60% of the time… (1)
- A/B Testing (2)
- Ad placements (3)
- adops (4)
- adops vs sales (5)
- AdParlor 101 (43)
- adx (1)
- algorithm (1)
- Analysis (9)
- Apple (1)
- Audience (1)
- Augmented Reality (1)
- authenticity (1)
- Automation (1)
- Back to School (1)
- best practices (2)
- brand voice (1)
- branding (1)
- Build a Blog Community (12)
- Case Study (3)
- celebrate women (1)
- certification (1)
- Collections (1)
- Community (1)
- Conference News (1)
- conferences (1)
- content (1)
- content curation (1)
- content marketing (1)
- contests (1)
- Conversion Lift Test (1)
- Conversion testing (1)
- cost control (2)
- Creative (6)
- crisis (1)
- Curation (1)
- Custom Audience Targeting (4)
- Digital Advertising (2)
- Digital Marketing (6)
- DPA (1)
- Dynamic Ad Creative (1)
- dynamic product ads (1)
- E-Commerce (1)
- eCommerce (2)
- Ecosystem (1)
- email marketing (3)
- employee advocacy program (1)
- employee advocates (1)
- engineers (1)
- event marketing (1)
- event marketing strategy (1)
- events (1)
- Experiments (21)
- F8 (2)
- Facebook (64)
- Facebook Ad Split Testing (1)
- facebook ads (18)
- Facebook Ads How To (1)
- Facebook Advertising (30)
- Facebook Audience Network (1)
- Facebook Creative Platform Partners (1)
- facebook marketing (1)
- Facebook Marketing Partners (2)
- Facebook Optimizations (1)
- Facebook Posts (1)
- facebook stories (1)
- Facebook Updates (2)
- Facebook Video Ads (1)
- Facebook Watch (1)
- fbf (11)
- first impression takeover (5)
- fito (5)
- Fluent (1)
- Get Started With Wix Blog (1)
- Google (9)
- Google Ad Products (5)
- Google Analytics (1)
- Guest Post (1)
- Guides (32)
- Halloween (1)
- holiday marketing (1)
- Holiday Season Advertising (7)
- Holiday Shopping Season (4)
- Holiday Video Ads (1)
- holidays (4)
- Hootsuite How-To (3)
- Hootsuite Life (1)
- how to (5)
- How to get Instagram followers (1)
- How to get more Instagram followers (1)
- i don't understand a single thing he is or has been saying (1)
- if you need any proof that we're all just making it up (2)
- Incrementality (1)
- influencer marketing (1)
- Infographic (1)
- Instagram (39)
- Instagram Ads (11)
- Instagram advertising (8)
- Instagram best practices (1)
- Instagram followers (1)
- Instagram Partner (1)
- Instagram Stories (2)
- Instagram tips (1)
- Instagram Video Ads (2)
- invite (1)
- Landing Page (1)
- link shorteners (1)
- LinkedIn (22)
- LinkedIn Ads (2)
- LinkedIn Advertising (2)
- LinkedIn Stats (1)
- LinkedIn Targeting (5)
- Linkedin Usage (1)
- List (1)
- listening (2)
- Lists (3)
- Livestreaming (1)
- look no further than the new yorker store (2)
- lunch (1)
- Mac (1)
- macOS (1)
- Marketing to Millennials (2)
- mental health (1)
- metaverse (1)
- Mobile App Marketing (3)
- Monetizing Pinterest (2)
- Monetizing Social Media (2)
- Monthly Updates (10)
- Mothers Day (1)
- movies for social media managers (1)
- new releases (11)
- News (72)
- News & Events (13)
- no one knows what they're doing (2)
- OnlineShopping (2)
- or ari paparo (1)
- owly shortener (1)
- Paid Media (2)
- People-Based Marketing (3)
- performance marketing (5)
- Pinterest (34)
- Pinterest Ads (11)
- Pinterest Advertising (8)
- Pinterest how to (1)
- Pinterest Tag helper (5)
- Pinterest Targeting (6)
- platform health (1)
- Platform Updates (8)
- Press Release (2)
- product catalog (1)
- Productivity (10)
- Programmatic (3)
- quick work (1)
- Reddit (3)
- Reporting (1)
- Resources (34)
- ROI (1)
- rules (1)
- Seamless shopping (1)
- share of voice (1)
- Shoppable ads (4)
- Skills (28)
- SMB (1)
- SnapChat (28)
- SnapChat Ads (8)
- SnapChat Advertising (5)
- Social (169)
- social ads (1)
- Social Advertising (14)
- social customer service (1)
- Social Fresh Tips (1)
- Social Media (5)
- social media automation (1)
- social media content calendar (1)
- social media for events (1)
- social media management (2)
- Social Media Marketing (49)
- social media monitoring (1)
- Social Media News (4)
- social media statistics (1)
- social media tracking in google analytics (1)
- social media tutorial (2)
- Social Toolkit Podcast (1)
- Social Video (5)
- stories (1)
- Strategy (608)
- terms (1)
- Testing (2)
- there are times ive found myself talking to ari and even though none of the words he is using are new to me (1)
- they've done studies (1)
- this is also true of anytime i have to talk to developers (1)
- tiktok (8)
- tools (1)
- Topics & Trends (3)
- Trend (12)
- Twitter (15)
- Twitter Ads (5)
- Twitter Advertising (4)
- Uncategorised (9)
- Uncategorized (13)
- url shortener (1)
- url shorteners (1)
- vendor (2)
- video (10)
- Video Ads (7)
- Video Advertising (8)
- virtual conference (1)
- we're all just throwing mountains of shit at the wall and hoping the parts that stick don't smell too bad (2)
- web3 (1)
- where you can buy a baby onesie of a dog asking god for his testicles on it (2)
- yes i understand VAST and VPAID (1)
- yes that's the extent of the things i understand (1)
- YouTube (13)
- YouTube Ads (4)
- YouTube Advertising (9)
- YouTube Video Advertising (5)