TL;DR: User/visitor-based KPIs have never reflected actual users. They’ve only ever reflected clients (IOW, the combination of device and browser you visited the site from), making reporting on them a waste of pixels.
Continuing my series on dimensions in Google Analytics, I want to address the most misunderstood of them all: the User Type dimension. This dimension is supposed to measure new vs returning visitors** to your site. But, in reality, this dimension provides your report with a steaming pile of poo. I’ll demonstrate why in this post.
But I’m not just going to look at the User Type dimension. Since the issues that plague the User Type dimension apply to metrics as well, I’m going to broaden the scope of my investigation to all user-based key performance indicators (KPIs). This includes the Users and New Users metrics, which are accessible via custom reports, as well as the Overview reports (under Audience and Acquisition, respectively).
**Google still vacillates between using “user” and “visitor” in its report nomenclature.
Website owners, marketers, and analysts have had a long-standing love affair with these KPIs. But they were ill-conceived from the beginning. They always operated under the assumption that you use only one device and one browser or app on that device to surf the web. Even in the early days of the web that wasn’t true for many who had a computer at home and work or who bounced back and forth between Internet Explorer and Netscape. #flashback
Calling unique clients visitors or users is kind of like saying your social security number (or whatever number your country uses to identify its citizens) is interchangeable with your license plate. A single person may have multiple vehicles s/he uses to get around. But my car != me. Therefore, my girls can borrow my car without risk of being arrested for identity theft.
Likewise, your client is a vehicle you, the user, use to get to a site. It’s not you. The problem with user-based KPIs in Google Analytics is they measure license plates, not social security numbers.
Let’s Do an Experiment
My devices: I have two laptops (one Mac and one PC), a desktop (an iMac), an iPad, and an iPhone.
My browsers: On my laptops and desktop, I have a combination of four browsers: Chrome, Firefox, Safari, and Internet Explorer.
My Chrome habit: To complicate matters, I use multiple instances of Chrome (which I show you how to do in this post) because I have an affinity for Chrome.
My apps: I use a variety of apps between my iPad and iPhone.
My Quest for Redneck Wine Glasses (Phase 1)
Watch what happens when I search for redneck wine glasses in two different instances of Chrome. With the Google Analytics Debugger extension active, I clicked through to the One Hundred Dollars a Month blog, which had a post featuring redneck wine glasses. And then I looked at the data Chrome was capturing about me in the console. Notice that it indicates that it’s my first session (formerly known as a visit) and assigned me a Visitor ID of 712640405. This is accurate. I’ve never been to this site before. Or sipped wine from a redneck wine glass, regrettably. (But it’s on my bucket list.)
Note: This site is running the classic version of Google Analytics. If it had been running Universal, you would see clientId instead of Visitor ID, like you do with my site.
And why do you think Goog changed the nomenclature from Visitor ID to clientId? You got it! Google Analytics is measuring clients, not visitors. Unfortunately, this shift isn’t reflected in Google Analytics reports, as they still give you the ability (although just an illusion, IMO) to report on users.
Google Analytics does give us the ability to measure actual visitors now with Universal, but there are quite a few hoops you have to jump through to set this up, which we’ll get to.
Rinse and Repeat in a Different Instance of Chrome (Phase 2)
Now let’s see what happens when I go to the site using a different instance of Chrome from my same laptop.
This time I was assigned a Visitor ID of 1140255071. At the risk of stating the obvious 1140255071 != 712640405.
It also recorded it as my first session. Liar. You’re my witness. I just visited this site.
But it actually makes sense if you trade in your Warby Parkers for a pair of taped-up horn rims since, according to my cookie data, I’m a totally different visitor. This poor, unsuspecting site has no idea that I’m the same person who just found the site via a Google search.
It gets worse.
If I came back via Firefox I’d be a third visitor. Safari? Yet another. My iPad? Yep. Another. My iPhone browser? You guessed it. And an app on my iPhone or iPad? Throw in a couple more.
When you consider that I use different instances of Chrome on each of my computers, I could realistically be 10+ different visitors for sites I visit regularly.
For example, I bank from all of my devices, and I use my bank’s app on my phone and tablet. Because I use the same instance of Chrome for my banking, on any given day I could show up as coming from one of five Client IDs: my two laptops, my desktop, my iPhone app, or my iPad app.
Why I’m Calling BS
If I can’t be identified as the same person while using the same browser and device, how is Google Analytics going to associate me as the same user when I actually switch devices? It can’t. So I’m only a returning visitor if I come back to your site using the same instance of the same browser on the same device. And I’m a <air quote> new visitor </air quote> every time I visit your site with a different device, browser (or browser instance), app, or kiosk.
So when you add a KPI like new visitors to your monthly reports, you’re not exclusively celebrating acquiring, you know, new visitors (although I’m sure there are bonafide new visitors in the mix). You’re more realistically celebrating diversity. Some of your visitors want to visit your site using different devices and browsers. Woohoo! Drinks on me!
Who Can Safely Use User-Based KPIs
You can use user-based KPIs if you satisfy all of the following requirements:
1. Your site has a login option. This is the most critical component. There has to be some way to authenticate your unique visitors. A site like Facebook would have no problem identifying unique users because most people using the site are going to be logged in. So it doesn’t matter what device or browser or app I use, Facebook knows who I am. Going back to our car analogy, they know my social security number, not just my license plate.
2. Visitors actually log in to use it. My site has a login for subscribers to my dashboard course. However, the overwhelming majority of visitor visit the site logged out. My content isn’t behind a subscriber wall, so there’s no reason for visitors to log in just to read my blog posts. So guess how many effs I give about my new vs. returning visitors? My site isn’t a good candidate for these KPIs.
3. You are using Universal Analytics. You have to update to Universal to take advantage of Google Analytics’ new feature that actually enables you to track unique users. (More on that in the next point.)
4. You have modified your Google Analytics tracking code to capture your members’ User ID. You can use customer ID, login ID, member ID … whatever your CRM uses to identify your users. But it can’t identify the user to Google, so names, email addresses, and the like are out. Justin Cutroni wrote a great post about using the User ID feature if you want to come up to speed. But you will still need to read the help files on this one. That’s the only way I was able to really wrap my mind around it.
5. You have set up a view that uses User-ID to determine new/returning visitors instead of clients. After you’ve added the extra code to capture your site’s User ID, you’ll need to create a new view to enable this feature. Google doesn’t reprocess data like tools such as Kissmetrics does, so you start from scratch whenever you enable a feature like this. You can read more about this in the Configuration section of the User ID reference.
This new view will only track the visits of your logged-in visitors. A switch is flipped, and Google Analytics no longer uses the device to determine if a visitor has been to the site before; it now looks at that User ID. Because of this switch Google is able to provide you with new Cross-Device reports not found in views that don’t have User ID enabled.
Better But Still Not Perfect
Sooo if you are doing all of these things and using a User ID-enabled view, your user-based KPIs are infinitely more reliable. Why? Because now you have a way to identify me across different devices and browsers. I’m no longer just a visitor coming from my personal Gmail instance of Chrome on my 15-inch MacBook Pro; I’m Annie Cushing … a real human being with a social security number and proclivities for data, shoes, and lime green and aversions to country music, Brussels sprouts, and the color pink.
It’s still not foolproof. I mean, I could go to my bank’s website to research how to send a wire transfer without logging in. Also, if I’m a member of a site and have logged in with my current browser but am not logged in when I visit it, that session will not be routed to this view. Kissmetrics has it all over Google Analytics in its ability to identify unique users even if they’re not logged in. But it’s at least a step in the right direction.
If I had my druthers, Google Analytics would align its report nomenclature to reflect that these are clients, not users. I think it would help marketer assess if it’s worth reporting on.
If you’re doing analytics for a social networking site, bank, airline, online tool, or any other site where your visitors have incentive to log in — and you’re taking full advantage of Google Analytics’ newer User ID feature — you’re golden. And you should be rocking those user-based KPIs.
For everyone else, I’d shift your focus from Google Analytics’ traditional user/visitor KPIs and put the lion’s share of emphasis on session-based KPIs. There’s plenty of good data in them that’s not based on false assumptions.
If you want to become crazy marketable with advanced analytics skills, check out my Analtyics Audit Template, a self-guided, 147-page audit template that is regularly updated and will teach you how to do detailed analytics audits like a pro.