A little background … Penn State is very large. We have about 43,000 students at main campus and about 78,000 students throughout all of our campuses. Needless to say we have a lot of websites – I can’t even guess how many we have – 200? 400? 500? Not sure. It’s a lot, though. That much I know.

With so many websites owned by so many different units, community is so important. We have a great web community with a fantastic annual web conference and yesterday we started a different kind of web community – one devoted to web analytics.

Since most units at Penn State use Google Analytics, it was called the Google Analytics User Group kickoff event. Going forward we’re probably going to call it something less tool-specific, but we’ll see.

Some great folks at the Penn State Libraries – including doteduguru Nikki Massaro Kauffman – put the event together. It was an awesome event  and the attendee list was completely full only a few days after invitations went out.

We gathered in the morning at the libraries to kick off the event with an open panel – discussing how Google Analytics was being used at the university within different departments. It was so great to hear the different ways web analytics is having an impact at the university and how website owners are using it. Experience with the tool (and analytics in general) ran the gamut from just getting started to years of experience.

There were many sessions throughout the day including methodologies (which led into a discussion about privacy issues), Google Analytics implementation, new users, and reporting.

At the wrap up discussion we talked about how we will continue with the community. I’m very excited to continue the conversation within Penn State and we already have ideas of specific projects going forward. What a great way to end the week! I’m excited to continue the conversation.

Why do I think this? It became apparent after a couple conversations during the HighEdWeb conference I recently attended in Milwaukee. First, let me say that this has nothing to do with the conference. It was a fantastic conference and I learned a ton from the great presenters and attendees. I would go again in an instant.

More than once, however, I found myself in the midst of a conversation about how using web analytics is pretty much “useless.”

I doubt that the people in the conversation had any idea who I was (who would?) and that I was actually presenting at the conference about web analytics (both conversations took place before the presentation). So, I decided to keep my mouth shut and just listen. I wanted to see *why* they thought web analytics was useless on their sites before I jumped in to defend the practice.

Conversations like these happen all the time. They could have very easily happened at any of our campuses. In any of our offices. I’ve heard it all before and I’m sure I’ll hear it many times again. So, I’ll bring up some excuses I heard and offer some recommendations.

So … here goes …

We always meet our enrollment goals, so what’s the point?

The first excuse was that the university/college had a waiting list a mile long to get into the school, so why bother using web analytics to make the site more useful?

I’ve heard this excuse many times before, even by some people at my own institution. This, beyond anything else, is the most frustrating of excuses. Why? Because it screams, “I don’t care about my customers!”

For the sake of argument, let’s forget about using web analytics to optimize marketing efforts. Let’s just talk about usability in general.

To not use a tool that can help improve the usability of a website *and* help save money just because you’re meeting your [insert goal here] goals just doesn’t make sense. Without even mentioning the fact that you may not always meet your goals, let’s go straight to customer service. Do we not care about customer service anymore? Should the fact that “we aren’t Best Buy,” mean that we completely ignore the fact that our website might suck and that users might be frustrated?

But we run usability tests. We care about our users! Yes, usability tests are fantastic and we all should run them. But usability isn’t 1 and done. After your usability tests are done, you make your design changes, how do you know it worked? Making sure your website is useful is a never-ending task. Your content changes constantly, right? Then how do you know it’s continuing to be useful without continuing to look at the stats?

Secondly, web analytics can save you time and money. Do those in higher education care about whether they are losing money (hard and soft costs!) due to poor design? Your enrollment goals might be met, but how much money are you wasting on staff time when users call in because they couldn’t figure something out on the website?

We looked at the analytics, but all we saw was that *everyone* was clicking on [insert link here].

This is where segmentation comes in. When someone says, “everyone” did something, I always question that statement. Who is everyone? How many is everyone? Did you use segmentation? For example, did you segment out first time vs. returing visitors? Chances are, if that was a “current students” or “log in” link, the vast majority of the users clicking it were returning. So segment that group out. What were the *rest* of your users doing?

If you aren’t segmenting your data, then yes, in almost all circumstances the data will be pretty much useless.

Here is a very simple example:

So if I didn’t segment new vs. returning visitors here, I’d think that my bounce rate and average time on site might be ok (although 39% is pretty high for a bounce rate). I wouldn’t have a clue that we need to dig deeper with our new visitors and find out why their bounce rate is so high and they are not spending a lot of time on our site.

In essence, this data in aggregate is pretty much useless. When you segment, you begin to get great insights.

We don’t have time to really use web analytics.

I’ve written about this one before as well. If you truly want to be user-centric, you find the time. Period. Do you have *time* to run usability tests and analyze that data? Do you have *time* to read blogs, articles, and tweets about web design, development, or whatever, for your own professional development? No. But, you find the time. Why? Because it will benefit your users. The usability tests will help you find website issues. The blogs, articles, etc. will help you improve your skills thus helping you to create a better experience for your users. Isn’t that the idea?

That’s why I don’t buy the “I don’t have time” excuse. Nobody has time.

Recommendations

So none of my ranting probably changes the fact that some people still don’t think web analytics is useful. So, how do we solve this issue? Here are some quick recommendations:

1. Segment Your Data – As I mentioned before, if you are not segmenting your data, you will either a) make a potential wrong decision based on the *average* data or b) think the data is useless. Neither conclusion is good. Segment your data!!
2. Focus – Know your business goals and website goals. Develop key performance indicators. This will save you time and give you focus. Know what reports to look at and forget the rest. If one of your goals is to achieve better search engine rankings, know what reports will help you with that and only look at them for that goal. If one of your goals is to increase online course registration, know what reports drive users to that goal and look at those. And so on … forget the rest.
3. Create a Dashboard – This will also save you time. Again, look at your goals, know what metrics to look at to see if your website is achieving those goals (KPIs!) and create a dashboard using those reports.
4. Look at Your Quantitative and Qualitative Data Together – When you’re running your usability tests, surveys, focus groups, remember to look at your on-site web analytics as well (your quantitative data – the *what*) to see if it jibes with your usability/survey/focus group results (your qualitative data – the *why*).

What I usually find is that most people who think web analytics is useless don’t know exactly how to use it effectively. That’s fine. There’s nothing wrong with that. Don’t make the mistake of thinking that it’s useless, however, without finding out more about it – without making sure you’re getting the best insights you can possibly get out of your tool.

Customer-focused. Remember that is our mantra. Web analytics can help you get there.

]]> http://www.trendingupward.net/2009/10/frustrating-conversations-we-dont-need-web-analytics/feed/ 11 http://www.trendingupward.net/2009/06/help-fix-our-analytics/?utm_source=subscriber&utm_medium=rss&utm_campaign=rss http://www.trendingupward.net/2009/06/help-fix-our-analytics/#comments Wed, 10 Jun 2009 03:26:41 +0000 Shelby Thayer http://www.trendingupward.net/?p=1530 On Monday, Joshua Ellis (a co-worker and Google Analytics guru — and someone I’m hoping to get to guest blog here soon!) and I presented at the Penn State Web Conference. Our presentation was called Actionable Web Anaytics for Higher Education.

As attendees started asking question during and after the session, a theme developed – a theme that we are all familar with and is consistently brought up (especially in higher education).

Common Theme - I/we have issues with our analytics set up/implementation/limitations – what do we do? Help!

I am confident, especially in higher education, that *every* web site has issues with 1 or both of the following:

1. The web analytics implementation.
2. Web site or system limitations that, in turn, limit the potential of the web analytics tool.

I want to reiterate that we *all* have issues with 1 or both of the above (hopefully more of #2 than of #1).  :)

First and foremost, take care of #1 above. Make sure your implementation is as sound as possible.

1. Make sure all your pages are tagged (use the WASP Firefox plug-in to help).
2. Make sure your Google Analytics account number is unique (from other colleges or units within your university). This might seem obvious, but we’ve found instances of duplicate account numbers on college or unit web sites within the same university. This obviously depends on your school.
3. Create some basic filters to make your reports as clean as possible.
4. In the words of Linda Bustos, Don’t do stupid things.

Secondly, data collection on the web is imperfect. Numbers will never, ever tie. In higher education, we have web sites that are glorified portals, use archaeic application systems, and, in the case of large universities, are fragmented and siloed. We have many limitations.

There are limitations that every web site deals with as well. When IT folks tell you the data is crap because of  deleted cookies, users using different browsers/computers, blocking javascript, etc., etc. … smile and say, “get over it.” There is nothing we can do about these and the data will never be perfect, but that’s ok. That’s why trending is important.

Take a look at this recent post by Avinash Kaushik about being comfortable with incomplete and imperfect data. Read. Heed.

Remember some basic facts …

Fact 1. Web analytics is in its infancy. In higher education, it isn’t even born yet. I’m not a biologist, but I’d say it’s about a zygote.

Fact 2. Web analytics is not cookie-cutter. Each site implementation/set up and site analysis will (and should!) be different.

Fact 3. Especially in higher education, measuring through to the actual conversion (enrollment, registration, membership, etc.) is very hard and, in some instances, impossible.

In higher education, we need to focus on what we *can* do … and there is so much we can do.

Use micro-conversions:

• clicking on “register”
• using the knowledgebase
• watching a video

Look at the data that is screaming at you:

• top landing pages with high bounce rates (find out why!!)
• 404 page referrers (clean them up!)
• top internal search keywords (terminology) you’re not using (or not using prominently!) on your web site

I also think that there are some (yes, rare) instances when we realize that an implementation might be so hosed that we need to blow it up and start over.

There may be many reasons for this, but whatever the reason, the data that we’re getting now is really useless.

Obviously there are many instances where this is *not* a good solution, but depending on your circumstances, it might be time to start over … as painful as that seems. This is my opinion and I do realize that a lot of people disagree with me here, but there you go. I said it.

Whatever the issues (and we *all* have them!), get over it. Find something that *can* be done. You’ll be surprised at how many actionable insights you can get out of your web analytics, even with all our limitations.

The greatest thing about all this? The simple fact that you care about this stuff means you’re way ahead of the game in higher education. That’s fantastic and makes data folks like me smile on the inside. Baby steps, baby steps. Do what you *can* do.

Our presentation, Actionable Web Analytics for Higher Education is part of the Web Project and Information Management Track and is, of course, among the last group of presentations of the day. I hope people don’t bail before that time slot. We’ll see.

We did a “dry run” to our co-workers in Outreach Marketing and Communications this past Thursday and I think it went pretty well.

One thing we are emphasizing in the presentation is the fact if the attendees are working with/on a Web site that can be found in a search engine, or if they send out emails with links back to their Web site, then they are marketing that Web site whether they know they are or not.

I’m also looking forward to a lot of other sessions that will be taking place. Mark Greenfield always has awesome sessions and I’m sure this year will be no different. Always look forward to attending his sessions. 

Follow the whole conference on Twitter using #psuweb2009.

I wholeheartedly agree. Especially in higher education, the recipient of an analytics report rarely knows what the term *unique *really means in this context or the problems with measuring unique visitors.

“Unique visitors” is misleading. Why? According to Brian’s post:

Firstly, cookies get lost, blocked and deleted. Research has shown that after a period of four weeks, nearly one third of tracking cookies are missing, which means the visitor will be incorrectly considered a new unique visitor should they return to the same website.

The longer the time period, the greater the chance of this happening, which makes comparing year-on-year data invalid for example. In addition, browsers make it very easy these days for cookies to be removed – see the new ‘incognito’ features of the latest Firefox, Chrome and Internet Explorer browsers.

However, the biggest issue for counting uniques faced by both on and off-site web analytics tools is how many devices people use to access the web.

The problem I see with unique visitors is the name. Most (and by most I mean the recipients of your report!) *assume* unique visitor means a person. Unique implies … well … that it’s just that … unique – one – a person.

Even the Web Analytics Association definition of unique visitors says it’s an “inferred” person.

The number of inferred individual people (filtered for spiders and robots), within a designated reporting timeframe, with activity consisting of one or more visits to a site. Each individual is counted only once in the unique visitor measure for the reporting period.

Using the term “inferred” makes the statement, I guess, more true, but it’s misleading nonetheless. The document does go on to note the problems with measuring unique visitors, so that’s good.

The only way to true measure of “unique visitors” is by using authentication, but the majority of our sites and pages don’t do this.

In the past, when I’ve been asked for unique visitor rates because we want to measure how many *people* have come to the website, I’ve pushed back.

The argument for using unique visitors has been that we want to create conversion rates for enrollments, which are “people” so using “unique visitors” would be apples to apples.

That’s not true, though. That *assumes* it’s apples to apples, but it isn’t.

Use visits instead. It’s a more accurate number and more importantly doesn’t “infer” that it’s something it’s not.

Last week while taking a training session at Omniture Summit, I was shocked that our instructor said that unique visitors meant people. He preceded the statement with something like, “yeah, yeah, there are a ton of things that can skew unique visitors, but …” and then proceeded to say that they are essentially people.

I almost flipped my lid.

I know. I take this stuff too seriously.

This is big, though. Remember who the recipients of your reports are. Management? Leadership? Will they assume unique visitors is people? Probably. That’s not only misleading, it’s irresponsible on our part.

If you absolutely must use unique visitors, make a big, red, flashing label on your report that spells out why that term does not mean people.

Better yet, just use visits instead.

This was my first year at the summit and it was completely worth every penny. I learned so much, mostly in the Omniture University course I took last Monday.

Other than learning about classifications, VISTA rules, and advanced campaign strategies at the Omniture University course, I went to sessions on advanced use of SiteCatalyst, tracking Web 2.0 technologies, and video tracking.

I’ve always said that the one reason web analytics hasn’t made it into the higher ed community at the rate it should is that it has always been considered a *marketing* function – something that those marketers do and we as admissions, help desk, library, whatever, don’t have time or the need to pay attention to.
This couldn’t be more wrong.

If you own or have anything to do with a website, whether you like it or not, you are a marketer. Do you want your users  do something on your website? Then you’re a marketer. Plain and simple.

At the Omniture Summit, most of the sessions were geared toward e-commerce sites. At first, I was a bit frustrated at their focus on them, but then I got thinking – this is still very much relevant to us. Why? Because, again, we have goals on our site and we want users to do something (download a request information form, sign up for a course, apply for a degree). Why can’t this be like buying a widget? No, users don’t go through a check out process per se, but  they do go through a *process* of some sort.

That “every site is a customer service website” mentality came back to my mind. This is what we need to keep in mind. The conversion from visitor-to-widgit buyer may not make sense to us, but the conversion from click-to-visitor-to-prospect or visitor-to-applicant does.

It’s the same exact thing and we track it in almost the same way.

At the summit we talked about goals and key performance indicators – something every website should and *needs* to have to be successful, regardless of the type of website.

We talked about conversions – something we all should be tracking regardless of how our *conversion* is defined.

There were many tactical things I learned at the summit as I stated at the beginning of this post, but my biggest takeaway is this: when it comes to websites, everything is connected. Offisite mentions, onsite processes, search engine optimization, usability, relevant content – it all fits together. Anlaytics is how we know it’s working or not working and how we prioritize tasks and projects to fix issues. It’s how we know where to put our money and resources.

Just like setting goals for a website, I find goals helpful going into a conference.

So, I’m going to focus on:

1. Social media metrics
2. Video metrics
3. Moving to the next level with Discover (Omniture’s advanced segmentation tool)

What *aren’t* we taking advantage of and how can we start? For example, I have yet to truly dive into video metrics, so I’m very excited about getting my hands dirty with video metrics.

One thing I am alsoo excited about is live blogging throughout the conference. This will be my first time live blogging, so we’ll see how it goes.

Stay tuned … off to Salt Lake.