February 2nd, 2012
Posted by Thanh Huynh

Have you ever been on a site where cryptic labels or jargon were used for the navigation options? Or have you ever resorted to entering a Google search instead of exploring a site because the content was scattered and not organized in a way that you found intuitive? It’s highly possible that these sites were designed with little consideration for the user and user input was not collected during the life cycle of the project. Spending time with users or performing user research was probably viewed as overhead or a “nice to have” rather than incorporated into the overall project budget. Often, user research is not viewed as mission-critical and instead is seen as a step standing in the way of getting the project completed on time.

Fortunately, there are ways to include user research without blowing your budget. A simple yet eye-opening approach to gather user input is card sorting. The beauty of the card sort is to see how representative users of your site would naturally organize content from your site.

There are two primary types of card sorts:

  • Open card sort. Users are given a sample content list and asked to create groups as they see fit and label the groups they created. This is typically done for a new website project and conducted before the information architecture has been finalized.
  • Closed card sort. Users are given a sample content list as well as predefined groups that they will need to use to group the content into. This may be done to validate assumptions made after an open card sort or to refine existing categories (e.g., top navigation options) during a website redesign.

Card sorting is flexible and easy to administer because it can be conducted in person or remotely. And all you need is either a set of index cards or an online card sorting tool. Index cards work well because users easily work with the cards by spreading them out in any manner they choose as they begin to assemble the groups. However, there are some online tools that are effective in emulating the experience you get with index cards:

  • Optimal Sort is one of the most popular card sorting tools. It has an easy-to-use drag-and-drop interface. And you can also benefit from some analysis reports that are automatically generated by this tool. Here is a video on how to create an Optimal Sort study: http://vimeo.com/14204460
  • iCardSort is an iPad app that was created to replicate the feeling of physical cards.

While online card sorting tools can give you the flexibility to conduct the study without a moderator, it’s beneficial to have a moderator there to gain a deeper understanding of the user’s rationale and interpretation of the content. So as users organize content into what they deem are the logical groupings, they are asked to describe their thought process through a think-aloud protocol. This gives insight into their mental models and answers the following types of questions:

  • What do users find easy to organize?
  • What do users find difficult to organize?
  • How do users describe the content or what do they name groups?
  • How do people want information grouped – by subject, process or content type?
  • Do they understand the content?
  • What content would they organize into more than one group?
  • Were there any patterns (e.g., similar content groupings or labels)? Or what were the areas of high disagreement across users?

With answers to the questions above, the team will be better equipped to design an experience that is positioned to meet the needs and expectations of your target users. Integrating user research approaches like card sorting can act as useful checkpoints, but, more importantly, provides the team with valuable insights that will make them better advocates for your customers and help create a positive experience for your brand. This is a technique that is best conducted early in the process – ideally before wireframes are created or before any development begins – when there is less risk to the project at large. With simple card sorting exercises, incorporating user research into the overall project shouldn’t be a daunting overtaking and is actually fairly easy to integrate.

More on Card Sorting

December 21st, 2011
Posted by Thanh Huynh

Do your customers find your brand’s digital presence easy to use and learn or difficult and confusing? Are your customers satisfied with how your products work? The answer to this question can be traced to the usability of your offerings. ISO defines usability as “The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.”

Usability is important to customers and often differentiates brands among their competitors. Look at Apple’s and Google’s success. Many of our clients ask their search to work like Google’s, and many of us can’t resist Apple’s products because of their commitment to simplicity, making them easy to use.

When customers use certain products or offerings, they have specific goals they need to accomplish or want access to information they believe those offerings provide. When customers experience your digital offerings, they don’t want to spend a lot of time figuring out how to use them. If they have a difficult time, they won’t be successful and will likely leave unhappy.

Get Your Design in Front of Customers

Usability testing is one of the most common techniques to assess usability and put your design to the test. During this process, observation is key – what customers say is often not what they do. You will find where the design hits the mark, as well as where users struggle that you may not have expected they would or find out there are aspects of the design they overlook.

Unlike focus groups where the collective wisdom of many is gathered, usability testing has customers participate in one-on-one sessions and asks them to complete common tasks as they normally would. Therefore, recruiting participants for usability testing based on relevant behaviors rather than marketing segments is key to ensure that you are seeing the right customers completing tasks they have performed before or would likely do on their own.

Based on what was learned from usability testing, you might need to make some simple tweaks like creating more intuitive link labels. Or perhaps you might find there were some showstoppers; for example, finding that most users were unable to complete the registration process. By seeing customers interact with your brand, your team is able to make more informed decisions on behalf of your customers.

Usability Is Not an Add-On

But usability should not simply be assessed with one usability test. It’s an approach that should be incorporated throughout the life cycle of the design process. When usability is a priority throughout the process, inefficiencies in design interactions can be caught early enough to make the necessary fixes when they are less costly and easier to make. Or once your product is in your customer’s hands, usability improvements can contribute to a reduction in customer support calls because customers can use your products without anyone’s assistance or they experience productivity improvements because they complete tasks more quickly.

As 2012 approaches, I’d like to propose that you make usability one of your New Year’s resolutions. Separate your brand from other competitors by making usability an integral part of your digital brand presence.

December 7th, 2011
Posted by James Hering

At least one thing is pointing up in this economy. And that’s the level of digital ad spending. The Interactive Advertising Bureau released its Q3 Internet Advertising Revenue Report reflecting a 22% increase compared to the same period last year. This is the eighth consecutive quarter of year-over-year positive growth. In fact, the third quarter is always a strong performer, with increases occurring every year except three since 1996.

The major categories in terms of overall digital spending include Search at 49%, Display-Related at 37%, Classified at 8%, Lead Generation at 5% and Email Revenue at 1%. Within Display-Related, Display/Banner Ads make up 23% of the total pie, Rich Media makes up 5%, Digital Video is at 6% and Sponsorship rounds out at 3% of the total spend.

So does this mean you need to adjust your budgets to reflect overall spending trends? Here are some points and checklists to take into account before you jump on the digital budget bandwagon.

Where to Spend, Where to Spend?

As you begin to finalize planning for 2012, you might consider using the following as a budgeting checklist. To help with allocations or achieving business goals, you might assign a value to each effort – say 1 is “Extremely Important” and 7 is “Not Important”:

Owned Media
Company intranet/extranet
Brand website
Brand channels in social media (example: Facebook, YouTube, Twitter)
Branded content (content created by a brand)
Mobile site
Email marketing
Smartphone applications
Tablet applications
Search engine optimization

Paid Media
Paid search
Display ad units (including display ads on social networks)
Online video
Mobile display
Online sponsorships (online or integrated with cross-platform programs)
Branded content (produced as a part of a paid media effort)
Games (in-game advertising or sponsorships)

Earned Media
Digital word of mouth (including blogger outreach)
Social monitoring/listening
Online video distribution
User-generated content via social media
User-generated content via other online channels

Customer Behavior Matters, Too

You might also ask how the following consumer media behavior trends are occurring across your target audience and how they may impact the receptivity of your communications:

  • Customers who migrate from device to device throughout their day
  • Spending more time accessing the Internet via mobile device
  • Customers who participate/access user reviews
  • Watching more video and programming via a mobile device
  • Increased use of location-based services
  • In-store use of mobile devices (tablets and smartphones)
  • Adoption and usage of digital couponing
  • Downloading of apps or sampling content via apps
  • Use and adoption of smartcodes or Quick Response (QR) codes
  • Customers who are time shifting a majority of their TV watching

Now is also a great time to revisit your reporting and information resources so that you are making the best possible business decisions. Major digital measurement tools include:

  • Digital campaign performance reporting
  • Online brand impact studies
  • Website analytics/reporting
  • Search (PPC and SEO) reporting
  • Mobile site analytics/app activity reporting
  • Social media analytics
  • Online listening/conversation analysis tools
  • E-commerce sales performance
  • Online audience measurement tools
  • Online competitive site activity
  • Online competitive advertising activity

Also consider some of the following major technology trends and how you might allocate proper resources to take advantage of the “next wave” of innovation:

  • HTML5 development
  • Mobile app development
  • Tablet apps
  • Near-field communications (NFC)
  • Augmented reality
  • Social gaming/social apps
  • Gesture-based computing (think Xbox 360 Kinect)
  • Digital displays/point of sale interactive experiences
  • Digital imaging/exterior installations

Finally, you might want to check the last time you implemented a digital strategy. Has more than 24 months gone by? Not yet even taken the step to implement one? Well, now’s an excellent time to establish a current digital strategy to help with spending decisions…and so much more. And we know several savvy digital strategists who are ready to help (hint, hint).

June 14th, 2011
Posted by Click Here

I haven’t watched Major League Baseball in several years, but, as bandwagoners often do, I started to follow the Texas Rangers after their recent trip to the World Series. I’ve also noticed how on-screen graphics for sports presentations have evolved to display a vital set of information about the game in a very small area so that viewers aren’t distracted or overwhelmed with information. As an average fan of the game, I can get right up to speed after tuning in to a game already in progress.

In a very small “FOX Box,” as FOX Sports calls it, six crucial pieces of information can quickly get a viewer up to speed in a matter of seconds, with no extraneous information. While taking up a minute amount of space on the screen I can see:

  • The teams playing – using three characters mirroring those found in newspaper box scores
  • The current score
  • The current inning (top and bottom)
  • The number of runners on their respective bases
  • The pitch count
  • The number of outs
FOX Sports Southwest FOX Box graphic display during a Texas Rangers game

FOX Sports Southwest FOX Box graphic display during a Texas Rangers game

FOX uses this same box in their NFL presentations. Again, a small box in the upper left-hand corner displays:

  • The teams playing, using team colors and logos only
  • The current score
  • The time remaining in the current quarter
  • The down and distance
  • The team in possession of the ball
  • The number of timeouts remaining for each team
FOX NFL FOX Box graphic display during an Atlanta Falcons game

FOX NFL FOX Box graphic display during an Atlanta Falcons game

Providing context is also vital on the Web. One of the primary heuristics of user interface design is called “Visibility of System Status” – We need to ensure the user knows where they are within a process, taxonomy or time frame. Users don’t always enter a site from the home page, so the interface should provide some sort of context within the page where they have landed.

When moderating user testing for an interface, I like to have the participant turn away from the screen while I go to a random page. I’ll then ask them to look at the screen and tell me where they are on the site. Without visual cues, the user is completely lost. Without context, the site appears disjointed.

Some solutions: The global navigation elements need an “ON” state, enabled when any of the pages within its section are being viewed. Each page could also include breadcrumbs (text or text links that give users a way to keep track of their location within programs or documents). Dallasnews.com does a great job with both. They highlight the current section and subsection and provide a breadcrumb so that users can see where they are and easily navigate back to a primary section or page.

Dallasnews.com’s multilevel tabbed navigation and breadcrumb text

Dallasnews.com’s multilevel tabbed navigation and breadcrumb text

Amazon does a good job matching titles within the browser’s header, tab and URL.

Amazon.com URL

Amazon.com URL

Even a simple “Back to Category XYZ” text link on a detail page within category XYZ will help users understand where they are.

While visibility of system status is only one of many heuristics we evaluate for each interface we build, it is probably the most important and simplest of concepts to implement. The next time you are on one of your favorite sites, try closing your eyes, clicking somewhere on the screen and then seeing if you can identify where you are in the site. It’s a fun way to test for system status visibility.

May 26th, 2011
Posted by Joe Wilson

Sony has recently experienced what is shaping up to be the most disruptive and dangerous public cyber-attack to date. According to Sony, they discovered their PlayStation and Qriocity networks had been compromised on Tuesday, April 19. The following day, Sony took steps to bring these networks offline, leaving millions of gamers isolated without access to online play or any of the other offered online services. It was two days later before Sony issued a public statement clarifying that the downtime was because of an outside incursion to their network. On April 26, the public received its first glimpse of the full extent of the damage. In a statement released on the PlayStation Blog, Sony revealed that user information, including name, address, country, email address and birthdate, had been obtained by a third party. In this statement, Sony also stated that it had no reason to believe that users’ credit card information had been compromised. However, three days later, hackers claimed to have 2.2 million credit card numbers, including CVV security codes, belonging to Sony users.

It is now a month after the intrusion and Sony is still struggling to provide users with online services. This attack underlines why online security is essential for any brand managing a user network, especially those that store sensitive information. Yet perhaps the most valuable lesson to take away from the Sony cyber-attack is crisis management. Any digital security expert will tell you that no system is unhackable. Even the most secure network can be penetrated given enough time and effort. Many feel Sony’s customer frustration stems more from mismanaged PR and a general lack of information. Sony has continued to share information with users through the PlayStation Blog and has also offered other services for customers, including free identity theft protection for subscribers. However, PS3 users attempting to log on to the network received only an error message announcing that the server is currently undergoing maintenance. In the weeks following the PlayStation Network intrusion, Sony lost over $2.08 billion in stock value as investors sold in droves. This crisis is likely to change the way consumers view the brand, and it’s still yet to be seen if gamers will be willing to trust Sony again. Without a doubt, this cyber-attack has wreaked havoc at Sony and the aftermath is still to be determined.

Ever since the Web went 2.0 and users started liberally handing out personal information, privacy has become a hot-button issue. Mark Zuckerberg and Facebook continue to be pressed by the public and government on privacy. Even Internet giant Google faced harsh criticism when it launched the social service Buzz last year. Google admitted fault right away and released updates to address the public’s privacy concerns within a week of launch. When Google experienced an attack against Gmail, it not only provided information on the kind of the attack, but also detailed steps outlining what was to be done to prevent such attempts in the future.

Digital crimes are no longer just the stuff of sci-fi movies. Today, almost everyone from the biggest brands to individual bloggers store user data in one form or another. It could be a simple email newsletter distribution list or sensitive financial information like credit card or bank account numbers. More than ever, it is important to know that your data is secure. Sony is a cautionary tale for brands. Users expect that if they share their information with a company, every measure will be taken to protect that data and handle it in an ethical fashion. Equally important is for companies to have a contingency plan in place for how to handle an attack or misstep when one takes place. Quick communication and transparency are key when handling user privacy issues. And always, always remember that there is someone out there smarter than you. Don’t ever assume that your data is safe. Ask questions of those that manage your network and stay informed as to the security measures you have in place to protect your users’ data.

In summary, if you haven’t put any thought to your network’s digital security plan, now is the time to start. A harmful attack can tarnish your brand image just as much as it can compromise user information. Just keep the following things in mind:

1. How is the user information on your network stored and what safeguards are in place to protect it? If you don’t know, find out.

2. What is your plan if an attack takes place? Knowing how to react can mean the difference between public sympathy and scorn.

3. Be transparent. Always tell your customers how you plan to use their information and if it is ever compromised, be sure you tell them every step of the way what is being done to prevent such missteps in the future.

Users just want to know that you have their best interests in mind when dealing with sensitive personal information. Due diligence is required to protect it, and honesty and open communication are the key when managing user expectations.

April 14th, 2011
Posted by Click Here

“Back in my day, son, we had to create all our wire frames in Visio! We had to link everything by hand if we wanted to create some kind of interactive version, or even hand-code them in static HTML…” – A 40-something information architect

Nowadays programs like Axure have made it incredibly easy to render small and medium-sized wire frame documents and interactive prototypes. While they can provide realistic, testable user experiences – there still are limitations, especially when creating a large catalog of pages or when the project has a quick turnaround time.

In Axure, the longer the list of pages, the longer the render time. One of our recent projects topped out at well over 100 pages. It took almost an hour to render a prototype. Even after all the work had been finalized and approved, the code Axure generated could not be reused by the development team. Developers still had to recreate the experience from scratch.

Generating....

Recently, I was asked to create a set of wire frames and render a prototype for a client’s blog. There were numerous revisions to the documents, and development was left with a shortened timeline. For another client, I had to build an interactive prototype based on a 150+ page site map/content matrix. Axure was just too cumbersome.

So lately I have been experimenting with a new method of prototyping – building the prototype in the actual content management system (CMS) that my team’s developer will use to render the final version of the site.

For example, a client might use WordPress as the CMS for their blog/static page site. I simply created a default, grayscale template and built the prototype using a few additional plug-ins. Even if some pages will ultimately contain more robust toolsets, the page structures, template types, site map and naming conventions are already complete. Only the presentation layer, styles and additional features or custom scripting remain.

Simple Add Pages plug-in

Above - Using the “Simple Add Pages” CMS plug-in creates an infinite amount of nested pages and corresponding navigation in one step.

I’ve found this method superior for making changes on the fly while collaborating with clients. Change page order, move pages to another section, rename entire areas – no problem. Different navigation systems can be tested on the fly and rapid iterative designs are much more, well, rapid. And again, all of the code, databases and XML data can be reused for the final version of the site.

The primary benefit – time savings for both information architecture and development. The 150+ page site that might take several weeks to create in Axure took a few days. Still, I would not recommend this solution for every project. Axure is far more nimble for prototyping more experiential sites, and the supporting specifications generator can be easily printed for client and development review.

While the CMS method of prototyping has yet to be fully vetted and tested, several of my colleagues in development are eager to collaborate with me on this type of prototyping. I’ll post updates in this blog with a client-approved example later this year.

April 4th, 2011
Posted by James Hering

In today’s social media mayhem, everyone is busy “friending” everyone and “liking” just about anything. Just take a look over your shoulder at all your “likes and follows” in your Facebook, LinkedIn and Twitter accounts (to name the obvious), and you’ll quickly wonder “just who the heck are some of these people?” And what exactly were you thinking when you hooked up with that certain brand? It does make you wonder if something in the virtual world might be out of whack.

Sir Isaac and His Third Law

So with all this liking going on, there has to be some type of equalizing force to bring the world back in balance, right? Remembering Newton’s Laws of Motion, for every action there is an equal and opposite reaction. But does this hold up in the world of social interactions? Well, it appears so. In fact, there’s even report, Top Reasons Why Consumers Unsubscribe Via E-Mail, Facebook & Twitter, that outlines the motivations behind people “disconnecting” from all their connections.

The Motive

I actually pondered this same point just a few weeks ago, wondering what would be different if one didn’t have all those prized confirmations. So I decided to implement a social media behavioral study. After all, what exactly happens when someone gets “defriended”? Does the other person notice? How would they know? And what is their reaction if they figure it out?

And since trying to recruit respondents to participate in such a study would either be highly difficult or really expensive, I decided to be the virtual lab rat and offer up my own network connections.

The Approach

Drawing a complete data set of all my friends and acquaintances from within my Facebook and LinkedIn accounts, I quickly grouped my peeps into three categories:

  1. Super-Close Friends and Family – Defined as people that I personally connect with at a minimum of once every month. And, no, a drive-by status update with a clever quip does not qualify as a personal connection. To be in this group, I have to have a meaningful exchange, such as passing along information or sharing genuine care and concern.
  2. Important Business Connections – These are defined as clients, business associates or industry connections for which I have some type of interaction within the last six months. Could be anyone from an important client to a trusted industry resource.
  3. Indirect Connections – We all have these, don’t we? The distant friends from high school or college we haven’t talked to in years or “what’s-his-name” from that company we did a project with a while back…yet we flippantly confirm on the fly when asked to connect.

The Sample

Randomly sorting names from each category in a spreadsheet, I then even more randomly selected numbers from a hat to pick 20 numbers that corresponded to line items on the spreadsheet. Presto, I had my list of 60 unsuspecting respondents.

The Moment

Without hesitation, I immediately deleted the 60 contacts. And waited.

Surely I’d get a quick text from a puzzled friend…or a call from someone wondering, “What’s up?”

Hours and days went by…seven to be exact, before someone noticed. At that point, I kept a careful log of thedate, time and method of contact. I also noted how the respondents figured out we were no longer connected. So while it’s a small sample size, I was intrigued by how people came to the realization and was able to group them into four general buckets:

53% Used social media to access my contact information
27% Planned to initiate a social outing or activity
13% Needed to communicate directly with me
7% Wanted to arrange a business meeting

Also interesting is that it has taken nearly seven months to hear back from every respondent. Which could mean either I’m just not that important (most likely) or we rarely consider and review the lists of contacts we amass. I’ll go with the latter.

One final note – how did people feel about being deleted? Here again are four general buckets:

69% Totally fine with being a respondent
21% Quickly laughed it off
7% Seemed slightly annoyed
3% Were totally incensed

Believe it or not, two of the respondents were so offended they were deleted that, to this day, they will no longer speak with me. Really? My social currency is that valuable? Or is a “defriend” really that distasteful?

Personally, I’m puzzled. But maybe that’s just part of the new social etiquette. In any event, something to consider before you start a little spring cleaning in your social sphere.

January 17th, 2011
Posted by Click Here

There is a certain level of prestige that comes with coming up with a “big idea” or campaign. We want our ideas to stand apart from the rest of the marketplace, but let’s be frank – it’s a big marketplace. When executing a website, even the little details matter.

My Country, Right or Wrong
There are a number of reasons why you may need to collect someone’s address. For U.S.-only companies, this usually entails some sort of state selection drop-down menu. According to Jakob Nielsen, this annoys users, but they muddle through because they are accustomed to it.

“The drop-down menu is unpleasant, and we sometimes hear users sigh when they encounter it. That said, they know how to beat it into submission, because other sites have similarly annoyed them before.“

But at least the solution is simple (even if underused). It’s possible to still use a simple HTML element to make it easy for users to enter their state code. Amazon does it, they said. You should, too.

Country selectors, however, can be a bit more difficult (think 51 options vs. 195 options). To mitigate negative attitudes, then, we must make the experience easier than a drop-down menu, which means we need to go beyond the ready-made HTML elements at our immediate disposal.

This approach costs a little more time to implement, so if it is critical to your business to incorporate a country selection device, I highly recommend testing it with international users.

…But May We Always Be Right
A drop-down menu does that already, so – if we accept consumer reports that state drop-down menus are already annoying and prone to error – going from 51 options (50 states plus the District of Columbia) to 195 is even more of a hassle.

Advanced users will deftly point out that there are ways they can get to the country faster, such as by typing the name of the country when the menu is selected. However, in most cases, we must design for users who are less advanced, as well.

The best selection is the one the user doesn’t have to make, when the user is most likely to input inaccurate information. So, if it’s possible to prevent a mistake, use that option. Error prevention is always better than error recognition and correction, anyway. You may accomplish this feat in three ways.

  1. Don’t require information you don’t need. If you’re delivering nontransactional information and the delivery of that information does not REQUIRE certain information from the user (such as is the case with most electronic delivery needs), do not ask for it.
    If you’re not asking for information, the user cannot enter it incorrectly.
  2. Detect and select the option for the user. Geotargeting based on an IP address is an inexact science. Still, there are viable options, depending on your comfort level with uncertainty. As a failsafe, give the user the ability to both change the country if it does happen to be wrong and to instruct the system to remember their selection if you expect even a modest proportion of repeat users requiring this selection.
    If a decision made for users is not obviously incorrect, they are less likely to change it.
  3. Build country-specific portals. Sometimes the best thing to do is to let the user select a country of origin at the outset. When the user gets to the point that it’s time to fill in address information, the information required to determine address fields and formatting has already been made. FedEx and UPS both do this, and in their own ways make it easy for the site to “remember” the user’s preference for subsequent visits. Note, however, that this option works well for FedEx and UPS because the content is different for each country, but it may not fit with your business model.
    If users believe they will receive and do receive a payoff for selecting the correct country of origin early (such as geocentric content), they’ll be less likely to select the incorrect one and have cause for changing it later.

The last option (which can be used in conjunction with the other methods) is to utilize a different device for country selection than a standard drop-down menu that is part of HTML. They are used primarily when it’s necessary to select a single option from a list of options. HTML’s other standard single-selection devices, radio buttons, are impractical for use as a country selector (when all 195-ish countries are options), because it would take up too much space.

Instead, simply give the user a better mechanism to select from a big list. Break it into chunks, organized alphabetically, in a modal window less subject to dexterity deficiencies.

(In this example, the business requirements dictated that U.S. be the default choice, and although Great Britain and Canada were secondary choices because of relatively higher volume of traffic than other countries, the other countries could not be excluded.)

Why is this important? This all seems like a lot of work for what seems to be a minor thing. Why bother? The users will figure it out, won’t they?

While one minor annoyance isn’t usually a big deal (though sometimes it is), many minor annoyances compound the negative attitude with which a user regards your brand. Even a good idea can be choked out by sloppy execution of the smaller annoyances. First, decide to fix them. Then, create a plan to find and root them out through continuous monitoring and testing.

References
Jakob Nielsen’s Alert Box
Rocket Surgery Made Easy
Expedia on how one extra data field can cost $12 million
How to Take Your Website from “Good to Great”

November 29th, 2010
Posted by Alex Kenney

As brands are increasingly shifting media dollars online, it’s becoming more and more important to measure the effectiveness of online campaigns beyond just clickthrough rates. One way to do this is with advertising effectiveness studies, which take us beyond standard metrics to show the impact of a specific campaign on brand metrics and perceptions. InsightExpress and Dynamic Logic are the two leading vendors in the field to conduct such survey research, and they’re the ones that Click Here recommends and partners with as well.

How Do They Work?

At the start of a campaign, our vendor partner provides a tracking tag to append to the online creative. Visitors are surveyed on the websites where the campaign is running and classified as “control” (not exposed to the creative) or “exposed” (exposed to the creative). Exposure is not self-reported; instead, we know a respondent is exposed because of the implemented tracking tag. The survey asks respondents to rate the brand and four competitors on metrics such as brand awareness, message association, brand favorability, purchase intent and brand attribute agreement. Since the control group is weighted to match the exposed group on demographics and product usage and is recruited in the same digital footprint, any changes in brand metrics and perceptions are attributed to the campaign itself.

Source: InsightExpress AdIndex Methodology

What Do We Learn?

While the overall results are an incredible learning in and of itself in terms of how well the campaign performed compared to industry and category norms, the final report from the vendor also includes more specific analyses such as results among a target segment and between publisher sites, ad types (video vs. display), sizes (728×90 vs. 300×250) and frequency of exposure. If two creative executions ran, we are able to tag these individually to determine which one resonated more strongly with respondents. Let’s also not forget that these studies can be executed on a mobile platform, too. The main difference there is that the control sample is recruited on the mobile site precampaign launch (because cookies are not as easily captured on mobile, so this is done to ensure a clean control), while the exposed group is recruited during the campaign.

Can I Measure the Effectiveness of a Campaign on Multiple Platforms?

Yes. Taking online ad effectiveness studies a step further, cross-media advertising research gives a more holistic view of how a campaign is working across multiple platforms together such as online, print, TV and outdoor. Online exposure is determined by the tracking tag – just as in the online-only studies – but since this is not possible for offline media, an “Opportunity to See“  (OTS) methodology is employed. Endorsed by the Advertising Research Foundation, OTS is the most reliable way to identify offline media consumption control vs. exposed groups. It surveys respondents about their media habits (such as which TV shows they watch regularly) and determines based on the media plans whether the respondent was likely to have been exposed to the offline ads on various platforms. Results show how well each platform worked individually as well as synergistically.

Behavioral Tracking

Some of the most interesting online advertising research being done right now overlays a behavioral tracking component on top of the attitudinal survey results we get from the above methodologies. InsightExpress and Dynamic Logic both partner with Compete, a TNS media company that has the largest U.S. consumer behavioral panel. By adding this additional layer, we are able to see how campaign exposure affects search behavior (for the brand and/or category) as well as visiting the brand’s (and competitors’) websites and even making purchases online. This is an ideal way to combine attitudinal and behavioral results.

Source: Dynamic Logic and Compete

There are even more options, overlays, vendors and methodologies out there that provide invaluable insights into how a campaign resonates with consumers. Of course, there are impression thresholds and budget limitations, but as we pioneer into a 2011 that’s likely to bring even further media proliferation, this kind of research is worth doing when possible to guide crucial media budget allocation decisions as well as future campaign development.

October 18th, 2010
Posted by Click Here

No one knows everything.

Ask Jim Collins, author of Built to Last and Good to Great, who observed that companies that have risen above the rest understood that such progress was cultivated over time through focused discipline, not through big personalities and initiatives. He called this discipline “The Flywheel Effect.”

The term is based on the process for getting a large, heavy flywheel to rotate on an axis. From a resting position, it takes enormous effort to get it to move an inch. After a lot of time and many individual pushes — all to the same design – the flywheel eventually builds speed momentum until the its own weight is doing most of the work.

If someone came at the tail end of this process and saw the flywheel spinning rapidly and unstoppably, what could they say caused it?

“You wouldn’t be able to answer; it’s just a nonsensical question. Was it the first push? The second? The fifth? The hundredth? No! It was all of them added together in an overall accumulation of effort applied in a consistent direction. Some pushes may have been bigger than others, but any single heave – no matter how large – reflects a small fraction of the entire cumulative effect upon the flywheel.”

Even the best web designers cannot anticipate every scenario, so the best initial design is going to be, in some form, incomplete.

Websites that must last more than a few months should be tested and refined at regular intervals to root out and correct problems that occur — even if “genius designers” have built it.

This cumulative effect of all this testing acts on a website like the tiny pushes on the flywheel. Over time, it becomes an incredible force.

The relative cost of doing nothing

Which costs more:

  1. A modest monthly budget for regularly testing and refining a website to make sure it doesn’t anger, frustrate or bore your audience, or
  2. Having a website that angers, frustrates or bores your audience?

At the risk of equivocating, the answer depends, really, on who you are, what you’re building and why. If people consuming your content or using your website for you to fulfill or significantly enhance your core mission, your inability to deliver it can cost tens of thousands — if not hundreds of thousands of dollars in lost opportunities and goodwill brand equity.

Does having a good designer help? Of course, but as Jakob Nielsen points out:

“Design is an inexact science; even if you have a superb designer, not all of his or her ideas will be equally great. It’s only prudent to reduce risk and subject design ideas to a reality check by user testing them with actual customers.”

Pushing your website’s flywheel

Remember: it isn’t about a big splash. The goal isn’t a single, grand transformation, but in putting in the work to make dozens of small improvements over time that accumulate into a significantly better product.

The simplicity of the solution may be deceptive. Like the small pushes on the flywheel, repeated enough times, these steps can produce dramatic results. A competent cross-discipline UX improvement team engages in the following activities:

  1. Decide what to test
  2. Choose the appropriate testing method
  3. Test
  4. Analyze the results
  5. Pick a problem area to fix
  6. Implement fix

If, in the course of creating or redefining a strategy, the company has accurately identified appropriate objectives, metrics and scenarios, testing will always result in some call for change, big or small.

If the tests are designed well and the solutions are executed well, this will mean the website is always getting better. Even if an experiment lands a dud (which will happen every once in awhile), the team is confident that their strategy of continuous monitoring, testing and refining will root even mistakes out at the next or subsequent tests.

What of the rest?

Although Collins didn’t mention it specifically in his book, it’s hard to miss the connection between the other attributes of a “Good to Great” company and a “Good to Great” website as well.

Level-5 Leadership – Humble leaders know that they don’t have all the answers. They understand results are more important than credit and are willing to do what it takes to get results. Thus, they have the humility to admit that the work isn’t done at launch, because they know that they don’t even know what all the questions are, let alone how to answer them.

First Who, Then What – Intelligent, collaborative, adaptable and creative problem solvers always trump strict adherence to bureaucratic rules. A good interdisciplinary UX team will be willing to look at their own processes critically and nimbly change directions if they lose confidence in the results.

Confront the Brutal Facts – Willingness to test ideas and correct ineffective choices prohibit becoming too infatuated with any one idea. It is the hallmark of any study. Ask the hard questions and let the data guide decisions. But always keep the faith that the overall efforts will ultimately be successful.

Hedgehog Concept — The UX improvement team focuses on one issue at a time. Preferably they improve those qualities that fall at the intersection of three things:

  1. What their users are deeply passionate about
  2. What drives their company’s economic engine
  3. What the website can help them accomplish

Culture of Discipline – Every week, twice a month, monthly – whatever interval the company deems appropriate — without fail — test, analyze, change. Without the discipline to keep pushing the flywheel at regular intervals, it’s easy to give up after a few pushes based on the lack of dramatic results.

Technology Accelerators – The nature of today’s age is that technology is always changing. As new tools become available for either a website or a testing method, UX improvement teams should be aware of what technologies will help a company become better at meeting its “hedgehog concept,” online or off.

Given the rapid changes in technology and markets, a website redesign should never be considered “completed.” Even the best teams with the best plans must take time to implement everything. Immediately after launch, the work begins to transform your good website into a great one.

Anyone who has ever been through a major website overhaul understands how tempting it is to catch their breath before you planning the next major redesign. However, in any case where your website plays a critical role in communications or distribution strategies, the best time to start planning to change a website is while still planning the initial redesign.

It doesn’t have to be a major change. In fact, every major change to the exclusion of smaller ones forbids the minor course corrections necessary to refining an idea. More appropriate is a series of smaller changes determined through the disciplined efforts of the right people asking the right questions and addressing the problems those questions uncover… fearlessly.