#SXSWi Session Notes: Metrics-Driven Design

Here are my raw notes from this session.Presenter tells the story of Doug Bowman leaving Google because of the 41 shades of blue type testing. Data showed that there certainly was a difference in how the blues were used. The click through rate for green content is less than the click through rate for blue links. Data suggests that the darker the link, the better the click through.Bing's blue was worth $80 million to them. These discussions matter.Intuition driven design verses data driven design. Most designers are intuition driven. Aesthetics are how they sell their work.When it is data driven, every design decision is tested. You don't do things like trust your gut.Reasons metrics are a designer's best friend:1. Metrics reduce arguments based on opinion.2. Metrics give you answers about what really works.3. Metrics show you where you are strong (and weak) as a designer.4. Metrics allow you to test anything you want.5. Clients love metrics.Which metrics to use? They will be as unique as your business.Google Analytics are high level views; there is not any actionable data for design decisions. They are vanity metrics, like hit counters of the 90s.The Usage Lifecycle: for example, interested, trial/beta user, customer, passionate customer (varies depending on your customer and business). And there are stages/hurdles in between each one: acquisition, conversion, engagement, satisfaction.With metrics, you are measuring how you move people through this cycle.Acquisition Metrics:How much didn't cost for you to acquire that person? What is the lifetime value of that customer? And, how do these two compare.Performance acquisition metrics: comparative metrics, revenue by channel, revenue by acquisition. Email is the best performing acquisition channel.Conversion funnel analysis: How many people make it through each step/page of the conversion and where are you seeing the drop off and how does it need to be fixed to increase conversion?Engagement Metrics:HitsPage viewsVisitsUnique visitorsReturning visitorsRegistered usersCustomersFrequencyTime on siteDaily active usersCohort analysis: break up your users based on segments when they started. So you can see if engagement is improving over time.On Facebook, a design change on the deactivation page prevented 1 million people from not leaving the site. They added a message that said: these people are going to miss you.Satisfaction Metrics:Referral through Net Promotors Score. This one question is a very good indicator of how satisfied your customers are: how likely are you to recommend this product/service to a friend?Emergent Metrics:Once a FriendFeed user found five friends, they became very active users. Once FriendFeed understood that, they made a design change to integrate a find friends tool. It was part of the stream.At Blogger, they looked at the number of posts as a key metric.Look for the one thing that really drives a lot of other activity. (Like a tipping point.)