Social media measurement, after Madrid

What, exactly, is the value of social? This was the question I sought to help answer in my slidestack ahead of the AMEC European Summit in Madrid earlier this month. And it was the overarching question that informed much of the three days of debate, discussion and deliberation.

This post is about two related developments – the latest from "The Conclave" (aka the #SMMStandards Coalition), and "A New Framework for Social Media Metrics and Measurement".

Measurement standards

"Perhaps the most important Social Media launch of the year" is how Katie Delahaye Paine portrays it. This is so-Katie that I can actually hear her saying it right now (as she might hear me cry "the most exciting development in PR since the Cluetrain"!)

Katie refers to a suite of social media measurement standards that represents the work of a collection of organisations (including AMEC, a full list is appended here) informally referred to as The Conclave. Following 18 months of long conference-calls, meetings, slidestacks and email threads, we have posted standards for:

1. Content and Sourcing
2. Reach and Impressions
3. Engagement and Conversation
4. Influence
5. Opinion & Advocacy
6. Impact & Value

Congratulations to everyone involved! A PDF of the definitions and status is available here. Comments remain open until the end of July, so dive in! I'll be feeding back some of the points I've stumbled across in considering a new framework...

A new framework

Having consistent definitions represents significant progress. It reminds me of the progress the Web Analytics Association (now the DAA) made more than a decade ago in formalising the analysis of website performance. And indeed of every other more established profession, not least my first love – engineering.

But what to do next?

The Oxford English defines a framework as a basic structure underlying a system, concept, or text. Now that we're close to having the component parts, subject to comments, we can turn our attention to the framework for their assembly so to speak.

Don Bartholomew and Richard Bagnall took on the mantle of reviewing the current landscape and presented their recommendations for a new framework at the AMEC summit. As Don writes in his accompanying blog post, their work seeks to answer the question: How do we take the standards work coming from The Conclave and operationalize it to create proper social media measurement?

Readers of my blog and my book The Business of Influence will know that I advocate integration of social media measurement into business performance management (BPM), of which the Balanced Scorecard is the dominant framework. If you're not familiar with this management approach, I recommend my slidestack for the Madrid summit, "What, exactly, is the value of social?", and perhaps my May 2013 ebook Attenzi - a social business story.

If you want an even shorter explanation of my advocacy, it goes like this – why reinvent the wheel?

So you can imagine my interest in the framework proposed by Don and Richard was piqued when the only mention BPM secured in Don's post is as follows: "There are certainly other ways to think about this (e.g. Business Performance Management) and we intend to possibly add others based on industry feedback and suggestions."

Remaining an advocate of BPM as the logical conclusion to this debate, I will review this proposed framework in that light. Here goes...

Comments on the proposed new framework

It's worth noting Don's clarification of the difference here between model and framework: "When we use the word model, we are referring to a representation of a system, in this case social media. In the original Valid Metrics Framework, the model used was the traditional sales funnel. A framework adds additional dimensions to the model and is operationalized with metrics."

1. Associated models

First up, I couldn't agree more heartily with the conclusion that the sales funnel is an inappropriate model in this context. As Don writes, it's "not the best way to model common uses of social media like customer relations and building relationships with stakeholder groups". And as I wrote in The Business of Influence, "the application of the marketing funnel to the horizontal axis [of the existing AMEC framework] may turn out to be a weakness in its ability to help to identify appropriate metrics if ‘action’ is translated simply in terms of ringing up the cash register."

However, there appears then to be a contradiction, at least in emphasis. Having referenced the criticality of 'social' when it comes to all stakeholder groups, Don and Richard pick out Forrester’s Customer Lifecycle and McKinsey’s Customer Journey models in the same paragraph as the phrase "social media marketing"; three references to a continued focus on the customer above all other stakeholders. I say there appears to be a contradiction, but fortunately the resulting model does not reference the customer exclusively. I fear however that the couching of the model may lead many to read it with the customer solely in mind.

Here's the proposed new model:

2. Audience

The model references the "audience". It quotes the new measurement standards definition of impact – "effect of a social media campaign, program or effort on the target audience". It also qualifies exposure – "create potential audience exposure".

I know I can be pedantic when it comes to words and their definitions, but our thoughts and behaviours are influenced by the very language we employ. In this instance, the Oxford English explains that audience means "the assembled spectators or listeners at a public event". With this in mind, I always think references to audience in the context of social media are inappropriate. It's only appropriate in terms of broadcast. I usually defer to the word stakeholders instead, and some prefer publics or constituencies (ie, those having a say). And at the end of the day perhaps "people" or "people with an interest or that matter" says it all.

3. Influence

The model clarifies "influence" by reference to the definition in the measurement standards, but the measurement standards as they stand only recognise the top level definition of influence presented in WOMMA's Influencer Guidebook, and not the accompanying qualification: "There are two distinct states of influencer measurement that are relative to the point in time an influencer marketing program begins: 1. The potential to influence (before); 2. Actual, observed influence (during/after)."

As I wrote in my post about the guidebook, I'd prefer to emphasise the second of these over the first if only for the simple reason that it's the very outcome that so-called influencer marketers seek. Indeed, what is the intended outcome of your marketing (customer) and PR (all stakeholders) campaigns, and the design of your organisation overall, if it's not to get stakeholders to think and behave as you'd like, and to be sensitive to how they'd like you to think and behave? Put like this, influence is the final outcome of any model in the context here, the consequence of which creates mutual value (and business profit, in the context of for-profit entities), or not.

4. Value

Under "impact" the proposed model defines value as "the financial impact". This is counter to the current standards documentation, which, following some last minute tweaks admittedly, defines value as "the importance, worth, or usefulness of something." ROI is one but not the sole way to express the value of something.

5. Advocacy

In my view, any model that acknowledges the possibility of advocacy emerging must also recognise opposition and advocacy for competing agenda. I also don't quite understand why advocacy comes after impact.

6. Programmatic-level, channel-specific and business perspectives

Here's the proposed new framework:

It recognises the need for tiered metrics ("programmatic-level, channel-specific and business perspectives") – a similarity it shares with or perhaps borrows from business performance management. Interestingly, the Balanced Scorecard also talks about perspectives, although with an emphasis on aligning human, information and organizational assets, processes and corresponding performance metrics with strategy.

6. Paid Owned Earned

Don writes that an alternative version of the framework employs the Paid Owned Earned media (POEm) taxonomy to address the framework's 'phases', and that this will be published shortly. The model also references "owned" and "earned" under "engagement".

I dislike this taxonomy for the simple reason that I haven't seen it contribute any strategic value. In fact, all I have seen it do is prop up 20th Century organizational silos and reinforce misconceptions (eg, that PR is focused on earned / unpaid media when in fact it's agnostic in this respect). See "The Influence View of Content" for more on this.

I wouldn't mind then if this alternative version didn't surface, but I do seem to be alone in my dislike of POEm.

7. Impact

The standards assert that "impact and value represent the ultimate outcome of a social media effort." I don't believe then that any framework can consider programmatic-level or channel-specific metrics in terms of impact. It just doesn't make sense to me.

The influence scorecard

Don, Richard and I have spoken at various lengths about this topic, and indeed the Influence Scorecard framework I have outlined to connect social media measurement into the Balanced Scorecard and similar BPM frameworks. Blog posts are a great way to exchange ideas and air points of view, and I hope such "working out loud" continues, but I wonder if AMEC and The Conclave might not host a workshop on just this topic.

Mr. Leggetter? Katie?

---

Members of the Conclave

• Association for the Measurement and Evaluation of Communication
• The Chartered Institute of Public Relations
• The Institute for Public Relations
• The Public Relations Society of America
• The Council of PR Firms
• The Global Alliance for Public Relations
• The International Association of Business Communicators
• The Society for New Communications Research
• The Digital Analytics Association (previously the WAA)
• The Word of Mouth Marketing Association
• The Advertising Research Foundation
• Federation Internationale des Bureauxs d'Extraits de Press.

The Conclave's #SMMstandards initiative also includes:

• The American Association of Advertising Agencies
• The Association of National Advertisers
• The Interactive Advertising Bureau

and the following "client organizations":

• Dell
• GM
• McDonalds
• Ford
• P&G
• SAS
• Southwest Airlines
• Thomson Reuters.

---

21st June 2013

Don posts a reply on his blog: New Framework for Social Media Measurement – Update & Debate.