Measuring communications and reconciling models, after Amsterdam

The AMEC International Summit on Measurement played out in Amsterdam last week, and I tuned in from afar. On 18th June 2013 I published my thoughts on last year's events in Madrid, and I'll do the same now exactly one year on. Gladly. Gladly because I love the direction the AMEC community is going.

I don't intend to repeat any of the substance and lengthy and valuable commentary to my post last year – which I just enjoyed rereading, thank you. But I have taken the opportunity to append here the Slideshare that accompanied my assertions and that has accrued over three thousand views would you believe.

Perhaps one of my responses to the comments on last year's post is worth noting quickly, a response to Don Bartholomew:

I don't think of myself as a member of the measurement industry for the simple reason that I'm not! Rather, my company is a management consultancy helping organisations benefit from social media and related technologies. Our purview is very much about business performance, about organisational alignment for brilliant execution.

It's not about media

I believe the focus on outcomes in recent years is getting people to look up from media. AMEC is the Association for the Measurement and Evaluation of Communication, not "of Media", and I'd go further than that. Here are some of my core assertions of recent years:

From Attenzi – a social business story:

We don't have media for media's sake but to communicate. And we don't communicate for communication's sake but to influence and to be influenced.

From The Business of Influence:

You have been influenced when you think something you wouldn't otherwise have thought or do something you wouldn't otherwise have done.

From Attenzi – a social business story:

There’s influence in everything an organization does, and sometimes in what it does not do.

From Influence – request for comments:

What is the intended outcome of your marketing and PR campaigns, and the design of your organization overall, if it’s not to get stakeholders to think and behave as you’d like, and to be sensitive to how they’d like you to think and behave?

From The complexity of influence is a challenge – and an opportunity, The Guardian, 15th Feb 2012:

Right now, we have no scalable facility to ascertain or infer who or what caused someone to change their mind or behavior, so how then can we pretend to score an individual’s likelihood to exert that influence, and as if they did so with apparent Newtonian simplicity? We’ve barely even attempted to correlate proxies for influence, assuming that universal correlates even exist.

It is about your organization

As you can see, I believe understanding how hearts, minds and behaviours are changed is central to the success of your organization – it may in fact be the very definition of organization. And it is the primary object of people working in marketing and PR – the likely functional roles of people reading this particular post. Moreover, as the Six Influence Flows portray, this requires a wider and deeper domain of understanding and context than simply seeking to measure and evaluate your "campaigns".

A revised and enhanced AMEC social media measurement framework

Just prior to the summit, Don Bartholomew wrote about the updated AMEC social media measurement framework that Richard Bagnall then presented in Amsterdam. I commented a couple of days later, and I'll make the same observations here.

Exposure, engagement and advocacy are all part and parcel of influence flow, as are non-communications activities such as delivering a great product and service, or indeed being a considerate employer and customer, and the influence system (the flows of influence) contributes to the bottom line business impact.

It’s this complicated cause-and-effect that led me to propose the Influence Scorecard in augmenting the Balanced Scorecard and similar approaches to business performance management, and I noted several references throughout Don's post that indicate a trend towards this philosophy. Notably:

• The cascade down from overarching business objectives
• The reliance on the selection of non-fiscal leading indicators
• The cross-departmental program elements – resonant of the emphasis the Balanced Scorecard and related Strategy Maps place on identifying the organizational, human and informational assets in which to invest to help align the org structure and processes
• Reference to the measurement story – Kaplan and Norton assert that an acid test of a good Balanced Scorecard is that it should ‘tell the story of the business unit’s strategy’.

Measure AND evaluate

Fish and chips. Laurel and Hardy. Park and ride. Some things just go together so naturally we consider them one, but measurement and evaluation are different things, as I explain on the first page of Chapter 6 of The Business of Influence by reference to their respective dictionary definitions:

Evaluation – the making of a judgement about the amount, number, or value of something.

Measurement – the action of measuring something; ascertaining the size, amount, or degree of something by using an instrument or device; assessing the importance, effect, or value of something.

... it appears that ‘evaluation’ doesn’t really add any meaning that ‘measurement’ doesn’t convey alone. Rather than consider it redundant however, I like to think that ‘evaluation’ takes the qualitative role, leaving ‘measurement’ to focus on the quantitative aspects. Measurement and evaluation – quantitative and qualitative.

Professor Jim Macnamara PhD, Professor of Public Communication, University of Technology, Sydney, presented new thinking (PDF) at the International Summit in Amsterdam, and this includes a similar observation, specifically regarding the problems of conducting measurement and evaluation concurrently:

First, evaluation conducted at this point is based on a narrow and limited range of data. Evaluation fused on to or conducted concurrently with measurement is focussed exclusively on metrics collected in the measurement processes undertaken by the organization – what is referred to as endogenous data (inside data). It does not consider other information that might be available and useful, referred to as exogenous data...

Second, and even more fundamentally, while they are related, measurement and evaluation are two quite different processes. Measurement is the taking of measures such as counting items, collecting ratings on a scale, or recording comments in interviews, and analyzing these. Measurement involves two key stages – data collection and data analysis.

Prof Macnamara insists that measurement should collect qualitative data as well as quantitative data, so different semantics to my conclusion, but to identical effect I believe.

The MAIE model

Further, Prof Macnamara proceeds to champion the "forward looking", marrying directly in my opinion with the leading indicators of the Influence Scorecard. In fact, I find it difficult to identify any significant differences between his MAIE (measurement, analysis, insights, evaluation) model and the Influence Scorecard, although I would add that the Scorecard, by augmenting business performance management approaches such as the Balanced Scorecard, goes further by being ready-made to plug into the wider organizational context. This is critical as more organizations appreciate that the structure of the 20th Century organization – with the dominant functional silos we still see manifest today – is no longer fit for the 21st.

Everyone is in communications. Everyone is in the business of influence.

I wholeheartedly recommend you read A New Paradigm and Model for Measurement and Evaluation (PDF), not just because it corroborates my outlook, but because it isn't informed by my outlook (my work isn't referenced). This means it includes slightly different ways of justifying similar things, of describing a similar view of the world. Therefore, I too will be doing my best to learn more about it.

Connecting models and sense making

The buzz around Prof Macnamara's contribution betrayed a general feeling that MAIE is an alternative lens on our problem space here to the AMEC social media measurement framework. However, in hoping to show here the commonalities between each and the Influence Scorecard, I hope I've shown that they aren't irreconcilable at all. I also hope I've encouraged those who share our fascination for this area to pick up a copy of The Business of Influence :-)

In Attenzi – a social business story, I was able to reframe subtly the context in which business performance management is viewed. While I have never considered it the sole domain of the command-and-control organizational hierarchy, far from it in fact, some do. I was able then to present the measurement and evaluation process discussed here as sense making, as a critical sensitizing feedback loop in the continuous evolution of the organization, as supportive of emergent strategy in striving for greater organizational adaptability than traditional deliberate strategy approaches allow.

If this presses your buttons, you might like this conversation between Adam Pisoni, Stowe Boyd and me, and indeed my recent video presentation, The Future of Organization. For now, I said up top that I'd conclude with this slidestack, so here it is:

[iamsterdam Photo by Neil Thompson, CC BY-SA.]