Can We Measure The Quality of a Healthcare Conference Twitter Stream?

We’ve recently looked at a couple of record-breaking healthcare conferences (one here in the States and the other in Europe) as well as the value of social media for a healthcare conference specifically. The first two mentioned blogs looked at tweet volume, and the third used participant numbers as a metric. Tweet volume is a fun and very easy-to-use comparison metric, but unfortunately it doesn’t really give us much insight into the quality of the conversations taking place.

Measuring healthcare conference tweet quality by quantitative means

If tweet volume as a metric fails to measure quality, are there other quantitative metrics that can? Are there ways to measure this other than reading through all tweets and making a subjective judgement? I’m sure glad we have several excellent curators providing summarized blog posts about the conference and a short list of carefully selected tweets from the conference stream. Human curation can never be replaced.

Healthcare Conference Engagement

Let’s compare some metrics that may tell us something about the social media engagement of healthcare conference participants. We’re going to look at three very interesting conferences that recently took place: #himss12, #TEDxMaastricht and #TEDMED.

Can we measure how many attendees at the healthcare conference were socially active?

Obviously, there is no way for us to measure who was actually physically present and who was attending virtually. Unfortunately (at least for this study), less than 1% of tweets have a geo tag associated with it.

What is a good metric value? It really depends on the type of the conference. #himss12 is not just presentations and sharing of ideas; it’s also partly a very large trade/exhibition event from which one should expect less healthcare social media engagement. #TEDxMaastricht and #TEDMED are both pure idea- and knowledge-sharing conferences, highly conducive for social media sharing.

we should expect the social participation of a conference to exceed the physical participation as a new norm.

I would like to suggest that a 1:1 ratio of physical and social participants should be considered a success for such conferences. As the rapid growth of healthcare social media continues, we should expect the social participation of a conference to exceed the physical participation as a new norm.
healthcare conference social active metric

Engagement distribution in a healthcare conference

We’ve looked at tweet volumes and participation ratios, but what about the distribution of the conversation? Volume and participation metrics can still give us the wrong story if the distribution is highly skewed. The assumption we would like to make here is that conversations that are more equally distributed among the participants are of higher quality compared to a tweet stream that is highly skewed by a few participants with a very high tweet volume. This assumption may not hold true, since the top tweeters by volume may indeed provide much value in all their sharing.

Average Tweets per Participant and Median

One very simple metric is the average tweets per participants, which is easily skewed by the top tweeters. The median, the numeric value separating the higher half of tweets per participants from the lower half, can in many cases give a more accurate picture with less “bias”.
healthcare conference participation average and median

Social Engagement Distribution Quintiles

The average and median are two very simple metrics loved for their simplicity (a single number to compare), but they very often don’t tell the real story. For a better impression of the engagement distribution, we’re going to visualize the percentiles of participant tweets, more specifically the quintiles. By comparing each 20% block of all the participants ranked by tweet volume, we can perhaps more easily compare the equality of conversation distribution across conferences.

healthcare conference participation distribution bar chart and trendline
From the colums above, we can quickly observe that the tweet volume of the top 20% tweeters of the #himss12 and #TEDMED conferences is proportionaly much larger than the next next 20%. There is clearly less difference for the #TEDxMaastricht conferene. This insight becomes a little more clear when we look at the trendlines. Our assumption is that more equally distributed tweet volume is better; in other words a flatter trendline is more desirable. We can visually see that the green line (#TEDxMaastricht) is flatter. That can also be confirmed by observing that the slope of equation behind the trendline is less steep (-2.664x) compared to the other trendline equations.

Pie Chart Visualization of Healthcare Conference Participation Distribution

The best visual for comparing tweet distribution is perhaps good old pie charts. In these pie charts we can see the proportion of tweets from each quintile in the context of the total number of tweets from the whole healthcare conference. Again, we can observe that #TEDxMaastricht had a more equally distributed conversation which may signify a broader participation and perhaps higher quality with less outliers.

healthcare conference participation distribution pie chart

Audun Utengen

Audun Utengen - @audvin

Job to be done: Connect the dots in healthcare social media. Co-founder of @symplur.

What do you get when you combine the world's largest database of social healthcare conversations with the world's largest database of healthcare influencers and power it all by machine learning?

The most fun and effortless path to understanding healthcare social media.

Symplur Signals is used by pharma, medical device companies, governments and research institutions.