We’ve recently looked at a couple of record-breaking healthcare conferences (one here in the States and the other in Europe) as well as the value of social media for a healthcare conference specifically. The first two mentioned blogs looked at tweet volume, and the third used participant numbers as a metric. Tweet volume is a fun and very easy-to-use comparison metric, but unfortunately it doesn’t really give us much insight into the quality of the conversations taking place.
Measuring healthcare conference tweet quality by quantitative means
If tweet volume as a metric fails to measure quality, are there other quantitative metrics that can? Are there ways to measure this other than reading through all tweets and making a subjective judgement? I’m sure glad we have several excellent curators providing summarized blog posts about the conference and a short list of carefully selected tweets from the conference stream. Human curation can never be replaced.
Healthcare Conference Engagement
Let’s compare some metrics that may tell us something about the social media engagement of healthcare conference participants. We’re going to look at three very interesting conferences that recently took place: #himss12, #TEDxMaastricht and #TEDMED.
Can we measure how many attendees at the healthcare conference were socially active?
Obviously, there is no way for us to measure who was actually physically present and who was attending virtually. Unfortunately (at least for this study), less than 1% of tweets have a geo tag associated with it.
What is a good metric value? It really depends on the type of the conference. #himss12 is not just presentations and sharing of ideas; it’s also partly a very large trade/exhibition event from which one should expect less healthcare social media engagement. #TEDxMaastricht and #TEDMED are both pure idea- and knowledge-sharing conferences, highly conducive for social media sharing.
we should expect the social participation of a conference to exceed the physical participation as a new norm.
I would like to suggest that a 1:1 ratio of physical and social participants should be considered a success for such conferences. As the rapid growth of healthcare social media continues, we should expect the social participation of a conference to exceed the physical participation as a new norm.
Engagement distribution in a healthcare conference
We’ve looked at tweet volumes and participation ratios, but what about the distribution of the conversation? Volume and participation metrics can still give us the wrong story if the distribution is highly skewed. The assumption we would like to make here is that conversations that are more equally distributed among the participants are of higher quality compared to a tweet stream that is highly skewed by a few participants with a very high tweet volume. This assumption may not hold true, since the top tweeters by volume may indeed provide much value in all their sharing.
Average Tweets per Participant and Median
One very simple metric is the average tweets per participants, which is easily skewed by the top tweeters. The median, the numeric value separating the higher half of tweets per participants from the lower half, can in many cases give a more accurate picture with less “bias”.
Social Engagement Distribution Quintiles
The average and median are two very simple metrics loved for their simplicity (a single number to compare), but they very often don’t tell the real story. For a better impression of the engagement distribution, we’re going to visualize the percentiles of participant tweets, more specifically the quintiles. By comparing each 20% block of all the participants ranked by tweet volume, we can perhaps more easily compare the equality of conversation distribution across conferences.
From the colums above, we can quickly observe that the tweet volume of the top 20% tweeters of the #himss12 and #TEDMED conferences is proportionaly much larger than the next next 20%. There is clearly less difference for the #TEDxMaastricht conferene. This insight becomes a little more clear when we look at the trendlines. Our assumption is that more equally distributed tweet volume is better; in other words a flatter trendline is more desirable. We can visually see that the green line (#TEDxMaastricht) is flatter. That can also be confirmed by observing that the slope of equation behind the trendline is less steep (-2.664x) compared to the other trendline equations.
Pie Chart Visualization of Healthcare Conference Participation Distribution
The best visual for comparing tweet distribution is perhaps good old pie charts. In these pie charts we can see the proportion of tweets from each quintile in the context of the total number of tweets from the whole healthcare conference. Again, we can observe that #TEDxMaastricht had a more equally distributed conversation which may signify a broader participation and perhaps higher quality with less outliers.
104 Responses to “Can We Measure The Quality of a Healthcare Conference Twitter Stream?”
Angela Dunn
Very interesting analysis, Audun!
It would also be interesting to analyze content, the number of RTs as a ratio, and the reach of the Twitter handles. TEDMED is a bit of an outlier because they had many physical locations spread throughout the country that had free access to a livestream, but would have been considered virtual.
If we are looking for quality and the actual spread of ideas, I would also like to see all the tweets eliminated from the analysis that said, “Get this conference to trend now!” :-)
Audun Utengen
Thanks so much Angela :)
Those are excellent ideas. In some ways RTs can be seen as “citations”, so we can use similar citation rankings as in published journal articles etc. I will definitely take a look at that.
I also have some other ideas for slicing these datasets in hope to discover more insight. Keep the ideas coming!
Audun
Heather N. McKenzie
RT @PracticalWisdom: RT @hjluks: Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/GtUxaOVZ Good question!
Lisa Fields
RT @hjluks: Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/IzIelNTF Good question!
cc: @Healthcarewen
Lisa Fields
Wen, This is interesting: Can we measure the Quality of a Healthcare Conference http://t.co/EfACEDap
@healthcarewen
Audun Utengen
Thank you for your comment @blogbrevity, I've responded :) http://t.co/CmXQ4x9G #hcsm #hcsmeu
Harris Lygidakis
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/U6CBhGMC @rigon_sara @rqgb #hcsm
Mike Clark
'Can We Measure The Quality of a Healthcare Conference Twitter Stream?' – looks at recent events http://t.co/kNsDJ7Hq via @symplur #nhssm
Steven de Heer
Can We Measure The Quality of a Healthcare Conference Twitter Stream? Met oa die van #TEDxMaastricht http://t.co/sWPL0QUj via @zite
David Thompson
Can We Measure The Quality of a #Healthcare Conference #Twitter Stream? http://t.co/7Y9rMFRp #hcsm
GlobalHealthHub
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/PPfmlXwI #hcsm #hcsmeu
Ryan Madanick, MD
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/PPfmlXwI #hcsm #hcsmeu
Gene Hobbs
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/PPfmlXwI #hcsm #hcsmeu
Ben Nwomeh MD
+1 RTs equivalent to citations! MT @audvin: Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/mSOBJKwG
PhysicianDigital
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/PPfmlXwI #hcsm #hcsmeu
rqgb
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/U6CBhGMC @rigon_sara @rqgb #hcsm
Bas Idzenga
RT @StevendeHeer: Can We Measure The Quality of a Healthcare Conference Twitter Stream? Met oa die van #TEDxMaastricht http://t.co/AocPvjr8
Eva Alisic
Interesting for all: MT @lygidakis: Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/TryejzFX #phdchat
Lucien Engelen
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/IbMHcyH8 #tedxmaastricht
Salmaan Sana
interesting RT @lucienengelen: Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/8kmiyjEb #tedxmaastricht
Nikhil D. Patil
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/PPfmlXwI #hcsm #hcsmeu
Nikhil D. Patil
RT @audvin: Can We Measure The #Quality of a Healthcare Conference Twitter Stream? http://t.co/LwPB4EuO #hcsm #hcsmeu #socialmedia
Pam Ressler
RT @lucienengelen: Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/CTUpdcnC #tedxmaastricht. #hcsm
Jake Pringle
Reading. RT @cmaer: RT @symplur: Trying to measure quality of a Healthcare Conference Twitter Stream? http://t.co/jnSsdmI3 #hcsmca
Steve Sisko
I think it would be hard to every measure true social media value without some sort of adjustment for “qualitative reach” and “qualitative consumption.” By qualitative reach and consumption I mean some sort of adjustment for the “content of the tweet” in relation to the “topic knowledge level” and “perceived authority” of those who are doing the tweeting and those reading the tweets.
“Content of the tweet” – What is the tweet about?
“Topic knowledge level” – How pertinent, germane, “useful,” etc. is the tweet topic to sender and receiver?
“Perceived authority level” – What is the level of skill, experience and “authority” of the sender and/or receiver?
For instance, I consider myself somewhat knowledgeable about ICD-10 and healthcare IT systems integration. I believe many people follow me because I share that kind of information and have been involved in healthcare IT for a long time. In terms of my followers (my reach), I ‘prune’ XXX, MLM and other worthless-type of followers in an attempt to cleanse my overall “follower quality” – although I’m beginning to question the value of this pruning effort. :’)
So if I’m at HIMSS12 and I tweet something about ICD-10 then I think the ‘quality’ of that social media interaction – for my followers – should be measured higher than if I’m tweeting tips on how to knit socks from the HIMSS exhibit floor.
Conversely, if I’m a Vegas hooker cruising the halls and tweeting about ICD-10 to her Secret Service followers, then the value of that social media interaction as it relates to the ICD-10 topic would likewise score lower; regardless of any latent interest said followers may have about ICD-10. And if I, ShimCode, were to tweet about great places to eat in Cartagena Columbia from the HIMSS12 exhibit floor, the value of that social media interaction is essentially worthless – at least to my followers at that point in time.
So I think it will be very hard to ever truly measure social media value on quantity alone.
:’)
Audun Utengen
Steve, thanks for the excellent insight you’ve shared with us.
You’ve described a dream scenario :) If we would have the data points you mentioned, we would be able to quantify much more accurately the “quality” of the twitter stream.
Two of the data points I believe we could be able to get hand of; “Perceived authority level”, and “Topic knowledge level”. Given a large enough database, which we have, this can be extracted.
However, what I actually perceive as a difficult is to extract the topic of an individual tweet. In so many tweets, without the context, it often impossible to know the true topic of the tweet. Often times, the most valuable tweets, have no topical keywords in the at all.
Challenges to be solved :)
Liesbeth Meijnckens
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/xRhH2CJK via @zite #tedxmaastricht
Ali
Interesting RT @symplur: Trying to measure the quality of a Healthcare Conference Twitter Stream? http://t.co/Y3zpuGz0 #hcsmca
Vinny Arora
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/VF5esHYs #hcsm interesting
Colleen Kraft
"@FutureDocs: Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/1CURQWAr #hcsm interesting"
Susan Eller
interesting RT @lucienengelen: Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/8kmiyjEb #tedxmaastricht
Camilo Erazo
Can we measure the Quality of a Healthcare conference Twitter Stream? http://t.co/EejMa4SK @asalinaf @artangelo @claudionavarro #viralizando
P. Mimi Poinsett MD
Can we measure the Quality of a Healthcare conference Twitter Stream? http://t.co/EejMa4SK @asalinaf @artangelo @claudionavarro #viralizando
Mark Harmel
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/aWPZQrP7 via @symplur
Luis Avila
@HannaIsBack mira lo que me mandaron :3 http://t.co/STmHwOAo
Luis Avila
@claudionavarro cachaste esto? http://t.co/ShQY9Etf
Dr Amer Sharif
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/VF5esHYs #hcsm interesting
Armand Krikorian
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/VF5esHYs #hcsm interesting
BlueNovius
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/K5MOMIZQ
ssmithaahpm
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/sKGINP51 via @symplur #hpm #aamse
Elin Silveous
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/8f3dPUX8 #himss12 #TEDMED #HCSM #PCORI
Elin Silveous
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/8f3dPUX8 #himss12 #TEDMED #HCSM #PCORI
Alfredo Calderon
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/doChbTLv via @symplur #SMCPR
Alfredo Calderon
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/doChbTLv via @symplur #SMCPR
Alfredo Calderon
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/42gJuesc
Alfredo Calderon
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/42gJuesc
Alfredo Calderon
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/doChbTLv via @symplur #asppr
Suzana Makowski
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/sKGINP51 via @symplur #hpm #aamse
Suzana Makowski
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/sKGINP51 via @symplur #hpm #aamse
Katherine Burns
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/sKGINP51 via @symplur #hpm #aamse
Thomas M. Lee
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/C42EK15z #hcsm #hcsmeu
Gustaf von Dewall
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/iyYW5WyP via @symplur
Joe Hage
Nicely done, Audun.
I wonder if there is something prescriptive here so social participants can get a sense *before* the event which will be best to follow.
P.S. I could not find a way to enlarge the text as I read. Perhaps you might consider a larger default font?
Craig Fukushima
RT @tmlfox: Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/cuemE0D7 #hcsm #hcsmeu
Audun Utengen
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/CmXQ4x9G #hcsm #hcsmeu
Audun Utengen
@ShimCode Thanks for your comment! I've responded. http://t.co/CmXQ4x9G @blogbrevity
Wen Dombrowski MD
Wen, This is interesting: Can we measure the Quality of a Healthcare Conference http://t.co/EfACEDap
@healthcarewen
Wen Dombrowski MD
#HCSM #infographic http://t.co/8bb6tEiA @TEDMED @HIMSS by @Audvin via @PracticalWisdom cc @aahcp @nychbl @mayoclinic @larrylin @ctsinclair
ClancyClark
Measure The Quality of a Healthcare Conference? http://t.co/WYSTTYXA
Emily Peters
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/hkTTcODQ via @symplur > Love this!
Christine Soya
Some interesting comparisons @audvin: Can We Measure Quality of a Healthcare Conference Twitter Stream? http://t.co/F3l5KkAP #hcsm #hcsmeu
Chris Graf
;) Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/FHjwS48p #ISMPPAM12
Jonathan Bloor
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/INcajb44
Lionel REICHARDT
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/VHIYHlqm via @symplur
Paul J. Swider
RT @carimclean: Can we measure the quality of a healthcare conference Twitter stream? http://t.co/f7ETGX6r via @symplur #HIMSS12 >Of course
Partha
RT @carimclean: Can we measure the quality of a healthcare conference Twitter stream? http://t.co/f7ETGX6r via @symplur #HIMSS12 >Of course
Shea Steinberg
Can we measure the quality of a #healthcare conference twitter stream?http://t.co/UV91ydut #socialmedia
HL7 Standards
Can healthcare conferences be measured by its Twitter stream? Interesting insights on @symplur http://t.co/voHd4q9x #HITsm #hcsm
Keith W. Boone
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/iZPvundm
Jon Mertz
Fascinating! > Can We Measure The Quality of a Healthcare Conference Twitter Stream? by @symplur http://t.co/VrREMfEL via @carimclean #HITsm
Erik Pupo
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/iZPvundm
Carla N. Berg
Can We Measure The Quality of a Healthcare Conf. Twitter Stream? #hcsm #s4pm http://t.co/xN4toFek (via @symplur)
Lionel REICHARDT
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/VHIYHlqm via @symplur
MissLondres
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/VHIYHlqm via @symplur
ANPA
+1 RTs equivalent to citations! MT @audvin: Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/mSOBJKwG
Elizabeth Ruscitto
Some Fantastic Data. "Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/xR2UDLV2" #himss12 #TedMed
Robert L. Oakes
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/PPfmlXwI #hcsm #hcsmeu
PharmaDecisions LLC
RT @hjluks: Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/fPFfUFQI Interesting … thx @symplur
Andrea Marlow
Very interesting… RT @tmlfox Can we measure the quality of a healthcare conference Twitter stream? http://t.co/cJXvrmvs #hcsm #hcsmeu
HP Healthcare
Do you think we can measure the quality of a healthcare conference’s @Twitter stream? http://t.co/RKos1g52 cc @audvin #hcsm #hcsmeu
Audun Utengen
MT @HPHealthcare: Can we measure the quality of a healthcare conference’s tweet stream? http://t.co/jVqzIMDc @audvin #hcsm #hcsmeu
Ed Butler
Can we measure the quality of a healthcare conference’s tweet stream? http://t.co/Uu9L9uyK /via @audvin #hcsmanz
H2Online
Can we measure the quality of a healthcare conference’s tweet stream? http://t.co/ZURlsN02 #hcsm #hcsmeu RT @audvin @HPHealthcare
Steve Zonner
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/mKMFZvhi via @symplur
CLOUDHealth
Interesting #socmed analysis from @symplur of #TEDMED #HIMSS12 & #tedxmasstricht http://t.co/mpMFD87W | #HITsm #hcsm
Ritesh Patel
Excellent post – Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/qQX05JNt
Ritesh Patel
Excellent post – Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/8sPYHMWh
Stacey Bailey
Excellent post – Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/8sPYHMWh
dearsarah
Excellent post – Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/8sPYHMWh
Health Literacy
Can We Measure the Quality of Healthcare Conference Twitter Stream? http://t.co/YnC4ERmQ RT @symplur
Why_We_Do_That
Can We Measure the Quality of Healthcare Conference Twitter Stream? http://t.co/6NTDJX7x RT @symplur
HIV Insight
Can We Measure the Quality of Healthcare Conference Twitter Stream? http://t.co/GHNevdNx RT @symplur
Louisa Bassant
Can We Measure the Quality of Healthcare Conference Twitter Stream? http://t.co/GHNevdNx RT @symplur
Health Literacy
Can We Measure the Quality of Healthcare Conference Twitter Stream? http://t.co/YnC07hdG RT @symplur
Steve Sisko
@RossMartin @theEHRGuy Idea: Measuring quality of #HCSM participants/votes (paraphrased) (Via @Symplur) http://t.co/xC6MPbx4 #HIT100 #HITsm
Steve Sisko
.@ehrandhit Some ideas for measuring quality of social media participants & #HIT100 votes next year? http://t.co/xC6MPbx4 #HIT100 #HITsm
Thomas M. Lee
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/8wq8TZID #hcsm #hcsmca #hcsmeu #healthcare
Dale Loberger
Interesting.. Can We Measure The Quality of a #Healthcare Conference Twitter Stream? http://t.co/Yngth2Mp #hcsm #hcsmca #hcsmeu via @tmlfox
High Performance EMS
Interesting.. Can We Measure The Quality of a #Healthcare Conference Twitter Stream? http://t.co/Yngth2Mp #hcsm #hcsmca #hcsmeu via @tmlfox
MDWebPro
Nice article. RT @daleloberger: Interesting.. Can We Measure The Quality of a #Healthcare Conference Twitter Stream? http://t.co/EpO9DtX3…
Regina Holliday
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/xvaY4s1r via @symplur
kait b. roe
RT@ReginaHolliday: Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/Oab3RXae via @symplur
Brian Norris
Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/xvaY4s1r via @symplur
Analyzing Twitter Conversations from Healthcare Conferences
[…] an amazing number! We’ve commented on the growth of Twitter use in healthcare conferences in the past, and we plan to publish some more healthcare social media analytics in the time to come. Conference […]