Can We Measure The Quality of a Healthcare Conference Twitter Stream?

We’ve recently looked at a couple of record-breaking healthcare conferences (one here in the States and the other in Europe) as well as the value of social media for a healthcare conference specifically. The first two mentioned blogs looked at tweet volume, and the third used participant numbers as a metric. Tweet volume is a fun and very easy-to-use comparison metric, but unfortunately it doesn’t really give us much insight into the quality of the conversations taking place.

Measuring healthcare conference tweet quality by quantitative means

If tweet volume as a metric fails to measure quality, are there other quantitative metrics that can? Are there ways to measure this other than reading through all tweets and making a subjective judgement? I’m sure glad we have several excellent curators providing summarized blog posts about the conference and a short list of carefully selected tweets from the conference stream. Human curation can never be replaced.

Healthcare Conference Engagement

Let’s compare some metrics that may tell us something about the social media engagement of healthcare conference participants. We’re going to look at three very interesting conferences that recently took place: #himss12, #TEDxMaastricht and #TEDMED.

Can we measure how many attendees at the healthcare conference were socially active?

Obviously, there is no way for us to measure who was actually physically present and who was attending virtually. Unfortunately (at least for this study), less than 1% of tweets have a geo tag associated with it.

What is a good metric value? It really depends on the type of the conference. #himss12 is not just presentations and sharing of ideas; it’s also partly a very large trade/exhibition event from which one should expect less healthcare social media engagement. #TEDxMaastricht and #TEDMED are both pure idea- and knowledge-sharing conferences, highly conducive for social media sharing.

we should expect the social participation of a conference to exceed the physical participation as a new norm.

I would like to suggest that a 1:1 ratio of physical and social participants should be considered a success for such conferences. As the rapid growth of healthcare social media continues, we should expect the social participation of a conference to exceed the physical participation as a new norm.
healthcare conference social active metric

Engagement distribution in a healthcare conference

We’ve looked at tweet volumes and participation ratios, but what about the distribution of the conversation? Volume and participation metrics can still give us the wrong story if the distribution is highly skewed. The assumption we would like to make here is that conversations that are more equally distributed among the participants are of higher quality compared to a tweet stream that is highly skewed by a few participants with a very high tweet volume. This assumption may not hold true, since the top tweeters by volume may indeed provide much value in all their sharing.

Average Tweets per Participant and Median

One very simple metric is the average tweets per participants, which is easily skewed by the top tweeters. The median, the numeric value separating the higher half of tweets per participants from the lower half, can in many cases give a more accurate picture with less “bias”.
healthcare conference participation average and median

Social Engagement Distribution Quintiles

The average and median are two very simple metrics loved for their simplicity (a single number to compare), but they very often don’t tell the real story. For a better impression of the engagement distribution, we’re going to visualize the percentiles of participant tweets, more specifically the quintiles. By comparing each 20% block of all the participants ranked by tweet volume, we can perhaps more easily compare the equality of conversation distribution across conferences.

healthcare conference participation distribution bar chart and trendline
From the colums above, we can quickly observe that the tweet volume of the top 20% tweeters of the #himss12 and #TEDMED conferences is proportionaly much larger than the next next 20%. There is clearly less difference for the #TEDxMaastricht conferene. This insight becomes a little more clear when we look at the trendlines. Our assumption is that more equally distributed tweet volume is better; in other words a flatter trendline is more desirable. We can visually see that the green line (#TEDxMaastricht) is flatter. That can also be confirmed by observing that the slope of equation behind the trendline is less steep (-2.664x) compared to the other trendline equations.

Pie Chart Visualization of Healthcare Conference Participation Distribution

The best visual for comparing tweet distribution is perhaps good old pie charts. In these pie charts we can see the proportion of tweets from each quintile in the context of the total number of tweets from the whole healthcare conference. Again, we can observe that #TEDxMaastricht had a more equally distributed conversation which may signify a broader participation and perhaps higher quality with less outliers.

healthcare conference participation distribution pie chart

Audun Utengen

Audun Utengen - @audvin

Sharing research from the Healthcare Social Graph®. Co-founder, CEO of @symplur. Advisor to @MayoClinicSMN. ❤️ #Parkinsons, #Schizophrenia, #PancreaticCancer

104 Responses to “Can We Measure The Quality of a Healthcare Conference Twitter Stream?”

  1. Angela Dunn

    Very interesting analysis, Audun!

    It would also be interesting to analyze content, the number of RTs as a ratio, and the reach of the Twitter handles. TEDMED is a bit of an outlier because they had many physical locations spread throughout the country that had free access to a livestream, but would have been considered virtual.

    If we are looking for quality and the actual spread of ideas, I would also like to see all the tweets eliminated from the analysis that said, “Get this conference to trend now!” :-)

    Reply
    • Audun Utengen

      Thanks so much Angela :)

      Those are excellent ideas. In some ways RTs can be seen as “citations”, so we can use similar citation rankings as in published journal articles etc. I will definitely take a look at that.

      I also have some other ideas for slicing these datasets in hope to discover more insight. Keep the ideas coming!

      Audun

      Reply
  2. Heather N. McKenzie

    RT @PracticalWisdom: RT @hjluks: Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/GtUxaOVZ Good question!

    Reply
  3. Lisa Fields

    RT @hjluks: Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/IzIelNTF Good question!
    cc: @Healthcarewen

    Reply
  4. Lisa Fields

    Wen, This is interesting: Can we measure the Quality of a Healthcare Conference http://t.co/EfACEDap
    @healthcarewen

    Reply
  5. Audun Utengen

    Thank you for your comment @blogbrevity, I've responded :) http://t.co/CmXQ4x9G #hcsm #hcsmeu

    Reply
  6. Harris Lygidakis

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/U6CBhGMC @rigon_sara @rqgb #hcsm

    Reply
  7. Mike Clark

    'Can We Measure The Quality of a Healthcare Conference Twitter Stream?' – looks at recent events http://t.co/kNsDJ7Hq via @symplur #nhssm

    Reply
  8. Steven de Heer

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? Met oa die van #TEDxMaastricht http://t.co/sWPL0QUj via @zite

    Reply
  9. David Thompson

    Can We Measure The Quality of a #Healthcare Conference #Twitter Stream? http://t.co/7Y9rMFRp #hcsm

    Reply
  10. GlobalHealthHub

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/PPfmlXwI #hcsm #hcsmeu

    Reply
  11. Ryan Madanick, MD

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/PPfmlXwI #hcsm #hcsmeu

    Reply
  12. Gene Hobbs

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/PPfmlXwI #hcsm #hcsmeu

    Reply
  13. Ben Nwomeh MD

    +1 RTs equivalent to citations! MT @audvin: Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/mSOBJKwG

    Reply
  14. PhysicianDigital

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/PPfmlXwI #hcsm #hcsmeu

    Reply
  15. rqgb

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/U6CBhGMC @rigon_sara @rqgb #hcsm

    Reply
  16. Bas Idzenga

    RT @StevendeHeer: Can We Measure The Quality of a Healthcare Conference Twitter Stream? Met oa die van #TEDxMaastricht http://t.co/AocPvjr8

    Reply
  17. Eva Alisic

    Interesting for all: MT @lygidakis: Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/TryejzFX #phdchat

    Reply
  18. Lucien Engelen

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/IbMHcyH8 #tedxmaastricht

    Reply
  19. Salmaan Sana

    interesting RT @lucienengelen: Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/8kmiyjEb #tedxmaastricht

    Reply
  20. Nikhil D. Patil

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/PPfmlXwI #hcsm #hcsmeu

    Reply
  21. Nikhil D. Patil

    RT @audvin: Can We Measure The #Quality of a Healthcare Conference Twitter Stream? http://t.co/LwPB4EuO #hcsm #hcsmeu #socialmedia

    Reply
  22. Pam Ressler

    RT @lucienengelen: Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/CTUpdcnC #tedxmaastricht. #hcsm

    Reply
  23. Jake Pringle

    Reading. RT @cmaer: RT @symplur: Trying to measure quality of a Healthcare Conference Twitter Stream? http://t.co/jnSsdmI3 #hcsmca

    Reply
  24. Steve Sisko

    I think it would be hard to every measure true social media value without some sort of adjustment for “qualitative reach” and “qualitative consumption.” By qualitative reach and consumption I mean some sort of adjustment for the “content of the tweet” in relation to the “topic knowledge level” and “perceived authority” of those who are doing the tweeting and those reading the tweets.

    “Content of the tweet” – What is the tweet about?

    “Topic knowledge level” – How pertinent, germane, “useful,” etc. is the tweet topic to sender and receiver?

    “Perceived authority level” – What is the level of skill, experience and “authority” of the sender and/or receiver?

    For instance, I consider myself somewhat knowledgeable about ICD-10 and healthcare IT systems integration. I believe many people follow me because I share that kind of information and have been involved in healthcare IT for a long time. In terms of my followers (my reach), I ‘prune’ XXX, MLM and other worthless-type of followers in an attempt to cleanse my overall “follower quality” – although I’m beginning to question the value of this pruning effort. :’)

    So if I’m at HIMSS12 and I tweet something about ICD-10 then I think the ‘quality’ of that social media interaction – for my followers – should be measured higher than if I’m tweeting tips on how to knit socks from the HIMSS exhibit floor.

    Conversely, if I’m a Vegas hooker cruising the halls and tweeting about ICD-10 to her Secret Service followers, then the value of that social media interaction as it relates to the ICD-10 topic would likewise score lower; regardless of any latent interest said followers may have about ICD-10. And if I, ShimCode, were to tweet about great places to eat in Cartagena Columbia from the HIMSS12 exhibit floor, the value of that social media interaction is essentially worthless – at least to my followers at that point in time.

    So I think it will be very hard to ever truly measure social media value on quantity alone.

    :’)

    Reply
    • Audun Utengen

      Steve, thanks for the excellent insight you’ve shared with us.

      You’ve described a dream scenario :) If we would have the data points you mentioned, we would be able to quantify much more accurately the “quality” of the twitter stream.

      Two of the data points I believe we could be able to get hand of; “Perceived authority level”, and “Topic knowledge level”. Given a large enough database, which we have, this can be extracted.
      However, what I actually perceive as a difficult is to extract the topic of an individual tweet. In so many tweets, without the context, it often impossible to know the true topic of the tweet. Often times, the most valuable tweets, have no topical keywords in the at all.

      Challenges to be solved :)

      Reply
  25. Liesbeth Meijnckens

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/xRhH2CJK via @zite #tedxmaastricht

    Reply
  26. Ali

    Interesting RT @symplur: Trying to measure the quality of a Healthcare Conference Twitter Stream? http://t.co/Y3zpuGz0 #hcsmca

    Reply
  27. Vinny Arora

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/VF5esHYs #hcsm interesting

    Reply
  28. Colleen Kraft

    "@FutureDocs: Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/1CURQWAr #hcsm interesting"

    Reply
  29. Susan Eller

    interesting RT @lucienengelen: Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/8kmiyjEb #tedxmaastricht

    Reply
  30. Camilo Erazo

    Can we measure the Quality of a Healthcare conference Twitter Stream? http://t.co/EejMa4SK @asalinaf @artangelo @claudionavarro #viralizando

    Reply
  31. P. Mimi Poinsett MD

    Can we measure the Quality of a Healthcare conference Twitter Stream? http://t.co/EejMa4SK @asalinaf @artangelo @claudionavarro #viralizando

    Reply
  32. Mark Harmel

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/aWPZQrP7 via @symplur

    Reply
  33. Dr Amer Sharif

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/VF5esHYs #hcsm interesting

    Reply
  34. Armand Krikorian

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/VF5esHYs #hcsm interesting

    Reply
  35. BlueNovius

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/K5MOMIZQ

    Reply
  36. ssmithaahpm

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/sKGINP51 via @symplur #hpm #aamse

    Reply
  37. Elin Silveous

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/8f3dPUX8 #himss12 #TEDMED #HCSM #PCORI

    Reply
  38. Elin Silveous

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/8f3dPUX8 #himss12 #TEDMED #HCSM #PCORI

    Reply
  39. Alfredo Calderon

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/doChbTLv via @symplur #SMCPR

    Reply
  40. Alfredo Calderon

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/doChbTLv via @symplur #SMCPR

    Reply
  41. Alfredo Calderon

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/42gJuesc

    Reply
  42. Alfredo Calderon

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/42gJuesc

    Reply
  43. Alfredo Calderon

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/doChbTLv via @symplur #asppr

    Reply
  44. Suzana Makowski

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/sKGINP51 via @symplur #hpm #aamse

    Reply
  45. Suzana Makowski

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/sKGINP51 via @symplur #hpm #aamse

    Reply
  46. Katherine Burns

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/sKGINP51 via @symplur #hpm #aamse

    Reply
  47. Thomas M. Lee

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/C42EK15z #hcsm #hcsmeu

    Reply
  48. Gustaf von Dewall

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/iyYW5WyP via @symplur

    Reply
  49. Joe Hage

    Nicely done, Audun.

    I wonder if there is something prescriptive here so social participants can get a sense *before* the event which will be best to follow.

    P.S. I could not find a way to enlarge the text as I read. Perhaps you might consider a larger default font?

    Reply
  50. Craig Fukushima

    RT @tmlfox: Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/cuemE0D7 #hcsm #hcsmeu

    Reply
  51. Audun Utengen

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/CmXQ4x9G #hcsm #hcsmeu

    Reply
  52. Wen Dombrowski MD

    Wen, This is interesting: Can we measure the Quality of a Healthcare Conference http://t.co/EfACEDap
    @healthcarewen

    Reply
  53. ClancyClark

    Measure The Quality of a Healthcare Conference? http://t.co/WYSTTYXA

    Reply
  54. Emily Peters

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/hkTTcODQ via @symplur > Love this!

    Reply
  55. Christine Soya

    Some interesting comparisons @audvin: Can We Measure Quality of a Healthcare Conference Twitter Stream? http://t.co/F3l5KkAP #hcsm #hcsmeu

    Reply
  56. Chris Graf

    ;) Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/FHjwS48p #ISMPPAM12

    Reply
  57. Jonathan Bloor

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/INcajb44

    Reply
  58. Lionel REICHARDT

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/VHIYHlqm via @symplur

    Reply
  59. Paul J. Swider

    RT @carimclean: Can we measure the quality of a healthcare conference Twitter stream? http://t.co/f7ETGX6r via @symplur #HIMSS12 >Of course

    Reply
  60. Partha

    RT @carimclean: Can we measure the quality of a healthcare conference Twitter stream? http://t.co/f7ETGX6r via @symplur #HIMSS12 >Of course

    Reply
  61. Shea Steinberg

    Can we measure the quality of a #healthcare conference twitter stream?http://t.co/UV91ydut #socialmedia

    Reply
  62. HL7 Standards

    Can healthcare conferences be measured by its Twitter stream? Interesting insights on @symplur http://t.co/voHd4q9x #HITsm #hcsm

    Reply
  63. Keith W. Boone

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/iZPvundm

    Reply
  64. Jon Mertz

    Fascinating! > Can We Measure The Quality of a Healthcare Conference Twitter Stream? by @symplur http://t.co/VrREMfEL via @carimclean #HITsm

    Reply
  65. Erik Pupo

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/iZPvundm

    Reply
  66. Carla N. Berg

    Can We Measure The Quality of a Healthcare Conf. Twitter Stream? #hcsm #s4pm http://t.co/xN4toFek (via @symplur)

    Reply
  67. Lionel REICHARDT

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/VHIYHlqm via @symplur

    Reply
  68. MissLondres

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/VHIYHlqm via @symplur

    Reply
  69. ANPA

    +1 RTs equivalent to citations! MT @audvin: Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/mSOBJKwG

    Reply
  70. Elizabeth Ruscitto

    Some Fantastic Data. "Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/xR2UDLV2" #himss12 #TedMed

    Reply
  71. Robert L. Oakes

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/PPfmlXwI #hcsm #hcsmeu

    Reply
  72. PharmaDecisions LLC

    RT @hjluks: Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/fPFfUFQI Interesting … thx @symplur

    Reply
  73. Andrea Marlow

    Very interesting… RT @tmlfox Can we measure the quality of a healthcare conference Twitter stream? http://t.co/cJXvrmvs #hcsm #hcsmeu

    Reply
  74. HP Healthcare

    Do you think we can measure the quality of a healthcare conference’s @Twitter stream? http://t.co/RKos1g52 cc @audvin #hcsm #hcsmeu

    Reply
  75. Audun Utengen

    MT @HPHealthcare: Can we measure the quality of a healthcare conference’s tweet stream? http://t.co/jVqzIMDc @audvin #hcsm #hcsmeu

    Reply
  76. Ed Butler

    Can we measure the quality of a healthcare conference’s tweet stream? http://t.co/Uu9L9uyK /via @audvin #hcsmanz

    Reply
  77. H2Online

    Can we measure the quality of a healthcare conference’s tweet stream? http://t.co/ZURlsN02 #hcsm #hcsmeu RT @audvin @HPHealthcare

    Reply
  78. Steve Zonner

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/mKMFZvhi via @symplur

    Reply
  79. Ritesh Patel

    Excellent post – Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/qQX05JNt

    Reply
  80. Ritesh Patel

    Excellent post – Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/8sPYHMWh

    Reply
  81. Stacey Bailey

    Excellent post – Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/8sPYHMWh

    Reply
  82. dearsarah

    Excellent post – Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/8sPYHMWh

    Reply
  83. Health Literacy

    Can We Measure the Quality of Healthcare Conference Twitter Stream? http://t.co/YnC4ERmQ RT @symplur

    Reply
  84. Why_We_Do_That

    Can We Measure the Quality of Healthcare Conference Twitter Stream? http://t.co/6NTDJX7x RT @symplur

    Reply
  85. HIV Insight

    Can We Measure the Quality of Healthcare Conference Twitter Stream? http://t.co/GHNevdNx RT @symplur

    Reply
  86. Louisa Bassant

    Can We Measure the Quality of Healthcare Conference Twitter Stream? http://t.co/GHNevdNx RT @symplur

    Reply
  87. Health Literacy

    Can We Measure the Quality of Healthcare Conference Twitter Stream? http://t.co/YnC07hdG RT @symplur

    Reply
  88. Steve Sisko

    @RossMartin @theEHRGuy Idea: Measuring quality of #HCSM participants/votes (paraphrased) (Via @Symplur) http://t.co/xC6MPbx4 #HIT100 #HITsm

    Reply
  89. Steve Sisko

    .@ehrandhit Some ideas for measuring quality of social media participants & #HIT100 votes next year? http://t.co/xC6MPbx4 #HIT100 #HITsm

    Reply
  90. Thomas M. Lee

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/8wq8TZID #hcsm #hcsmca #hcsmeu #healthcare

    Reply
  91. Dale Loberger

    Interesting.. Can We Measure The Quality of a #Healthcare Conference Twitter Stream? http://t.co/Yngth2Mp #hcsm #hcsmca #hcsmeu via @tmlfox

    Reply
  92. High Performance EMS

    Interesting.. Can We Measure The Quality of a #Healthcare Conference Twitter Stream? http://t.co/Yngth2Mp #hcsm #hcsmca #hcsmeu via @tmlfox

    Reply
  93. MDWebPro

    Nice article. RT @daleloberger: Interesting.. Can We Measure The Quality of a #Healthcare Conference Twitter Stream? http://t.co/EpO9DtX3

    Reply
  94. Regina Holliday

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/xvaY4s1r via @symplur

    Reply
  95. kait b. roe

    RT@ReginaHolliday: Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/Oab3RXae via @symplur

    Reply
  96. Brian Norris

    Can We Measure The Quality of a Healthcare Conference Twitter Stream? http://t.co/xvaY4s1r via @symplur

    Reply
  97. Analyzing Twitter Conversations from Healthcare Conferences

    […] an amazing number! We’ve commented on the growth of Twitter use in healthcare conferences in the past, and we plan to publish some more healthcare social media analytics in the time to come. Conference […]

    Reply

Leave a Reply