Following on from my post last week on the burning need to investigate the effectiveness of constitutional approaches of TCM, we come to a critical part of this kind of investigation, the reliability of diagnoses.
Most people are under the impression that diagnostic reliability has already been investigated, but it remains one of the most under researched areas in all health modalities.
Almost all published diagnostic reliability studies investigate specific diseases. See (O’Brien 2009) for a review in the TCM context on this topic. Sadly, very often these studies use the wrong statistics (Popplewell 2018), so very little at all is known in this important field.
An open population is one that is fully inclusive and is not restricted to a single or small numbers of diseases or diagnostic options.
The published research carried out as part of my PhD (Popplewell 2015) appears to be the first attempt to try to measure diagnostic reliability in open populations in any modality.
Diagnostic reliability in an open population context is meant to quantify how reliable and repeatable practitioners diagnose in real world clinical settings and is a crucial consideration for quality registry data.
My first published study (Popplewell MC 2018) should be a cause of grave concern. We reported diagnostic reliability of a mere 19% in open population settings, using a semi-abbreviated list of 57 contemporary TCM diagnostic options. If the ICD-11 was used (a far more complex set of diagnostic choices), the result would have to have been even worse.
A novel format for recording TCM diagnoses called the Traditional Chinese Medical Diagnostic Descriptor (or TCMDD) was developed and trialled, as part of my PhD. This format facilitated 80% inter-rater agreement, with little loss in diagnostic detail (Popplewell MC 2018).
I am very proud to say that Stephen Birch, a leading academic in Oriental medicine, wrote a positive commentary on my papers (Birch 2019).
For this reason, the TCMDD is the diagnostic format used within the TCMCR. We are however not blindly wedded to this diagnostic format; if new approaches are found that further improve diagnostic reliability, we will enthusiastically adopt them.
To be confidently draw any conclusions and recommendations regarding TCM clinical practice, we need to use strategies that enable the highest levels of diagnostic reliability possible in real-world settings.
Any registry that does not pay particular attention to the diagnostic reliability issue is likely to provide misleading conclusions.
References
Birch, S. (2019). “Inter-Rater Agreement in Traditional Chinese Medicine: On the Potential Contribution of Popplewell’s Work.” J Altern Complement Med 25(11): 1077-1079.
O’Brien, K., .Birch, S. (2009). “A Review of the Reliability of Traditional East Asian Medicine Diagnosis.” The Journal of Alternative and Complimentary Medicine 15(4): 353-366.
Popplewell, M., Reizes J, Zaslawski C. (2018). “Appropriate Statistics for Determining Chance Removed Inter-practitioner Agreement ” Journal of Alternative and Complementary Medicine
Popplewell, M. C. (2015). Improving diagnostic reliability in Chinese medicine. Sydney, University of Technology, Sydney. PhD: 295.
Popplewell MC, R. J., Zaslawski C. (2018). “Consensus in Traditional Chinese Medical Diagnosis in Open Populations.” Journal of Alternative and Complementary Medicine.
Popplewell MC, R. J., Zaslawski C. (2018). “An Effective System to Describe Traditional Chinese Medical Conditions:
The ‘Traditional Chinese Medical Diagnostic Descriptor’.” Journal of Alternative and Complementary Medicine.
