Why It Matters
Once again, the folks at OneDrop are trying to pass off self-reported data as proof their way cool whiz bang toy produces better outcomes. This time it comes from a “study” published in JMIR MHEALTH AND UHEALTH. According to this “study”:
“On June 6, 2017, we identified people with type 1 (T1D) or type 2 diabetes (T2D) using the One Drop | Mobile app on an iPhone and Apple Watch who had manually entered at least two HbA1c values in the app with HbA1c test dates 60 to 365 days apart. We did not recruit participants. Instead, we analyzed data collected from real users who elected to use the One Drop | Mobile app on their mobile phone and smartwatch devices.
Gender, diabetes type, and year of diagnosis are self-reported in the app.
Test results and test dates of HbA1c were self-reported in the app. Self-reported recall of a HbA1c test is highly sensitive (99%) to medical records and claims data documenting an actual HbA1c test . A self-reported HbA1c result is sensitive (79%) to a lab HbA1c test result . Further, we used mean blood glucose measured before the second HbA1ctest date to exclude invalid HbA1c measurements and, subsequently, validate self-reported HbA1c at that time point (see Analyses section).”
Now we could go on here quoting more explanations as to why the authors believe the use of self-reported data is ok. We could also quote the explanation for how they used fuzzy math to determine this way cool whiz bang toy works. Yet the question remains why the authors went to such extraordinary lengths explaining why using self-reported data and fuzzy math is ok.
Would it not be easier just to do a REAL study with independently verified data? Would it not eliminate any questions on the legitimacy of the conclusions drawn from this “study” had this data been independently verified? Did the authors consider that the “study” participants were already highly engaged with their diabetes management and this group may not be a representative sample of patients with diabetes?
This is the reason we will continue to hammer away every time OneDrop publishes this crap. This is the reason we will continue to call out the diabetes community for their silence on this issue. Seriously isn’t it about time someone other than Diabetic Investor speak up about the use of self-reported data combined with fuzzy math.
But let’s take this a step further and explain why it matters, why it is important to use independently verified data, why fuzzy math has no place in our wacky world.
First and foremost, independently verified data eliminates any questions, removes any doubts and provides a more accurate picture of what did and what did not happen. Listen we are not claiming that patients in this “study” fudged the data or lied. What we are saying is had their self-reported data been confirmed by independently verified data the comfort level increases. Independently verified data would in its own way prove the legitimacy of the self-reported data.
Second in a real study we would have participants who more accurately reflect the patient population. A patient population that reflex’s real-world conditions. We have a big problem when any company tout’s results with patients who are already actively engaged with their diabetes management. As we keep saying these patients are the low-hanging fruit when it comes to achieving better outcomes. These patients simply put get it.
Now if OneDrop recruited a group of patients who had never used their system before and used independently verified data and this data showed an improvement in outcomes that would-be news. It would be even better if this group would be compared to their existing group of patients, who also had their data independently verified. Perhaps this type of study would show what we believe that it’s not that difficult to get engaged patients to do better but it’s a Herculean task to get non-engaged patients more actively engaged with their diabetes management.
One of the biggest issues here and this is not just for OneDrop but any of these way cool whiz bang cloud enabled systems, is to draw a straight line between the use of these systems and better outcomes. This why a real study is needed as it would more accurately reflect the impact of the system. Therefore, real studies use a control group, a group which does not use the system versus one that does use the system.
Basically, what the folks at OneDrop are doing is playing very fast and very loose with some very shaky data and lots of fuzzy math. The diabetes community by their silence is basically saying that it’s ok to use self-reported data combined with fuzzy math. Hence the reason why OneDrop will continues to get away with this crap.
Folks if it walks like a duck, squawks like a duck then it’s a duck. Or as Momma Kliff used to say; “You can dress up an ugly woman, put on lots of make-up but all the cosmetic enhancements will only disguise the fact that she is ugly. That when you take away the fancy dress and remove the make up it will reveal what was present all the time.” Or as Winston Churchill famously said; “Madame I may be drunk, but you are ugly. When I wake up tomorrow I won’t be drunk, but you’ll still be ugly.”