Reading Vendor Evidence Critically

Case studies

A vendor case study is a narrative account of one site's experience, usually co-authored with the site's marketing team. It is not evidence. Useful signals to extract from one:

• The site's baseline state (what did they replace?).
• The scope of the deployment (which units, how many rooms, over what period).
• Any named clinicians and whether they hold vendor advisory roles.
• Whether outcome metrics are attributed to a defined study protocol or to internal dashboards.

ROI decks

Return-on-investment claims are the most heavily massaged category of vendor evidence. Common softening patterns:

• Loaded labor rates: converting minutes saved into dollars at a blended clinician cost, without accounting for the fact that saved minutes are rarely reallocated to billable work.
• Selective time windows: the "post" period ends before the honeymoon effect fades.
• Excluded costs: interface engine work, additional headcount, or ongoing licensing.

The strongest ROI evidence uses all-cause length of stay, readmission rates, or contract labor spend — outcomes that appear on a hospital's actual income statement.

'Peer-reviewed' claims

Being published in a peer-reviewed journal is a floor, not a ceiling. Check whether:

• The journal is indexed in PubMed and Scopus.
• The study was pre-registered (for RCTs, on ClinicalTrials.gov).
• The vendor's role is disclosed in a conflict-of-interest statement, not buried in acknowledgments.
• Independent groups have replicated the result. A single-site retrospective study is a lead, not a proof.

A short reader's checklist

1. Who paid, and who wrote it?
2. What is the comparator, and is it fair?
3. What outcome would have counted as a negative result?
4. Would the effect show up in the hospital's audited financials?
5. Has anyone unaffiliated with the vendor reproduced it?