From an expert-driven paper guideline to a user-centred decision support system: A usability comparison study

Ellen Kilsdonk, Linda W. Peute, Rinke J. Riezebos, Leontien C. Kremer, Monique W.M. Jaspers

Research output: Contribution to journalArticlepeer-review

31 Citations (Scopus)


Objective: To assess whether a user-centred prototype clinical decision support system (CDSS) providing patient-specific advice better supports healthcare practitioners in terms of (a) types of usability problems detected and (b) effective and efficient retrieval of childhood cancer survivor's follow-up screening procedures compared to an expert-driven paper-based guideline. Methods and materials: A user-centred design (UCD) process was employed to design a prototype CDSS. Usability problems in information retrieval with the paper-based guideline were assessed by think-aloud analysis with 13 participants. Both simple and more complex tasks were applied. The analysis provided input for the UCD process of the prototype. The usability of the prototype CDSS was subsequently evaluated by think-aloud analysis with the same participants. Usability problems of the paper-based guideline and the prototype CDSS were compared by using the classification of usability problems scheme. In addition, efficiency (time to complete task) and effectiveness (completeness of retrieved screening procedures) of information retrieval of participants in the expert-driven paper-based guideline and the user-centred prototype CDSS were compared. Results: Usability problems in both the paper-based guideline and the CDSS prototype were mainly classified as 'incongruent with participants' mental model'. The prototype CDSS reduced this type of problem from 17 to 6 problems. The time to perform simple information retrieval tasks increased by 58. s when using the prototype CDSS, however, it resulted in a 58% improvement in task completeness compared to the paper-based guideline. The time to perform complex scenarios decreased by 3:50. min with the prototype CDSS, with 17% higher completeness compared to the paper-based guideline. Conclusion: Analysis showed that usability problems experienced by healthcare practitioners when using a paper-based guideline could be overcome by implementing the guideline in a user-centred CDSS design. Although different types of usability problems were experienced with the prototype CDSS, they did not inhibit effective and efficient performance of tasks in the system. The usability problem analysis of the paper-based guideline effectively supported comparison of usability problems found in the two information retrieval systems and it supported the UCD of the CDSS.

Original languageEnglish
Pages (from-to)5-13
Number of pages9
JournalArtificial Intelligence in Medicine
Issue number1
Publication statusPublished - Sept 2013
Externally publishedYes


  • Childhood cancer survivors
  • Clinical decision support system
  • Screening guidelines
  • Usability analysis
  • User-centred design


Dive into the research topics of 'From an expert-driven paper guideline to a user-centred decision support system: A usability comparison study'. Together they form a unique fingerprint.

Cite this