Assessing Health Information Systems: Frameworks & Tools
Updated: Jan 16
This blog post was written with support and inputs from Nehal Jain and Vunnava Rao who have extensive experience deploying PRISM and Routine DQA respectively. **Updated to include the HRIS assessment toolkit (see below)
According to the WHO, health information is one of the key building blocks of a health system. Getting routine health information systems right means you have a reliable data source for planning and problem solving at the facility and district level – an important step in quality improvement and decentralization.
I have spent a lot of time focusing on health information systems. I have been called upon to assess state-level human resource information systems, a national level mother and child tracking system, a state level hospital information system, and most recently an electronic medical record system in East Africa.
[caption id="attachment_892" align="alignright" width="300"] The WHO Health Systems Framework[/caption]
There are a number of excellent frameworks available for health information system assessments. I have never used one in its entirety, always pulling parts from different ones to meet the objectives of the particular assignment.
However, a familiarity with these different frameworks is helpful – it provides an established vocabulary to describe what you see, and highlights areas that need attention. I have summarized some of the frameworks that I am familiar with below, also drawing upon the experience of other public health colleagues.
I would love to hear from others if there are other frameworks that you have found useful.
1. Assessing the National Health Information System, HMN & WHO
What does it cover: Covers all the different data systems that would contribute to a national HIS (vital registry, service delivery data, census, household survey data). Builds stakeholder support and a common vision for the HIS.
Who is it meant for: For external assessors, with broad consultation.
How is it implemented: Structured, with quantitative (a score for adequate/ present but not adequate/ not adequate at all) and qualitative (space for details) components across all domains. Prescriptive: describes a process with three workshops and a steering committee.
2. PRISM, JSI & MEASURE Evaluation
What does it cover: The PRISM tools do the following:
Assess the performance of a routine health information system.
Identify technical, behavioral, and organizational factors that affect RHIS performance.
Aid in designing and prioritizing interventions to improve performance.
Support ongoing efforts to monitor and evaluate data quality and use in decision-making.
Who is it meant for: Internal and external assessors.
How is it implemented: Contains a number of tools, RHIS Performance Diagnostic tool, Management assessment tool, the Overview and Facility/Office Checklist and the Organizational and Behavioural Assessment Tool, Data Entry and Analysis Tool (DEAT) – which outputs charts and graphs. Includes sampling guidelines to ensure results are representative.
3. Design and Implementation of HIS, WHO
What does it cover: Written like a textbook, describes a broad approach for conducting an assessment. No tools provided.
Who is it meant for: For internal use.
How is it implemented: The chapter suggests the following data collection methods: reviewing and extracting methods, checking equipment and supplies, observing and recording consultations, interviewing staff and fgds with staff and clients.
4.The Real Access/ Real Impact framework, Bridge
What does it cover: This framework sets out the determining factors of ICT access and social impact within the development sector. Takes into account socio-cultural and systems factors. Provides 12 assessment domains (listed below). It does not prescribe a specific process, nor does it provide tools.
Who is it meant for: For external assessors.
How is it implemented: Has assessment domains, but does not outline a process or include tools. Can be used in a number of ways to help focus the work of ICT initiatives in developing countries.
5. Data Quality Audit Tool, MEASURE Evaluation, GFATM
What does it cover: Focuses on verifying the quality of reported data, and assessing the underlying system. Is implemented at service delivery sites, the district level (intermediate aggregation level) and the national M&E unit.
Who is it meant for: For external assessors, a shorter version “Routine DQA” is meant for internal use.
How is it implemented: Contains a system and data assessment tools, to be implemented at different levels of health system. Requires review of documents, system description, and “spot check” and “cross check” verification of data aggregates. Includes a close-out meeting at the end of the process. Includes sampling guidelines to ensure results are representative.
6. HRIS Assessment Tool, Intrahealth
What does it cover: This toolkit assesses dedicated HRH information systems and includes an inventory checklist, a survey tool and data quality considerations.
Who is it meant for: For external assessors
How is it implemented: The tool provides a survey to conduct a baseline assessment to precede any system improvements, so improvements can be evaluated. It includes a questionnaire and inventory tool, alongside guidance on how these can be adapted to a particular context.