Where are my rules?

We rarely get the opportunity to design the IT systems that support our laboratory operations. More often we inherit the IT systems passed down to us, without the benefit of adequate knowledge transfer. And with a growing number of systems in the lab today’s lab information systems, middleware systems, instrument on-board rules and even the electronic health record (EMR/EHR), lab result decision rules can be anywhere and everywhere. To complicate matters further, IT systems may be shared across hospitals and clinics but the rules at each location are implemented differently to support policy and procedure variations. Or consider how you might go about replacing or adding a system for managing decision rules if your knowledge of the current system is not well documented. In short, many labs lack a ‘map’ to help pinpoint where the result decisions rules are located, what they actually do and what data they act upon.

Rules can exist in pockets of the EMR/EHR, LIS, middleware systems, or instrument on-board software that support instrument platforms and interface consolidators. Over time, the lab loses sight of its result-based rules, how they are harmonized between the various systems and how they actually work together to qualify and release results. The lack of visibility means that changes and additions to the current rule sets can have unintended, perhaps disastrous consequences downstream. When rules are changed in one system but not in the corresponding system, they become out-of-synch, introducing unintentional risk to the quality of laboratory result reporting. Understanding the layers of rules within a rules ecosystem ensures that the rules are coordinated and working well together.

Taking Stock of Your Rule Inventory

Laboratory result decision rules act upon data received from the instrument. The addition of a process called ‘auto-verification’ is where results are evaluated based on predefined computer logic to govern the release of results to the system. More sophisticated software applications with rules and auto-verification logic can generate customized interpretative comments or text that can add value to the result. However, auto-verification rules do not determine a diagnosis or indicate to any medical professional a course of treatment. Laboratory result decision rules are used for result evaluation and consider many different criteria but do not direct any actions to predict the likelihood of a specific disease or condition.

There are many layers of result review decision rules that can exist across systems that support laboratory results. As the systems grow and encompass more and more functionality, it is a good idea to inventory the IT systems that contain result decision rules and how they interact with each other.

High level goals include the following:

• Verify that all rules required for full result review and reporting are present within or across your required systems

• Identify gaps in rules functionality along the continuum of result qualification

• Isolate duplicate rules that may inactivate or over trigger rules

• Opportunity to standardize rules across clinical practices

The IT and Rule inventory process should include three phases of identification and analysis.

Table 1: Phases of Rule Inventory Process

Let’s review each phase to dig deeper into the rule identification, mapping, and rule pruning process and final verification suggestions.

Phase I-Identify where all your Rules are Located

All healthcare organizations today use asset inventory management tools to protect the investment in equipment, hardware and software. However, as critical as rules are to producing accurate patient results, there are no rule asset management systems that track rules, including the types of rules, how they are applied, in which equipment and systems they reside and across which locations they are used. Why is this?

• Rules operate in silos – The rule ‘systems’ are rarely visualized as a comprehensive organizational asset detailing how the rule sets interact and their intended and expected outcomes. The operation of rules is typically defined and deployed in silos and not as one process that acts upon specimen result and workflow management throughout the healthcare organization.

• Haphazard rule implementation – IT and instrument technologies are implemented at different times as an organization grows and matures.  There is minimal effort to align organizational decision rule processing needs with software selection.  Rules management becomes an afterthought until system replacements upset the ecosystem and issues become apparent.

• Failing to coordinate rules operation globally- The lack of planning of how rules should be coordinated and their impact throughout the environment introduces areas of risk that becomes unmanageable as the rule bases age. Lack of collaboration results in fragmented rule operation and often ‘in-operation’ as systems are added and replaced.

Inspire you.

Reduce the risk that your decision rules might not be operating as you expect by creating a high-level diagram that maps your rules across the various systems. A rules map will help guide you in the ongoing development and maintenance of your rules by giving diverse stakeholders (IT and Clinical) a common, understandable view.  A shared rules map that everyone can understand helps tell the ‘story’ of your decision rules and gives the team the confidence to make rules improvements over time.  

Diagram 1: Example of High Level Rule Inventory by System

Phase II-Analyze rule Form and Function

After identifying where your rules are located, the rules within each rule set themselves should be organized into rule categories based on their intent and function – what do they actually do and how do they interact with other systems.  

Rules generally act upon results from the analytical instruments no matter where the rules reside. However, order type rules can add or remove a test or profile based on checking of previous orders and results or based on the demographic characteristics of the patient. All other rules act upon the result received from the instrument and then evaluated based on many different criteria. Not every analytical test meets every criterion for a rule based on the categories below. The determination of what types of rules should be defined by test analyte should be reviewed with the clinical subject matter expert(s) (SME) for that instrument and test category.

Group your rules the categories listed in the table below which follows the CLSI Auto-15 document for auto-verification of medical laboratory results for specific discipline (CLSI, 2019).

Evaluate your rule types based on their relevancy and where in the process they trigger. Follow these recommendations to help you evaluate if a rule is should be added or if it needs to be inactivated:

• Only add rules types at each level that are appropriate for the action or data they are acting on

• Coordinate rules so that the approval status is clear between systems-only one system should approve results as final

• Harmonize rules that my conflict in reporting of decimal places, rounding or equivalent units

• Remove rules that may inactivate other downstream rules that are of higher priority

• Add rules to suppress data that is not needed for the final reporting system - these rules keep the granular data for interlaboratory use and review

• Add rules for third party systems or other systems of record that require special reporting fields

Table 2: Rule Types by Form and Intent

Phase III - Identify Cross Rule Operation

The next important step in the inventory process of your result decision rules is to determine the rule dependencies and cross-talk that may occur between systems. The purpose of this exercise is to determine the flow of rule operation by type and identify any duplication of rules that might deactivate or adversely act upon results that may change or alter the intended result. In short – are there rules that can conflict with each other with overlapping criteria and either cause a continuous loop of triggering the same rule or inactivate one or more rules with similar criteria.

Key tips to consider:

• When tracking data across disparate systems and when tracking rule operation, review the rules at every level (instrument, LIS, middleware, EHR) to determine that the data is being transformed appropriate to the final system of record

• Variation in analytical methods may lead to different interpretation of the data and values and may NOT be equivalent when crossing software systems. Evaluate your result decision rules to make sure the data transformation is equivalent and consistent across systems

• Implementation of a rule traceability matrix across IT systems along with a periodic quality assurance and quality review process will identify any data transformation inconsistencies based on rule origin, actions, and triggers

• We recommend you test on a regular basis.  Testing result transmission from each point of contact to the next system will help you confirm the accuracy of your patient results as rules are applied across multiple systems

Phase IV - Rule Stewardship to Drive Value

Once you have identified where your rules reside and have cross-checked for rule types and duplication across systems, the next step is to ensure that your result decision rules represent your clinical best practices and meet your operational needs. Use of auto-verification rules is important to driving strategic value within your laboratory. Rule stewardship bridges the gap between the clinical and operational aspects of laboratory operation. Your laboratory should establish a rule quality assurance program to track and monitor rule performance through measurable quality objectives that should include:

• Monitoring of your auto-verification rate within and across systems to ensure your auto-verification rate is maintained

• Tracking lab productivity and capacity rates to identify any degradation in lab performance

• Recording and investigating rule-associated patient care issues or errors that might have an adverse impact on your rule quality

The practice of periodically evaluating your rules inventory and rule performance is prioritizing rule stewardship to align with your organization’s strategic priorities. Optimizing your rule performance through periodic rule inventory maintenance programs is key to a highly functional auto-verification and rule program. New tests procedures, clinical guidelines and analytical performance criteria is always changing and impacting your decision rule program integrity. A quality assurance program that includes rules mapping across systems is a pro-active response to inevitable change. Those labs that employ these techniques see tangible benefits both in clinical and operational outcomes to ensure each test result is handled properly across the entire organization.

Let’s keep talking!

References:

1.  CLSI (2019). AUTO-15: Auto-verification of medical laboratory results for specific disciplines, Clinical and Laboratory Standards Institute (CLSI), 2019.