A FOCUS ON PROVIDERS

IllumiCare serves hospitals by collecting real-time clinical data about inpatients and wholesale cost data (not charges) on medications and lab tests. The data is used to calculate a real cost of every medication and lab order, attribute that cost to the ordering provider, and aggregate those costs at the patient admission and DRG levels.

This study shifts the focus off patient characteristics and onto physician habits, a change from previous studies that utilized highly advanced models including artificial neural networks, ridge regression models, and other techniques to predict the individual patient’s cost. Those studies used patient demographics such as age, gender, ethnicity, and comorbidities as predictors of individual costs.ⁱ

Leep Hunderfund et al. surveyed physicians and medical students, finding that 88% of respondents moderately or strongly agreed that “trying to contain costs is the responsibility of every physician.” ⁱⁱ The AMA Code of Medical Ethics states, “Physicians also have a long-recognized obligation to patients in general to promote public health and access to care. This obligation requires physicians to be prudent stewards of the shared societal resources with which they are entrusted.” ⁱⁱⁱ

Despite calls for provider stewardship, studies show large variation in provider spending on similarly sick patients, even within the same specialty and in the same hospital.^iv Higher spending may be an acceptable tradeoff for better outcomes, although it does not correlate with better outcomes.

What is needed is a method to directly measure behaviors that correlate with higher spending and to intervene on those behaviors in a way that reduces both the behaviors specifically and spending generally.^v

Materials and Methods

Stewardship Opportunities Considered

Over the past six years, IllumiCare has accumulated a set of medication and lab orders that represent very likely wasteful care (Stewardship Opportunities or “Opportunities”). A medication example is the use of IV Tolvaptan over oral Urea when the patient’s sodium is 128 or above. A laboratory example is repeat daily ordering of B-type natriuretic peptide (BNP) whether or not the test result is normal or abnormal.

A single provider may have responsibility for some of these decisions, such as repeat BNP orders. Other decisions may represent missed opportunities for cost stewardship among multiple providers. For example, switching from IV to PO represents an opportunity where the original IV order was appropriate, but later the patient commenced other PO meds and the PO medication was clinically equivalent. Often, multiple providers failed to make the switch. In this analysis, we included only opportunities that were the responsibility of a single provider. We evaluate the relationship between a provider’s Opportunity per patient (calculated by summing the costs of the wasteful orders of that provider) - the independent variable denoted \({ x }\)- and that provider’s overall medication and laboratory spend per patient - the dependent variable denoted \({ y }\).

Data Collection and Preprocessing

Data on patients discharged between 01 November 2020 to 30 June 2021 was collected through the participating health systems’ HL7 data feeds. Only pharmacy-verified medication orders and resultant laboratory orders were evaluated. Costs for services were attributed at the CMS allowable rate, and cost for medications were attributed at the average wholesale price obtained from Medi-Span. Charge codes were not used in this analysis.

Only orders placed by providers registered with CMS under taxonomy code 208M00000X (Hospitalist), 207R00000X (Internal Medicine), or 207Q00000X (Family Medicine), or who was identified as a Hospitalist, Internal Medicine, or Family Medicine provider by the hospital system under the terms of an IllumiCare SmartRibbon user were included in this analysis. All 284 included providers will be referred to as “hospitalists” in this analysis.

Only orders associated to a patient encounter discharged as “Inpatient” were included. A provider is defined as “caring for” an inpatient encounter if they placed at least 25% of the orders on the discharged patient. Only orders in which the hospitalist was the direct ordering provider, per HL7 order messages, were included in the calculation. So, for example, if a hospitalist instructed a nurse practitioner to place an order, that order was not attributed to the hospitalist.

In sum, this analysis considered 17,032 patient encounters and 30,930 “violating” orders (28,448 lab orders and 2,482 med orders).

Independent Variable

The entire CMS allowable cost of a laboratory order was designated as a cost-saving opportunity. For medication orders, the opportunity was calculated as the difference between the daily unit cost of the ordered medication and the daily unit cost of the lower-cost medication, multiplied by the timestamp interval difference between prescription start and prescription end. This opportunity, for example, is only presented for IV to oral switches when the patient is on another oral medication. A hospitalist’s opportunity per patient is calculated as the sum of all orders that were identified as opportunities divided by their total number of “cared for” inpatient admissions.

Dependent Variable

Hospitalists’ risk-adjusted spend per patient is calculated as \({\frac{1}{n} \sum^n_{i=1} \frac{y_i}{w_i}}\), where n is the distinct number of patients treated, y is the subject hospitalist’s total medication and laboratory spend on the patient, and w is the MS-DRG weight of the patient. In other words, it is the sum of the value of all medication and laboratory orders placed by that provider on a patient that they cared for divided by the assigned MS-DRG weight, divided by the total number of inpatient discharges that they cared for.

Outcome Measures

Because we are evaluating the relationship between the two variables, our outcome measures were focused on directional relationship, variation, and correlation. Our key measures were the Coefficient of Determination (R²), Root Mean Squared Error (RMSE), and Mean Absolute Percentage Error (MAPE). R² is calculated as \({ \frac{\sum (\hat{y}-\bar{y})^2}{\sum y-\bar{y})^2} }\), or the sum of squares regression divided by the sum of squares total. This value, often expressed as a percentage, gives us the proportion of variability explained in the model among the total variability.vi In our case, how much does a provider’s opportunity per patient explain their overall spend per patient? RMSE is calculated as \({ \sqrt{ \frac{1}{n} \sum^n_{i=1} e^2_i } }\) where \({ e_i }\) is the error of samples \({ i }\). RMSE was chosen because it is more sensitive to variances within the model error than the mean absolute error by penalizing errors with a larger absolute value.vii MAPE is calculated as \({ \frac{1}{n} \sum^n_{i=1} \left| \frac{\hat{y}_i - y_i}{y_i} \right| }\), meaning the average absolute percentage error a predicted point was from the true value, across the data. MAPE is utilized for its intuitive interpretation on the quality of a regression model, its handling of relative variation instead of absolute variation, and its ability to handle both positive and negative errors within our model.^viii

Statistical Analysis and Dataset Preparation

The relationship between a hospitalist’s spend per patient and opportunity per patient was assessed through an ordinary least squares (OLS) regression, residuals analysis using diagnostic plots, train-test validation of the model, and comparison of time series.

In the first dataset used for the OLS regression model, the unit of analysis was an individual hospitalist. To prepare the dataset, a hospitalist’s independent and dependent variables were calculated using data over the entire evaluated timespan. For a provider to be included in this dataset, they must have placed at least 50 orders and cared for at least 15 patients. Using random assignment, the hospitalist data was split into a training set (80%) and test set (20%). The training set included 215 hospitalists and the test set included 54 hospitalists. The independent variable x was right skewed, causing the model to be heteroskedastic. OLS regression assumes \({ V(e_i) = \sigma^2 }\) for all values i, meaning the error terms have a constant variance (homoskedasticity).^ix  A logarithmic transformation was performed on the independent variable using \({ log_e(x+1) }\) to account for this. By adding the constant of 1, we were able to apply the logarithmic transformation to values \({ x=0 }\).

In the second dataset used for time series analysis, the unit of analysis was the mean value among hospitalists per month, i.e., the mean hospitalist opportunity per patient each month or the mean hospitalist spend per patient each month. Costs were attributed to the month in which a patient was discharged. A hospitalist’s independent and dependent variables were calculated as the sum cost for that month, divided by the number of patients cared for and discharged during that month. The mean value per month was calculated to achieve the monthly rate for each variable. A provider must have cared for at least five patients per month to be included.