• Applying Bayesian network approaches to study health outcomes

      Lee, Sun-Mi; Abbott, Patricia A., Ph.D., M.S. (2003)
      Background. In today's healthcare environment, the proliferation of information systems has facilitated the growth of large clinical and administration databases. Innovative knowledge discovery approaches in large healthcare databases via data mining techniques have been actively used in the analysis of these data. Of particular interest are Bayesian networks, which have recently emerged as powerful data mining algorithms for pattern recognition and classification. Purpose. The purpose of this study was to explore the feasibility of using Bayesian networks (BN) in studying health outcomes. The specific aims were to develop a BN model to identify predictors of limited health service utilization in HIV positive persons and evaluate the model by comparing it to the predictive performance of Naive Bayes (NB) and logistic regression (LG) models. Methods. This study used the HIV Cost and Services Utilization Study dataset consisting of 2,864 HIV positive adults. A total of 36 variables including two service utilization variables (hospitalization and outpatient visits) were selected. HUGIN Researcher(TM) 6.3 was used to develop the BN and NB models; SAS/STAT PROC LOGISTIC was used to develop LG models. Results. The BN model successfully captured relationships explaining complex patterns of human behavior in health service utilization. The area under the receiver operating characteristic curve (AUC) measuring the BN model's discriminatory power when predicting hospitalization was .72 (CI: .70, .74). The AUC of the BN model was statistically higher than that of the NB model (.68; CI: .66, .70), but no higher than that of the LG model (.70; CI: .67, .72) using the 8 variables from a previous study by Shapiro and colleagues (1999b). In a second analysis using the 10 influential variables discovered by the BN approach, the NB and LG performance improved (NB: .74 (CI: .72,.76); LG: .74 (CI: .72, .75)).;Conclusion/implication. The BN approaches contributed to the discovery of the influential predictors that lead to an increase of the models' predictive performance. This study provided new insight in working with large healthcare databases. When attempting to discover unknown relationships that might be missed by traditional analysis methods alone, investigators should consider the use of BNs.
    • Business-information technology alignment between chief nurse executives and informatics nurse specialist: An instrument development

      Lasome, Caterina Elisa Martinez; Mills, Mary Etta C. (2008)
      Background. The Informatics Nurse Specialist (INS) makes significant contributions to the health information management function. The ability to function effectively can be affected by the degree to which the INS and the Chief Nurse Executive (CNE) are aligned on business and information technology (IT) objectives, processes, priorities, and outcomes. The failure of this dyad to achieve alignment can seriously retard the information-handling needs of a nursing department. No reliable or valid measure existed to measure business-IT alignment among the CNE/INS dyad. Purpose. This study aimed to produce a reliable and valid instrument (Index of Business-Information Technology Alignment in Nursing (IBITA-N) for assessing business-IT alignment between CNEs and INSs in acute healthcare facilities and estimate its psychometric properties. Subquestions included: (a) What are the individual and organizational demographic and practice characteristics of the sample?, (b) What relationships exist between IBITA-N subscale and total scores and selected individual and organizational characteristics?, and (3) What set of individual and organizational characteristics best predicts a high degree of alignment between the CNEs and INSs? Design/methods. A cross-sectional survey design was used. Data were obtained through mailed surveys from a sample of 102 CNEs and INSs. Results. Principal components analysis (PCA) provided initial evidence of a parsimonious four-factor structure that was consistent with the proposed theoretical dimensions of alignment. Seven items loaded high on Factor I, titled "strategic communication." Six items loaded high on Factor II, titled "enabling processes". Seven items loaded high on Factor III, titled "prioritization." Nine items loaded high on Factor IV, titled "measures of success/outcomes." Significant differences between CNEs and INSs were found for the Strategic Communication and Prioritization subscales and the total scale score as well as between subscale and total scores and individual and organizational characteristics.;Conclusion. This study provided initial evidence to support the theoretical dimensions of the business-IT alignment construct among CNEs and INSs. The initial 81-item measure was reviewed by a panel of experts and subsequently reduced to 70-items; PCA suggested that a fewer number of items (29) may be sufficient to measure business-IT alignment in the CNE/INS dyad. Further development of the measure is needed.
    • Clinicians' perceptions of usability of an electronic medical record over time

      Seckman, Charlotte; Mills, Mary Etta C. (2008)
      The study evaluated clinicians' usability perceptions of an electronic medical record (EMR) over time in a large east coast research medical center. The descriptive retrospective longitudinal design used secondary data obtained from a performance measurement plan. Electronic survey data were collected at five different points in time over four years. The anonymous responses made matching of the 1148 individuals impossible so thirty cases were defined grouping clinicians by age category, gender, and role. Each response was considered to be a repetition of measurement of the case. The research questions explored the overall patterns of change in usability related to an EMR implementation, the short term and long term change in usability, and if the patterns of usability were influenced by frequency of use, gender, age, length of service (LOS), or role. The short term impact included evaluations at baseline, four months, and 16 months after the implementation of an EMR. The long term impact included evaluations before, four months, and 16 months following a system upgrade. Linear mixed model analysis was used to examine patterns of change over time including the contribution of frequency of use and user characteristics on the outcome variables. Each variable had curvilinear and linear patterns following the initial EMR implementation but not after the system upgrade. Usefulness, ease of use, supports clinical care, and supports research had significant linear and/or quadratic changes over the short term but no significant changes in long term except for ease of use. Frequency of use was a significant predictor for usefulness and supports clinical care but not for ease of use. Age, LOS, and role contributed significantly to usefulness, ease of use and supports clinical care but gender did not. Role and LOS combined were significant predictors for usefulness, supports clinical care, and ease of use over the short term. The findings indicate significant short term changes related to the initial EMR implementation but not the upgrade. This had implications for system development related to education, clinical practice and research.
    • A comparison of paper-based data submission to remote data capture for minimizing data entry errors in cancer clinical research

      Meadows, Beverly Jane; Mills, Mary Etta C. (2006)
      Background. Patient data are essential for judging the safety and efficacy of cancer clinical trials. The current process of paper-based data entry provides opportunities for incurring data discrepancies. Automated systems have shown potential to reduce the number of data entry errors and preserve the quality of clinical trial data. To test this potential, this study examined case report forms (CRFs) to test for differences in the proportion of discrepancies and the time to resolve these discrepancies between a paper-based data entry and OracleRTM Clinical (OC) Remote Data Capture (RDC). Objective. The purpose of this study was to examine differences in the proportion of errors and the time to resolve specific errors between a paper-based CRF and an electronic RDC format. Reason's conceptual framework for error detection and recovery feedback loop was used to guide this research where the warning environmental cueing function provided feedback to the end-user. Results. The sample consisted of 445 RDC and 445 paper-based CRFs submitted to the Cancer Trial Support Unit (CTSU) from March 12, 2004 through March 28, 2005. There was a significant reduction in the proportion of overall data discrepancies for RDC as compared to paper-based CRFs (46.5% vs. 31.7%, p<.001). Similar results were found for univariate (58.6% vs. 41.2%, p<.001) and multivariate (64% vs. 36%, p<.001) discrepancies. Of the total sample of 890 CRFs analyzed for this study, 509 (57.2%) had no discrepancies. For the 381 (42.5%) forms with discrepancies there was no difference in the mean number of days to resolve discrepancies between RDC and paper-based (43 vs. 35). However, RDC had a greater proportion of resolved discrepancies (52% vs. 48%, p<.001).;Conclusion. The results from this study supported Reason's concept of error detection and recovery. RDC data entry format decreased overall, univariate and multivariate data discrepancies for patient information collected on a colon cancer study; however, there was no difference in the timeline for discrepancy resolution between the two formats. Further studies are recommended to test alternate definitions of discrepancy resolution time points. Results from this study can only be generalized to automated systems that use Oracle RTM Clinical and the instance configuration specific for the programmed edit checks used for the colon cancer study.
    • The impact of Medline usage on nurses' research utilization and decision-making confidence: A study of computer usage applying the theory of reasoned action

      Prin, Patricia Leonard; Mills, Mary Etta C. (1996)
      This exploratory study in the field of nursing informatics examined the usage of information technology, namely on-line access to Medline in clinical settings, by a convenience sample of 121 nurses from a large university hospital. A descriptive correlational design was used. Guided by the conceptual framework of Nurse-Computer Interaction (Staggers and Parks, 1991) and based on variables set forth in the Theory of Reasoned Action (Azjen and Fishbein, 1980), the study tested hypotheses regarding attitudinal and normative influences on reported use of on-line bibliographic retrieval systems. It was also hypothesized that using MEDLINE could increase and improve nurses' decision-making confidence and their adoption of nursing research findings. Consistent with the Theory of Reasoned Action, multiple regression analyses were conducted on nurses' responses to survey questions to test hypotheses--that those who register more favorable attitudes towards computers and nursing research and those who perceived subjective norms more supportive of computers and nursing research would have a higher reported use of the MEDLINE system. Findings were significant (F = 9.12, p =.003) and supported the hypothesis that nurses' attitudes towards research influenced MEDLINE usage. To determine if there was a significant relationship between MEDLINE usage, decision-making confidence and research use, regression analyses were also performed. Findings indicated that MEDLINE usage was significantly related to both nurses' research utilization (F = 7.39, p =.0076) and nurses' decision-making confidence (F = 40.62, p =.0000). MEDLINE usage accounted for 25.4% of the variance in decision-making confidence. This study represents a beginning work in the field of nursing informatics to address the many complex variables inherent in human-computer interactions. Findings suggest the need for further research to document the benefits of information technology for nursing professionals.
    • Information system integration in a geographically distributed organization

      Charters, Kathleen G.; Mills, Mary Etta C. (1998)
      As health sciences advance, individuals increasingly specialize in fields of study with refined and limited focus making it is difficult to solve very large and complex problems. This qualitative study evaluated the process of integration review. This case study with multiple units of analysis describes information system key integration concerns, an integration process, and closure factors. Content analysis of twenty-two "System Change Requests," five focused interviews, and two program office documents addressed the following: (a) What concerns emerge when multidisciplinary communities discuss changes to an information system used by all the health care communities? (b) What is the process for integration of a change across professional communities? (c) What are the triggers for closure of the integration review? The theoretical foundation for this study was derived from Systems Theory. The case selected was the final version in the life cycle of the Composite Health Care System-a complex worldwide health care information system used by the U. S. military. Of proposed modifications to this system, those selected for this study met the following criteria: (a) will be implemented, (b) have comparable documentation, (c) discuss integration, and (d) reflect interactions among people. Since the organization was geographically distributed, electronic mail was used for integration review. Following review analysis, a qualitative survey was created. A pilot study refined the survey, and Institutional Review Board approval was obtained. Potential interviewees were identified by their participation in the selected reviews. Following content analysis of the interviews, program office documentation was obtained and analyzed for integration processes. Content analysis yielded ten categories of concern: scheduling/prioritization, extending user support, clarification, managing and reducing error, including stakeholders, user community impact, accountability, financial consideration, technology, and benefit. The integration process achieved a common understanding of what must be done to enhance user support without detracting from existing support. Triggers for review closure included time, stakeholder participation and issue resolution. This study confirmed that integration review by health care professionals solves complex problems spanning diverse areas of expertise. This study also revealed the need for a way to move "universal" recommendations for change out of a legacy system.