1999 AERC Proceedings

Development and Validation of an Instrument to Measure Critical Thinking in Respiratory Care

Lynda Thomas Goodfellow
Thomas Valentine
Margaret E. Holt

University of Georgia

Abstract:Purpose - to establish the reliability and validity of an instrument to measure critical thinking among respiratory therapists. Questions derived from Mishoe's (1995) study, expert therapists and the literature. Content validity established. Pilot instrument tested with reliability = 0.94 (Cronbach's alpha). Instrument can be used to measure critical thinking in respiratory care.

Critical thinking helps us to decide what really matters, what is important and how to achieve our goals and aspirations. The ability to think has always been important and critical thinking is required for all professionals, most certainly so in the health care professions. Basic technical skills may no longer be enough to process and use information. Not only is a well grounded knowledge base necessary, but practitioners must possess critical thinking abilities that incorporate logical reasoning skills, problem solving, reflection, decision making, and lifelong learning (Mishoe, 1995). Health care practitioners must become thinkers who know a great deal and who can continually adapt, refine, and use their knowledge. The ability to think critically is the main proficiency necessary that will enable not only health care practitioners, but all professionals to meet the demands of a rapidly changing environment.

There is a need to investigate critical thinking in professional practice and to be able to make summarizations about the nature of critical thinking which can be tested in further research. The majority of the literature in adult education on critical thinking focuses on the development of critical thinking and related issues regarding theory, learning, teaching, ethical considerations and sociopolitical concerns (Mishoe, 1995). Mishoe (1995) identified the critical thinking skills and traits of respiratory therapists in acute care settings. Respiratory therapists provide treatment and diagnostic care to patients with respiratory difficulties. They provide this care under the direction of a physician in either hospital, physician office, extended care, or home care settings. But, how one specifically measures critical thinking behaviors in clinical practice is not well understood. The purpose of this study was to develop a reliable and valid instrument that will measure critical thinking in respiratory care practice.

Mishoe's (1995) study focused on the expert practice of 18 respiratory therapists employed in acute care hospitals settings. She determined that critical thinking in respiratory practice involves the abilities to prioritize, anticipate, troubleshoot, communicate, negotiate, reflect, and make decisions. The traits that affect critical thinking in practice include willingness to reconsider and challenge others, appreciation of multiple perspectives and continued learning, understanding of departmental and professional perspectives that impact the profession, and openness to continuing change in their personal and professional lives (Mishoe, 1995). The when, how, and why respiratory therapists are able to use these critical thinking skills is influenced by dispositional traits and organizational factors. Mishoe (1996) further reports that the work context and the role of the organization, including managers, must be addressed when attempting to explain or facilitate critical thinking in respiratory care practice.

Critical thinking in the practice of respiratory therapy is the cognitive process described by logical reasoning, problem-solving, and reflection (Mishoe 1995). This working definition incorporates the reflective, communicative, practical, and experiential aspects of critical thinking in respiratory care practice. To advance research and transfer these qualitative concepts into a quantitative empirical process, critical thinking is further defined as observable, measurable behaviors from situations and circumstances that occur within the context of professional clinical respiratory care practice which require logical reasoning, problem-solving and reflection. Mishoe's study (1995) can be shown to overlap the ways of knowing described in the expert practice of nurses (Benner, 1984) and to the professional performance goals of physicians (Miller, 1980). Given these works, the antecedent relationship of critical thinking and critical thinking skills to expert practice was suggested.

Instrument Development Process

The purpose of this research was to develop a valid instrument to measure critical thinking in respiratory care practice. Since an appropriate instrument that would gather desired data could not be found, a survey questionnaire was developed (Table 1). Survey research is considered to be a branch of social scientific research and is used to accurately assess the characteristics of whole populations of people (Kerlinger, 1986). This is the intent when using a survey instrument to measure the critical thinking skills of respiratory therapists.

Several sources were used in generation of the item pool. An in depth interview with Mishoe and review of her work (1995) generated several items. From the literature, Benner's (1984) study on the novice to expert practice among nurses and Brookfield's (1987) four stages of the critical thinking process were helpful. Other sources used for item generation and for content validity came from an expert panel of six respiratory therapists who currently work in various specialized areas of respiratory care. These experts were chosen based on recommendations by Nunnally & Bernstein (1994) that participants in content validation efforts should be as representative as possible of the types of individuals who will eventually be studied with the instrument. The expert panel consisted of a technical director of a large urban research-based hospital from a metropolitan area, two respiratory care educators from a baccalaureate respiratory therapy school housed in a state university, a neonatal/perinatal specialist, a pulmonary function laboratory supervisor and a clinical coordinator for a respiratory home health company which covers a third of the state. These particular individuals were chosen because they understand the practice of respiratory care and represent as close as possible the population that will be studied. The panel was provided with definitions of the critical thinking skills and trigger examples of how critical thinking could be operationalized into possible questions. They were then asked to give an example from their clinical experience, a situation that would illustrate the logic of the critical thinking skill described by Mishoe (1995).

A total of 215 raw items for possible inclusion in a survey instrument were generated from these four sources. This process intended to find saturation among the items gathered concerning critical thinking behaviors. Refinement of the item pool began by deleting redundancies which decreased the item pool to 167 questions. Content validity was assessed by placing the remaining 167 questions randomly on a survey instrument and was mailing it to the expert panel for their input and rating of each item in terms of importance. This expert panel survey was necessary to decrease the number of items for possible inclusion without infusion the researcher's own subjectivity. Results were analyzed for means and rank of each item.

Two factors were necessary for an item to remain in the pool: a) an item must receive a score of "4" or above from a possible scale of 1 to 6 from all experts and b) the mean of all scores must be equal to or greater than "5". By process of elimination, the item pool was further decreased to 90 items. The ideal survey measurement instrument requires sufficient but not excessive indicators, because too many indicators is a wasteful measurement of the construct. Ninety total items proved to provide too many questions in some constructs while other constructs did not have enough. Therefore, criteria for item inclusion was further refined to: no construct will have less than 8 items and no construct will have more than 12 items. This ranking method resulted in some ranks having more questions needed from that particular set. When these "ties" occurred, the best items from that particular rank were chosen. This resulted in a further reduction of items from 90 to 70. It was felt that the questions remaining indicated saturation of content areas for the construct.

Determination of Construct Validity

The term construct is used in psychology to refer to something that is not observable but is literally constructed by the investigator to summarize or account for regularities or relationships in observed behavior (Thorndike, 1997). For instance, we speak of a therapist's ability to prioritize as a way of summarizing observed consistency in past behavior in relation to patient care. This construct (prioritizing) then can be used to predict how individuals will act on future occasions. Items for inclusion on this instrument were derived from Mishoe's (1995) study and these skills provided the constructs needed in order to operationalize critical thinking into measurable behaviors. These skills (constructs) are prioritizing, anticipating, troubleshooting, communicating, negotiating, decision making, and reflecting.

Several methods for construct validity are described in the measurement and evaluation literature (Kerlinger, 1996; & Thorndike, 1997). Construct validity for this survey instrument was assessed by a modified Q sort. The remaining 70 items were randomized, numbered from 1-70 for tracking, and cut into small pieces of paper. Seven envelopes with a conceptual definition of each construct on separate envelopes was provided to six faculty members of a baccalaureate respiratory therapy school. They were asked to sort the random questions into a construct or critical thinking dimension that best identified which construct the item belonged. This type of sorting procedure was different from traditional Q sorts in that this was not a forced distribution of the items into an equal number per construct, but the items could be sorted into a construct with no limitations on the number per construct. Five of the six Q sort packets were completed. This further reduced the pool to 48 items because in order for an item to be accepted for construct validity, at least 80% of the responses had to be sorted into a particular construct. However, two of the seven constructs were left with too few questions. The researchers went back to the original expert panel item pool to see if there were any items that should possibly be put back into the pool. Twenty three items were retrieved and rewritten, then re-sorted by the same faculty to ensure construct validity. All remaining items were carefully scrutinized for clarity and some questions rewritten. This resulted in enough items per construct and the final item pool was further decreased to 44 questions.

The research sample for this study consisted of registered respiratory therapists working in various clinical settings throughout the United States. These particular practitioners are well suited for this study because of their experience in respiratory care practice. This includes respiratory care in the non-ICU areas, in the ICU, in ambulatory care, in extended care facilities, for the homebound patient, in the emergency room, in the diagnosis of cardiopulmonary disease, and in disease management. A random sample of 100 registered or registry eligible respiratory therapists were drawn from a membership list obtained from the American Association for Respiratory Care.

A total of 100 surveys were mailed with two follow-up mailings for non-respondents. A cover letter describing the research and a self-addressed, stamped envelope was included with each survey. Sixty surveys were returned for a 60% response rate. Each question was scored from a Likert six point scale corresponding to how well the respondent did each of the tasks in their clinical practice.

Several computations were reviewed for validation of the instrument. If an item revealed that more than 80% of the responses were clustered around one or two points on the Likert scale, then insufficient variance was indicated. Enough variance was found for all items using this criteria. Possible redundant questions were noted by intercorrelations. Five sets of questions were reviewed and two were re-worded to eliminate any potential duplication. The determination of the reliability of the survey instrument was performed using Cronbach's alpha. Cronbach's alpha measures the internal consistency of an item to determine the extent to which items categorized within a particular critical thinking behavior measure that construct. Table 2 includes a summary of the means, standard deviations and coefficient alpha results for the total instrument as well as for each construct.

This process demonstrates that critical thinking behaviors are measurable and accountable by the framework described by Mishoe (1995). The results showed encouraging evidence of reliability and validity for the total instrument and the results further indicate that each construct measures what it theoretically is supposed to measure. The reliability computations indicate that there is internal consistency to the instrument and that the items are homogenous. The instrument at 0.94 proves to rate high in all reliable assessments which is important in determining the accuracy of an instrument. This is the first attempt to develop a mechanism for assessing the critical thinking behaviors of respiratory therapists. With minor revisions, this instrument can provide a useful tool for future research in assessing the critical thinking behaviors of respiratory therapists.

TABLE 1

Overview of Instrument Development and Validation Process

TABLE 2

[Home] [Search] [Conference Proceedings] [Back] [Top]