From Polit, Beck and Hungler: Essentials of Nursing Research (5th Edition). Use these as they relate to the article you wish to critique. Refer to the appropriate chapters in this text for further information. Print this page and keep it handy as you critique your research article. (This is a long page; make sure it loads completely on your computer. You'll know you've got it all when you see The End at the bottom of the page.)

The Ethical Aspects of a Study

Research Problems, Research Questions, and Hypotheses

Research Literature Reviews

Conceptual and Theoretical Frameworks

Research Designs in Quantitative Studies

Qualitative and Integrated Designs

Quantitative Sampling Designs

Qualitative Sampling Designs

Self-Reports

Observational Methods

Biophysiologic Methods

Data Collection Procedures

Data Quality in Quantitative Studies

Data Quality in Qualitative Studies

Quantitative Analyses

Qualitative Analyses

Conduct of a Written Research Critique

The Interpretative Dimensions of a Research Report

The Presentation of a Research Report

The Clinical Relevance of a Body of Research

The Implementation Potential of an Innovation

Return to Learning Activity 06, Step 01

The Ethical Aspects of a Study Were the study participants subjected to any physical harm, discomfort, or psychological distress? Did the researchers take appropriate steps to remove or prevent the harm? Did the benefits to participants outweigh any potential risks or actual discomfort they experienced? Did the benefits to society outweigh the costs to participants? Was any type of coercion or undue influence used in recruiting participants? Were vulnerable subjects used? Were participants deceived in any way? Were they fully aware of participating in a study, and did they understand the purpose of the research? Were appropriate consent procedures implemented? Were appropriate steps taken to safeguard the privacy of participants? Was the research approved and monitored by an Institutional Review Board or other similar ethics review committee?

top

Research Problems, Research Questions, and Hypotheses What is the research problem? Has the researcher appropriately delimited the scope of the problem? Does the problem have significance for the nursing profession? How will the research contribute to nursing practice, nursing administration, or nursing education? Is there a good match between the research problem and the paradigm within which the research was conducted? Is the problem to be addressed formally stated as a statement of purpose, research question, or hypothesis to be tested? Is this information communicated clearly and concisely? Are the purpose statements and research questions worded appropriately (e.g., are key concepts or variables identified and the study group or population of interest specified)? If the report does not formally state any hypotheses, is their absence justifiable? Do the hypotheses (if any) flow from a theory or from previous research? If not, what is the basis for the researcher's predictions? Are hypotheses (if any) properly worded (i.e., do they state a predicted relationship between two or more variables)? Are hypotheses directional or nondirectional? Is there a rationale for the manner in which they were stated? Are hypotheses stated as research hypotheses or null hypotheses?

top

Research Literature Reviews Does the coverage of the literature seem thorough? Does it appear that the review includes all or most of the major studies that have been conducted on the topic of interest? Are recent research reports cited? Does the review rely on appropriate materials (e.g., mainly on research reports, using primary sources)? Is the review organized in such a way that the development of ideas is clear? If the review is part of a research report for a new study, does the review support the need for the new research? If the review is designed to guide clinical practice, does the review of the evidence support the need for(or lack of need for) changes in practice? Does the review conclude with a synopsis of the state-of-the-art knowledge on the topic? Is the style of the review appropriate? Does the reviewer paraphrase, or is there an overreliance on quotes? Does the review appear unbiased? Does the reviewer use appropriately tentative language?

top

Conceptual and Theoretical Frameworks Does the research report describe an explicit theoretical or conceptual framework for the study? If not, does the absence of an explicit framework detract from the usefulness or significance of the research? Does the report adequately describe the major features of the framework so that readers can understand the conceptual basis of t he study? Is the framework consistent with the research paradigm? Do the research problem and hypotheses naturally flow from the framework, or does the purported link between the problem and the framework seem contrived? Are conceptual definitions of the concepts in the study provided? Are the conceptual definitions consistent with the framework? Are the definitions clear and sufficiently detailed? Did the framework guide the study methods? For example, do the operational definitions correspond to the conceptual definitions? Were hypotheses tested statistically? Does the researcher tie the findings of the study back to the framework at the end of the report? How do the findings support or undermine the framework? Are the findings interpreted within the context of the framework?

top

Research Designs in Quantitative Studies Given the nature of the research question, what type of design is most appropriate? How does this correspond to the type of design used? Does the design used in the study involve an intervention? If yes, was an experimental, quasi-experimental, or preexperimental design used--and was this the most appropriate design? If the design is nonexperimental, why didn't the researcher manipulate the independent variable? Was the decision regarding manipulation appropriate? Was the study longitudinal or cross-sectional? Was the number of points of data collection appropriate, given the research question? What type of comparisons were called for in the research design (e.g., was the study design within-subjects or between-subjects)? Are these comparisons the most appropriate for illuminating the relationship between the independent and dependent variables? Can the study be described as a survey, an evaluation, or outcomes research? What procedures, if any, did the researcher use to control external (situational) factors and intrinsic (subject characteristic) factors? Were these procedures appropriate and adequate? To what extent is the study internally valid? What types of alternative explanations must be considered (i.e., what are the threats to the study's internal validity)? Does the research design enable the researcher to draw causal inferences about the relationship among study variables? To what extent is the study externally valid? What are the major limitations of the design used? Are these limitations acknowledged by the researcher and taken into account in interpreting the results?

top

Qualitative and Integrated Designs Is the research tradition within which the qualitative study was undertaken identified? What was the tradition? If no research tradition is identified, can one be inferred? Does the research question appear to be congruent with the research tradition (i.e., is the domain of inquiry for the study congruent with the domain encompassed by the tradition)? Do the data sources and general methods of the study appear consistent with the tradition? How well is the research design described? Are design decisions explained and justified? Does the design appear thoughtful and appropriate? Does the design lend itself to a thorough, in-depth, intensive examination of the phenomenon of interest? What design elements might have strengthened the study (e.g., would a longitudinal perspective have been preferable, although a cross-sectional design was used)? Is the study exclusively qualitative, or were both qualitative and quantitative data collected? Could the study have been strengthened by the inclusion of some quantitative data? If both qualitative and quantitative data were collected, were they used in complementary fashion? How (if at all) did the inclusion of both types of data contribute to enhanced theoretical insights, enhanced validity, or movement toward new frontiers?

top

Quantitative Sampling Designs Is the target or accessible population identified and described? Are eligibility criteria specified? to whom can the study results be generalized? Are the sample selection procedures clearly described? What type of sampling plan was used? How adequate is the sampling plan in terms of yielding a representative sample? Did some factor other than the sampling plan affect the representativeness of the sample (e.g., a low response rate)? Are possible sample biases identified? Is the sample size sufficiently large? Was the sample size justified on the basis of a power analysis or other rationale?

top

Qualitative Sampling Designs Is the setting or study group adequately described? Is the setting appropriate for the research question? Are the sample selection procedures described? What type of sampling strategy was used? Given the information needs of the study, was the sampling approach appropriate? Were dimensions of the phenomenon under study adequately represented? Is the sample size adequate? Did the researcher stipulate that information redundancy was achieved? Do the findings suggest a richly textured and comprehensive set of data without any apparent "holes" or thin areas?

top

Self-Reports Does the research question lend itself to a self-report method of data collection? would an alternative method have been more appropriate? Is the degree of structure consistent with the nature of the research question? Given the research question and respondent characteristics, did the researcher use the best possible mode for collecting the data (i.e., personal interviews, telephone interviews, or self-administered questionnaires)? Do the questions included in the instrument or topic guide adequately cover the complexities of the problem under investigation? If a composite scale was used, does its use seem appropriate? Does the scale adequately capture the target research variable? If a vignette, projective technique, or Q sort was used, does its use seem appropriate?

top

Observational Methods Does the research question lend itself to an observational approach? Would an alternative method have been more appropriate? Is the degree of structure consistent with the nature of the research question? To what degree were observers concealed during data collection? If there was no concealment, what effect might the observers' presence have had on the behaviors being observed? What was the focus of the observation? How much inference was required on the part of the observers, and to what extent did this affect the potential for bias? Where did the observations take place? To what extent did the setting influence the naturalness of the behaviors observed? How were data actually recorded (e.g., on field notes, checklists)? Did the recording procedure appear appropriate? What was the plan by which events or behaviors were sampled? Did this plan appear appropriate? What steps were taken to minimize observer biases?

top

Biophysiologic Methods Does the research question lend itself to a biophysiologic approach? Would an alternative method have been theoretically more appropriate? Was the proper instrumentation used to obtain the biophysiologic measurements? Would an alternative instrument or method have been more appropriate? Does the researcher appear to have the skills necessary for proper interpretation of the biophysiologic measures?

top

Data Collection Procedures Who collected the research data? Were the data collectors qualified for their role, or is there something about them (e.g., their professional role, their relationship with study participants) that could undermine the collection of unbiased, high-quality data? How were data collectors trained? Does the training appear adequate? Where and under what circumstances were the data gathered? Were other people present during the data collection? Could the presence of others have created any distortions? Did the collection of data place any undue burdens (in terms of time or stress) on participants? How might this have affected data quality?

top

Data Quality in Quantitative Studies Does there appear to be a strong congruence between the research variables as conceptualized (i.e., as discussed in the introduction) and as operationalized, i.e., as described in the method section)? Do the rules for the measurement of the variables seem sensible? Were the data collected in such a way that measurement errors were minimized? Does the report provide evidence of the reliability of the data? does the evidence come from the research sample itself, or is it based on a prior study? If the latter, is it reasonable to believe that reliability would be similar for the research sample (e.g., are the sample characteristics similar)? If there is reliability information, what method of estimating reliability was used? Was this method appropriate? Is the reliability sufficiently high? Does the report provide evidence of the validity of the measures? Does the evidence come from the research sample itself, or is it based on a prior study? If the latter, is it reasonable to believe that validity would be similar for the research sample (e.g., are the sample characteristics similar)? If there is validity information, what validity approach was used? Was t his method appropriate? does the validity of the instrument appear to be adequate?

top

Data Quality in Qualitative Studies Does the research report discuss efforts the researcher made to enhance or evaluate the trustworthiness of the data? If so, is the description sufficiently detailed and clear? Which techniques (if any) did the researcher use to enhance and appraise the credibility of the data? Was the investigator in the field an adequate amount of time? Was triangulation used, and, if so, of what type? Did the researcher search for disconfirming evidence? Were there peer debriefings or member checks? Do the researcher's qualifications enhance the credibility of the data? Which techniques (if any) did the researcher use to enhance and appraise the dependability, confirmability, and transferability of the data? Were the procedures used to enhance and document data quality adequate? Given the procedures used (if any), what can you conclude about the trustworthiness of the data?

top

Quantitative Analyses Does the report include any descriptive statistics? Do these statistics sufficiently describe the major characteristics of the researcher's data set? Were the correct descriptive statistics used? (e.g., were percentages reported when a mean would have been more informative?) Does the report include any inferential statistical tests? If not, should it have (e.g., were groups compared without information on the statistical significance of group differences)? Was a statistical test performed for each of the hypotheses or research questions? Do the selected statistical tests appear to be appropriate (e.g., are the tests appropriate for the level of measurement of key variables)? Were any multivariate procedures used? If not, should multivariate analyses have been conducted--would the use of a multivariate procedure strengthen the internal validity of the study? Were the results of any statistical tests significant? Nonsignificant? What do the tests tell you about the plausibility of the research hypotheses? Was an appropriate amount of statistical information reported? Were important analyses omitted, or were unimportant analyses included? Were tables used judiciously to summarize statistical information? Is information in the text and tables totally redundant? Are the tables clear, with a good title and carefully labeled headings? Is the researcher sufficiently objective in reporting the results?

top

Qualitative Analyses Based on; information in the report regarding either the analysis strategy or the research tradition, what type of analysis style appears to have been used? Is the initial categorization scheme described? If so, does the scheme appear logical and complete? Was the scheme validated in any way (e.g., did more than one person develop it collaboratively)? Who coded the data--the researcher or assistants? Did the report indicate that efforts were made to determine interrater reliability of the coding? Does the report describe the process by which an integrated thematic analysis was performed? What major themes emerged? If excerpts from the data are provided, do the themes appear to capture the meaning of the narratives (i.e., does it appear that the researcher adequately interpreted the data and conceptualized the themes)? Is the analysis parsimonious--could two or more themes be collapsed into a broader and perhaps more useful conceptualization? What evidence does the report provide that the researcher's analysis is accurate and replicable? Were data displayed in a manner that allows you to verify the researcher's conclusions? Was a conceptual map, model, or diagram effectively displayed to communicate important processes? Was the context of the phenomenon adequately described? Does the report give you a clear picture of the social or emotional world of the study participants? If the result of the study is an emergent theory or conceptualization, does it seem trivial or obvious? does the scheme yield a meaningful and insightful picture of the phenomenon under study?

top

Conduct of a Written Research Critique Be sure to comment on the study's strengths as well as its limitations. The critique should be a balanced consideration of the worth of the research. Each research report has some positive features. Be sure to find them and note them. Give specific examples of the study's strengths and limitations. Avoid vague generalizations of praise and fault finding. Try to justify your criticisms. Offer a rationale for how a different approach would have solved a problem that the researcher failed to address. Be as objective as possible. Try to avoid being overly critical of a study because you are not particularly interested in a topic or because you have a world view that is inconsistent with the underlying paradigm. Be sensitive in handling negative comments. Try to put yourself in the shoes of the researcher receiving the critical appraisal. do not be condescending or sarcastic. Suggest realistic alternatives that the researcher (or future researchers) might want to consider. Don't just identify problems--offer some recommended solutions, making sure that the recommendations are practical ones. Evaluate all aspects of the study--its substantive, theoretical, methodologic, ethical, interpretive, and presentational dimensions.

top

The Interpretative Dimensions of a Research Report Does the discussion section offer conclusions or interpretations for all the important results? Are the interpretations consistent with the results? Do the interpretations give due consideration to the limitations of the research methods? What types of evidence in support of the interpretation does the researcher offer? Is that evidence persuasive? Are the results interpreted in light of findings from other studies? Are alternative explanations for the findings mentioned, and is the rationale for their rejection presented? In quantitative studies, does the interpretation distinguish between practical and statistical significance? Are generalizations made that are not warranted on the basis of the sample used? Does the researcher offer implications of the research for nursing practice, nursing theory, or nursing research? Are the implications appropriate, given the study's limitations? Are specific recommendations for practice or future studies made? Are the recommendations consistent with the findings and consistent with the body of knowledge on the topic?

top

The Presentation of a Research Report Does the report include a sufficient amount of detail to permit a thorough critique of the study's purpose, conceptual framework, design and methods, handling of critical ethical issues, analysis of data, and interpretation? Is the report well written and grammatical? Are pretentious words or jargon used when a simpler wording would have been possible? Is the report well organized, or is the presentation confusing? Is there an orderly, logical presentation of ideas? Are transitions smooth, and is the report characterized by continuity of thought and expression? Is the report sufficiently concise, or does the author include a lot of irrelevant detail? Are important details omitted? Does the report suggest overt biases? Is the report written using tentative language as befits the nature of disciplined inquiry, or does the author talk about what the study did or did not "prove"? Is sexist language avoided? Does the title of the report adequately capture the key concepts and the population under investigation? Does the abstract (if any) adequately summarize the research problem, study methods, and important findings?

top

The Clinical Relevance of a Body of Research Does the research have the potential to help solve a problem faced by practicing nurses? Does the research have the potential to help with clinical decision-making with regard to (1) making appropriate observations, (2) identifying client risks or complications, or (3) selecting an appropriate intervention? Are clinically relevant theoretical propositions tested by the research? If the research involves an intervention, does the intervention have potential for use in clinical settings? Can the data collection measures used in the research be applied to clinical practice?

top

The Implementation Potential of an Innovation Transferability of the Findings Will the innovation "fit" in the proposed new setting? How similar are the target populations in the research and in the new setting? Is the philosophy of care underlying the innovation compatible with the philosophy prevailing in the new setting? Is there a sufficiently large number of clients in the new setting who could benefit from the innovation? Feasibility Will nurses have the authority to carry out the innovation and to terminate it if it is considered undesirable? Is there reasonable consensus among staff, administrators, and medical personnel that the innovation be tested? Are there major pockets of resistance that could undermine efforts to implement and evaluate the innovation fairly? Are the skills needed to carry out the utilization project--both the implementation and the clinical evaluation--available within the nursing staff? If not, are there avenues to collaborate with others with the necessary skills? Does the organization have the equipment and facilities necessary for the innovation? If not, is there a way to obtain needed resources? If nursing staff need to be released from other practice activities to learn about and implement the innovation, what is the likelihood that this could happen? Cost-Benefit Ratio What are the risks to which clients would be exposed during the implementation of the innovation? What are the potential benefits? What are the risks of maintaining current practices (i.e., the cost of not trying the innovation)? What are the benefits? What are the short-term and long-term material costs of implementing the innovation? What are the material costs of not implementing the innovation (i.e., could the new procedure result in some efficiencies that could lower the cost of providing care)? What are the potential nonmaterial costs of implementing the innovation to the organization (e.g., lower staff morale, staff turnover, absenteeism)? What are the potential nonmaterial benefits of implementing the innovation (e.g., improved staff morale, improved staff recruitment, positive community publicity)?

top