Research Evaluation Worksheet

Select  a research article of interest to you, preferably related to your Research Proposal, and use the Research Evaluation Worksheet to analyze the article. You can use this information to help you form the literature review section of your research proposal.

Research Evaluation Worksheet

Title:

Full Article Reference (APA style):

Abstract

Introduction

a. Is the need for the study clearly stated in the introduction? Explain by using information presented in the literature review.

b. What is the research hypothesis or question?

c. What are the variables of interest (independent and dependent variables)?

d. How are the variables operationally defined?

Method

a. Sample Size (Total): ________________ Size Per Group/Cell: _______________

b. Were the methods and procedures described so that the study could be replicated without further information? What information, if any, would you need to replicate or reproduce this study?

Participants

a. How were participants selected and recruited? b. Were subjects randomly selected? c. Were there any biases in sampling? Explain

d. Were the samples appropriate for the population to which the researcher wished to generalize?

e. What are the characteristics of the sample populations?

Research Design (check which design applies)

_______ Single group, time series study

_______ Multiple baseline (sequential) design: ______________

_______ Single group, no measurement

_______ Single group with measurement: Pre ______ During _____ Post _____

_______ Two groups classic experimental versus control group, randomly assigned

_______ (quasi-experimental) two groups experimental versus control group,

not randomly assigned

_______ Correlation research, not manipulated, degree of relationship

_______ Descriptive research (qualitative study)

_______ Natural observation

_______ Analytical research

_______ Interview research

_______ Historical study

_______ Survey research

_______ Legal study

_______ Ethnography research

_______ Policy analysis

_______ Fieldwork research

_______ Evaluation study

_______ Phenomenology

_______ Grounded theory

_______ Protocol analysis (collection and analysis of verbatim reports)

_______ Case study, no measurement

_______ Case study, with measurement: Pre _________ During _______ Post _________

_______ Developmental research

_______ Longitudinal (same group of subjects over period of time)

_______ Cross-sectional (subjects from different age groups compared)

_______ Cross-sequential (subjects from different age groups, shorter period of time)

_______ Correlation, more than two groups: control, treatment, and other treatment comparisons

_______ Factorial design, two or more groups: other treatment differences, no untreated controls

_______ Two or more dependent variables (MANOVA)

_______ Other design: __________________________________________________________

Consider the Following Questions:

a. Was a control group used? Yes ______ No ______ If yes, complete b, c, and d below. b. Was the “control” method for the study appropriate?

c. What variable was being controlled for?

d. In the case of an experimental study, were subject randomly assigned to groups?

Measures

a. Describe the Dependent Measure(s)/Instruments used:

b. Describe the Measurement/Instrument Validity Information:

c. Describe the Measurement/Instrument Reliability Information:

Consider the Following Questions:

a. For all measures (measures to classify subjects, dependent variables, etc..) was evidence of reliability and validity provided, either through summarizing the data, or by referring the reader to an available source for that information?

b. Do the reliability and validity data justify the use of the measure?

c. Are the measures appropriate (if not, why not)?

d. Are multiple measures used, particularly those that sample the same domains, or constructs but with different methods (e.g., self-report, rating scales, self-monitoring, or direct observation)?

f. If human observers, judges, or raters were involved, was inter-observer or inter-rater agreement (reliability) assessed? Was it obtained for a representative sample of the data? Did the two raters do their ratings independently? Was their reliability satisfactory?

Independent and Dependent Variables

a. What is/are the Independent Variable(s):

b. What is/are the Dependent Variable(s):

Data Analysis

Scales of Measurement (check those that apply):

Nominal _______ Ordinal _______ Interval _______ Ratio _______

a. What type of statistical techniques are used?

b. What type of tables and graphs are used?

Consider the Following Questions:

a. Were tests of significance used and reported appropriately (e.g., with sufficient detail to understand what analysis was being conducted)?

b. Do the researchers report means and standard deviations (if relevant) so that the reader can examine whether statistically significant differences are large enough to me meaningful? c. Other comments on the reported statistical analyses?

Discussion

Evaluate the Summary and Conclusions of the study (Usefulness):

Describe the Strength(s) and Limitation(s) of the Study:

Describe what you learned from the study:

List any remaining questions you have about the study:

*Adapted from form created by Dr. Randy Buckner, University of Phoenix Instructor