Comparison of Activities: Research versus Evaluation, Assessment, Quality Improvement, and Quality Assurance

Summary

Determining whether a project requires IRB review depends on whether it constitutes research involving human subjects. The table below is intended to help determine whether a project requires submission to the IRB as a research project involving human subjects. Many evaluation projects look like research, but do not fall into the regulatory definition of research.

Full Description

If the project involves at least some characteristics of a research project, the IRB should be consulted before implementation. If you would like a formal determination as to if the assessment, evaluation, or QI/QA project requires IRB review and approval, please submit an “Oversight Determination” form in SPARCS Cayuse. Please contact the W&M IRB Office with any questions.

Table compares research studies to assessment and evaluation projects.
Topics	Research	Evaluation/Assessment/QI/QA
Definition	A systematic investigation, including research development, testing, and evaluation, designed to develop or contribute to generalizable knowledge.	A systemic process that uses empirical data to refine outcomes related to programs, initiatives, learning, behaviors, and processes. As a continuous process, evaluation establishes measurable and clear outcome, provisioning a sufficient amount of opportunities to achieve these outcomes, implementing a systematic way of gathering, analyzing and interpreting evidence to determine how well the outcomes match expectations, and using the collected information to inform improvement in programs, initiatives, learning, behaviors, and processes. Evaluation, Assessment, Quality Improvement (QI), and Quality Assurance (QA) can be synonymous though there may be some nuanced differences among definitions.
Intent	Intent of project is to develop or contribute to generalizable knowledge Seeks answers to questions to generate new knowledge or understanding.	The intent of this project is to improve a specific program or support student learning. In some cases, W&M may be asked to participate in national research initiatives, during which data will be collected from the W&M population. In such cases, W&M will receive de-identified data from the national research initiative. Though the national organization is implementing research, they have their own IRB responsibilities. Because W&M is not engaging in research, and instead simply implementing an assessment, no IRB approval through W&M is required to implement the data collection procedures or receive the de-identified data. Data generated is used to inform and guide decision making, to determine if a program or initiative is making progress towards its goals, to determine long term effects of a program or initiative
Motivation for Project	Project occurs in large part because of individual professional goals and requirements (e.g., seeking tenure; obtaining grants)	Project not initiated by the evaluator and occurs regardless of whether individual(s) conducting it may benefit professionally from conducting the project
Design	Designed to develop or contribute to generalizable knowledge; may involve randomization of individuals to different treatments, regimens, or processes	Not designed to develop or contribute to generalizable knowledge; does not usually involve randomization of individuals, but may involve comparison of variations in programs
Mandate	Activities not mandated by institution or program	Activity mandated by the program, usually it's funder, as part of its operations
Effect on Program or Practice Evaluated	Findings of the study are not expected to directly affect institutional or programmatic practice	Findings of the evaluation are expected to directly affect the conduct of the program and identify improvements
Population and Sample	Usually involves a subset of individuals - universal participation of an entire clinic, program, or department is not expected; generally, statistical justification for sample size used to ensure endpoints can be met	Information on all or most participants within or affected by receiving a particular treatment or undergoing a particular practice or process expected to be used exclusion of information from some individuals significantly affects conclusions.
Benefits	Participants may or may not benefit directly – benefit, if any, to individuals incidental or delayed	No benefit to participants expected; evaluation concentrates on program improvements or whether the program should continue
Dissemination of Results	Intent to publish or present generally presumed at the outset of project as part of professional expectations, obligations; dissemination of information usually occurs in research/scientific publications or other research/scientific forum; results expected to develop or contribute to generalizable knowledge by filling a gap in scientific knowledge or supporting, refining, or refuting results from other research studies	Intent to publish or present is generally presumed at the outset of the project; dissemination of information to program stakeholders and participants; may be publicly posted (e.g., website) to ensure transparency of results; when published or presented to a wider audience – it is usually an assessment conference or journal, the intent is to suggest potentially effective models, strategies, assessment tools or provide benchmarks or base rates rather than to develop or contribute to generalizable knowledge
Clinical Settings and Standard Practice: Use of Placebo	Use of placebo may be planned	N/A
Clinical Settings and Standard Practice: Deviation from Standard Practice	May involve significant deviation from standard practice	N/A
Records Retention	Minimum 5 years from completion unless there is a contractual obligation, scientific justification, or participant protection as to why it would be retained for a shorter or longer period.	Minimum 5 years from completion.
Applicable Regulations and Standards	45 CFR 46 , FDA , DoD , Federal and State Laws, University Policy, Cultural Context	CAS , CDC , Joint Committee on Standards for Educational Eval
IRB Review Required	Yes	No
W&M Contact	W&M IRB Office	W&M IAE Office - Academic Assessment W&M Planning, Strategy & Integrative Practices Team W&M Career Office

Completing the IRB Application

Below, you will find guidance and template language regarding “general considerations” for when research and assessment, eval, or QA/QI overlap and how to communicate what is occurring in the IRB application and study materials. Specifically, the below guidance addresses:

When you want to use already existing assessment/evaluation information for research
When you want to implement assessment/evaluation activities and use the same information/collection methods for research purposes
When you want to implement assessment/evaluation activities and research activities, and some activities overlap and some activities do not overlap

Guidance and Template Language