Internal and External Validity
I need a response to student discussion
an explanation of a threat to internal validity and a threat to external validity in quantitative research. Next, explain a strategy to mitigate each of these threats. Then, identify a potential ethical issue in quantitative research and explain how it might influence design decisions. Finally, explain what it means for a research topic to be amenable to scientific study using a quantitative approach.
Be sure to support your Main Issue Post and Response Post with reference to the week’s Learning Resources and other scholarly evidence in APA Style. Respond to a classmate by offering a strategy to address the ethical issue she or he identified.
Student discussion response:
There are four main types of Quantitative research: Descriptive, Correlational, Causal-Comparative/Quasi-Experimental, and Experimental Research. attempts to establish cause-effect relationships among the variables. These types of design are very similar to true experiments, but with some key differences.
In quantitative research, the term validity is often used to discuss measurement validity, which refers to the degree to which an instrument or test measures what is supposed. Research is an indispensable part of modern reality, facilitating the progress and development of societies, economies, and individuals. Thus, it is crucial to ensure the quality of research at all stages. The most critical aspect of quality is research validity, or whether the results of studies are interpreted and understood correctly. Threats to validity are characteristics of research designs that lessen the degree to which results are interpreted correctly. This entry first discusses conceptual frameworks of research validity and then looks at specific types of validity threats and ways of avoiding validity threats.
Please note that validity discussed here is in the context of experimental design, not in the context of measurement.
Internal validity refers specifically to whether an experimental treatment/condition makes a difference to the outcome or not, and whether there is sufficient evidence to substantiate the claim.
External validity refers to the generalizability of the treatment/condition outcomes across various settings.
Efficacy studies in experimental settings are conducted to address the issue of internal validity whereas effectiveness studies in naturalistic settings (the “real” world) are employed to examine the external validity of the claim.
Factors that jeopardize internal validity
History–the specific events which occur between the first and second measurement. Maturation–the processes within subjects which act as a function of the passage of time. i.e. if the project lasts a few years, most participants may improve their performance regardless of treatment. Testing–the effects of taking a test on the outcomes of taking a second test. Instrumentation–the changes in the instrument, observers, or scorers which may produce changes in outcomes.
Statistical regression–It is also known as regression to the mean. This threat is caused by the selection of subjects on the basis of extreme scores or characteristics. Give me forty worst students and I guarantee that they will show immediate improvement right after my treatment.
Selection of subjects–the biases which may result in the selection of comparison groups. Randomization (Random assignment) of group membership is a counter-attack against this threat. However, when the sample size is small, randomization may lead to Simpson Paradox, which has been discussed in an earlier lesson.
Experimental mortality–the loss of subjects. For example, in a Web-based instruction project entitled Eruditio, it started with 161 subjects and only 95 of them completed the entire module. Those who stayed in the project all the way to end may be more motivated to learn and thus achieved higher performance.
Selection-maturation interaction–the selection of comparison groups and maturation interacting which may lead to confounding outcomes, and erroneous interpretation that the treatment caused the effect.
John Henry effect–John Henry was a worker who outperformed a machine under an experimental setting because he was aware that his performance was compared with that of a machine. Factors that jeopardize external validity Reactive or interaction effect of testing–a pretest might increase or decrease a subject’s sensitivity or responsiveness to the experimental variable. Indeed, the effect of pretest on subsequent tests has been empirically substantiated (Willson & Putnam, 1982, Lana, 1959). Interaction effects of selection biases and the experimental variable Reactive effects of experimental arrangements–it is difficult to generalize to non-experimental settings if the effect was attributable to the experimental arrangement of the research. Multiple treatment interference–as multiple treatments are given to the same subjects, it is difficult to control for the effects of prior treatments.
Threats to validity include
Selection–groups selected may actually be disparate prior to any treatment.
Mortality–the differences between O1 and O2 maybe because of the drop-out rate of subjects from a specific experimental group, which would cause the groups to be unequal.
Others–Interaction of selection and maturation and interaction of selection and the experimental variable.
Campbell, D. & Stanley, J. (1963). Experimental and quasi-experimental designs for research. Chicago, IL: Rand-McNally.
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston, MA: Houghton Mifflin Company.
Threats to Research Validity, The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation https://dx.doi.org/10.4135/9781506326139.n701