Building a Validity Argument While Developing and Using an Assessment: A Concurrent Approach for the Winsight® Summative Assessment

We describe the summative assessment component within a K–12 assessment program and our development of a validity argument to support its claims with respect to intended uses and interpretations. First, we describe the Winsight ® assessment program theory of action, a logic model elucidating mechanisms for how use of the assessment components can lead to improved student learning in mathematics and English language arts (ELA) and defining the roles of key groups of stakeholders. We explain our process of developing the validity argument that enumerates sources of validity evidence supporting the Winsight summative assessment claims drawn from the theory of action. We examine candidate structures for the argument, describe the steps in which we unpacked stakeholder‐level claims and defined subclaims to operationalize components, and discuss types and sources of validity evidence to meet the goals of professional and federal guidelines. Finally, we discuss the research agenda and operational analyses that will provide this validity evidence.