Navigate the WM Web W&M Home Contact the College Directions & Maps W&M A-Z Prospective Students Parents Alumni & Friends Current Students Faculty & Staff Campus Visitors About W&M Academics   Undergraduate   Graduate Administration Admission Athletics Board of Visitors Campus Construction Campus Life Development Office Diversity @ W&M Economic Development Employment Opportunities Events/Calendars Financial Aid Gateway W&M Giving to W&M Human Resources Libraries News & Media Office of the President Public Service Research Technology myWM Calendar view Submit an event Selected calendars Academic Alumni Athletic Box Office Dance Law School Lively Arts Series Muscarelle Museum of Art Music department The Andrews Gallery Theatre VIMS

School of Education: NCATE Focused Visit 2005

Standard 2: Assessment System and Unit Evaluation Addendum

This addendum to the Assessment System and Unit Evaluation section of the Institutional Report is intended to provide additional information concerning the operation of the Assessment System, particularly in regard to the flow and usage of data at the Unit level. The Standard 2 section of the Institutional Report presented the general assessment plan elements, including the transition points for the School of Education. Table 2A.1 was presented as the model for the major assessments. Table 2A.1 serves as the template for the Assessment System in the School of Education. Individual programs populate the template with the standards and assessments that are most appropriate for them. Thus, the specific elements of the system differ by program and are delineated in each program’s specific assessment plan.

Table 2A.1. Major Assessments for the School of Education

Key Assessment Points	Standards Addressed	Types of Assessments	Sources of Information	Stage of Implementation
1.    Admissions	(Conceptual Framework, Specialty Organization, State, Program Specific)	• GPA • Admissions Essay • References • Employment History (graduate only) • GRE or MAT (graduate only)	• Transcripts • Application • Reference Forms • Application • ETS	Complete.
2. Preparation Program	-	-	-	-
A.  Coursework and Practica	(Conceptual Framework, Specialty Organization, State, Program Specific)	• Tests • Papers • Projects • Practicum Evaluation	• Instructor • Instructor • Instructor • Field Supervisor and University Faculty	Substantially complete. (Aggregation of data at Area and Unit level to be completed.)
B.  Clinical Experience	(Conceptual Framework, Specialty Organization, State, Program Specific)	• Application • Test Scores (Praxis I) • Mid-cycle and Final Evaluation • Portfolio Review	• Student • ETS/Student • Field and Univ. Supervisors • Faculty	Substantially complete. (Aggregation of data at Area and Unit level to be completed.)
3. Graduation (Exit)	(Conceptual Framework, Specialty Organization, State, Program Specific)	• Degree Audit • Verification for License • Test Scores (Praxis II) • Portfolio Review • Project / Thesis / Dissertation Review (Graduate only)	• SOE Director of Records and Matriculation • SOE Licensure Specialist • ETS/Student • Faculty • Faculty	Substantially complete.  (New Praxis requirement requires implementation. Portfolio requires aggregation.)
4. Post-Graduation       (Follow-up)	(Conceptual Framework, Specialty Organization, State, Program Specific)	• Survey of Graduates • Survey of Employers • K-12 Student Outcomes	• Graduates • Employers • Virginia Unified Data System	In progress. (Several surveys of graduates have been conducted. Employer and Outcomes surveys to be developed.)

Figure 2A.1 (see link below) shows a diagram of the intended flow of assessment data. It is important to note that Figure 2A.1 is not a decision-structure diagram. The figure is intended to indicate how data are considered within the organizational structures of the School. The orange triangle represents the changes that may occur as a result of consideration of the data at any level. Program, Area, and Unit groups are expected to consider data and make proposals for changes that may impact at any level. The diagram indicates that assessment data are expected to move through multiple analysis and synthesis points before being considered at the Unit level. All elements in the blue, dotted rectangle on right side of the diagram represent Unit-level consideration. Assessment system data enter the system through any of the evaluations indicated in the Program-level assessment plans based on the template in Table 2A.1 above. Some of the data, such as admissions data, are initially processed by School of Education Offices.  Other data, such as portfolio analyses, are brought into the system by faculty. Regardless of how the data initially enter the system, they are first considered at the Program level. Program faculty use the data for individual student evaluation and monitoring of student progress through key transition points. They also use these data for program evaluation as evidenced in the Assessment Portfolios. Based on their findings, Programs make recommendations for improvements through the appropriate structures of the School.

Programs are also responsible for providing summaries of their data and actions to the Areas with which they are affiliated. The actual processing of the data may be centralized within an administrative office (e.g., aggregation of student teaching evaluations by the Office of Professional Services) or it may be done by program faculty (e.g., summary data on student portfolios). Areas consider the information to evaluate the functioning of their programs and to gain perspective on the functioning of the Area as a whole. Although these data are shared on an ongoing basis, at least once each academic year (no later than September) Areas meet to discuss aggregate information. Based on their findings, Areas make recommendations for improvements through the appropriate structures of the School. For example, changes in Program requirements or courses are presented first to the Area Faculty for support, then to the Curriculum Committee, and finally to the Faculty of the Whole.

Figure 2A.1 Click here to view Unit Assessment System Data Flow

Areas also have the responsibility to report to the Unit level. Again, the actual processing of the data may be centralized to an administrative office or it may be completed by Area faculty. As indicated in the Standard 2 section of the Institutional Report, each Area will produce an annual report detailing the results of its assessments. These reports will focus on the Conceptual Framework, common dispositions, diversity, professional standards, and technology. The Areas will report on their recommendations for improvement and update the status of previous recommendations.

The Unit level is represented by the Committees of the School, the Dean, and the Faculty of the Whole. The Assessment System has been designed to utilize the existing governance and committee structure of the School, with the addition of the Standards and Assessment Committee. All of the existing committees function at the Unit level and have responsibilities to report to the Dean or the Faculty of the Whole. Our Assessment System brings a new focus and responsibility to the current operation of these Committees.

Relationship of School of Education Committees to the Assessment System

Standing Committees

The Assessment System utilizes the existing committee and governance structure of the School. The Curriculum Committee, Admissions and Financial Aid Committee, Faculty Affairs Committee, and the Executive Committee are standing committees of the School. The roles of these committees are described in the By-Laws of the School. The Curriculum Committee is charged with the review of curricular proposals for all programs of the School. The Committee is also charged to “conduct continuing studies of educational developments and present recommendations as appropriate to the Faculty.” In this function, the Committee will fit into the Assessment System as the Unit body reviewing and studying the curricular outcomes and suggestions of the Programs and Areas. The Curriculum Committee will be a primary recipient of Exit and Follow-up data from the Assessment System.

The Admissions and Financial Aid Committee is already charged to “review admissions and financial aid decisions to ensure that they comply with the approved School of Education and Area policies.”   The role of the Admissions and Financial Aid Committee in the new Assessment System will be to evaluate the admissions process in terms of its alignment to the Conceptual Framework and other Unit-level standards. The Committee will report to the Faculty on the alignment and success of the admissions process on a yearly basis.

The Faculty Affairs Committee is charged to “advise the Dean on faculty compensation, nominations and appointments, and examine other issues which may affect the welfare of the Faculty collectively or individually.” The Faculty Affairs Committee will be expected to monitor the information from the Assessment System and advise the Dean on issues they feel relevant to Faculty welfare. This Committee should provide advice on faculty development opportunities they perceive as necessary based on the data from the system.

The Executive Committee “regularly advises the Dean on all matters pertaining to the School and shares responsibility for governance of the School of Education.” This Committee is central to the functioning of the Assessment System at the Unit level. With the Dean, this Committee plays a significant role in shaping the policies and direction of the School. Members will review the reports of all other committees concerning data from the Assessment System. Within their governance role, the members will strive to offer advice in a manner that is reflective of the Conceptual Framework and standards of the School. The Committee will consider outcome evidence from the Assessment System in advising the Dean on plans, priorities, and resource allocation.

Working Committees

The Diversity, Educational Technology, and TEAC committees are long-standing working committees of the School. Working committees of the School are appointed by the Dean and can continue their function as needed. The Diversity Committee is a recent reorganization of the Affirmative Action Committee with a broader focus on issues of diversity beyond employment and admissions. This Committee will have the role of examining programs’ efforts to ensure that candidates develop the knowledge, skills, and dispositions to ensure that all students learn. The Educational Technology Committee has the primary responsibility to advise on the allocation of technology resources. This Committee will continue in that role but will add the responsibility of examining the use of technology in instruction, assessment, follow up, and program evaluation. TEAC has been the committee used to obtain formal input and feedback from Arts and Sciences and K-12 Partners, and it will continue in this role within the Assessment System. TEAC will examine data related to the outcomes of candidate and program assessments related to their roles as School of Education partners. In particular, these groups will review data related to content area assessments (e.g., Praxis results) and clinical performance (e.g., student teaching assessments). TEAC will continue to serve as the advising panel on Teacher Education related issues.

Standards and Assessment Committee. The Standards and Assessment Committee is a new committee that represents an extension of the NCATE Assessment Committee. The original Assessment Committee and the School recognized that our existing committees did not cover all of the necessary aspects of Unit-level analysis that are required in the new Assessment System. Therefore, the Standards and Assessment Committee was proposed as the Unit-level committee with primary responsibility to address the Unit as a whole on the standards of quality. The committee will have the same composition as the Executive Committee with two representatives from each of the Areas. In addition, the Associate Deans for Academic Programs and Professional Services will serve on the Committee with the Associate Dean for Academic Programs acting as Chair.

A primary role of the Standards and Assessment Committee will be to receive and review the annual reports of the Areas. The Committee will provide a summary to the Dean and Faculty on the performance of the School on the aspects of the Conceptual Framework, common dispositions, diversity, professional standards, and technology. This summary will focus on Unit aggregate functioning on the elements that are common to the School. The Committee will make recommendations for improvement to the appropriate groups based on their analysis of the Area Reports.

The Standards and Assessment Committee will also monitor the reports of other committees of the School in order to gather information on Unit functioning. In addition, the Committee will review the Dean’s yearly progress report for information concerning Unit-level functioning, planning, and decision making.

The Committee will act as a conduit for alignment of standards and assessments within the School. Recommendations for changes to the Conceptual Framework, general assessment system, and common dispositions will be brought to the Faculty by this Committee. The Committee will examine the consistency between the plans of the School and the Conceptual Framework.

The Committee will make recommendations on the systems to collect and review data. They will also recommend and review studies related to the technical adequacy of assessments. The Committee will primarily act in a quality assurance role with a focus on making recommendations that promote the production of high quality candidates.

The Unit Assessment System and Data Flow

The features delineated in Table 2A.1 and Figure 2A.1 operate as an overall Assessment System that will allow for the assessment of individual candidates and provide the basis for Program, Area, and Unit analyses and evaluation. Essentially, the features in the Table and the Figure can be viewed as describing two functions within the same System. Table 2A.1 is the template for the School’s assessment systems, and describes the key transition points that are designed to monitor the progress of individual candidates through their programs of study and beyond. The assessments that populate the system can be used both to evaluate the candidates and to provide data concerning the programs that prepare them. These data are the essential input to the data flow process illustrated in Figure 2A.1. The attention in this latter process is on the functioning of the Unit and its programs. The data from the Assessment Systems of the Programs are passed from Program to Area to Unit. At each point in the data flow, the focus and analysis is successively higher. The data used for the analysis at each successive step also changes. While Programs will be interested in how their candidates perform compared to specific SPA and state standards, Areas are comprised of multiple Programs with differing specific standards. Therefore, Area-level analyses will focus on common elements among its Programs. Likewise, Unit-level analysis will concentrate on aspects that are common to the Unit. The Conceptual Framework, dispositions, diversity, and technology are elements that have common threads throughout the Unit. There are also School and College regulations (e.g., academic continuation requirements) that apply to all students. These will be the focus of Unit-level evaluation.

Ultimately as the Unit level, the Dean and Faculty of the Whole are responsible for the interpretation and use of assessment data. A strength of the Assessment System and the general governance structure of the School is the participation of the Faculty of the Whole and the Dean in decisions impacting the School. This shared governance structure, deeply rooted in the culture of William and Mary, encourages Unit participation by including the Faculty of the Whole in all decisions regarding policy. The committee structure has been established to be representative of the Areas, but those Committees must report and bring change recommendations to the Faculty of the Whole. Through this system, the Faculty of the Whole monitors and participates in the operation of the School. The Faculty works in concert with the Dean to evaluate all aspects of the Unit. The Dean has the general responsibility for the Faculty of the School and for the direction of the work of the Faculty. The Dean also has leadership responsibilities in the development of educational programs. Additionally, the Dean is the conduit of information about the School to the Provost, President, and the Board of Trustees. Therefore, it is essential that the Dean be the recipient of data from Assessment System that provides information concerning Program, Area, and Unit functioning. The Dean of the School has the responsibility to report to the Faculty, Provost, President, and the Board of Trustees on the status of the School. While this responsibility is currently carried out on a regular basis through periodic meetings, the Dean provides a more formal annual report on the state of the School to the Faculty. Given the new Assessment System, the Dean’s statement will now address the evidence of the School’s progress in meeting the expectations of the Conceptual Framework, common dispositions, diversity, professional standards, and technology. Based on the analysis of the data, the Dean will make recommendations for improvements to Programs, Areas, or the Unit. Additionally, the Dean will use the results of the analysis in working directly with the Provost on academic planning, personnel matters, actions relating to faculty, and on curricular, budgetary, and educational policy matters.