Guidelines for Academic Program Assessment
Institutional Research
Appendix D
Assessment at Oswego is structured and carried out in the context of statewide and SUNY mandates for assessment, the ongoing requirements of both regional and professional accrediting bodies, and standards established by disciplinary societies and organizations. As a result, the assessment of our academic programs addresses both student learning outcomes and other quality indicators. Specifically, the State University of New York links the assessment of major programs to a cycle of departmental program reviews, with participation of teams of external reviewers, i.e., consultants or accrediting teams. Through this program review process, quality indicators appropriate to disciplines or programs are identified by departments, and a systematic data collection/analysis scheme provides departments with information that can be used to improve curricula and other services to students.
The most compelling reason to engage in meaningful assessment is the improvement of teaching and learning. The assessment process itself often evokes meaningful discussion among faculty about educational intent, outcomes they wish to see in their students, and identification of ways to accomplish and measure attainment of the outcomes: debate about the best way to realize educational goals is in itself a desired and documentable benefit of assessment. The ideal, however, remains for outcomes assessment to be used to affect program improvements (to "close the assessment loop"), and as an opportunity to tell our story: to discuss the accomplishments of our students.
Principles Underlying Program Assessment
Primary emphasis should be placed on assessment as a means of improving academic program, pedagogy and student learning.
Assessment results should never be used to punish, publicly compare students, faculty, courses, program, department, or to make public comparisons among groups of students based on gender, race, or ethnicity or other demographic characteristics.
While individual programs are free to use their own assessment results in ways they see fit, campus may publicly disseminate assessment ONLY through aggregate data for the institution as a whole.
Department/programs should have maximum autonomy in the development of their assessment plans for academic majors and should include the input of faculty and students.
Whenever feasible, assessment of academic programs should respond to standards established by appropriate external organizations.
Assessment programs should include, but not be limited to, those other review activities associated with general and specialized accreditation conducted by professional and accrediting bodies.
Program assessment should take place on a regular basis and there should be periodic institutional review of programs.
Program assessment, where possible and appropriate, should use institutional data and data from established assessment programs.
Assessment programs may include nationally available instruments, campus designed measures or a combination of the two. The choice of measures will vary depending on the particular goals and objectives of individual programs.
The assessment of major programs must include the assessment of student learning.
Guidelines for the Development or Revision of Departmental Assessment Plans
In order to establish some commonality in our assessment programs, academic departments are asked to follow the following general outline in preparing their Assessment Plan Documents (APD):
1. a clearly defined statement of your program mission (cf. mission statement of the institution);
2. a specific description of the program's most salient educational goals, i.e., intended learning outcomes for students;
3. an identification of activities intended to address the program's intended educational goals and learning outcomes;
4. a description of assessment methods/instruments you will utilize to determine how successfully each educational goal is being met. SUNY mandates that student learning be assessed over time, using multiple methods/instruments;
5. an estimate of cost and resources required to implement your assessment plan;
6. a timetable for implementation of your assessment plan;
7. a description of the processes by which you will regularly review and revise your academic program in light of the results obtained in #4 above.
Determining Educational Goals and Student Learning Outcomes
The educational goals and student learning outcomes for students are multi-faceted:
1. some goals are intended to result in "content" outcomes e.g., descriptions of what you intend students to know;
2. other goals are focused on how you wish students to be able to think e.g., establishing fundamental analytic skills or more broad-based attitudes towards a discipline;
3. still other goals are directed at what students are able to do (behavior) after completing a program e.g., research or applications of the discipline.
Clearly no one academic department or program can define everything it wishes to accomplish in its educational mission. Departments should identify the most important goals of their educational efforts, those which they believe all students would attain upon completion of the academic program. It is also reasonable to articulate outcomes at an operational level at which faculty can best address and assess. This strategy has already been utilized in the General Education Program by identifying goals relative to Knowledge (cognitive), Skills (behavior), and Attitudes. While it might be difficult to translate departmental educational goals into these three categories, trying to do so usually helps faculty in understanding the interrelationship of courses, curriculum, and the departmental mission. When educational goals and student learning outcomes expectations are direct, simple, and clear it is easier for faculty to share their ideas with each other and develop ways to document their students learning. Three to five goal statements are often sufficient to direct the assessment effort.
Determining Assessment Procedures
Each intended student learning educational outcome should have at least one means identified to assess and document success. Qualitative as well as quantitative indicators are appropriate to employ in your plan.
Generally, the attainment of educational goals and student learning outcomes can be measured by testing students, observing students, surveying students, reviewing student performance, examining student productions, or a combination of such methods. Tests and other analytic techniques are commonly used to measure educational outcomes relevant to "student knowledge", and "conceptual understanding"; observation techniques are common methods of measuring educational outcome defined in terms of "student behavior"; and many educational outcomes relevant to student attitude and interest are measured by "asking" or surveying. In terms of the means by which appropriate data are collected, the department may choose to use commercially developed instruments; for example, American College Testing, Educational Testing Service, Riverside Publishing and other companies have developed/standardized a wide range of instruments to assess a variety of educational goals. Depending on the nature of the educational goal, the department may wish to develop its own instruments to collect assessment data. For example, some departments have developed comprehensive exams to measure student learning outcomes, some others utilize capstone projects or seminars. Finally, the data routinely collected at the college may be utilized by departments in their assessment plan. In this case, the Office of Institutional Research and Assessment will be the primary source of the information.
Implementation
A Departmental Assessment Plan is expected to be implemented within the time frame specified in the plan. Departments will be asked to submit an annual report regarding the status of their Assessment Plans and their progress in implementing them. For assessment report time table and reporting format see Appendix E.
Some Further Considerations
It is helpful to recognize that not all student outcomes are measurable, nor do all educational outcomes occur by the time students graduate; however, it is also true that some successful educational outcomes are easily assessed and do occur during the time that students are here at our college. Every public institution of higher education is facing this task--some are further along than others. Given the demands of our various constituencies, and our need to act responsibly, we will address their concerns and at the same time promote the success of our students using quality indicators of performance. This again does not mean that we seek quantifiable outcomes to the exclusion of qualitative outcomes; success is simply measured differently in some fields. You are free to choose ways that are appropriate to your discipline. Nevertheless, in your assessment plan, you should be able to identify the fundamental goals of your program by providing information relative to the degree of attainment of the student learning outcomes to which you are committed.