Outcomes Assessment Handbook

 


The Basics

Why Should I Do Outcomes Assessment?

What is My Role?

What Outcomes Assessment Resources are Available?

How Do I Do Outcomes Assessment? What's the Process?

The Basics

What Is outcomes assessment?

Outcomes assessment is a collaborative process of inquiry regarding student learning outcomes, followed by analysis, reflection, and action. The goal of outcomes assessment is to improve student learning and improve instructional programs. Outcomes assessment is not individual student, faculty, course, or program evaluation. Student learning outcomes are statements of what students know or can do upon successful completion of a course or program.

Outcomes Assessment Cycle

Outcomes assessment is a continuous cycle.

Here at Clark, we also like to think of it has a slinky toy that has been linked into a spiral, suggesting that the cycle continues to loop in distinct and continuous iterations.

                                                                             Outcomes Assessment Continuous Spiral

It can be easy to get caught up in the data-gathering phase and lose sight of the fact that outcomes assessment is a holistic process. Data-gathering is just one step in this process.

Back to top

How can I be involved with outcomes assessment? What is the IPT charge?

All full-time faculty members at Clark participate in Outcomes Assessment activities. The 2015-2016 IPT (Instructional Planning Team) charge is as follows:

The IPT-defined course- and program level-outcomes assessment activities for faculty
for 2015-2016 will consist of the following:

Glossary of terms

Click here to view a pdf glossary of commonly used terms in outcomes assessment at Clark College.

Back to top

What is a learning outcome?

Student learning outcomes (SLOs) provide direction for all instructional activity. They are statements of what students know or can do upon successful completion of a course or program.
SLOs should specify an action that is:

SLOs should be:

Back to top

How is outcomes assessment different from classroom assessment?

As an educator, you regularly assess your students' learning within your own classrooms, a process called classroom assessment. Outcomes assessment, by contrast, refers to the broader, collaborative activity of assessing student learning within a degree/certificate program (e.g. a Welding certificate or the AA degree). Additionally, faculty may also engage in outcomes assessment for a course (e.g. all sections of English 101), though our accreditation depends on our conducting outcomes assessment on our program-level outcomes.

Back to top

Why aren't grades enough?

Grades and outcomes assessment (OA) are used for different purposes. Grades represent the degree to which a student has met a faculty member's requirements and expectations for a class; however, some of these expectations—for example, attendance, participation, homework completion—may not be direct measures of whether the student has achieved the student learning outcomes for the course. In addition, where multiple sections of a course exist, there may be inconsistencies in grading practices between sections that make grades an inappropriate measure for student learning across all sections. Outcomes assessment, on the other hand, is a direct measure of student learning. The best OA plans use multiple points, tools, and methods to assess progress and achievement of SLOs.

Back to top

What is Clark's approach to outcomes assessment?

At Clark, our outcomes assessment mission is to improve student learning through the college-wide practice of meaningful, sustainable assessment of learning outcomes. OA gives faculty the opportunity to engage in the scholarship of teaching by collaboratively examining student learning within Clark College programs, following the assessment cycle. OA is not used to evaluate individual students, faculty, courses, or programs.

Back to top

Program assessment versus course assessment

Student learning outcomes (SLOs) should provide direction for all instructional activity. These outcomes can be assessed at different levels: they are statements of what students know or can do upon successful completion of a course or program.

Program-level assessment is used to determine how well the program as a whole prepares students to achieve the learning outcomes. It can also be used to identify curricular gaps.

Example program-level assessment questions include:

Common program-level assessment tools include:

Course-level assessment is used to determine how well all sections of a course prepare students to achieve course learning outcomes.

Example course-level assessment questions include:

Back to top

Why Should I Do Outcomes Assessment?

Who benefits from outcomes assessment?

The short answer is: everyone. The purpose of outcomes assessment (OA) is to enhance student learning. It's that explicit focus on student learning which makes OA a tool with many uses.

Primarily, OA produces reliable information that allows faculty to have meaningful conversations about how students are learning in the classes, then make informed decisions about how to better produce such learning in the future. By documenting student learning, OA projects also provide a picture of how that learning can be impacted by areas outside of instruction: staffing, facilities, scheduling, advising, etc. In this way, administrators and staff are able to see how their support of instruction impacts student learning and can make similarly informed decisions about how best to continue that support.

For this reason, OA provides valuable evidence to external accreditors that the college is engaged in meaningful reflections that place student learning at the heart of its purpose. The broader community benefits from OA in this same way, though explicit evidence of the knowledge and skills that the college is producing to the benefit of its surroundings.

Lastly, but crucially, OA benefits students. After all, their learning is the focus of OA work. Our underlying question--how we can best enhance student learning—is the common engine for all OA projects, ensuring that students' experience is given absolute priority in the daily work of the college.

Back to top

Student learning, retention, and completion

Retention and completion are often thought of as separate issues from outcomes assessment, unrelated to the learning that takes place in a classroom. The reality, however, could not be further from the truth. More accurately, student retention and completion are better understood as products, or effects, of student learning.

Barriers to student learning are undeniable and obvious barriers to course and degree completion. What's often missed, however, is the way that enhancing student learning can lead to increased retention and completion. A robust body of research suggests that a student who can successfully learn and progress in a class is far more likely to persist in, and complete, his or her path of chosen study. In this way, OA – by providing evidence of student learning and identifying barriers to that learning – is a key element of any successful bid to raise student retention/completion. Put simply, the learning comes first.

Back to top

From teacher-centered to learning-centered

Perhaps the most important benefit to participating in OA is that it sponsors meaningful, valid conversations among faculty about their teaching practices. A great deal of faculty development, however, is framed as "teaching improvement." While this is undeniably a vital element to maintaining qualified and effective practitioners, by focusing on student learning, OA offers a subtle, but critical, shift in thinking.

Generally, this shift in thinking moves our attention away from the means (teaching) and toward the end (learning). While a faculty member might walk away from a lesson fully confident in the effectiveness of her pedagogical approach, OA instead focuses on the students' experience of that lesson. Teaching and learning are, admittedly, intrinsically-linked sides of a single coin. However, the learning-centered approach made possible by OA allows faculty to look past the strategies they employ in the classroom and to think more concretely about the effects of those strategies on their students.

Back to top

Strategic planning and outcomes assessment

Clark College aims to employ a coordinated set of strategies to produce an agreed-upon, shared set of desired outcomes. However, one challenge with any large-scale human enterprise of this type is the difficulty in maintaining a sense of connection to, or consistency with, what are often abstract goals that are variously understood and supported.

Given the size of our college, Clark is particularly vulnerable to these challenges. However, the promise of OA is that is affords every member of the college community a common foundation to rest upon: student learning. Every strategy and initiative undertaken as a part of the 2015-2020 Strategic Plan can, and will, be assessed by its impact on student learning. OA, in this way, provides a consistent baseline for what can otherwise become a tangled mass of unconnected, vague attempts at "improvement." Student learning—as measured through outcomes assessment—will be an overriding indicator of the success of Clark's strategic plan.

Accreditation and outcomes assessment

External accreditors benefit from, and therefore require, outcomes assessment. For this reason, accreditation is frequently cited as the main impetus for conducting OA work. This "do it because we have to" rationale, however, is too simplistic: faculty who participate in OA simply out of fear of punishment, rather than for the invaluable insights OA can afford, are less likely to have high morale or engagement. It is true that OA is required by external accreditors, but it is not true that the work should be done only to suit their needs. OA is a tool with many uses and should be approached as such.

Back to top


Outcomes assessment is not another administrative fad or buzzword!

The practice of outcomes assessment can be traced back to the U.S. Department of Education's April 1983 report A Nation at Risk. While the push for accountability in higher education has taken many forms since then, OA is the only version of this initiative that places the values for student learning squarely in the hands of faculty. After several failed attempts to rate the effectiveness of a college from without, OA allows the faculty to take ownership of the learning that they themselves provide.

For this reason, and because of the many other stakeholders OA projects can support (see I.a), the work has been increasingly required by accreditors in recent years. Beyond just the work itself, new accrediting standards are asking colleges to engage in meaningful, widespread reflection on OA data. Locally, and nationally, faculty-led OA is seen as the solution to the accountability "crisis" in higher education. For this reason, it should not be seen as a new initiative that will eventually "go away." Rather, the future will bring greater emphasis on this work and its role in documenting the college's impact on students' lives.

Community colleges in general are likely to use OA data to:

At Clark we also use OA to:

Back to top

Goals of outcomes assessment

Outcomes Assessment Goals for 2015-16

Back to top

What is My Role?

Full-time faculty role

Each year, IPT determines which OA activities the college will focus on to achieve institutional goals.

2015-16 IPT Charge to Full-Time Faculty

The IPT-defined course and program–level outcomes assessment activities for full-time faculty for 2015-2016 will consist of the following:

Part-time faculty role

Clark encourages and invites adjunct faculty to participate in OA. Since the majority of faculty at Clark are adjunct, and since part-time faculty often have OA experience at other institutions, their insights about student learning are especially valuable. Programs rely to varying degrees on adjunct faculty, and the more a program relies on part-time faculty, the more adjuncts should be encouraged to take part in outcomes assessment. We recommended that the full-time faculty detail explain the protocols for assessment plans as well as emphasize the importance of participating in OA for program improvement.

Back to top

Vocational Certification (applies to full-time Career & Technical faculty)

Certification under the standards specified in Washington Administrative Code (WAC) is a condition of continued employment for all professional-technical education personnel. At Clark, all new Full-Time CTE faculty and those who will be renewing their vocational certification are required to choose an Outcomes Assessment activity as part of their plan. Those activities include:

Standard B
B.1 Identify, evaluate, and modify current outcomes.

Standard C
C.3 Identify, evaluate, and modify program outcomes and assessments.

Standard D
D.4 Modify instructional material and methods based on student and industry assessments and feedback.

Back to top

Compensation

All full-time faculty are required to participate in Outcomes Assessment work as stipulated by their contract. However, faculty can be compensated for Outcomes Assessment work which, in the judgment of the Outcomes Assessment Committee, requires effort above and beyond ordinary contractual Outcomes Assessment duties. All compensation for Outcomes Assessment work must be approved ahead of time by the Outcomes Assessment Committee; one condition for compensation will be successful completion of the Outcomes Assessment project in question.

All adjunct faculty will be compensated for Outcomes Assessment work, assuming that the work has been approved ahead of time by the Outcomes Assessment Committee.

Back to top

What Outcomes Assessment Resources are Available?

Faculty Outcomes Assessment Liaisons

Clark has two faculty members here to support your assessment activities. Think of them as your program assessment consultants and technical experts. The OA Liaisons offer an array of assessment workshops throughout the academic year. They are also available by appointment for individual program consultation; they can provide advice on how to assess your program, what to assess, and how to act on the assessment results.

 

Back to top

Office of Planning and Effectiveness

The Office of Planning and Effectiveness is home to Clark's Institutional Research (IR) services. Institutional Researchers are available to support your assessment projects by helping with the practicalities of research, such as:

Institutional Researchers can help make your project more efficient and take some of the research-related workload off of faculty shoulders. You are encouraged to get in touch in the early stages of your project, or even before you have a project in mind.

Back to top

Instructional Deans and Leadership

The Outcomes Assessment team works closely with the instructional deans. In collaboration with the OA Committee and Liaisons, the deans play a role in creating direction and processes in OA. Faculty should go to the deans as a resource when needed. In addition, faculty are encouraged to share project results as part of "closing the loop" so that deans can use OA data to inform decision-making.

Back to top

Outcomes Assessment Committee

The primary responsibility of the Outcomes Assessment Committee (OAC) is to oversee and coordinate the development of program-level assessment plans for all transfer and CTE degree and certificate programs. In addition, duties of the OAC include:

The OAC is currently made up by the following people:

Back to top

Outcomes Assessment Toolbox (OAT)

The Outcomes Assessment Toolbox (also known as the "OAT") is a collection of web applications and reports designed to facilitate Outcomes Assessment at Clark College. It was developed by Clark College staff and faculty. The tools are intended to make Outcomes Assessment clearer and more productive for faculty, and the Outcomes Assessment team welcomes feedback so that the tools can be continually improved.

The best way to learn more about the OAT is just to head over to the site and begin exploring. One of the most commonly used tools is the Course Outcomes Editing Tool, which is used to revise, add, or delete course outcomes. Another tool is the Assessment Project Form, which is used to plan and report outcomes assessment projects.

All faculty should have access to these and a number of other tools, using their regular Clark log-in information. If you are having difficulty accessing the OAT, please contact the system administrator .

Back to top

Where can I find program and course learning outcomes?

Learning outcomes for degree and certificate programs and for courses are documented in the OAT. Look under the headings "Course Outcomes" and "Program Outcomes" in the OAT to find these tools.

Back to top

How do I report my projects?

Use the Assessment Project Form, located within the OAT, to plan and report outcomes assessment projects.

Back to top

Will I receive feedback on my report/project?

An assessment project doesn't end with a report. Results should produce compelling information and questions for you to further examine with your colleagues in order to improve your program's effectiveness. The Faculty OA Liaisons generally review projects over the summer and provide feedback during the first weeks of fall term. They may also review your projects throughout the year. The OA Liaisons will contact the individuals listed on each report in order to support with "closing the loop." This involves looking at how you can use the results to make changes to improve student learning within your program. "Closing the loop" is the final step in the assessment cycle, and each program will be required to document this step for our accreditation. You can request feedback or additional information anytime by contacting the Liaisons.

Back to top

How Do I Do Outcomes Assessment? What's the Process?

Nine principles of good assessment practice

The American Association for Higher Education (AAHE) has published nine key considerations to bear in mind before engaging in the work of outcomes assessment. We highly recommend that you familiarize yourself with all of these prior to engaging in OA work, in order to ensure that your experience will be as productive and rewarding as possible.

Back to top

Creating a plan: how do we choose what to assess?

It's important to remember that faculty retain the authority to direct OA work. Often, when just starting out, the best way to begin is with a complaint: what are you and your fellow faculty least satisfied with when it comes to student learning? Informal conversations (gripe sessions) around the departmental water cooler are a good way to discover your most pressing problem with student learning.

Once faculty have a sense of what aspect of student learning is most pressing, the second question to answer is: what outcome(s) would best provide us a picture of the student learning that we want to enhance? The key to any successful OA project is to keep the outcome(s) clearly in mind. Not only does an outcome clarify the learning in question, the language of the outcome will direct the work of the project itself.

Back to top

The assessment cycle

 The assessment cycle is a continuous process, and consists of the following steps:

Back to top

What kind of assessment tool can I use?

An assessment tool is an assignment completed by students that reflects their learning in a course or program. Faculty have a wide variety of assessment tools to choose from: exams, essays, performances, surveys, interviews, portfolios, quizzes—the list goes on. Any tool that can highlight how well students are meeting stated learning outcomes is appropriate. It's crucial that an OA project, though, be able to clearly address the learning outcomes in question. For example, a compelling essay prompt that is not in some way related to a learning outcome may not be useful for your project.

Back to top


How do I choose an instrument?

An assessment instrument is used to assess how well a student has demonstrated evidence of learning. This can range in complexity from an answer key on a multiple choice test to an essay or performance rubric. The tool you choose will ultimately direct you to the best instrument to assess your results. It's important to note, however, that those instruments that may involve subjective interpretation on the part of faculty —a rubric, for example – should be applied as consistently as possible.

Back to top


How do I choose a sample?

When assessing a group of students, it's important that your results be valid. That is, the results should give you a reliable foundation upon which to base your conclusions about student learning. For that reason, it's ideal that you choose a representative sample of students to assess from a group. That number depends upon the total number of students in your sample size (whether that be a single class, series of classes, or a still larger sample).

With that said, it's also important to recognize that OA takes place in real time, with real constraints (particularly in relation to money and time). If you feel that you do not have sufficient resources to assess a valid sample size for your project, please contact Clarks' Office of Planning and Effectiveness.

Back to top

How should I conduct the assessment?

Assessments can be embedded (given to students to complete as part of their graded requirements) or external (given to students separately from their graded requirements). When conducting the assessment, bear in mind these considerations:

Logistics: How many classes are you assessing? What will be necessary to ensure that the students complete the assessment? Who will administer the assessment? Who will collect the assessment?

Validity: How seriously will the students take the assessment? How will both students and instructor perceive the assessment?

Often, administering the assessment involves a series of decisions, dependent on your sample size, resources, needs, and other project variables.

Back to top


What will the results tell me?

The results of a successful OA project will present valid data about how well students are meeting stated learning outcomes. They should sponsor a meaningful conversation among faculty as to what steps should be taken in future to enhance student learning relative to the outcome(s) in question.

Back to top

What is the timeline?

A common timeline for an OA project is to Plan in Fall, Assess in Winter, Reflect in Spring. This approach ensures that a project is broken up into manageable parts which can be completed over the course of an academic year. However, this is not the only timeline faculty may choose for their project. As always, your resources and needs should play a part in the project timeline.

Keep in mind, for example, that faculty energy and participation can vary. Enthusiasm tends to be highest in Fall, whereas in Spring many faculty are distracted with year-end responsibilities. As well, you may wish to conduct multiple OA projects throughout the year. Like precious snowflakes, or precocious students, no two OA projects are alike. Your goals and resources should dictate your timeline.

Back to top

Applying results to practice

"Closing the loop" is the catchphrase used to refer to the last step in an OA project. This step addresses the following question: Based on the results of the assessment, what changes are planned?

Applying results to practice is absolutely vital to a sustainable approach to OA. Not only does it satisfy explicit accreditation requirements, but implementing planned changes demonstrates for faculty how OA can impact their practice. Reflections on OA data must be more than conceptual, they should result in action—as collectively determined by the faculty involved in the project. That action is where the rubber meets the road (to employ another metaphor) when it comes to enhancing student learning.

Back to top

Do my outcomes assessment results play a role in decision-making?

Closing the loop should mean more than just changes to faculty practice. The OA "loop" can—and should—be closed on an institutional scale. Administrative decisions that impact student learning have, in OA, a clear and common baseline in their impact on student learning. As OA serves many uses and multiple audiences, its incorporation into college-wide decisions will be made explicit for other stakeholders as well.

Back to top

What is the role of outcomes assessment in curriculum?

OA plays a role in curriculum development at every level:

Program design – Program outcomes identify the broader learning goals for students upon completion of a course of study. In addition, they govern the distribution areas of those programs by providing a framework of learning within which required courses should fit. The courses required for a program should explicitly relate to the student learning stated in the program outcomes. For this reason, OA is central in determining the designation of courses with regard to program requirements and related distribution areas.

Course design – By articulating the goals for student learning in the form of course outcomes, OA provides faculty an opportunity to begin with the end in mind. More commonly known as "backward design" (link?), course outcomes allow faculty to structure students' experience in ways that lead them to mastery of stated learning goals. Rather than just being "about" the course topic, outcomes lead faculty to consider specifically what students will know or be able to do upon successfully completing the class.

Lesson design – It is important to note that OA should never infringe upon academic freedom. At the same time, outcomes provide a helpful focus in aligning classroom activities with stated learning goals. While stated learning outcomes need not encompass the totality of a student's experience in a class, at the same time they can provide faculty members with key guideposts for planning – and assessing – individual lessons.

Back to top

How does Clark keep outcomes assessment data safe and secure?

Since outcomes assessment often entails using student data such as names, course records, and assignment scores, faculty must take care to keep this information safe and secure. Additionally, since outcomes assessment data must not be used to evaluate individual faculty members (see the Memorandum of Understanding section below), it is important to remove identifying information about individual faculty from assessment reports.

For guidance on data security for your specific project and needs, please contact the Office of Planning and Effectiveness.

Here are some generally applicable instructions:

Back to top