3_3_1

Section 3: Comprehensive Standards

3.3.1 The institution identifies expected outcomes for its educational programs and its administrative and educational support services; assesses whether it achieves these outcomes; and provides evidence of improvement based on analysis of those results.

Judgment of Compliance:
Compliance

Narrative/Justification for Judgment of Compliance:
CNU is in compliance with this comprehensive standard. Educational, administrative, and educational support services state expected outcomes, assess the degree to which these outcomes are achieved, and explain how these results were used for improvement. Collectively, through two processes at CNU, data are gathered from a variety of sources using a broad range of assessment measures:

Yearly Assessment: Each academic and administrative unit submits an Assessment Plan for the upcoming year that includes intended outcomes for that unit, how these outcomes will be measured, and how the outcomes are related to guiding documents. After the year has concluded the unit submits an Assessment Record, which reveals the results of the assessment outlined in the Assessment Plan. Units interpret these results and explain how they used the results for improvement.
Program Review: Academic departments also participate in a comprehensive, self-study every five or six years. Departments report status and progress on numerous elements related to their effectiveness including relevance to university, learning outcomes, teaching, scholarship, service, organization, curricula, size, and resources.

The methods of assessment used by departments/units vary considerably and are customized to the intended outcomes. Overall, however, the academic departments tend to use direct measures of student learning such as licensure tests, nationally standardized tests, locally developed tests, and performance assessments such as internship on-site evaluations, writing samples, oral presentations, and portfolios. Surveys, interviews, and focus groups are also used to complement direct measures of assessment or to assess constructs that are difficult to assess otherwise. In the aggregate through yearly assessment efforts and the program review process, data are collected from:

current students—all academic departments and many administrative units collect data from current students via direct and indirect measures of assessment.
past students—in program review, departments report on accomplishments of past students. This information is obtained through alumni surveys.
employers—several departments collect information from employers to identify the direction of program outcomes and/or to help evaluate students (e.g., Master’s of Applied Teaching program, School of Business, Government, Communication Studies).
faculty—evaluations of students’ work through portfolio review or other performance assessment. Faculty also make formal and informal investigations of the curriculum.

The balance of this section explains the yearly assessment process and program review in greater detail.

Yearly Assessment
Beginning in 2000, academic and administrative units (i.e., administrative and educational support units) began participating in yearly assessments. As part of this process, departments and units submit Assessment Plans whereby they (1) reference guiding documents such as CNU’s Mission and Vision and the unit’s goals, (2) articulate three or more intended outcomes that result from its services, (3) provide two means of assessing each intended outcome, and (4) state criteria by which to evaluate the degree to which the intended outcomes were achieved. For academic departments, outcomes are stated in terms of what a student is expected to know, think, or do. For administrative units, the framing of outcomes varies, reflecting the diversity of the functions of these departments. After departments and units collect data throughout the year, they submit Assessment Records where assessment results are compared against stated criteria. Each department interprets whether or not the intended outcomes have been achieved and often provides a description of factors that influenced the results. Academic departments and administrative units take part in this process. Units with mature assessment processes often provide longitudinal data, insightful interpretation of results, and descriptions of how the department used assessment results for improvement. (See Assessment Records of English (annotated), Master’s of Arts of Teaching, Social Work and Residence Life office for good examples of department and unit assessment at CNU and the Yearly Assessment Timeline to see the sequence of the process.)

It is this last point—use of assessment results for improvement—that is the ultimate goal of CNU’s assessment processes. Several examples are listed below.

The Social Work program discovered that students during internships were rated low or not at all on several core internship areas by their internship supervisors. For example, 8 out of 23 internship supervisors rated the item “use of groups in practice” as not applicable. Further research revealed that many internship sites were not offering certain experiences to students that CNU requires. As a result, the Social Work program initiated a more extensive training session for internship sites to explain expectations regarding experiences for interns.

Residence Life often has to balance issues of safety and convenience. Prior to 2003, a CNU residential student visiting a friend at another CNU residence hall had to sign in as a guest. Seen as too restrictive, this policy was removed in 2003; however, there was concern that students would feel less safe. But longitudinal survey results (below) revealed students did not report a decrease of feeling safe. In fact, they reported feeling safer. In sum, easing the policy on visitation among residence hall students had no negative impact on safety but increased the ease with which friends could see each other.

12) I feel safe in the residence hall.
	S Agree	Agree	Neutral	Disagree	S Disagree
Fall 02	52.52	38.07	5.71	2.35	0.78
Fall 03	57.63	34.74	5.48	1.35	0.51
Fall 04	63.15	31.93	2.94	1.03	0.24
Fall 05	64.6	31	3.4	0.5	0.5

In 2003-2004, the Philosophy and Religious Studies department noted that fewer than 10% of seniors passed their senior theses with distinction. The next year, a department faculty member created and distributed a “thesis handbook.” With this extra guidance, students performed markedly better – 47% passed with distinction in 2004-2005.

To facilitate the use of results and its precursors in the assessment process, the Student Assessment Committee was established in 2000. This committee helps guide academic and administrative units by providing feedback on Assessment Records to assessment liaisons. Liaisons are persons designated by a unit who coordinate assessment within their units and interface with the Student Assessment Committee.

Effective 2005-2006, the committee was renamed the University Assessment and Evaluation Committee (UAEC) to reflect the breadth of assessment activities. The UAEC is composed of 11 members appointed by the provost. The members include the Director of Assessment and Evaluation, five faculty members, and five administrators. Trained on the UAEC Assessment Review form, each committee member rates units’ Assessment Records on specific elements: congruency to guiding documents including Vision 2010; CNU’s Strategic Plan; statement of outcomes; criteria used to determine success; methodology, results, and interpretation and use of results; and the degree to which departments have implemented the suggestions provided by past UAEC evaluations. Copies of the UAEC reviews of Assessment Records are sent to respective department chairs and assessment liaisons. (see Reviews of English, Social Work, and Master’s of Arts of Teaching). The Director of Assessment and Evaluation meets with the department liaisons to discuss the evaluation ratings and improve future assessment. Thus, CNU’s yearly assessment process is systematic and improving. With input from executive officers, department liaisons, UAEC members, and interested faculty, over the last six years the following changes have been implemented: assessment training for liaisons, revisions of reporting forms, reformulation of committee structure, update of committee charge, rater training of UAEC members and more communication with the Director of Assessment and Evaluation.

All academic departments and most administrative units (see academic and administrative assessment matrices) have participated, many continuously, in assessment over the last six years. Some of the nonparticipation in the last year can be attributed to a year’s absence of the Director of Assessment and Evaluation position at CNU. In July 2005 a new Director of Assessment and Evaluation was hired to provide leadership in assessment and move toward 100% participation from units. To this end, the Director and executive administrators (see executive e-mails) put forth a concerted effort to elicit greater participation. They targeted administrative units that had not recently submitted any assessment documents. Subsequently, representatives from the majority of these non-participating units attended an assessment workshop held by the Director of Assessment in the spring of 2006. These units then submitted an Assessment Plan “Sketch,” which includes intended outcomes, congruency of outcomes to guiding documents, and ideas about how these outcomes will be assessed. In essence, the non-participating units took a practical, incremental step toward integrating assessment into their operations.

Also related to the absence of a Director of Assessment & Evaluation, administrative units’ Assessment Records have not been reviewed by the UAEC in recent years. In 2005-2006 academic year, the UAEC made plans (see UAEC minutes) to review administrative Assessment Records again in 2006-2007. They will be treated with the same extensive process as their academic counterparts (i.e., reviewed by multiple raters, feedback given to unit heads and assessment liaisons, meeting with Director of Assessment and Evaluation).

Program Review
Academic programs also participate in a more comprehensive Program Review. Each program conducts a self study and produces documentation and a report that is reviewed by the Undergraduate Program Review Committee (PRC). In this year-long and comprehensive process, departments reflect upon and analyze all facets of their operation including, but not limited to, relationship to university mission, unit mission, goals, teaching, scholarship, service, enrollment, retention, graduation rate, budgetary efficiency, input from alumni, and student learning outcomes assessment. Essential to these analyses is consideration of the degree to which departmental goals are accomplished. Other parties to this process are the Dean, and the Provost. Program Review Guidelines were recently updated to include a description, purposes, process, timelines, and responsibilities of department under review.

Beginning in 1982, the Program Review process has long been an important component of institutional effectiveness at CNU. Indeed, many departments have undergone four program reviews (a five to six-year reporting cycle), the Program Review Calendar set so that three departments are reviewed each year.

Perhaps the most important aspect of program review is that results are used. In other words, decisions are made, actions undertaken, goals reconsidered, and resources allocated, all to improve departments based upon findings of program review. The academic deans play an important role. Considering the results of department’s program review and the recommendations of the program review committee, each dean sets up an action agenda for the department to accomplish prior to the next program review. The intent of the action agenda is to help the department reach its goals. Each year, every department reports on its progress on the action agenda via its Annual Report to the Dean (see Sociology’s Self-Study, the Program Review Committee’s Report on the Self-Study, and a Subsequent Annual Report from the Sociology Department to the Dean). Changes within departments that were influenced by program review include curricular modifications, additional resources, changes of goals, and improved assessment. Note: graduate programs take part in a similar process (see Graduate Program Review Policy).

Additionally, several academic programs at the university (Luter School of Business, Master’s of Arts and Teaching program, Engineering, Music) also report on their effectiveness to relevant external governing bodies, adding to the rigorous oversight of their academic programs.

In sum, at CNU educational, administrative, and educational support services state expected outcomes, assess the degree to which these outcomes are achieved, and use these results for improvement.

Support Documentation:
Academic Assessment Matrix
Administrative Assessment Matrix
Assessment Plan
Assessment Plan Sketch
Assessment Record
Assessment Workshop
English Assessment Record (Annotated)
English Review
Executive E-mails
Graduate Program Review Policy
MAT External
MAT Record
MAT Review
Music External Accreditor
PCSE – ABET Interim Report
PRC 05-06 Handbook
Program Review Guidelines Final 2006
Residential Life Assessment Record
SOB External ( School of Business)
Social Work Record
Social Work Review
Sociology Annual Report 04-05
Sociology Self-Study for PRC
Sociology PRC Report
Student Assessment Committee 04-05 Handbook
UAEC Review Form
UAEC Minutes 05-06
University Assessment and Evaluation Committee 05-06 Handbook
Yearly Assessment Timeline

Additional Live Web Resources:
None