Public sentiment seems to hold higher education increasingly accountable for providing students with quality education and skills necessary to earn high salaries and contribute to the global community. As a result of this increased accountability and demand for quality education, those in higher education have come to regard assessment as a very popular word. Academic advising on campus plays a large role in student learning and success at an institution, and the profession has received progressively more attention in the last decade. Research demonstrates that improved academic advising directly correlates with an institution's successful retention programming efforts and positive student learning outcomes (Beal & Noel, 1980; Keeling, 2004; Hunter, McCalla-Wriggins, & White, 2007).
The University Academic Advising Center/University Studies (UAAC/US) at Virginia Tech has practiced assessment and has evaluated programming, service, and student learning for quite some time; but now an intentional, cyclical plan and reporting system exist to guide our efforts and our practice. Academic advising is viewed as a process that is designed to produce specific outcomes and lends itself to evaluation (Gordon & Habley, 2000). Specifically, the assessment cycle in our unit is designed to capture information on the advising process, what student learning outcomes are being emphasized in the UAAC/US, how our unit can demonstrate contributions to student success, and what opportunities we need to enhance or develop for increased student learning.
A plethora of information and literature explains that good assessment is a process and is cyclical in nature. To begin the process, it was necessary for me to take purposeful steps, suggested by a variety of assessment experts, to create an effective assessment plan in our unit. I used staff meetings to educate advisers on assessment practices and emphasized that assessment was a positive process used for improvement as well as accountability. Specifically, the topics covered in the meetings were: (1) understanding assessment and evaluation, (2) the assessment cycle, (3) accountability and improvement, (4) the relationship and importance of unit mission, (5) establishing student learning outcomes, (6) various ways to gather and report data, and (7) providing feedback for improvement (Banta, 2004; Maki, 2004; Suskie, 2004; Van Dyke, 2007).
I will risk the use of a metaphor and say that it was imperative to create a culture of assessment within the UAAC/US. As is often the case, it was this first step that was the most important and most challenging. Maki (2004) proposed that it is communication, commitment, and collaboration that are effective tools in building a culture and capacity for assessment. To create a culture of assessment, I found that the tools stated above were helpful guidelines to follow.
To begin, it was necessary to define and articulate the meaning of assessment. Defining assessment was one way to clearly communicate its purpose and helped to substantiate why the UAAC/US should make a commitment to the assessment process. A definition of assessment was used in staff development meetings to clearly communicate the purpose and establish a commitment to the process. Suskie (2004, p. 3) states:
Assessment is the ongoing process of:
- Establishing clear, measurable expected outcomes of student learning.
- Ensuring that students have sufficient opportunities to achieve those outcomes.
- Systematically gathering, analyzing, and interpreting evidence to determine how well student learning matches our expectations.
- Using the resulting information to understand and improve student learning.
Next, as a team we needed to clearly define our understanding of academic advising in terms of our mission, goals, and objectives. It was also necessary for us to examine how we deliver our services and acknowledge that our methods of operation directly impact our students (Gordon & Habley, 2000). Professional standards served as a guide in creating our unit's mission statement to ensure that the statement accurately defined our purpose and goals.
Evaluating the mission statement as a team was the best way to define our academic advising. This step was actually a self-assessment for the unit and demonstrated a collaborative commitment to our mission. By establishing and communicating mission objectives, we determined specific evaluation needs as we moved toward a culture of assessment. Professional standards along with our mission statement provided clear objectives and formed the basis of our assessment plan.
To develop the assessment process we planned a timeline and the methods we would use to collect data. It was necessary to decide on the population we would assess, how the data would be collected, when we would schedule the assessment, and who would review and interpret the results (Maki, 2004). For example, in some cases all of the students enrolled in the University Studies major were included in the sample, while other assessments needed to look at the results from a specific cohort of students, such as our entering first-year students. Careful review of our programming and advising efforts aided in identifying assessment criteria and specific student populations, such as our students on academic probation.
The methods used for collecting data varied as well. Using institutional resources can be very valuable in retrieving and interpreting existing data. The Office of Institutional Research and the Office of the University Registrar were very helpful in providing quantitative data necessary to improve service, and the Office of Academic Assessment provided continued support and guidance. In addition to this support, our unit determined that online surveys and paper-and-pencil surveys would be useful at different times during the academic year. It was determined that the paper-and-pencil surveys could best be administered during times of high student traffic periods in our office. Online surveys could be used at anytime, and focus groups would be possible during the spring semester so we could gather information on services offered throughout the year. When data are collected, one person in our unit is responsible for ensuring the assessment is complete and reporting the results to all of the staff. It is imperative that we interpret the results and discuss them as a team for the most beneficial outcomes (Suskie, 2004).
Since a good assessment plan is one that improves teaching and learning (Keeling, 2004; Suskie, 2004; Van Dyke, 2007), it is collaboration within the unit that determines the usefulness of the assessment and the means to move forward to improve our work. Once the assessment process begins, the results are valuable only if they are shared. By sharing the results, we interact and collaborate to reach decisions that improve practice through professional judgment and not allow the results alone to dictate our course of action.
It is useful to remember that clear communication within the unit is necessary when conducting assessment so that others are comfortable and choose to collaborate in the process (Maki, 2004; Suskie, 2004). Good communication enhanced the assessment effort in UAAC/US because it has accentuated the positive and fostered the support of all advisers. One way in which we established effective communication on assessment issues was to include a brief discussion of unit assessment at regular staff meetings. This simple step helped to create a culture in which the topic of assessment became conversational and recognized as an integral part of our unit function.
Effective communication of assessment issues creates collaboration among advisers and enables our team to set clear unit goals, gather data, and provide feedback on ways to improve our advising. Our efforts have had a positive outcome on the quality of advising and services we provide to our students. For example, each adviser is now well aware of the components of our unit mission statement and understands the meaning behind the statement. The mission statement has become a part of our unit language, and it has become clear how we need to work toward increasing positive student learning outcomes through advising and programming. The assessment effort has indeed made our work much more intentional and educational in nature.
Overcoming the negative view of assessment was necessary for success and support of the process. It was essential for our staff to understand assessment and evaluation in terms of purpose, the reasons for doing it regularly, the characteristics of good assessment, and the value it would serve within our practice. To gain support for the assessment process, it was important for everyone to know that we were already doing a good job, but we needed a way to substantiate our claims of excellence or discover a better way to educate our students and improve our service.
Keeling (2006) explains that external demands for accountability can often create fear of failure and accentuate shortcomings or lack of value, because those who don't know our work create the requirements. Keeling (2006) encourages the internal assessment process and explains that internal assessment puts control into the hands of those who do the work and are familiar with the value of the goals and mission of the unit. This thinking was helpful in explaining the positive aspects of assessment and establishing our ownership of the process. It was exciting to think that we could take control and determine and assess the most important aspects of our practice, while getting valuable feedback to share with stakeholders and improve our work.
Assessment of academic advising emphasizes the importance of advising on campus and its connection to other advising issues such as adviser training and recognition (Cuseo, 2003; Gordon Gordon & Habley, 2000). Perhaps more importantly, assessment reinforces the link between the levels of quality found in academic advising on a campus and student satisfaction and retention issues (Beal & Noel, 1980; Keeling, 2004; Hunter, McCalla-Wriggins, & White, 2007). Assessment not only measures our effectiveness but has been a powerful tool to motivate and enhance our efforts in working with students. Initiating the assessment process in the UAAC/US has increased our awareness of the value of our work and supports the university community in communicating the message that student success is important to Virginia Tech.