Meta-Assessment and Stakeholder Learning: Proposed Rubrics and Relevance to Academic Advising
DOI:
https://doi.org/10.26209/mj2261870Keywords:
assessment, meta-assessment, academic advising, learning outcomesAbstract
Assessment in higher education has expanded from examining academic disciplinary programs to all aspects of student learning, including academic advising. But, how do we know whether the assessment process is effective, appropriate, accurate, and useful? This article (a) reviews the relatively new and little used approach of meta-assessment, which evaluates the quality of the assessment processes and methodologies to ensure outcome data validity and (b) argues that meta-assessment should include assessment of stakeholder learning resulting from participation in the assessment process. Academic advising professionals, as well as others in higher education, should adopt both the established approach to meta-assessment as well as the proposed focus on stakeholder learning when considering assessment processes and results. Sample rubrics are provided for both of these components of meta-assessment.
References
Angelo, T. (1995, November). Reassessing and defining assessment. AAHE Bulletin.
Baehr, M., (2005). Distinctions between assessment and evaluation. In Coe College Faculty Guidebook (pp. 441–444).
Banta, T. W., Ewell, P. T., & Cogswell, C. A. (2016, October). Tracing assessment practice as reflected in assessment update [Occasional paper]. National Institute for Learning Outcomes Assessment.
Bresciani, M. J., Gardner, M. M., & Hickmott, J. (2009). Demonstrating student success: A practical guide to outcomes-based assessment of learning and development in student affairs. Stylus Publishing.
Campbell, S. (2008). Using assessment to understand what and how students learn through academic advising [Plenary presentation]. 2008 NACADA Assessment of Academic Advising Institute, San Diego, CA, United States.
Council for the Advancement of Standards. (2019). CAS standards for academic advising programs (10th ed.). http://standards.cas.edu/getpdf.cfm?PDF=E864D2C4-D655-8F74-2E647CDECD29B7D0
Creamer, E. G., & Scott, D. W. (2000). Assessing individual advisor effectiveness. In V. N. Gordon & W. R. Habley (Eds.), Academic advising: A comprehensive handbook (pp. 339–348). Jossey-Bass.
Crookston, B. B. (2009). A developmental view of academic advising as teaching. NACADA Journal, 29(1), 78–82. https://doi.org/10.12930/0271-9517-29.1.78 (Reprinted from “A developmental view of academic advising as teaching,” 1972, Journal of College Student Personnel, 13, 12–17).
Cuseo, J. (n.d.). Assessment of academic advisors and academic advising programs. NACADA Clearinghouse of Academic Advising Resources. https://www.nacada.ksu.edu/Portals/0/CandIGDivision/documents/assessment%20of%20advising%20resources/Cuseo_Marymount1.pdf
Cuseo, J. (2008). Assessing advisor effectiveness. In V. N. Gordon, W. R. Habley, and T. J. Grites (Eds.), Academic advising: A comprehensive handbook (2nd edition, pp. 369–385). Jossey-Bass.
Ender, S. C., Winston, R. B., & Miller, T. K. (1984). Academic advising reconsidered. In R. B. Winston, Jr., T. K. Miller, S. C. Ender, & T. J. Grites (Eds.), Developmental academic advising (pp. 3–34). Jossey-Bass.
Ewell, P., & Cumming, T. (2017). Introduction: History and conceptual basis for assessment in higher education. In T. Cumming & M. D. Miller (Eds.) Enhancing assessment in higher education: Putting psychometrics to work (pp. 3–26). Stylus Publishing.
Fulcher, K. (2018). Assessing the quality of assessment practice [PowerPoint slides]. The Center for Assessment & Research Studies, James Madison University. https://sacscoc.org/pdf/2018SIHO/Keston%20Fulcher_Assessing%20the%20Quality%20of%20Assessment%20Practice%20(Presentation).pdf
Habley, W. R. (2004). The status of academic advising: Findings from the ACT sixth national survey. National Academic Advising Association (NACADA).
Hemwall, M. K., & Trachte, K. C. (2003). Academic advising and a learning paradigm. In M. K. Hemwall & K. C. Trachte (Eds.), Advising and learning: Academic advising from the perspective of small colleges and universities (pp. 13–19). National Academic Advising Association.
Hemwall, M. K., & Trachte, K. C. (2005). Academic advising as learning: 10 organizing principles. NACADA Journal, 25(2), 74–83. https://doi.org/10.12930/0271-9517-25.2.74
Huba, M. E., & Freed, J. E. (2000). Learner-centered assessment on college campuses: Shifting the focus from teaching to learning. Allyn and Bacon.
Hurt, R. L. (2007). Advising as teaching: Establishing outcomes, developing tools, and assessing student learning. NACADA Journal, 27(2), 36–40. https://doi.org/10.12930/0271-9517-27.2.36
Keup, J. R., & Kinzie, J. (2007). A national portrait of first-year students. In M. S. Hunter, B. McCalla-Wriggins, & E. R. White (Eds.), Academic advising: New insights for teaching and learning in the first year (pp. 19–38). National Resource Center for the First-Year Experience and Students in Transition.
Kuh, G. D. (2008). Advising for student success. In V. N. Gordon, W. R. Habley, & T. J. Grites (Eds.), Academic advising: A comprehensive handbook (2nd ed.). Jossey-Bass.
Lowenstein, M. (2005). If advising is teaching, what do advisors teach? NACADA Journal, 25(2), 65–73. https://doi.org/10.12930/0271-9517-25.2.65
Lynch, M. L. (2000). Assessing the effectiveness of the advising program. In V. N. Gordon & W. R. Habley (Eds.), Academic advising: A comprehensive handbook (pp. 324–338). Jossey-Bass.
Macaruso, V. (2007). From the co-editors: Brief report on the NACADA Commission on the Assessment of Advising 2004 survey results. NACADA Journal, 27(1), 3–8. https://doi.org/10.12930/0271-9517-27.2.3
Maki, P. L. (2002). Developing an assessment plan to learn about student learning. Journal of Academic Librarianship, 28(1–2), 8–13. https://doi.org/10.1016/s0099-1333(01)00295-6
Maki, P. L. (2004). Assessing for learning: Building a sustainable commitment across the institution (1st ed.). Stylus Publishing.
Maki, P. L. (2010). Assessing for learning: Building a sustainable commitment across the institution (2nd ed.). Stylus Publishing.
Marchese, T. (1993). AAHE continuous quality improvement project: Profiles of campuses. The Assessment Institute.
McDonald, B. (2010). Improving learning through meta assessment. Active Learning in Higher Education, 11(2), 119–129. https://doi.org/10.1177/1469787410365651
McGill, C. M. (2016). Cultivating ways of thinking: The developmental teaching perspective in academic advising. New Horizons in Adult Education & Human Resource Development, 28(1), 50–54. https://doi.org/10.1002/nha3.20131
Middle States Commission on Higher Education. (2014). The 2014 standards for accreditation and requirements of affiliation: A Guided review. https://www.msche.org/?Nav1=EVALUATORS&Nav2=TRAININGMATERIALS&Nav3=VIDEOS&strPageName=VIDEOS
Miller, M. A., & Alberts, B. (1994). Developmental advising: Where teaching and learning intersect. NACADA Journal, 14(2), 43–45. https://doi.org/10.12930/0271-9517-14.2.43
NACADA: The Global Community for Academic Advising. (2006). Concept of academic advising. NACADA Clearinghouse of Academic Advising Resources. https://nacada.ksu.edu/Resources/Pillars/Concept.aspx
NACADA: The Global Community for Academic Advising. (2017a). NACADA Academic Advising Core Competencies. https://www.nacada.ksu.edu/Portals/0/Resources/documents/Core%20Competencies%20BASIC%20Flyer%20%28update%205-10-17%29.pdf
NACADA: The Global Community for Academic Advising. (2017b). NACADA Core Values of Academic Advising. https://www.nacada.ksu.edu/Portals/0/Resources/Pillars/Core%20Values%20Handout.pdf
NACADA: The Global Community for Academic Advising. (2018). NACADA Center for Research at Kansas State University. https://www.nacada.ksu.edu/Resources/Research-Center.aspx
Ory, J. C. (1992). Meta-assessment: Evaluating assessment activities. Research in Higher Education, 33(4), 467–481. https://doi.org/10.1007/bf00973767
Palomba, C. A., & Banta, T. W. (1999). Assessment essentials: Planning, implementing, and improving assessment in higher education. Jossey-Bass Publishers.
Pellegrino, J. W., Chudowsky, N., & Glaser, R. (2001). Knowing what students know: The science and design of educational assessment. National Academies Press. https://doi.org/10.17226/10019
Powers, K. L., Carlstrom, A. H., & Hughey, K. F. (2014). Academic advising assessment practices: Results of a national study. NACADA Journal, 34(1), 64–77. https://doi.org/10.12930/nacada-13-003
Rinck, C. N. (2006, June 28). Student engagement and academic advising. The Mentor: An Academic Advising Journal, 8(1). http://dus.psu.edu/mentor/060628cr.htm
Robbins, R. (2009). Utilizing institutional research in the assessment of academic advising. NACADA Clearinghouse of Academic Advising Resources. http://www.nacada.ksu.edu/Resources/Clearinghouse/View-Articles/Assessment-and-Institutional-Research.aspx
Robbins, R. (2011). Assessment and accountability of academic advising. In J. Joslin and N. Markee (Eds.), Academic advising administration: Essential knowledge and skills for the 21st century (pp. 53–64). NACADA.
Robbins, R. (2016a). Assessment of academic advising: Gathering outcome evidence and making changes. In T. Grites, M. Miller, & J. Givans Voller (Eds.), Beyond foundations: Developing as a master academic advisor (pp. 289–303). Jossey-Bass.
Robbins, R. (2016b). Assessment of academic advising: Overview and student learning outcomes. In T. Grites, M. Miller, and J. Givans Voller (Eds.), Beyond foundations: Developing as a master academic advisor (pp. 275–288). Jossey-Bass.
Robbins, R. (2017, February 2–4). Developing a rubric [Paper presentation]. NACADA’s 2017 Assessment of Academic Advising Institute, Daytona Beach, FL, United States.
Robbins, R., & Adams, T. (2013). Assessment of peer advising. In H. Koring and D. Zahorik (Eds.), Peer advising and mentoring: A guide for advising practitioners (2nd ed., pp. 129–140). NACADA.
Robbins, R., Miller, M., & Zarges, K. (2019, October 20–23). Using NACADA pillar documents as a foundation for student learning outcomes and academic advising program assessment [Paper presentation]. NACADA’s 43rd Annual Conference, Louisville, KY, United States.
Robbins, R., & Zarges, K. M. (2011). Assessment of academic advising: A summary of the process. NACADA Clearinghouse of Academic Advising Resources. http://www.nacada.ksu.edu/Resources/Clearinghouse/View-Articles/Assessment-of-academic-advising.aspx
Rodgers, M., Grays, M. P., Fulcher, K. H., & Jurich, D. P. (2013). Improving academic program assessment: A mixed methods study. Innovative High Education, 38, 383–395. https://doi.org/10.1007/s10755-012-9245-9
Schoepp, K., & Benson, S. (2016). Meta-assessment: Assessing the learning outcomes assessment program. Innovative Higher Education, 41(4), 287–301. https://doi.org/10.1007/s10755-015-9347-2
Schulenberg, J., & Lindhorst, M. (2008). Advising is advising: Toward defining the practice and scholarship of academic advising. NACADA Journal, 28(1), 43–53. https://doi.org/10.12930/0271-9517-28.1.43
St. Cloud State University. (2015). Meta-assessment at St. Cloud State University. http://assessmentinstitute.iupui.edu/overview/institute-files/2015-institute/tuesday-2015/ewing_2015.pdf
Stevens, D. D., & Levi, A. J. (2005). Introduction to rubrics: An assessment tool to save grading time, convey effective feedback, and promote student learning. Stylus Publishing.
Suskie, L. (Ed.). (2001). Assessment to promote deep learning: Insight from AAHE’s 2000 and 1999 assessment conferences. American Association for Higher Education.
Suskie, L. (2009). Assessing student learning: A common sense guide (2nd ed). Anker Publishing.
Troxel, W. G. (2008). Assessing the effectiveness of the advising program. In V. N. Gordon, W. R. Habley, & T. J. Grites (Eds.), Academic advising: A comprehensive handbook (2nd edition, pp. 386–395). Jossey-Bass.
Upcraft, M. L., & Schuh, J. H. (1996). Assessment in student affairs: A guide for practitioners. Jossey-Bass.
Wholey, J. S., Hatry, H. P., & Newcomer, K. E. (1994). Handbook of practical program evaluation. Jossey-Bass.
Young-Jones, A. D., Burt, T. D., Dixon, S., & Hawthorne, M. J. (2013). Academic advising: Does it really impact student success? Quality Assurance in Education, 21(1), 7–19. https://doi.org/10.1108/09684881311293034
Published
Issue
Section
License
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
