HIED 840

The information contained on this page is designed to give students a representative example of material covered in the course. Any information related to course assignments, dates, or course materials is illustrative only. For a definitive list of materials, please check the online catalog 3-4 weeks before the course start date.

HIED 840 Assessing Student Outcomes & Evaluating Academic Programs (3 credits): Academic program assessment/student outcomes in accountability and accreditation processes.

Overview

This course pulls together the many threads that add up to educational effectiveness: evaluating academic programs and curricula, assessing student learning outcomes, coping with accountability and performance reporting requirements, responding to the demands of both regional and disciplinary/vocational accreditation bodies. The course acquaints students with strategies and instruments for conducting outcomes studies of programs, students, and alumni alike. Assessment topics include studies of students' basic skills, general education, knowledge in the major, personal growth, and alumni outcomes.

Thus, the course is designed for higher education professionals who seek to a fuller understanding of Student Outcomes Assessment, Program Evaluation, and Institutional Effectiveness. The on-line experiences are aimed at applying the readings, obtaining hands-on experience in analyzing data, and developing reporting skills. Each Unit lists supplementary readings and weblinks where you can find additional information to explore the topic in greater depth. The course has linkages to the other courses in the Institutional Research Certificate program. For example, the opening weeks of HIED 840 expand upon some of the assessment and evaluation readings and materials covered in selected units of the Foundations course (HIED 801). Both this assessment course and the course on Studying Students and Student Affairs (HIED 802) draw upon and discuss relevant literature and theories of student outcomes. Persistence models and theories referred to in these courses are relevant also to Enrollment Management and Forecasting (HIED 860). The Research Design course (HIED 830), with its emphasis on measurement issues and survey research, provides an analytical foundation for all these other IR courses.

HIED 840 summarizes the best of what we know about assessing student outcomes. Few topics are more complicated than outcomes assessment. The needs of students and the areas of their learning vary highly among institutions and degree programs. Students are diverse and the dimensions of the learning processes in American Higher Education are extremely complex. Likewise, assessing student performance is complex and hence difficult to summarize. Moreover, before we travel into the real content of assessment, we need to place our journey within a context, and within an environment that is heavily shaping what we do. Thus, before we focus on evaluation academic programs and assessing student outcomes, we will examine accountability, accreditation, and performance reporting. In recent years, evidence of student outcomes has become one of the key indicators of institutional effectiveness, especially as it is viewed by accrediting associations and many state higher education governing boards.

Course Objectives

This course examines institutional effectiveness, accountability and accreditation, academic program evaluation, and student outcomes assessment. Curricular goals for students in HIED 840:

• to understand concepts, methodologies, research practices, instruments, and information systems that support campus evaluation and assessment activities;
• to discuss with others the challenges associated with planning and implementing assessment studies in different types of institutional settings;
• to design and conduct studies of student and alumni outcomes such as basic skills, general education, knowledge in the major, and personal growth;
• to utilize SPSS software, make PowerPoint presentations, and produce effective reports on selected assessment topics;
• To outline the contents of a campus assessment plan

Required Course Materials

Most World Campus courses require that students purchase materials (e.g., textbooks, specific software, etc.). To learn about how to order materials, please see the Course Materials page. You should check LionPATH approximately 3–4 weeks before the course begins for a list of required materials.

Library Resources

Many of the University Libraries resources can be utilized from a distance. Through the Libraries website, you can

• access magazine, journal, and newspaper articles online using library databases;
• borrow materials and have them delivered to your doorstep—or even your desktop;
• get research help via email, chat, or phone using the Ask a Librarian service; and
• much more. 

You must have an active Penn State Access Account to take full advantage of the Libraries' resources and service.  The Off-Campus Users page has additional information about these free services.

Technical Requirements

Technical Requirements

Operating System	Canvas, Penn State's Learning Management System (LMS), supports most recent versions of Microsoft Windows and Apple Mac operating systems.  To determine if your operating system is supported, please review Canvas' computer specifications.
Browser	Canvas supports the last two versions of every major browser release. It is highly recommended that you update to the newest version of whatever browser you are using. Please note that Canvas does not support the use of Internet Explorer. Students and instructors should choose a different browser to use.    To determine if your browser is supported, please review the list of Canvas Supported Browsers. Note: Cookies must be enabled, and pop-up blockers should be configured to permit new windows from Penn State websites.
Additional Canvas Requirements	For a list of software, hardware, and computer settings specifically required by the Canvas LMS, please review Canvas' computer specifications.
Additional Software	All Penn State students have access to Microsoft Office 365, including Microsoft Office applications such as Word, Excel, and PowerPoint. Students will need a PDF reader, such as Adobe Reader.
Hardware	Monitor: Monitor capable of at least 1024 x 768 resolution Audio: Microphone, Speakers Camera (optional, recommended): Standard webcam - many courses may require a webcam for assignments or exam proctoring software.
Mobile Device (optional)	The Canvas mobile app is available for versions of iOS and Android. To determine if your device is capable of using the Canvas Mobile App, please review the Canvas Mobile App Requirements.

---

Student Education Experience Questionnaire (SEEQ)

During the semester you will receive information for completing the Student Education Experience Questionnaire (SEEQ). Your participation is an opportunity to provide anonymous feedback on your learning experience. Your feedback is important because it allows us to understand your experience in this course and make changes to improve the learning experiences of future students. Please monitor email and course communications for links and availability dates.

---

If you need technical assistance at any point during the course, please contact the Service Desk.

For registration, advising, disability services, help with materials, exams, general problem solving, visit World Campus Student Services!

Course Requirements and Grading

This course is offered in a compact 12-week format in the summer. Each lesson begins on Monday and ends on Sunday, but assignments and interactions with classmates often extend beyond these boundaries. Working online requires dedication and organization. Arrange your work schedule, family obligations and vacation plans smartly. We suggest working ahead to prepare for planned conferences and vacations, as well as for unplanned illness or job stress. You are expected to log in to the course each week and complete the assignments and activities on or before the due dates. You must check your Penn State email messages regularly for course announcements, which may include reminders, revisions, and updates. Please communicate any questions or problems to your professors promptly.

Most of our on-line IR courses have at least one large data set, and this one has two, but we offer students the alternative of substituting datasets from their own campuses where appropriate and approved by the instructor. There are three LIVE sessions on designated days, tentatively at 7:00 pm EST. During these sessions, we examine the datasets and review the analytical steps for two multi-week assessment analyses. You need to be at a computer with a microphone in order to fully participate. Although these sessions are recorded for later review, please let your professors know ahead of time if you have a schedule conflict.

SPSS or an equivalent statistical software package like MINITAB or SAS is required for this course. We recommend the Statistics Premium Grad Pack for graduate students purchasing SPSS. In IR work, as well as in these courses, you eventually need to be able to do both OLS & Logistic regression modeling. Moreover, the premium Grad Pack gives deals easily with missing values, building custom tables, displaying decision trees, and other options that are useful in IR work. Although your instructors in this course do not have direct experience with MINITAB, students have access to several different software options through Penn State webapps including Minitab 16.2.

Grading Scale:

All course assignments and the associate points are listed in the Assignments Table. Grading rubrics for assignments have been embedded in the course.  The final course grade will be calculated based on the cumulative points: 

Letter Grade Values Table

Letter Grade	Point Value Range
A	160+
A-	150 - 159
B+	140 - 149
B	130 - 139
B-	120 - 129
C+	110 - 119
C	100 - 109
Failure	Below 100

Course Schedule

The schedule below outlines the topics we will be covering in this course, along with the associated time frames and assignments.

Lesson 1: Dissemination and Effective Reporting

Lesson 1

Readings:

Textbook: Assessing Student Learning: A Common Sense Guide Summarizing and analyzing assessment results Sharing assessment results with internal and external audiences Using assessment results effectively and appropriately ​Textbook: Assessing Student Outcomes: Why, Who, What, How?​ Reporting research results effectively eReserve Smart, J. C. (2005). Attributes of exemplary research manuscripts employing quantitative analysis. Research in Higher Education, 46(4), 461-477.

Assignments:

E-mail exercise Drop Box exercise Discussion Forum exercise What Is AIR? (Optional)

Lesson 2: The Assessment Context: Accreditation, Accountability, and Performance

Lesson 2

Readings:

Textbook: Assessing Student Outcomes: Why, Who, What, How?​ The assessment context: Accreditation, accountability, and performance eReserve Brittingham, B. (2009). Accreditation in the United States: How did we get to where we are. New Directions for Higher Education, Vol. 145, Issue 145, pp 7-27.  Dodd, A. H (2004). Chapter 2. Accreditation as a catalyst for institutional effectiveness. Chapter 2 in M. Dooris, J. Trainer, & W. Kelley, (Eds.), Strategic Planning and Institutional Research. New Directions for Institutional Research, (No. 123). San Francisco: Jossey-Bass Inc., Publishers. Ewell, P. (2009). Assessment, accountability, and improvement: Revisiting the tension.  (NILOA Occasional Paper No.1). Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment.  http://www.learningoutcomeassessment.org/documents/PeterEwell_005.pdf Provezis, S. (2010). Regional Accreditation and Student Learning Outcomes: Mapping the Territory.  (NILOA Occasional Paper No.6). Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment. http://www.learningoutcomeassessment.org/documents/Provezis.pdf

Assignments:

Accreditation Critique "Group" Discussion Forum

Lesson 3: A Model of Institutional Effectiveness

Lesson 3

Readings:	Textbook: Assessing Student Outcomes: Why, Who, What, How?​ A model for assessing institutional effectiveness eReserve Volkwein, J. Fredericks (2007).  Assessing Institutional Effectiveness and Connecting the Pieces of a Fragmented University.  Chapter 7, pp. 145-180, in Fixing the Fragmented University (Joseph Burke, Ed.). Bolton, MA: Anker Publishing.
Assignments:	Lesson 3 Quiz

Lesson 4: Institutional Effectiveness: Academic Program Evaluation

Lesson 4

Readings:

eReserve Barak, R.J. and Sweeney, J.D. (1995). “Academic Progam Review in Planning, Budgeting, and Assessment.” Chapter 1 in Robert Barak & Lisa Mets (eds.), Using Academic Program Review. New Directions for Institutional Research, No. 86. San Francisco: Jossey-Bass Inc., pp.19 – 36. Ewell, P, Paulson, K, & Kinzie, J. (2011). Down and in: assessment practices at the program level. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA). http://www.learningoutcomeassessment.org/NILOAsurveyresults11.htm Gardner, Don E. (1977). Five Evaluation Frameworks. Journal of Higher Ed. 48: (Sep/Oct) 571-593. Mets, L.A. (1995). “Progam Review in Academic Departments.” Chapter 2 in Robert Barak & Lisa Mets (eds.), Using Academic Program Review. New Directions for Institutional Research, No. 86. San Francisco: Jossey-Bass Inc., pp.19 – 36. Shirley, R. C., & Volkwein, J. F. (1978). Establishing academic program priorities. Journal of Higher Education, 49 (5), 472-488.

Assignments:

Lesson 4 Writing Assignment "Group" Discussion Forum

Lesson 5: Assessing Student Learning Outcomes: WHY, WHO, WHAT?

Lesson 5

Readings:

Textbook: Assessing Student Learning: A Common Sense Guide Understanding assessment Planning for assessment success Textbook: Assessing Student Outcomes: Why, Who, What, How? Assessment with open eyes Overcoming obstacles to campus assessment eReserve Kuh, G., & Ikenberry, S. (2009). More than you think, less than we need: Learning outcomes assessment in American Higher Education. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA). http://www.learningoutcomeassessment.org/NILOAsurveyresults09.htm Pascarella, E.T., Seifert, T.A., and Blaich, C. (2011). How effective are the NSSE Benchmarks in predicting important educational outcomes? Change, January/February, 2010, pp. 16-22. Volkwein, J. F. (2011, September). Gaining Ground: The Role of Institutional Research in Assessing Student Outcomes and Demonstrating Institutional Effectiveness  (NILOA Occasional Paper No.11). Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment. http://www.learningoutcomeassessment.org/documents/Volkwein.pdf Go to the AIR Website and click on the "Measuring Quality" button. Explore the inventory of assessment instruments compiled there by AIR and NILOA.  This online resource replaces the 2001 publication by Borden and Owens. Go to the NILOA Website and explore the Occasional Paper Series.  http://www.learningoutcomeassessment.org/occasionalpapers.htm

Assignments:

Lesson 5 Quiz

Lesson 6: Assessing Student Learning Outcomes II

Lesson 6

Readings:	Same as Lesson 5
Assignments:	Assessment Plan Outline "Group" Discussion Forum Analyze the Student Outcomes Datasets

Lesson 7: General Education and Liberal Learning

Lesson 7

Readings:

Textbook Suskie, Linda (2009). Writing a traditional test. Chapter 11 in L. Suskie, Assessing Student Learning: A Common Sense Guide. San Francisco, CA: Jossey-Bass. Suskie, Linda (2009). Selecting a published test or survey. Chapter 14 in L. Suskie, Assessing Student Learning: A Common Sense Guide. San Francisco, CA: Jossey-Bass. Volkwein, J. F. (2010). Assessing general education outcomes. Chapter 6 in “Assessing Student Learning: Why, Who, What, How?” New Directions for Institutional Research, (Assessment Supplement). San Francisco: Jossey-Bass Wiley, Inc., Publishers. eReserve Association of American Colleges and Universities (2012). Assessing liberal education outcomes using VALUE rubrics. Peer Review: Emerging Trends and Key Debates in Undergraduate Education. Vol. 13, No.4/Vol. 14, No. 1. Association of American Colleges and Universities (2010). Valid Assessment of Learning in Undergraduate Education (VALUE) Rubrics. Penn, J.D. (2011). The case for assessing complex general education student learning outcomes. New Directions for Institutional Research, 149, pp. 5-14. Shavelson, R. J. & Huang, L. (2003). Responding responsibly: To the frenzy to assess learning in higher education. Change, 35(1), 11-19. Go to the AIR Website and click on the "Measuring Quality" button. Explore the inventory of General Education assessment instruments compiled there by AIR and NILOA.

Assignments:

Lesson 7 Writing Assignment Analysis of Gen Ed Outcomes using NSSE Dataset

Lesson 8: Basic Skills

Lesson 8

Readings:

Textbook: Assessing Student Learning: A Common Sense Guide Using a scoring guide to plan and evaluate an assignment Textbook: Assessing Student Learning: Why, Who, What, How? Basic skills assessment eReserve Grawe, N.D. (2011). Beyond math skills: Measuring quantitative reasoning in context. New Directions for Institutional Research, 149, pp. 41-52. Hatcher, D.L. (2011). Which test? Whose scores? Comparing standardized critical thinking tests. New Directions for Institutional Research, 149, pp. 29-39. Possin, K. (2013). A fatal flaw in the Collegial Learning Assessment Test. Assessment Update, 25(1), pp. 8-9. Thomas, K. M. (2004). The SAT II: Minority/Majority test-score gaps and what they could mean for college admissions. Social Science Quarterly, 85(5), 1318-1334.

Assignments:

Lesson 8 Writing Assignment

Lesson 9: Attainment in the Major Field of Study

Lesson 9

Readings:

Textbook: Assessing Student Learning: A Common Sense Guide Creating an effective assignment Assembling assessment information into portfolios Textbook: Assessing Student Learning: Why, Who, What, How? Assessing student learning in the major field of study eReserve Banta, T.W., Griffin, M., Flateby, T.L., & Kahn, S (2009). Three promising alternatives for assessing college students' knowledge and skills. National Institute for Learning Outcomes Assessment (NILOA). Online at: http://www.learningoutcomeassessment.org/occasionalpapertwo.htm. Ewell, P. T. (2013). The Lumina Degree Qualifications Profile (DQP): Implications for Assessment. National Institute for Learning Outcomes Assessment (NILOA).

Assignments:

SPSS Analysis and Report

Lesson 10: Alumni Studies

Lesson 10

Readings:

Textbook: Assessing Student Learning: Why, Who, What, How? Assessing alumni outcomes eReserve Borden, V. M. H. (2005). Using alumni research to align program improvement with institutional accountability. Chapter 5 in D. Weerts, & J. Vidal, (Eds.), Enhancing alumni research: European and American perspectives. New Directions for Institutional Research, (No. 126). San Francisco: Jossey-Bass Inc., Publishers. Cabrera, A. F., Weerts, D. J., & Zulick, B. J. (2005). Making an impact with alumni surveys. Chapter 1 in D. Weerts, & J. Vidal, (Eds.), Enhancing alumni research: European and American perspectives. New Directions for Institutional Research, (No. 126). San Francisco: Jossey-Bass Inc., Publishers. Ewell, P. T. (2005). Alumni studies as instruments of public policy: The U.S. experience. Chapter 2 in D. Weerts, & J. Vidal, (Eds.), Enhancing alumni research: European and American perspectives. New Directions for Institutional Research, (No. 126). San Francisco: Jossey-Bass Inc., Publishers. Kuh, G. D. (2005). Imagine asking the client: using student and alumni surveys for accountability in higher education. In J. Burke (Ed.), Achieving accountability in higher education: balancing public, academic, and market Demands (pp. 148-172). San Francisco, CA: Jossey-Bass.

Assignments:

Survey Assignment

Lesson 11: Measurement Issues in Assessment

Lesson 11

Readings:

Textbook: Assessing Student Outcomes: Why, Who, What, How? Measurement issues in assessment eReserve Pascarella, E. T. & Terenzini, P.T. (1991). Appendix: methodological and analytical issues in assessing the influence of college. In Ernest T. Pascarella & Patrick T. Terenzini. How college affects students: Findings and insights from twenty years of research, pp. 657-691. Pascarella, E. T., Cruce, T., Umbach, P. D., Wolniak, G. C., Kuh, G. D., Carini, R. M. Hayek, J.C., Gonyes, R. M., & Zhao, C. (2006). Institutional selectivity and good practices in undergraduate education: How strong is the link? The Journal of Higher Education, 77 (2), 251-285. Pascarella, E.T., Seifert, T.A., and Blaich, C. (2011). How effective are the NSSE Benchmarks in predicting important educational outcomes? Change, January/February, 2010, pp. 16-22. Pike, G. R. (2002). Measurement issues in outcomes assessment. In T. Banta and Associates, Ed. Building a scholarship of assessment, pp.131-147. San Francisco, CA: Jossey-Bass. Terenzini, P.T. (1994). The case for unobtrusive measures.  In J.A. Stark, A. Thomas., Eds. Assessment and program evaluation: ASHE Reader Series, pp. 619-628. Boston: Pearson Custom.

Assignments:

Lesson 11 Writing Assignment

Note: If you are planning to graduate this semester, please communicate your intent to graduate to your instructor. This will alert your instructor to the need to submit your final grade in time to meet the published graduation deadlines. For more information about graduation policies and deadlines, please go to the Graduation Information on the My Penn State Online Student Portal.

Formal instruction will end on the last day of class. Provided that you have an active Penn State Access Account user ID and password, you will continue to be able to access the course materials for one year, starting from the end date of the academic semester in which the course was offered (with the exception of library reserves and other external resources that may have a shorter archival period). After one year, you might be able to access the course based on the policies of the program or department offering the course material, up to a maximum of three years from the end date of the academic semester in which the course was offered. For more information, please review the University Course Archival Policy.

Academic Integrity

According to Penn State policy G-9: Academic Integrity , an academic integrity violation is “an intentional, unintentional, or attempted violation of course or assessment policies to gain an academic advantage or to advantage or disadvantage another student academically.” Unless your instructor tells you otherwise, you must complete all course work entirely on your own, using only sources that have been permitted by your instructor, and you may not assist other students with papers, quizzes, exams, or other assessments. If your instructor allows you to use ideas, images, or word phrases created by another person (e.g., from Course Hero or Chegg) or by generative technology, such as ChatGPT, you must identify their source. You may not submit false or fabricated information, use the same academic work for credit in multiple courses, or share instructional content. Students with questions about academic integrity should ask their instructor before submitting work.

Students facing allegations of academic misconduct may not drop/withdraw from the affected course unless they are cleared of wrongdoing (see G-9: Academic Integrity ). Attempted drops will be prevented or reversed, and students will be expected to complete course work and meet course deadlines. Students who are found responsible for academic integrity violations face academic outcomes, which can be severe, and put themselves at jeopardy for other outcomes which may include ineligibility for Dean’s List, pass/fail elections, and grade forgiveness. Students may also face consequences from their home/major program and/or The Schreyer Honors College.

How Academic Integrity Violations Are Handled
World Campus students are expected to act with civility and personal integrity; respect other students' dignity, rights, and property; and help create and maintain an environment in which all can succeed through the fruits of their own efforts. An environment of academic integrity is requisite to respect for oneself and others, as well as a civil community.

In cases where academic integrity is questioned, the Policy on Academic Integrity indicates that procedure requires an instructor to inform the student of the allegation. Procedures allow a student to accept or contest a charge. If a student chooses to contest a charge, the case will then be managed by the respective college or campus Academic Integrity Committee. If that committee recommends an administrative sanction (Formal Warning, Conduct Probation, Suspension, Expulsion), the claim will be referred to the Office of Student Accountability and Conflict Response.

All Penn State colleges abide by this Penn State policy, but review procedures may vary by college when academic dishonesty is suspected. Information about Penn State's academic integrity policy and college review procedures is included in the information that students receive upon enrolling in a course. To obtain that information in advance of enrolling in a course, please contact us by going to the Contacts & Help page .

Accommodating Disabilities

Penn State welcomes students with disabilities into the University's educational programs. Every Penn State campus has resources for students with disabilities. The Student Disability Resources (SDR) website provides contacts for disability services at every Penn State campus. For further information, please visit the SDR website.

In order to apply for reasonable accommodations, you must contact the appropriate disability resources office at the campus where you are officially enrolled, participate in an intake interview, and provide documentation based on the documentation guidelines. If the documentation supports your request for reasonable accommodations, your campus's disability resources office will provide you with an accommodation letter. Please share this letter with your instructors and discuss the accommodations with them as early in your courses as possible. You must follow this process for every semester that you request accommodations.

Additional Policies

For information about additional policies regarding Penn State Access Accounts; credit by examination; course tuition, fees, and refund schedules; and drops and withdrawals, please see the World Campus Student Center website.

Veterans and currently serving military personnel and/or dependents with unique circumstances (e.g., upcoming deployments, drill/duty requirements, VA appointments, etc.) are welcome and encouraged to communicate these, in advance if possible, to the instructor in the case that special arrangements need to be made.