Search Views


Browse Archives

Views

Encouraging Assessment From the Ground Up

September 28, 2007

Share This Story

FREE Daily News Alerts

Advertisement

In this space last month, I wrote about how assessment from the ground up means that accountability for colleges and universities ought to flow from the improvement of student learning and not the other way around. In the responses to that article, and in the work that my colleagues at Alverno College have done with other institutions over the last three decades, a compelling question arises: How can we encourage one another as faculty to engage in assessment that will work for our students and for us, and not just be a bureaucratic chore?

Under pressure from accreditors and others, just about every college and university has declared that it has some form of measuring learning. But we also know that assessment data are gathering dust in file cabinets around the country, and that learning outcomes have gone into syllabi and quietly died. But when this has not been the case, when faculty have embraced assessment as central to their teaching, what has made the difference? How and why have faculty tied learning activities and assessment to course outcomes so that students themselves see achieving the outcomes as essential to success in a course or program?

My Alverno colleagues have conducted workshops on the improvement of teaching and assessing at colleges and universities in every state in the union and around the globe. And we have hosted a summer teaching and assessment workshop at the college for over 30 years. Our goal has been to share how assessing students’ performance has improved learning, and has provided us with evidence to document progress in individual student learning and the effectiveness of the curriculum as a whole.

In our experience, there are at least five things that have been helpful in engaging faculty in teaching for and assessing learning outcomes:

1. Draw on the expertise of professors who are already -- even without a formal assessment protocol -- doing effective work in teaching, and in understanding what helps students learn. Colleges should create permanent spaces and places for faculty to brag about teaching jobs well done, and offer rewards for individual faculty and programs where effective learning is taking place. Many attempts to institute assessment have unfortunately proceeded from a deficit model – “teaching needs to get better around here, and we are going to bring in consultants and faculty developers to show you the error of your ways.” It may be true that faculty members who have learned to teach on the job – as is true of the vast majority of college and university teachers today – have developed some bad habits and ineffective approaches. But it is also true that there is great teaching going on in every institution. Share and reward what’s working, and recognize faculty expertise as teachers as well as scholars of the disciplines. An important part of this discourse is developing and sharing assessment processes that get at the kind of learning that faculty expect of students and provide insights into how to improve teaching. Encourage that professor who tried something with her students last year that failed to explain to her peers how she learned from that experience and what she is doing now to get better results. This sort of sharing among faculty will do more to advance assessment that improves student learning than providing canned rubrics and requiring end-of-term assessment paperwork.

2. Move toward reward structures that encourage and recognize this kind of faculty collaboration. Develop criteria for excellence in teaching and assessment of student learning, and make them central – not peripheral – to faculty hiring, tenure and promotion. Voices in higher education have been saying this since Ernest Boyer’s work on the scholarship of teaching, but a real change in attitudes about the importance of teaching in faculty life has yet to occur. Take seriously the need to nurture good teaching throughout a faculty member’s career, and institute supportive apprenticeships for new teachers, who still get little to no attention to their development as teachers in their graduate preparation. Legitimize the scholarship of teaching, by recognizing its value alongside disciplinary research, but also create an institutional culture that prizes scholarly teaching. I take the term “scholarly teaching” from my Alverno colleague Tim Riordan, who has written and spoken extensively about the systematic and deliberate and joyous pursuit of the improvement of student learning that should be a part of every faculty member’s professional life.

3. Create communities of practice around teaching and learning issues that faculty themselves see as critical to their work. It can be difficult to reach across institutional divides to have interdisciplinary conversations about teaching and learning, but one way to overcome barriers is to capitalize on informal gatherings to provide sites for discussion of issues faculty are raising in the context of their teaching. Try teaching breakfasts, brown bag lunches for effective assessment, coffees about how to work with brand new or challenging students. At Alverno, our Teachers of New Students has been meeting every Friday over lunch for over 25 years to discuss work with freshman and new transfer students. Institutions can nurture these efforts by upping the catering budget, providing the publicity, and recognizing their worth in official pronouncements by institutional leaders. It is surprising but true that discussion of teaching and assessing student learning can be just as intellectually stimulating as discussing disciplinary research, does more to create a community of shared interest among the faculty, and validates good teaching efforts.

4. Emphasize that collaboration to improve the teaching and assessing of student learning need not violate academic freedom or faculty autonomy. Professors have a shared responsibility for student learning, and that responsibility depends in part on where we teach. We have a duty to take into account the make-up of the student body, the mission of the institution, and in the best cases, what’s consistent with the shared pedagogical approach of the faculty. Faculty agreement to establish and assess student learning outcomes can provide a framework for curricular coherence, while avoiding a restrictive rigidity for either faculty or students. It is vital to preserve for professors the space to teach their disciplines in light of their own understandings and interpretations, but equally vital to uphold the promise of effective teaching and learning that is extended in the mission of every college. In our experience faculty can collectively coordinate assessment across the curriculum to make learning more accessible, meaningful and rigorous for students, without having to give up the ways of knowing and methods of inquiry appropriate to their disciplines.

5. In working with institutions everywhere, we have also learned that leadership on behalf of improving learning and assessment – both formal and informal – is critical. Administrators can sometimes get things started, but it is faculty members who keep assessment going and meaningful. Without the expertise in and advocacy for assessment of faculty members who have their colleagues’ respect, there is little hope that assessment practices will persist when the accreditors have gone away or the provost has left the room. Recognizing a faculty member’s emerging expertise in assessment by making responsibility for assessment part of his or her load is commendable. Adding responsibility for assessment to the load of a talented but already overburdened faculty member means the job just will not get done. Be realistic about the time it will take for individuals, departments, and corporate faculties to work together in new ways to take responsibility for improving student learning, establish goals and timelines for the process, and celebrate milestones along the way.

Instituting assessment from the ground up takes resources in time, talent, attention, publicity, and developing faculty expertise. Cultural change will be required, as will long term leadership and support. But from the perspective of those who have been engaged in assessing from the ground up, its rewards in terms of student learning and, ultimately, faculty satisfaction at a mission achieved, are well worth it.

Donna Engelmann is professor and chair of the philosophy department at Alverno College, and past president of the American Association of Philosophy Teachers.

See all postings »
Advertisement
Advertisement

Matching Jobs

Comments on Encouraging Assessment From the Ground Up

  • Posted by kate , Professor of Sociology on September 28, 2007 at 8:05am EDT
  • Here's a few more. Don't let programs get away with calling 'failure,' 'success.' In every school, there are, for example, grad programs that swore by year 4 they'd be getting 100 new students in that are pulling in 20 every other year instead. For whatever reason, that program is not 'selling itself' well and if it is way too cost ineffective -- then don't let the periodic review be a compliment-fest -- cut the program. Let faculty see that that program evaluations and assessment are not just exercises in creative writing to show that we are teaching at Lake Wobegone! We can do the math; we know how much a program costs -- even just the faculty salaries -- and why sink money into programs that are not working? Shut down ineffective programs. Then maybe we'll take it seriously. But when the worst programs (worst in # of students, the easy 'A's and even then they can't pull in students, etc.) get rewarded -- we see how the game is played. And it is about programmatic presentation of self and CYA, not real assessment. So we'll play along too then. Why not?

  • Posted by Patrick Sanaghan on September 28, 2007 at 8:05am EDT
  • great article. If faculty don't really buy into assessment , it doesn't matter. It then becomes an exersize that needs to be done rather than a learning process that improves the quality of teaching and learning. You offer sound, practiacl strategies and advice.Thanks

  • Posted by Dave Stone on September 28, 2007 at 8:50am EDT
  • Donna Engelmann wrotes "when faculty have embraced assessment as central to their teaching, what has made the difference?"

    I think that's the wrong question. The question in my mind, and the mind of my fellow faculty, is "when faculty have embraced assessment as central to their teaching, WHAT DIFFERENCE HAS IT MADE?" I already assess student learning in my classes with time-honored rubrics called "exams" and "papers" and "talking to my students." What's the evidence that embracing the brave new world of assessment actually provides more benefit than it costs?

  • Assessing Writing
  • Posted by Bob Schenck on September 28, 2007 at 9:35am EDT
  • Dr. Engelmann suggests creating “spaces and places for faculty to brag about teaching jobs well done.” She writes of “the systematic and deliberate and joyous pursuit of the improvement of student learning.” She proposes “teaching breakfasts, brown bag lunches for effective assessment, coffees” and “upping the catering budget,” providing “publicity,” and praise for assessment in “official pronouncements by institutional leaders.” Schools will “celebrate milestones.”

    What is it about this exhortation that makes me weary?

    Regular student attendance might improve learning. Should we require attendance? Daily reading and writing might improve learning. If we require it, will students do it? And then there is the problem of actually assessing student writing in a discipline where writing is the primary outcome by which understanding is known. Is there any valid, reliable means of measuring this other than a slow, careful, deliberate sentence-by-sentence reading of student text? Is there any quick and convenient shortcut? I know of none. A teacher has done this once. Shall it be done again? By whom? And if the second assessment is in conflict with the first, shall student writing be assessed a third time? Of course we could simply count pages, or paragraphs, or words, or errors, or component parts, but a clerk could do that.

    These dilemmas have proven intractable at my institution, though we have duly completed and submitted the required forms. Evaluating and assessing writing demands an informed, engaged, careful reader who ideally is also familiar with a student’s academic performance in the course over a period of weeks and months. There is only one person who might possess all of these qualifications. He or she has assessed the writing once. Shall we make this teacher do it all again?

  • What difference has it made?
  • Posted by T-bone on September 28, 2007 at 10:10am EDT
  • You might try reviewing the research on Feedback Intervention Theory that showed that Feedback Interventions improved performance by d = .41.
    Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119(2), 254 - 284.

    You might also consider the work of Robert Marzano on "what works" in classroom instruction, as he has also studied the impact of feedback and goal setting on student learning.

    Studies on the impact of assessment at the program or institutional level are less common and still emerging - you might consider work by Banta, Lund, Black, & Oblander, or by Peterson & Vaughn, or others.

  • Accessing the time for Right assessment
  • Posted by Prof Priyavrat Thareja at Pb Engg Col, Chandigarh on September 28, 2007 at 11:00am EDT
  • Dr Engelmann's assessment is a good supplement to balancing of the time between teaching and (student) learning. Imcrease one, and the time for other gets starved.

    There is possibly a need to inculcate responsibility for watching their learning. In that case, the scope/purpose of assessment can be limited to ensuring the variation and efficacy of rightful-retention in learning.

    In our educational system in India, assessing the motivation and focus is a big challenge. So is the problem of attracting students in the class. Not exactly teaching Quality, but rather breaking off the abstractions.

    Priyavrat Thareja

  • Assessing Writing--programs
  • Posted by Jeffrey Klausman , English Professor at Whatcom Community College on September 28, 2007 at 1:00pm EDT
  • Bob Shenk is absolutely right about assessing writing--that a careful reading by an informed reader is all we've got, but we need to go a bit further. Who is that informed reader and in what and in what ways is she or he informed? These are tougher questions and support Engelmann's point about collaboration and community. Writing teachers who assess writing need a community in which to do that assessment; is it then possible to communicate the standards of that community to another audience and give examples of individual assessment? That's the question I'm grappling with in my program.

  • What difference has it made?
  • Posted by Dave Stone on September 28, 2007 at 1:35pm EDT
  • T-Bone suggests looking at Kluger and DeNisi for evidence of the good assessment can do.

    I think Kluger and DeNisi prove my point. They define feedback intervention as "actions taken by an external agent to provide information regarding some aspect of one's task performance." Any decent teacher will regard this as self-evidently necessary and an integral part of what teaching is about. The question remains: when we bureaucratize what good teachers already do, and have done for centuries, while stirring in a generous helping of eduspeak, what is the value added?

  • Going deeper
  • Posted by Robert Cape at Austin College on September 28, 2007 at 1:35pm EDT
  • Donna Engelmann is right about assessment being successful if it is driven by faculty “from the ground up.” However, I would propose that the process begin deeper down.

    For assessment to be most meaningful, it must play an important role in how a faculty member identifies herself/himself as a professional in the field. Whether we work at teaching or research institutions, we define ourselves according to the standards of our professional societies. When student learning and the assessment of student learning become an integral part of what it means to be a professor in a particular field, faculty members will come to embrace it.

    Of course, the process goes faster when a department needs to be accredited by its own professional association. Otherwise, assessment will always be imposed by people from the “outside.” In a recent discussion about student learning outcomes and the importance of undergraduate research, I learned about the American Chemical Society’s guidelines for departments desiring certification. It became no surprise that chemistry faculty have done more than many others in terms of assessment and undergraduate research, since it is part of what it means to be a chemistry professor.

    Until the major academic professional societies (at least the ones I am familiar with) make learning assessment an important part of their mission, along with consideration of what it means to be a practicing member, assessment will remain peripheral to a faculty member’s work in the field. And when professional societies make the assessment of student learning an integral part of their mission, they need to be in touch with the best teachers in the discipline, not necessarily the teachers at the best universities or those who typically drive their research mission. Faculty at teaching institutions can feel marginalized at professional meetings, yet they contribute enormously to their field. Guidelines and suggestions for teaching and assessment practices should come from the faculty who make real gains in student learning.

    As Englemann notes, the importance of good teaching and understanding how and why it is good has been around a while. She writes, “Voices in higher education have been saying this since Ernest Boyer’s work on the scholarship of teaching, but a real change in attitudes about the importance of teaching in faculty life has yet to occur.”

    I submit that the change has not occurred because it has been pushed at the institutional level, imposed on faculty for, apparently, someone else’s purpose. (That there is a common good involved should be obvious, but often isn’t.) Ideally, as Engelmann writes, the push should come from the faculty.

    Even more, if assessing student learning becomes a part of what faculty do as practitioners in their field, I think it is safe to say that it will be embraced more widely. Equally, if the professional societies take action on the initiative of faculty, they have the opportunity to provide direction about the best practices in their field, ensuring that when the outside agencies do come around, they must take into account what the professional society says are the appropriate standards.

  • Using Evidence from Your Students to Improve Learning
  • Posted by Steve Ehrmann , Dir. Flashlight Program at The Teaching, Learning, and Technology Group (TLT Group) on September 28, 2007 at 1:35pm EDT
  • Very nice article and a good discussion. One topic that sometimes is overlooked is the potential importance of using evidence from students to figure out how to make course activities work better. WHY aren't students doing that writing? HOW can I get better participation in my online discussions? Is there anything you can find out from students that could help you improve that course?

    All faculty learn from their students. But we're working on strategies to help busy, skeptical faculty learn some new tricks. I won't describe this program here. It's called "Asking the Right Questions" (ARQ) and it's described starting on this web page:
    http://www.tltgroup.org/Flashlight/ARQ/Index.htm

    If you have suggestions or questions about this brand new initiative, please get in touch.

  • Community Standards—Jeffrey Klausman
  • Posted by Bob Schenck on September 28, 2007 at 2:00pm EDT
  • One way might be to collect and maintain an online log or portfolio of student writing acceptable to a majority of the local discourse community. This portfolio might contain a variety of first-person narrative essays with formatted dialogue; a variety of exposition and argument both with and without citation and documentation; fiction; poetry; business and technical genres; and also a selection of blended forms. Both students and faculty could access the collection to learn what is expected and what is acceptable and to compare new student writing to it. Perhaps critiques of pieces in the collection could also be included, critics explaining why they do or do not concur with the selections. Another section could contain pieces of student writing which were nominated but did not receive consensus approval, their advocates explaining the reasons for the nominations, their critics explaining their objections.

  • Assessing Encouragement From the Ground Up
  • Posted by Frizbane Manley on September 28, 2007 at 9:30pm EDT
  • Over here at the College of Mother Theresa we have pretty much decided that an unusual obsession with assessment has a very detrimental impact on our students’ learning and self-esteem, so we have decided that every moment previously devoted to assessment will, in the future, be a moment devoted to encouragement. I was asked to write a paper about the subject, but, thanks to Professor Engelmann, her “Encouragement of Assessment From the Ground Up” can be converted into my paper, “Assessment of Encouragement From the Ground Up.” Just scroll through her paper and change every use of “assess” to “encourage” and every use of “assessment” to encouragement” ... and, lo and behold, there is absolutely no loss of intellectual content.

    I am a mathematician, and, back in the day, I used to spend a lot of time presenting the subject matter of my classes and interacting with my students in a manner that was likely to (1) enable them to love mathematics and (2) once they fell in love with the queen of the sciences, allow me to jump out of their way while some pretty damned exciting learning took place. What I’m telling you is that my primary objective – and I’m pretty sure it wasn’t a LEARNING objective – was inspiring love. If you love that stuff, you will just devour it in large quantities ... and continue to do so, albeit less intensively than as a college student, for the rest of your life.

    Then I went through a phase – like the one that apparently defines the culture for learning that IS Alverno College – in which I presented students with my learning objectives, employed one or more of the in-vogue teaching-learning strategies, and paused every few minutes to assess my students’ learning. I really got off on assessment. But, since I could not accurately measure the extent to which my students loved mathematics I decided it simply couldn’t be all that important and I removed “you will love mathematics” from my learning objectives.” Seriously, can you just imagine me standing in front of my class, shaking my fist raised on high, and shouting “You’re going to love this stuff if it takes me all semester!” You see, most of my important learning objectives were not about things ... they were about attitudes ... about principles ... about how my students feel and think about the world in which they found themselves. Their answers to problems of differentiation and integration were much less interesting to me.

    More than anything else, however, my students and I found teaching strategies built around assessment were just too boring. So, we’ve moved on to encouragement, thinking it would serve us well until the next educational or management flavor-of-the-month captures the attention of our local business school or college of education.

    But thanks Professor Engelmann, for letting us know what we’re missing. We’ll keep you apprised of our successes with and assessment of encouragement.

  • Frizbane Manley
  • Posted by Bob Schenck on September 28, 2007 at 10:05pm EDT
  • Hooray, F. Manley! Though I teach language and literature, you have expressed my sentiments exactly! Thank you.

  • just okay
  • Posted by PS on September 29, 2007 at 5:20am EDT
  • This well written and well intentioned article won't matter. As long as faculty continue to value parking over learning and laziness over hard work, they will never be able to articulate what their students learn (because at the classroom and program there is very little evidence that any of them are).

    My advice? Be patient and wait for the feds and/or state governments to make faculty use it to improve learning and tie their salaries and benefits to it. It is only a matter of time now.

  • Naivete, Inexperience, Ignorance
  • Posted by Bob Schenck on September 29, 2007 at 12:30pm EDT
  • PS believes the federal and state governments can and will demand assessments that can and will pressure faculty to teach in ways that can and will make students learn. PS need write nothing else. This belief alone is evidence of PS's utter and complete naivete, inexperience, and ignorance on the reality of learning.

    Government mandates will work in education exactly as well as did Joseph Stalin's five-year plans for Soviet agriculture and industry and Bill Kristol's five-year plan for the liberation and democratization the Nation of Islam. Official documents and reports will show that results far exceed even extraordinarily ambitious objectives and goals.

    Successful teachers and their government overseers will receive public honors, medals, and rewards at televised civic celebrations. Critics of the plan, those skeptical of official reports, and "unsuccessful," "noncompliant," "lazy" teachers and schools will be vilified and suffer political reprisals. Perhaps if we threaten to starve or to shoot the teacher, PS will wonder, maybe then he'll produce!

    Alone, behind the scenes of such theatre, the solitary teacher's tasl will remain unchanged: How to get a student to read and to think.

  • No Kudos To Those Who Oppose ...
  • Posted by Frizbane Manley on September 29, 2007 at 12:30pm EDT
  • PS is a person after my own heart.

    Seeing that there are some teachers who are not inclined to genuflect in the presence of Engelmann/Alverno-style assessment, PS gets right to the root cause of the problem, recognizing that those who oppose suffer from misplaced values and laziness.

    I am pleased to learn that, because I was operating under the misconception that there were intellectually and pedagogically sound instructional strategies (styles) that were every bit or more “effective” than the now famous Faculty Assessment & Review Technology.

    Frankly, I can hardly wait to get Form 633, Section A from the Feds to administer to my Advanced Structural Equation Models class. And, PS, will that be sent to me by the Department of “Education” or will there be another cabinet-level department created to help us assess our citizens from the ground up? And just as a matter of curiosity, does “from the ground up” suggest that we will have prenatal assessment?

  • Oops!
  • Posted by Bob Schenck on September 29, 2007 at 7:10pm EDT
  • In my last post I meant to write "awards," not "rewards," although there might be rewards, too, and I meant to write "task," obviously, not "tasl," whatever that is. Sorry, the mind goes.

  • Posted by Robert , PhD Student on October 1, 2007 at 8:25am EDT
  • Encouraging assessment from the ground up includes getting students to buy into the importance of assessment. At my undergraduate institution (Truman State), administrators made sure to stress assessment to students on a regular basis and how it helped better the quality of their education. As a result, students would actually be involved in the assessment process from filling out surveys to helping with the reaccreditation process.

  • Allllll Right Robert!!!
  • Posted by Frizbane Manley on October 1, 2007 at 3:45pm EDT
  • And I just can’t tell you how thrilled I am to discover that the students at Truman State University are (1) filling out surveys and (2) helping with the reaccreditation process.”

    Damn, that just warms my heart.

  • Guys, look around you
  • Posted by Grocheio , Asst VP Planning and Institutional Effectiveness at Shorter College on October 1, 2007 at 4:35pm EDT
  • This conversation leaves out what I have come to consider an essential aspect of assessment: it's a communication process. As such it has to be reported in such a way that the intended audience can understand. The intended audience, namely most of the public as well as internal audiences, no longer accepts on faith how wonderful all professors are, because they didn't go to college, and now their children are. Another aspect of communication is expressed in the following quotation I collected years ago: "The best programs use assessments which help students recognize their own growth." Finally, we can have the communication among professors mentioned in the article.

    Dedicated teachers devotedly grading your papers and encouraging students, have you ever noticed patterns in student performance and wondered what those mean and whether they could be affected by any change in the environment? Or would you just prefer using the same methods that were effective when you were in college, despite the fact that college students are now different in many ways?

  • Time-honored tests and quizzes?
  • Posted by Shree Iyengar on October 19, 2007 at 5:15pm EDT
  • We have been using tests and quizzes since day one of education. Aren't there faculty who keep recycling their old tests and quizzes throughout their tenure? Are the grading scales linear? Why would a student get an A in one faculty's course and if taking it with another would get a B or a C? Do we link the test questions to what we think we want our students to learn?

    We don't need to go overboard with assessment. But, those who 'know it by experience when a student has learned something' need to rethink and show how they know this! When they convince their colleagues, they have succeeded in assessing their student learning. My students are demanding, aren't yours?