Blog
Blog

How can we reach teaching excellence?

How do we know we do a good job as teachers? This piece reflects on ways to reach teaching excellence in higher education.

Source: Clément Lafon Placette ,14. 7. 2021, magazine Vesmír,  Photo: Felicia Buitenwerf, Unsplash

 

How do you know you have done a good job? If you are familiar with neuro-linguistic programming (or if you are a student I had the pleasure to teach), you will recognize here a typing question [1]. People typically reply either “I know I did so. It fits my standard” (internal frame of validation) or “people/my boss/students/statistics told/showed me I did a good job” (external frame). What are you? While there is no good or bad answer, this question becomes tricky as a teacher.

Disclaimer: surprisingly, it seems that offending people is not a good way to stimulate a constructive dialogue, and is instead a great means to polarize opinions [2]. Because the topic in this piece is dear to my heart, my goal is more than ever to work together, and therefore I promise not to offend anyone (intentionally, that is).

How do we know we do a good job as teachers? Should we simply follow the method we think is right, independently of the effect on student learning? Or should we instead be only focused on what students tell us work for them? Should we aim at the students liking the topic (or us) or should we only make sure they learn, independently of their pleasure in doing so? Is student pleasure to learn actually linked to their learning outcomes? Is the degree of learning perceived by students a good indicator of their actual learning? How do we measure student learning anyway? And finally, what do we do to improve our teaching, and what does improving our teaching mean in the first place? If you are a teacher, all these questions have certainly crossed your mind. There is obviously no easy answer to these, but let us not get discouraged to seek one thing: providing the best education we can and improving it, trial after trial. This piece reflects on ways to reach teaching excellence in higher education, and how to evaluate its quality, both as persons and institutions.

Student evaluations are not enough

In most countries in the world, as in the Czech Republic, universities have found a one-size-fits-all solution to answer the question “are we doing a good job at providing excellent education?”: student evaluations. Courses and teachers are evaluated by the students at the end of the semester/year. This solution is understandable: this is a relatively time and cost-effective way to address a crucial question for a university. This also may provide useful feedback for the teachers themselves. Nevertheless, several studies [3, 4] have raised an important issue: the correlation between student evaluations and their learning outcomes is pretty weak. In other words, how well students evaluate a course/teacher does not tell much about whether they actually learnt what they were meant to. This came to me as disturbing news and yet, not really as a big surprise. Indeed, many factors may bias the way students evaluate a course and/or its teacher: difficulty of the topic, size of the class, or even gender bias, as women teachers seem less well rated than their male colleagues [3, 4]. I do not even mention that the pool of students taking part in the evaluation is biased [5]. These studies therefore called for implementing other ways to evaluate the quality of courses and teachers. How then to measure whether a course or teacher achieved their purpose?

Before answering this tricky question, one needs to clarify a few things: what do we actually mean by “learning”? That may seem trivial, but there is more than meets the eye. Also, what is actually the purpose of a course and its teacher? Answering this question requires to touch an important topic: the philosophy of teaching. As with different teaching philosophies come different teaching goals. Only after going through these points, we can consider how to improve the teaching quality in universities, which includes how we evaluate such quality.

On learning

Intuitively, we probably all have a vague idea of what “understanding” a topic means. But if we want to implement teaching strategies to provide the best possible education, we need to precisely know what the learning process entails. Luckily, this has been the subject of psychology and pedagogy research since decades [6].

One of the well-established concepts resulting from this research is Bloom’s taxonomy of learning [6]. In short, this taxonomy hierarchizes different levels of learning, from the simplest task to the most complex and demanding one.

Bloom’s taxonomy of learning [6].

You may have never seen this taxonomy, and yet, if you are an academic, you intuitively know that all these levels of learning are required to make a good scientific article, for example. How do we assess students’ level of learning then? Having this taxonomy in mind, we can identify which type of exam assesses which level of learning. Here is an example: “identify this plant species and describe its ecology”. This exam tested for the first two levels of learning only: “remember” and “understand”. Similarly, multiple choice questions (MCQs) test only for the most basic level of learning, i.e. “remembering”. This is the reason why leading figures in pedagogy proscribe the use of MCQs: certainly MCQs make the work of the evaluator easier when there are 100s of students; however, if MCQs or similar low-level tests are used across the whole curriculum, this produces students that can “remember”, but for whom we do not know their abilities for higher orders of learning. And yet, when these students become doctoral students, they are suddenly expected to be able to write scientific articles, which requires all levels of learning. The problem is that they have not been trained/tested for such skills. 

This is perhaps a strength of the French education system, what makes the French famous for their “critical thinking”: from high school onwards, students are evaluated via essays. By essay, I mean a work of several pages which supports a well-argued stand, and is structured with an introduction, arguments with examples, and a conclusion. Not far from a scientific article in a way. For example, at the end of high school, students have to pass a final exam for each course (biology, maths, philosophy…). Presented with a question such as “What is the point of explaining a work of art?” (2019 philosophy national exam), the students have to write an essay. Such essay is actually a very good way to evaluate all levels of Bloom’s taxonomy: the students have to use all of them, from “remembering” (philosophy concepts learnt during the year) to “create” (writing a structured essay ending with a stand, a conclusion which is personal to the students, based on their knowledge and reflexions that they developed along the essay). Easy, huh? Well, I miserably failed at that exam.

Anyway, the essay as an exam becomes the norm in French universities, at least for biology (what I took). This can be scary to students; at least the ones I tortured with essays told me so. Why scary? Well, the “create” part is scary: one needs to take a stand, without knowing if it is the “right” one or not. And “being wrong” seems a very scary thing for students. To have a no-blame culture during the class is thus crucial if we want students to take the risk of making mistakes. Another crucial point: if we want to keep students motivated, it is shown that we need to give them ambitious yet achievable challenges [7]. Not ambitious enough? The students get bored. Too ambitious? The students get discouraged. And as a student, I actually loved these essay exams: I saw them as ambitious challenges, as it was finally time to show I could do more than just vomit knowledge I was fed, so to speak, to show I was able to think for myself. And yet, these challenges remained achievable (given that the student I was, more interested in gothic rock than biology, studied enough to master Bloom’s “remember” level, of course).

Another thing made these challenges achievable: training. Before the final exam, the course would include learning activities to improve every level of Bloom’s taxonomy, from acquiring theoretical knowledge to producing work requiring higher order of learning. This congruence between learning activities and assessment of students’ skills, both aligned towards intended learning outcomes, has been extensively theorized and applied under the name “constructive alignment” [7], and is one of the “learning outcome-based” teaching strategies. Constructive alignment is used in universities in Sweden, where I spent some years.

A shift to learning outcomes-based teaching and to “what students do”

We all strive for education excellence, but what does such excellence depend on? Constructive alignment has a special answer to this question (7), but let us examine what sort of answers exist, each of them relying on different teaching philosophies. Importantly, from each of these philosophies stem different ways of evaluating the quality of courses and teachers.

“What students are”. This is not a secret for anyone: students enrol for different reasons, with different levels of engagement, different natural inclination for certain skills. Some may want a well-paid job, some do not know what else to do, and few others are natural scholars and interested in academic learning per se. The “what students are” approach to teaching is simple: some students are “good/motivated” (= the natural scholars), and the job of the teacher is to find those. If the other students cannot follow a course, it is not a big problem or even, “losing them” is the goal. I recently had an extremely brilliant student, able to think and solve problems way faster than I could during class time. However, writing essays appeared to be an excruciating task for her. Not the “natural scholar” type. The kind that would be discarded with the “what students are” approach. This approach does not change the status quo: the “natural scholar” students would make it without teachers, as they are intrinsically motivated to learn. As about all the others, they would leave university with the same level of engagement and skills as they entered. With this teaching approach, how are student evaluations considered? Well, a course/teacher being rated bad is actually a good sign: “this course is not for everyone, only for the good ones.” Excellence here is synonym of being selective, and is the tree that hides the forest: “look at this brilliant student, our teaching is working for the good ones”.

“What the teacher does”. In this approach, the teacher focuses on what he/she can do to improve the quality of education provided, how to make the message clear or how to motivate students, by varying the content of lectures, learning communication techniques… Compared to the previous approach, it is certainly a great step forward as the teacher spends time and efforts in improving his/her way of teaching. This approach is rewarded by the student evaluations: if the course and teacher are liked by the students, because thanks to the improvement efforts, the teacher explains clearly, is engaging etc., this must mean that the learning is good. Nevertheless, this approach to teaching does not focus on monitoring whether the students reached learning in all levels of Bloom’s taxonomy, and therefore a student evaluation of a course/teacher scoring high does not mean that the students really learned what they were supposed to.

“What students do.” This approach is focused on the end result: the learning outcome. It therefore consists in 1. setting intended learning outcomes; 2. designing teaching activities for students to reach the intended learning outcomes; 3. assess (exam) whether the students reached the intended learning outcomes at the end of the course. The learning activities consist in students performing tasks matching each level of Bloom’s taxonomy. Why is action important? Simply, it is the only thing measurable. What a student thinks is not, we cannot read minds. With courses where students act rather than listen, the students can evaluate whether they are progressing (self-assessment being more fruitful for learning [7]) and the teacher can do so too. Also, higher levels of learning in Bloom’s taxonomy imply active tasks. With “knowledge transfer” (powerpoint lecture), students never practice the “create” level of learning. Engaging students also keeps them motivated, including the ones that would be lost with other approaches. As mentioned above, giving achievable challenges to students is an efficient way to motivate them, and learning activities up to the highest levels of Bloom’s taxonomy are challenging, yet achievable if we take the time to bring the students there. Indeed, achieving excellence here means a strategy of teaching for a deep learning, but this takes time. Consequently, because more time is spent on “climbing” levels of learning, it means to a certain extent sacrificing on the amount of topics “covered”. It comes to an important point: do we want students to be living encyclopedias, containing a lot of raw information accessible on demand, or do we instead want them to be able to process this information and produce new ideas? Google is now taking over the encyclopedia role, while making sense of such information is, for now, still better done by humans than Google.

Certainly, the “what students do” approach relies on “what the teacher does”, as it requires the teacher to constantly monitor whether the teaching activities serve the intended learning outcomes and adjust them accordingly, but it goes beyond that, as it aims at the “end product” (= what students do) rather than the process to reach so (= what the teacher does). And this is precisely the blind spot of student evaluations of courses/teachers: as explained above, they evaluate only what the teacher does (clarity of lectures etc), and not whether the intended learning outcomes are reached by the students.

The need to evaluate learning outcomes instead of students’ impression. How to do so?

Everyone will agree that we need to know how well we perform at educating students. However, student evaluations of courses/teachers as they are now only assess the quality of what the teacher does, and while it is necessary, it is not enough: we need to know if what we do as teachers leads to students reaching the intended learning outcomes.

How to do that? Well, first of all, we teachers need to set intended learning outcomes. Courses where we set intended learning outcomes that span all the levels of Bloom’s taxonomy are important for our students to reach a deep learning of the topics we teach them.  

Then, a successful, high quality education means a maximum of students reaching the intended learning outcomes (again, = deep learning spanning Bloom’s taxonomy). How do we know if we manage to do so as teachers? Well, I do not have a perfect solution. Certainly, when we test students, this is what we do. But obviously, as we teachers design the course AND judge if the students reach intended learning outcomes, we face a conflict of interests. So we cannot use student results at the exam to evaluate if we did a good job at teaching them. What about having the students tested by another teacher? We could imagine pairs of teachers that would agree on the intended learning outcomes prior to the start of the course, and one teacher would teach and the other would test the students. The second teacher could be from a different university to reduce conflicts of interests at maximum. Essays as the form of exam would ensure that one exam assesses all levels of Bloom’s taxonomy of learning.

Perhaps, another more doable way to evaluate if we did a good job at teaching would be to use the student results at the final exam, but such exam would be done with a process called calibrated peer review . In brief, instead of the teacher testing the students, such examination is done by the students themselves (they assess each other, anonymously), after they get trained on how to evaluate if learning outcomes are reached or not. This has the advantage to remove the conflict of interests of the teacher being judge and designer of the course. This way, we could base the evaluation of courses/teachers on an average between students’ impressions (the current evaluation) and their success in reaching learning outcomes (from the results of their final exams done with calibrated peer review). What do you say?

In conclusion

What about having only courses that aim at providing all levels of learning up to higher orders? That would mean both training and testing the students for all levels of learning (and forgetting about MCQs and other “memory-only” exams; what about introducing my favourite type of exam, i.e. essays?). Doing that is best achieved with a learning outcome-based teaching approach (“what the students do”), and constructive alignment is a good one - but not the only one. And starting as early as possible may make sense, as to acquire skills related to higher levels of learning may take time (it took years for me). Also, for the sake of improvement, we need to know how well we perform as teachers and as universities. Nevertheless, could we evaluate the quality of courses/teachers less based on students impressions (“what the teacher does”) and more on their actual learning outcomes (“what the students do”)? Maybe via some inter-teacher examination of essays. Maybe via calibrated peer review. A perfect solution for the evaluation of courses/teachers remains to be found, and has not been found anywhere in the world yet. Are we up to the challenge?

 

References

[1] Charvet S. R.: Words That Change Minds: The 14 Patterns for Mastering the Language of Influence. Institute for Influence, 2019.

[2] Lafon Placette C.: Česká vědo, pozor na to, po čem toužíš. Vesmír 99, 240, 2020/4.

[3] Uttl B., White C. A., Gonzalez D. W.: Meta-analysis of faculty’s teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation. 54, 22–42, 2017, DOI: 10.1016/j.stueduc.2016.08.007.

[4] Boring A., Ottoboni K., Stark P. B.: Student Evaluations of Teaching (Mostly) Do Not Measure Teaching Effectiveness. ScienceOpen Research, 2016, DOI: 10.14293/S2199-1006.1.SOR-EDU.AETBZC.v1.

[5] Reisenwitz T. H.: Student Evaluation of Teaching: An Investigation of Nonresponse Bias in an Online Context. Journal of Marketing Education, 38, 7–17 , 2016, DOI: 10.1177/0273475315596778.

[6] BloomB. S., Engelhart M. D., Furst E. J., Hill W. H., Krathwohl D. R.: Taxonomy of Educational Objectives: The Classification of Educational Goals by a Committee of College and University Examiners (Handbook I: Cognitive Domain). Longmans Publishing, New York, 1956.

[7] Biggs J., Tang C.: Teaching For Quality Learning At University. Open University Press, Maidenhead, 4th Edition. 2011.

 

About the author

After a PhD in France in 2012 and a subsequent postdoc in Sweden, Clément Lafon Placette (born 1985) was hired in 2018 as assistant professor by the Department of Botany of Charles University in Prague, where he has been working since then. His research and teaching interests revolve around plant reproduction evolution, speciation and genomics.

https://vesmir.cz/cz/on-line-clanky/2021/07/jak-muzeme-dosahnout-vynika…

Login (6)
Tikketti

Trid lingwa oħra?

Dan id-dokument huwa disponibbli wkoll f’lingwi oħra. Jekk jogħġbok agħżel waħda hawn taħt.
Switch Language

Want to write a blog post ?

Don't hesitate to do so! Click the link below and start posting a new article!

L-aħħar diskussjonijiet

EPALE 2021 Thematic Focuses. Let's start!

We invite you to enrich what is set to be an intense year ahead with your contributions and expertise! Let's start by taking parto in this online discussion. The online discussion will take place Tuesday 9 March 2021 from 10 a.m. to 4 p.m. CET. The written discussion will be introduced by a livestream with an introduction to the 2021 Thematic Focuses, and will be hosted by Gina Ebner and Aleksandra Kozyra of the EAEA on the behalf of the EPALE Editorial Board.

More

Diskussjoni ta’ EPALE: Il-forniment ta’ tagħlim ta’ ħiliet bażiċi

Il-ħiliet bażiċi huma trasversali. Dawn mhumiex rilevanti biss għall-politika edukattiva, iżda huma rilevanti wkoll għall-politiki tal-impjiegi, tas-saħħa, soċjali u ambjentali. Il-bini ta’ miżuri ta’ politika koeżivi li jappoġġjaw lill-persuni bi ħtiġijiet bażiċi ta’ ħiliet huwa meħtieġ mhux biss biex il-Perkorsi ta’ Titjib tal-Ħiliet ikunu suċċess, iżda wkoll biex jgħinu fil-bini ta’ soċjetajiet aktar reżiljenti u inklużivi. Id-diskussjoni online se ssir f’din il-paġna fis-16/17 ta’ Settembru bejn l-10:00 u s-16:00 CET u se tkun immoderata mill-Koordinaturi Tematiċi ta’ EPALE tal-EBSN

More