By K. Puge and B. Wahlgren, Sustainable Culture for Change, School of Education, Aarhus University.

Abstract

In this article, we describe how we have implemented formative evaluation at seven vocational schools. We describe how it has strengthened the schools' change capacity. It has developed the leaders' ability to reflect on their experiences and to apply what they learn in an ongoing improvement process.

We argue that formative evaluation – in the form we have developed – can be an important element in sustainable school improvement.

This article is a rewritten and translated version of a Danish-language research article from the Sustainable Culture for Change project.

Research-practice collaboration for building change capacity 

The article describes the experiences from a project where an evaluation concept is implemented at seven vocational education and training schools in four Central European countries with the purpose of developing the schools' change capacity. This allows ongoing improvement in relation to the school's goals to becomes part of the daily practice at the schools.

Schools' change capacity is a central element in school improvement because schools, vocational education and training schools in particular, increasingly face demands for change in relation to the requirements placed on the school (Bell, 2008). This happens when businesses place new competency demands on students, when developments in technology leads to a change in work processes and collaboration forms in companies, when schools' collaborations with companies and local communities expand, and when political initiatives change the framework for school operations.

The school must be able to absorb these demands and be able to change in relation to them. This includes, among other things, that the school's employees must be able to reflect on the change process and be aware of why the changes are necessary and how they can be implemented. Seen from this perspective, we see change capacity as an integrated element in sustainable school improvement.

Learning is a central element in building change capacity (Bowen et al., 2006; Ertsås & Irgens, 2023; Karagiorgi et al., 2018; Lai, 2015; Stoll, 2009, 2020). Teachers' and leaders' learning in relation to the social, organizational, political, and cultural conditions for change is a prerequisite for the development of the school's capacity for change (Lai, 2015). Bowen et al. (2006) show that focusing on organizational learning strengthens the sustainability of schools' change initiatives. Such an approach means that the school achieves a learning culture where employees identify challenges and test solutions, which are evaluated, thereby continuously creating new knowledge (Bowen et al., 2006, p. 99). Ertsås & Irgens (2023) show that building change capacity involves the school's development of knowledge. It requires joint reflection and systematic project work if the school's employees are to learn from their experiences. According to Fullan, there is a need to work with 'reculturing', meaning approaches that allow teachers to challenge the beliefs and habits they act upon to succeed with educational change (2016, p. 23). Karagiorgi et al. (2018) emphasize that schools must learn from trials and reflection on experiences with these trials.

Summing up, change capacity must be developed to ensure sustainable school improvement. This requires systematic learning processes in which school leaders and staff continuously experiment and learn from their school improvement efforts. 

The cited research results point to the fact that strengthening schools' change capacity is a learning process that includes reflection and evaluation. This presupposes that the schools are able to initiate, implement, and evaluate actions that contribute to goal achievement in relation to the changed conditions. The institution must be able to reflect on the change process if it is to be carried out with quality. At the same time, the school must have an organization where leadership and staff participate in this change process. To the extent that leadership and staff are able to reflect on the changes - and to work together on their implementation - the school's change capacity is strengthened.

In the article, we argue that school improvement work, carried out according to the concept we have developed and tested in collaboration with the schools, will develop schools' change capacity.

Formative evaluation in change capacity building

To strengthen the schools' change capacity, we have implemented formative evaluation as part of the school's daily practice. We understand formative evaluation as a process where the school continuously reflects on its own activities with a view to systematically collecting experiences for qualifying the school's practice. Specifically, we have worked with three interventions as a concept for formative evaluation: clarification of goals, staff involvement, and systematic collection of experiences.

Clarification of goals

A prerequisite for being able to carry out a formative evaluation is clarity about the goal of the improvement project. The first intervention in the concept relates to the understanding of school improvement that inform the actions of both leadership and teachers. As a starting point, they must be aware of the need for change, they must be motivated to improve the work throughout the institution, and they must understand improvement of their work as the core of their activities (Joroff et al., 2003).

The first part of our formative evaluation concept therefore concerns activities qualifying the leadership so that it becomes able to locate the needs for school improvement. They must have an idea of which processes are expected to lead to an improvement of the school's practice. We work from the assumption that the more the leadership is aware of the connection between their project goals and activities that promote goal achievement, the greater their capacity to initiate and implement ongoing changes (Kenney et al., 2019; Schein, 2010).

Therefore, school leaders must formulate a thorough plan for their school improvement project. The plan contains the goals for the various activities and considerations on how they can be anchored in the school's organization. The purpose is to ensure that the schools become aware of what they mean by quality improvement and how they themselves perceive the connection between the initiated activities and their project goals. The school must think in terms of implementing changes in relation to daily operations. At this stage, the emphasis is on the school leaderships seeing their planned project activities as part of an improvement process.

Staff involvement

The second intervention within the formative evaluation concept concerns the involvement of the school staff in the school improvement processes. The leadership's task is to give teachers greater control over the processes that should lead to continuous improvement. Teachers can advantageously be involved in planning, implementation, and evaluation of project activities. Both because teachers have the necessary insight into how changes can affect the work, and because teachers are motivated when they reap the fruits of their own school improvement efforts (Joroff et al., 2003, pp. 294, 308).

The leadership team must ensure that staff know and share the vision for the work before they can constructively participate in the school improvement processes (Seo et al., 2004). This ensures that the employees can contribute to the activities that are initiated, improving the school in the direction of goal achievement.

It strengthens schools' change capacity when school improvement projects support the staff in learning during the process (Kools et al., 2020). Through the process, the focus shifts from project operations - for example, purchasing equipment or developing teaching materials - to learning, where the experiences gained through the activities in the project can be converted into new knowledge and more qualified actions.

To strengthen the staff’s engagement in and attitude towards change, we conducted a competence development of the teacher group. The competence development does not focus on the teachers' professional or didactical competence, but on their competence to participate in school improvement and their ability to reflect on it. The purpose is to ensure a broad understanding at the school that it - and its teachers - are involved in school improvement work, and that it is important that they actively participate in the activities and in the collection of experiences. At this stage, the emphasis is on making the staff, primarily the teachers, aware of and engaged in the change processes.

Systematic collection of experiences

The third intervention in our concept for formative evaluation concerns the systematic collection of experiences as part of the school improvement process. The collection of experiences must take place continuously and as part of the school's ordinary operations.

The essential element in this context is a continuous evaluation of the activities that are carried out (Wahlgren & Puge, 2022; Stufflebeam & Coryn, 2014). The evaluation of activities and processes makes visible that one learns something, that one improves. The evaluation provides an insight into the change process, which is central to building change capacity (Fullan, 2020; Gillon, 2018; Palmer et al., 2017).

The evaluation concept's intervention here concerns an ongoing collection of experiences at the individual schools. The schools are continuously collecting experiences and systematically reflecting on the project's progression.

Specifically, this is done by the schools writing reflection papers at regular intervals. The content of these notes is collected knowledge about what has happened in the school improvement work, and what they have learned from what has happened. School leaders are given a tool for collecting and reflecting on the experiences gained from the past two months of project work. They are supported in reflecting systematically and in writing on the development process when they have to fill out these reflection papers.

The school's assessment clarifies the school improvement processes for the school's teachers and leaders. They become wiser about what drives school improvement and how it can be planned and implemented in their school context. Through the systematic reflection on the changes, their cause and significance, the leaders' change capacity is developed.

When leaders are able to reflect on their change processes, they are able to know the need for change, and how it may be implemented successfully. This motivates change (Fullan, 2020).

Through systematic collection of experiences, the school is held accountable for ensuring that continuous school improvement take place. The school learns to change by engaging in change processes and by reflecting on them.

Collaboration between researchers and school leaders

The focal point for the research group's initiatives in relation to each of the involved schools is a continuous and systematic dialogue. Continuous in the sense that we follow the schools' improvement efforts on an ongoing basis through regular contact. Systematic in the sense that the process and collection of experiences takes place on the basis of tools developed by the research group. Dialogue in the sense that we provide the schools with systematic feedback on the results they present throughout the collaboration in conversations with the school leaders.

Effects of the concept for building change capacity

To document the effects of the concept for formative evaluation in school improvement projects, we have collected the school leaders' perception of the work through a survey, and we have interviewed the school leaders about their perception of the activities within the initiative.

The table below shows the leaders' responses to four variables that relate to school improvement and change capacity. Here, the school leaders indicate that the collaboration with the research team, which has focused on implementing the concept for formative evaluation as described above, has had a substantial impact on the school's work. It has contributed to experiences being implemented continuously in the school's daily work, and it has contributed to formative evaluation becoming part of the school’s daily practice. The collaboration has had the greatest influence on the school's ability 'to reflect on what has been learned' and on the leaders’ awareness 'that school improvement is an ongoing activity'. Based on the quantitative data, we conclude that school leaders have built change capacity in the sense that they have become better at harvesting and using experiences as part of ongoing school improvement.

This work "Table of impact " is licensed under Creative Commons Attribution 4.0 International

Based on the interviews with the school leaders, we can further understand the effect of the three interventions within the concept for formative evaluation. 

The collaboration on the schools' quality improvement plans makes it clearer to the school that they have a plan and a clear goal to work towards. A leader describes the importance of having made the quality improvement plan: "It was very important for our success that we started the project with a clear project plan and project management. We succeeded because we were able to inspire the project team with our ideas and convince all the stakeholders about our honest desire to make a change".

The competence development day was an intervention that both contributed to strengthening the collaboration between the school staff and leaders, and to enhancing the involvement of teachers in the projects. A project group member from one of the schools describes the day: "I liked that the day were done in a very active form. Everybody was involved. And everybody was forced to communicate to be involved and sometimes it was very positive that different opinions were negotiated. And [it was] very positive that the outcome at the end of the day was a kind of consent. That they had to find a consent in the groups".

Completing the reflection papers makes the school leader teams aware of the change processes and the effect of the different activities. A school leader says: "After a couple of months, we came to the conclusion that the evaluation gives us the opportunity to measure and reflect on the progress of activities and overall improvement of the school's appearance to the public. In the years to come: We know for sure that we will constantly keep doing evaluations (…)". The school's management has acquired skills to evaluate projects and experiences a value in conducting the evaluations.

When schools have built change capacity, they are able to keep up with development when it is deemed to be positive and necessary. The change-ready school has the capacity to assess change requirements, plan efficient activities, and implement them. This does not mean that the school should always be changing. It means that the school is able to meet necessary demands for change.

Conclusion

In this article, we have described how an external effort can implement formative evaluation at schools, which strengthens their change capacity. The formative evaluation concept consists of three interconnected interventions: goal clarification, staff involvement, and collection of experiences. We have described a concept that has been developed through collaboration between researchers and school leadership teams at seven vocational schools.

In the article, we have argued that the concept has strengthened both the internal collaboration at the schools, as well as the school's leaderships and teachers' ability to engage in reflection and evaluation practices in connection to school improvement projects.

On this basis, we conclude that the intervention has strengthened the schools’ capacity to respond to new and evolving demands – the concept for formative evaluation has contributed to building the schools’ change capacity.

References

Bell, D. (2008). Organisational planning and performance integration in a TAFE context. https://www.proquest.com/reports/organisational-planning-performance-in…

Bowen, G. L., Rose, R. A., & Ware, W. B. (2006). The Reliability and Validity of the School Success Profile Learning Organization Measure. Evaluation and program planning, 29(1), 97-104. https://doi.org/10.1016/j.evalprogplan.2005.08.005 (Evaluation and Program Planning)

Ertsås, T. I., & Irgens, E. J. (2023). Developing organizational knowledge in schools: The role of theory and theorizing in collective capacity building. Journal of Educational Change, 24(1), 25-48. https://doi.org/10.1007/s10833-021-09433-3 

Fullan, M. (2002). Principals as leaders in a culture of change. Educational Leadership. http://www.michaelfullan.ca/Articles_02/03_02.htm 

Fullan, M. (2006). The future of educational change: system thinkers in action. Journal of Educational Change, 7(3), 113-122. https://doi.org/10.1007/s10833-006-9003-9 

Fullan, M. (2016). The NEW meaning of educational change (Fifth edition. ed.). Teachers College Press. 

Fullan, M. (2020). Leading in a Culture of Change (2 ed.). John Wiley & Sons, Incorporated. 

Fullan, M., & Pinchot, M. (2018). The Fast Track to Sustainable Turnaround.(leadership strategies to transform a low-performing elementary school in California). Educational Leadership, 75(6), 48. 

Gillon, A. C. (2018). The Nature of Contemporary Organization Development. Taylor and Francis. https://doi.org/10.4324/9781351106818 

Joroff, M. L., Porter, W. L., Feinberg, B., & Kukla, C. (2003). The agile workplace. Journal of corporate real estate, 5(4), 293-311. https://doi.org/10.1108/14630010310812145 

Karagiorgi, Y., Afantiti-Lamprianou, T., Alexandrou-Leonidou, V., Karamanou, M., & Symeou, L. (2018). ‘Out of the Box’ leadership: action research towards school improvement. Educational Action Research, 26(2), 239-257. https://doi.org/10.1080/09650792.2017.1310052 

Kenney, M., Jordan, K., & Curnow, M. (2019). Five steps to achieve sustainable culture and behaviour change. Peopletoo. https://peopletoo.co.uk/five-steps-to-achieve-true-culture-change-and-b…

Koh, G. A., & Askell‐Williams, H. (2021). Sustainable school‐improvement in complex adaptive systems: A scoping review. Review of education (Oxford), 9(1), 281-314. https://doi.org/10.1002/rev3.3246 

Kools, M., & Stoll, L. (2016). What Makes a School a Learning Organisation? OECD Publishing. 

Kools, M., Stroll, L., George, B., Steijn, B., Gouëdard, P., & Bekkers, V. (2020). The school as a learning organisation: The concept and its measurement. European journal of education, 55(1), 24-42. https://doi.org/10.1111/ejed.12383 

Lai, E. (2015). Enacting principal leadership: exploiting situated possibilities to build school capacity for change. Research Papers in Education, 30(1), 70-94. https://doi.org/10.1080/02671522.2014.880939 

Palmer, I., Dunford, R., & Buchanan, D. A. (2017). Managing organizational change : a multiple perspectives approach (3. edition. ed.). McGraw-Hill Education. 

Schein, E. H. (2010). Organizational Culture and Leadership (4. Aufl. ed., Vol. 2). Jossey-Bass. 

Seo, M. G., Putnam, L. L., & Bartunek, J. (2004). Dualities and Tensions of Planned Organizational Change. In M. S. Poole, A. H. Van de Ven, & M. E. Holmes (Eds.), Handbook of Organizational Change and Innovation. Oxford University Press, Incorporated. 

Stoll, L. (2009). Capacity building for school improvement or creating capacity for learning? A changing landscape. Journal of Educational Change, 10(2-3), 115-127. https://doi.org/10.1007/s10833-009-9104-3 

Stoll, L. (2020). Creating capacity for learning: Are we there yet? Journal of Educational Change, 21(3), 421-430. https://doi.org/10.1007/s10833-020-09394-z 

Stufflebeam, D. L., & Coryn, C. L. S. (2014). Evaluation theory, models, and applications (2nd ed., Vol. 50). Wiley.

Wahlgren, B., & Puge, K. (2022). The process of implementing an internal formative evaluation. CEPRA-striben(31), 52-61. https://doi.org/10.17896/UCN.cepra.n31.507