"The first PISA results were yet another wake-up call for Germany, following the already sobering TIMSS studies of the late 1990s," recalls Ingmar Hosenfeld, now a professor of educational psychology and the executive director of the Center for Empirical Educational Research (zepf). Shaken by the repeated poor performance of German students in international comparisons, Germany began to take a closer look and ask itself: What is the actual state of the younger generation’s skills and competencies? What can they really do?
These questions triggered a necessary shift away from defining what must be taught through the curriculum (input) and toward assessing what knowledge and competencies have taken root in students' minds (output). Numerous assessments in Germany, including standardized tests known as VERA, now measure learning outcomes in classrooms. These comparative assessments are also a key component of the Standing Conference of the Ministers of Education, Science and Cultural Affairs (KMK) overall strategy for educational monitoring.
"We were a bit naive when we developed VERA 25 years ago."
Ingmar Hosenfeld
Hosenfeld is one of the two “fathers of VERA.” Together with Andreas Helmke, who was a professor of developmental psychology at the University of Landau at the time, and the Rhineland-Palatinate Ministry of Education, Hosenfeld designed the learning assessments that examine student’s proficiency in German and mathematics. VERA was initially designed for fourth-grade elementary school classes. In 2003, the standardized tests were administered for the first time in Rhineland-Palatinate. Beginning in 2004, additional seven federal states joined the project. Since then, all federal states have introduced centralized learning assessments in third and eighth grades. "These two grade levels are important turning points in terms of the transition to secondary school and to the intermediate school-leaving certificate," Hosenfeld explains, describing this development.
The foundation for objectively evaluating – and thus measuring – student performance was established over 50 years ago at the former Landau University of Education, with the establishment of the Center for Empirical Educational Research (zepf).
It’s founder Karlheinz Ingenkamp had revealed in his research significant differences in how teachers evaluated student performance, justifying the call for objective assessment procedures in schools. Ingenkamp established empirical diagnostic methods in educational research and the training of prospective teachers – a novelty at the time.
"His decision to bring together educational and psychological research under one roof was innovative and a sign of pioneering spirit," affirms Ingmar Hosenfeld. At that time in Germany, educational research was primarily conducted from backgrounds in history or philosophy, and empirical approaches – and thus the methodical and systematic collection of data – were uncommon. "By establishing educational diagnostics and its empirical methodological foundation, Ingenkamp established key elements of objective assessment of student performance at zepf. In doing so, he helped create essential standards that are now taken for granted and form the basis for empirical assessments of learning progress."
If PISA exists, why does Germany need another comparative assessment system like VERA? "International comparisons and studies take a broader approach, allowing us to learn about student performance at the system level," explains Hosenfeld. However, since only a small sample is drawn from each country, and the studies are conducted only every three years, the results only provide snapshots. While these provide a clear picture at the level of large units, such as nations and federal states, they do not do so at the level of day-to-day classroom activities, such as schools and classes, where instructional development must occur. "With the comparative assessments, we’re pursuing an approach that specifically measures performance at the class level every year," Hosenfeld emphasizes. This approach provides schools with feedback at the classroom level, which they can utilize for quality assurance and instructional development.
VERA aims to diagnose the strengths and weaknesses of classes by assessing students’ learning levels at a specific point in time. This provides schools with data-driven insights and suggestions for instructional development. This is done in reference to the cross-state educational standards of the Conference of Ministers of Education (KMK), which distinguish five proficiency levels. Standardized tasks allow for statewide and nationwide comparisons of classes' performance levels relative to the school’s overall results, those of parallel classes, or the entire cohort of students in the state.
"More than comparability, we're interested in the competencies of individual students – that is, what each student can do in mathematics and German and which tasks they are likely to struggle with," says Hosenfeld, explaining VERA's true value for schools. The five proficiency levels underlying the individual educational standards represent a developmental sequence. This way, schools receive feedback on the proficiency level achieved by each student. "If I know that my student has a reasonable grasp of what is required at Level 3, then the additional requirements introduced at Level 4 are the next tasks to tackle, and those are the areas where we should continue working," says Hosenfeld.
How do schools handle this treasure trove of data? "We were a bit naive when we developed VERA 25 years ago," says Ingmar Hosenfeld. "We thought we'd report the results, and teachers and schools would know how to use them for quality assurance and instructional development." However, it has become evident that even those teachers within schools who deal with student data from VERA often lack the knowledge necessary to read and interpret it. "Ideally, appropriate training should take place as early as college," emphasizes Hosenfeld. In his view, issues of quality development still play a minor role in training prospective teachers.
"VERA should be a more useful tool for schools, and schools should have more control over it."
Ingmar Hosenfeld
Even for someone without empirical expertise, extracting insights from data is not easy. They must not only understand the reported data and contextualize what they know about the class beyond the test results, but also form hypotheses, explains education researcher Hosenfeld. For example, if my class generally performs at the same level as others in my school but performs differently in geometry, I should ask myself how much time I spent on the topic, when I covered it, and how I explained it. "From there, I can develop hypotheses as to why my class performed worse in geometry. Ideally, I could discuss these with my colleagues."
Hosenfeld believes that meaningful outcomes can emerge from the VERA data in schools if there is a trusting exchange among teachers and a school administration that understands the importance of data-driven processes for instructional development. This is demonstrated by the initial data from the research project, "Impulse für die Weiterentwicklung von Schule und Unterricht – WeSU" (english: Impulses for the Further Development of Schools and Instruction), in which Hosenfeld's team uses qualitative interviews to investigate the instructional development measures implemented by schools working with VERA or similar learning assessment results. "VERA is designed precisely to foster a culture of collaborative learning with room for mistakes, errors, and experiments – even if not all of them are sucessful," says Hosenfeld. "Ideally, our tool will encourage schools to discuss instructional development among faculty members and learn from one another," he hopes.
However, the education researcher remarks that many schools now administer the standardized tests relatively quietly and without complaint but view them as an annual burden rather than an opportunity.
Hosenfeld sees the reason for this wasted potential in the framework. "VERA should be a more useful tool for schools, and schools should have more control over it," Hosenfeld urges policymakers. Currently, schools are required to participate but have little influence over the content or timing. "Schools need to be empowered in many ways to work with the data," says the researcher. This starts with teachers’ time budgets and extends to the need for a fundamental understanding that data-driven decisions are effective.
"In my view, the states have failed to establish a proper and reasonable framework for quality development in schools overall." While school inspections exist, they are staffed primarily by generalists who can comment on teaching in general but cannot provide teachers with detailed, subject-specific pedagogical guidance. Furthermore, classroom visits are conducted randomly and are usually brief. "We need a much more robust cycle of instructional development driven and managed from within the school," says Hosenfeld. While an outside perspective is helpful, internal school processes ultimately make the difference.
"As a society, we're too closefisted with approaches as team teaching or classroom observation and don't view them as strategic investments in the system's capacity for development."
Ingmar Hosenfeld
Hosenfeld also sees a problem in the lack of coherence among the various stakeholders in education who address school and instructional quality: universities, teacher training colleges, schools, state institutes, the ministry, school supervision, and school inspections. "Instead of working in unison, these are overlapping and largely independent initiatives." Breaking down or harmonizing these structural rigidities is a monumental task. However, Hosenfeld believes that applying strict, scientific, evidence-based criteria and making data-driven decisions regarding the management of the education system could identify clear patterns that could be useful in everyday school life.
Hosenfeld sees another problem in the German education system: it only offers approaches such as team teaching or classroom observation in exceptional cases. "As a society, we're too closefisted with such approaches and don't view them as strategic investments in the system's capacity for development." This limits opportunities for colleagues to collaborate and improve teaching practices.
"But I’m also convinced that it was a good idea over 20 years ago to implement standardized tests on a large scale," affirms Ingmar Hosenfeld. After all, these standardized tests have helped schools gain a much better understanding of educational standards and the types of tasks needed to convey them. For this reason, the empirical education researcher made it a priority early on to involve schools and give them options, such as selecting tasks or determining when to administer the tests. The idea of increasing flexibility is gaining ground, but it poses major challenges for the VERA team.
"Our goal is to use these options to motivate schools and integrate the standardized tests sensibly into their quality assurance efforts, which are always taking place in some form." To achieve this, opportunities for customization must be created for individual schools. Furthermore, Hosenfeld sees a growing need to support schools in quality development and assurance. Closer coordination between the state institutes and the standardized tests would be beneficial to this end. "The teacher training programs offered by the state institutes focus heavily on instructional content. We need more programs on how to develop and ensure instructional quality in schools," says Hosenfeld.
He therefore not only advocates embedding empirical research and quality development in the university education of prospective teachers, but also aligning continuing education programs for teachers and school administrators accordingly.
"School administrators are like conductors in the symphony of school development"; therefore, they must have a clear understanding of quality orientation and development.
The challenge is finding the time to carry out this task. In German schools, administrators fulfil three roles: they are teachers, supervisors, and personnel developers. This is not the case in other education systems. "In English-speaking countries, you no longer teach once you’re in a leadership position." There is a consensus among empirical education researchers. "Policymakers must ensure that the levers in the complex education system are adjusted in a coordinated manner," says Hosenfeld. "If only individual cogs are turned, the effort will fizzle out."
Zimmer-Müller, M. & Hosenfeld, I. (Hrsg.). (2013). Zehn Jahre Vergleichsarbeiten: Eine Zwischenbilanz aus verschiedenen Perspektiven. Empirische Pädagogik, 27 (4). (Themenheft). Landau: VEP.
Pietsch, M. & Hosenfeld, I. (Hrsg.). (2017). Inspektionsbasierte Schul- und Unterrichtsentwicklung, 31 (2). (Themenheft). Landau: VEP.
Wagner, I., Hosenfeld, I. & Zimmer-Müller, M. (2019). Vergleichende Analyse der Zusammenhänge von Akzeptanz, Auseinandersetzung mit und Nutzung von Ergebnissen bei Vergleichsarbeiten und Schulinspektionen. Psychologie in Erziehung und Unterricht. DOI: 10.2378/peu2019.art22d
