Deep Dives

• Project
• Materials
• Method
• About us
• Contact

• Project
• Materials
• Method
• About us
• Contact

• Project
• Materials
• Method
• About us
• Contact

METHOD

We use the criteria as guidelines to develop and improve materials for the project. We take an analytic approach where we assign weights to the evaluation criteria. Numerical values next to criteria express weights. If an idea for teaching materials seems promising based on evaluation using these criteria, we attempt to develop and disseminate the materials related to the idea.

For any particular set of materials, we evaluate them on the basis of the criteria outlines in the table above. Each criteria has a weight, and the combined weights is 100. For each criteria, materials can get a grade going from 1-3. The maximum score is therefore 300. Here we break down the criteria in component parts, so as to explain what we mean by them, and why they are important.Uptake
Three things are important to teachers and distributors for uptake of materials:
- Curriculum alignment
- Ease of use
- Pedagogical quality.Curriculum alignment
Teachers will not use teaching materials in a class if the content is misaligned with the curriculum for that class. In this section we outline the curricular requirements that are relevant to teaching materials in Norway. These include teaching goals, core elements, interdisciplinary topics, and exams. Teaching goals (læreplanmål) are used as a guide for what teachers focus on, but are also essential for grading the students. In addition to the teaching outcomes and core elements, the current course requirements say that three interdisciplinary topics should be integrated into the class during the year. If materials prepare students for exams in a way that is likely to make them do well, it is easier to use the materials. For the final oral exams, the student is given a general theme, and will have to create their own research question, usually presented through a ten-minute presentation. In the following twenty minutes, the teacher asks questions from the whole syllabus, and an examiner may ask follow-up questions. This exercise lasts for about 20-minutes, after which the examiner and teacher sets a grade on the basis of the teaching goals. Themes for exam questions are given locally (by the school), and will thus vary, but are always connected to teaching goals. In religion and ethics, ethics is usually always a given theme every year.Easy to use
Teachers will not use materials if they are not easy to use. To be easy to use, materials must be easy to understand, quick and easy to prepare (including reading and understanding the materials), and easy to actually implement and adapt in the classroom. The materials need to be clearly and concisely formulated and presented. Ideally, the lesson should be summarised in a single sentence at the very beginning. The concrete activities should be explained concretely so that the teacher can follow the lesson like a recipe. The lesson itself should be easy to prepare for. The teacher should not have to spend a lot of time reading to understand the topics for the lesson. Teaching materials like quizzes and presentations should be made easily available. If the teacher is excited about the topic and wants to use it for further student assessment, additional resources should be easily available. The content itself needs to be well formulated and presented, and only contain essential information to increase ease of implementation. The actual implementation of the lesson in a class should not be too demanding. There can be some room for interpretation so that the teacher experiences a level of flexibility and can adapt the content to their needs, however, this should not be in defiance of fidelity.Pedagogical
Teachers care about whether their teaching materials are pedagogical. That is, whether they are likely to be engaging to students, and whether they are likely to make students learn effectively. Exactly what constitutes a pedagogical lesson is disputed, so we break down this criterion to variables that give reason to think that teaching the lessons is going to be engaging and effective.Most teachers judge materials by the content itself according to their personal conception of good pedagogy. However, there are some patterns and principles of pedagogical practice that most teachers recognize and approve. Also, teachers talk to each other, and are more likely to use materials that have been successful in other classes. In other words, teachers are more likely to use materials that have traction. In addition, at least some teachers like original or novel materials that are dissimilar from what is typically used. Many teachers would like to switch things up more than they do.Teachers are much more likely to want to use materials that have traction. Traction can come in many forms, including: (i) Empirical validity. The materials have been tested out in classes with good results. (ii) Testimony. People who know about the materials vouch for them. This can be people who have helped us to develop them, or teachers who have tried them out. The testimony of prestigious people count for more than that of less well-known people.If some lesson or set of materials has traction, or is likely to garner traction, that is an important part of them being considered pedagogical by teachers. We outline the traction for each particular set of teaching materials in the document outlining the materials.Teachers do their best to keep up with the latest developments in pedagogical research. Teaching materials are good ways to implement popular pedagogical principles and concepts in practice. Therefore, if teaching materials hold a high-pedagogical standard, in the sense of instantiating one or more influential pedagogical concepts, it is more likely that teachers will want to use the materials. Some influential pedagogical concepts are those of active learning, deliberate practice, flipped classroom and formative assessment. Active learning involves activities in which students physically move about, express themselves or practise relevant skills.
Deliberate practice involves activities in which students practise with explicit goals in mind. Ideally, students practise in several iterations, with feedback after each attempt. Formative assessment involves activities where students get feedback on their performance in the process of learning. Assessments can be given by the teacher, or by other students. Flipped classrooms typically involve giving students learning resources before a lesson, and actively working with materials during the lesson.The following concepts have a demonstrated effect in serious academic metastudies, and will be of interest to some teachers. Metacognition is about making students think about how they think while learning. Typical learning activities include reflection on beliefs and values. Collaborative learning involves working with others, often to solve problems that it is not so easy to solve alone. Concept mapping is a visual approach to learning where students make diagrams to express concepts and how they relate to each other. Gamification, or gamified learning seeks to implement games or game-like features in learning activities.Here are also some ideas on what makes for a good lesson. We have these ideas from readings and conversations with teachers. These don't fit neatly into any of the categories below, but are nonetheless good to have in mind when evaluating the pedagogical quality of teaching materials for lessons specifically. A good lesson opens with a thought piece that provokes reflection. A thought piece could be a video or a short text, or a brief introduction to a topic by the teacher, but should take no more than 5-10 minutes. Good lessons typically involve concrete assignments that the students are to work on for the duration of the session. The assignment should ideally be on the blackboard before students enter the classroom, and they should be able to understand immediately what they are supposed to do by the end of the session. Good assignments result in a concrete product or presentation that the students get feedback on.
Good lessons are characterised by variation. There is often some reading, or other intake of information, but there should never be more than 5-10 minutes before the students get to talk and be actively engaged. Activities where students get to move around and interact with each other are good. There should always be a little writing, and always a little discussion or peer-feedback. Practice assignments where students get immediate feedback is also a good way for students to learn quickly, as are quizzes (especially interactive one’s like Kahoot!’s).If materials express content, or involve activities that are dissimilar to what teachers typically do, they might want to use it for the sake of variety.Upside conditional on uptake
If teachers adopt materials and teach them, the effects might still not be positive, or they may even have a negative impact. The two variables that are most relevant to the outcomes of the materials conditional on uptake are fidelity and inspiration.Fidelity
There are two steps of communication where the ideas that the teaching materials are intended to express can fail. Firstly, if the materials themselves are vague, ambiguous or otherwise unclear, teachers can interpret the relevant ideas in a plethora of ways. Secondly, the materials specify the way in which the relevant ideas should be presented, which is relevant to the likelihood of the students coming away with an accurate understanding of the ideas. If the teacher or the students interpret the content expressing ideas in a way that is misleading or false, the effects can be very bad. It is generally much harder to change someone’s mind on a topic that they already have an opinion on, even though that opinion is false or based on a misunderstanding. Therefore, it is very important to preserve the fidelity of the ideas expressed in the teaching materials. A concrete way to do this is by ensuring that the materials are expressed in a clear language and with examples or scenarios relevant to students' lives. Materials can also focus on high-fidelity ways to present ideas. It is also possible to include direct prompts to students in the materials where experts on the relevant ideas present them in a precise manner.Inspiration
Good inspiring ideas are impactful, plausible, and actionable. All of these elements are distinct and necessary. Something else that is important is to ensure that the materials don’t end up alienating students to these ideas. This can occur by coming off as arrogant or misrepresenting ideas, people or organisations in ways that damage their ethos.The ideas should inspire actions of a sort that might have a large positive impact. The purpose of ethics is to guide action. A crucial assumption of this project is that students will be more likely to do good if they understand ethical concepts and theories. Some materials are more likely to inspire virtue or action than others, and this dimension should be taken into account. Therefore, ideas should be action-guiding in the sense of them being actionable. Whether an idea is actionable mostly depends on the form in which it is presented, the examples that are used, and whether the idea is very abstract, or concrete in a way that ties to real life considerations and choices. Ideas need to be plausible to students. That is, the ideas should not be too weird, or else students might feel alienated and not be inspired to positive action. It might be a good idea to avoid weird ideas that most students won’t accept anyways.