James Holt lives in Manchester, UK and is Senior Lecturer in Religious Education* at the University of Chester. This involves training teachers to teach Religious Education in schools, as well as aspects of education for teachers training to teach in a secondary (11-18) school. James is also Chair of Examiners for GCSE Religious Studies for a major awarding organisation in the UK. He is the author of Religious Education in the Secondary School. An introduction to teaching, learning and the World Religions (Routledge, 2015). He is a trustee of the RE Council of England and Wales, his PhD (Liverpool, 2011) constructed a Mormon theology of religions and explored the resultant implications for inter-faith dialogue. James currently serves as Bishop and early morning seminary teacher in his home ward. His current research interests focus around a Christ-centered pedagogy of teaching, assessment, and all aspects of inter-faith engagement.
*Religious Education means something different in the UK than in the USA. All school children in England should have an RE class weekly that explores the beliefs and teachings of the different world religions. In state schools this is not confessional and is to help children understand the lived reality of those who have religious and non-religious beliefs. This will hopefully lead to more informed and respectful members of communities and society.
Seminary Assessments and Americacentrism
A couple of things coalesced this week to make me return to some thoughts I had a couple of years ago that haven’t really gone away. The first ‘thing’ was as I was scrolling through Facebook I came across a post from the Maxwell Institute highlighting what one of the themes being explored in their Summer Seminar was going to be. Norma Calabrese’s question seemed to be (or so the infographic said) ‘Many European LDS Church members detect a strong sense of Americanism in their Church experiences. How might relinquishing this America-centrism enrich the Church?’
At this point I need to confess that I am an Americaphile- I love most things American. I would much rather watch the Dallas Cowboys play than a local football team (though I still love them), my favourite TV series tend to be American, and it has always been easier to situate my politics in America rather than the very centrist and homogeneous political situation of the UK (up until recently anyway). My first experiences of the Church were infused with America and I loved it- missionaries who gave me fireball gobstoppers, Tootsie Rolls, root beer and played American Football with me.
Occasionally comments were made that made me bristle- for example the companion on my mission who suggested I might be apostate because I didn’t want to go to ‘God’s University’- i.e. BYU. Or when I served on a Stake Presidency a member visiting from the US saying that it must be great to live in the Mission field and have all these opportunities. Translation: you wouldn’t have that calling if you lived elsewhere! As for living in the mission field, I’m not sure what that means- we have Temples and everything in the UK? There have also been huge strides too to globalise the Church- a couple of years ago the General board of the YW reflected a worldwide make up; a couple of years before that though less well known was the inclusion of a few members from outside Utah and around the world on the Curriculum Writing Committee for the Gordon B. Hinckley manual- of which I was grateful to be one. So any perceived Americacentrism was easily adapted and amended by relying on the encouragement of Elder L. Tom Perry to adapt the handbook to local circumstances.
The second event that contributed to this train of thought was an email to do with one of my callings as an early morning seminary teacher. The email contained some training for me and a link to the purpose/objective of Seminaries and Institutes. This is:
Our purpose is to help youth and young adults understand and rely on the teachings and Atonement of Jesus Christ, qualify for the blessings of the temple, and prepare themselves, their families, and others for eternal life with their Father in Heaven.
This triggered a memory of a concern I have with some changes that were made to seminary a couple of years ago. This is not just sounding off in a blog post- I have raised them with S&I people in the UK as I felt that it was right thing to do. Who am I to have such concerns? I am a professional educator, I lecture in Religious Education, I train teachers and I am responsible for Religious Studies exams for a large portion of English school students at the age of 16. To some extent, when talking about the principles of education and assessment I can be considered to have some knowledge.
What were my concerns? Before I get to those, I want to say that the requirement to read the book of scripture being studied is great. There are lots of things that I love about seminary, and my own experiences as a student and as a teacher are very positive. The concern I have is about the test. It is a reflection of a particular (American) approach to teaching, learning and assessment. For me, it seems to be a very blunt instrument that is not measuring what it is supposed to measure. An assessment is supposed to be linked to the objectives/ purposes of the course. The multiple choice and writing framed question does not seem to measure in any way the stated purpose of S&I. The manuals do not break down any further learning outcomes or objectives, and so in analysing the test and the course it assesses I am only left with the stated purpose.
A couple of sample questions include:
In the New Testament, __________ fulfilled Malachi’s prophecy that a messenger would be sent to prepare the way before the Lord.
- John the Baptist
- Mary, the mother of Jesus
Section 2 is an ‘extended’ answer question
- The role of Jesus Christ in the Plan of Salvation. Include reasons why Jesus Christ had to be the one to make the Atonement. (You will receive one point for including this item.)
- The need for and the results of the Atonement of Jesus Christ. Include the significance of the Atonement in your everyday life. (You will receive one point for including this item.)
- A miracle, parable, or truth from the New Testament that helped strengthen your belief in Jesus Christ. Include your personal thoughts on why Jesus Christ is important in your life. (You will receive one point for including this item.)
- Something you have done this school year to draw closer to Jesus Christ. Include something you can do to continue to draw closer to Jesus Christ. (You will receive one point for including this item.)
This is slightly better but still a point marking system that can be answered with a check box approach.
The fact that it does not assess what it should is just one concern- and you may say that such an objective is impossible and perhaps inappropriate to assess. So I will move onto other concerns. I recognise that assessment is central to teaching and learning, and just as teachers should be clear about the purpose of why they are teaching they should similarly be clear about the purposes of assessment. I’m not sure there is much purpose behind the assessment apart from checking knowledge. The Teaching and Learning Research programme have summarised the main purposes of assessment into three broad categories:
- The use of assessment to help build pupils’ understanding, within day-to-day lessons.
- The use of assessment to provide information on pupils’ achievements to those on the outside of the pupil teacher relationship: to parents (on the basis of in-class judgments by teachers, and test and examination results), and to further and higher education institutions and employers (through test and examination results).
- The use of assessment data to hold individuals and institutions to account, including through the publication of results which encourage outsiders to make a judgment on the quality of those being held to account (Mansell, James, & the Assessment Reform Group, 2009, p. 8).
Although a secular approach to learning, elements of these principles seem to underpin the learning in seminary. It could be argued that these categories decrease in day to day importance, but the reality is that seminary while the first is pre-eminent, the introduction of this test makes number 3 central to the programme. If this becomes a major element of the course then it could be argued that we are preparing our youth for is a recitation of facts. The way the test is structured it seems that factual knowledge has become the purpose and aim of the test, and therefore seminary completion, rather than the deepening of knowledge and understanding of the Restored Gospel.
To enable assessment to take place it is important to note that there are two main types: formative and summative. Summative assessment takes place at the end of a piece of learning and is often the terminal or final assessment of learning. These tests would be examples of a summative assessment. Formative assessment takes place throughout the learning process, it helps students and teachers understand where they are and how they can make progress. Formative assessment is on-going, as such while these tests may be seen to be summative. They can become formative if the learning measured in the test is used to inform future learning and teaching, but this is not automatic or necessarily suggested. For learning to be effective the major proportion of assessment undertaken should be formative; indeed John Rudge has argued that “It is only formative assessment which benefits the pupils directly in their learning” (2000, p. 109). As an example, it is impossible for a teacher to be able to plan effectively for a lesson if they do not know the learning and progress which has already been made by pupils. Without this knowledge it is possible for the lesson to be inappropriately pitched and no progress to be made. I have no doubt that our teachers are using formative assessment where a pupil’s prior knowledge, experiences and abilities are used as a basis on which to build future learning. Indeed, Seminary teachers are generally very good and adapting lessons to ensure all the students are challenged and the aims of seminary are fulfilled. It could be argued that the scriptures suggest this type of assessment where students learn ‘line upon line’ (2 Nephi 28: 30).
I completely agree that assessment is important in seminary, but not through this blunt instrument. The underlying principles of assessment should underpin all the planning and teaching that takes place; the Assessment reform Group suggested ten such principles, Assessment for Learning:
- is part of effective planning
- focuses on how students learn
- is central to classroom practice
- is a key professional skill
- is sensitive and constructive
- fosters motivation
- promotes understanding of goals and criteria
- helps learners know how to improve
- develops the capacity for self-assessment
- recognises all educational achievement.
Although these, again, are important in secular education I think that all of these are important in the assessment of student learning in seminary. If I just take a selected one:
Is sensitive and constructive/ recognise all educational achievement
The feedback that the student receives in this test will generally focus on their pass mark. One Seminary teacher reports a student who got 50%. For this young man who is a convert of a year and has dyslexia this is an amazing achievement. However, all he knows is that if we go by his mark is that he failed. This is not the case he has done extremely well. If I am honest all he has done is not do very well in a factual recall test (i.e. a pub quiz). His learning curve has been substantial and however much praise is given the test is neither sensitive nor constructive. S&I do discuss how to meet the needs of all students, but it is still a pass/fail approach that has been adopted.
If we need a test, consider a test that enables students to utilise their skills in using scriptures or scriptural passages to show their understanding of the principles they have learnt. This would be much more valuable. It seems as though the type of tests administered in the USA are being implemented here, and they are of debatable educational value. They can tell you how much a person knows but not how much they understand. Elder Bednar has consistently taught the difference between knowing and understanding, and this test seems to focus almost completely on knowledge.
In my ‘day job’ I teach a subject that explores attitudes, values and beliefs- and the debate about how to assess it has been ongoing for the last thirty years. I have argued “It is difficult to see how a percentage in a factual recall exam could be linked to progress in anything except factual knowledge. Understanding the central concepts in individual religions should be at the core of good RE. It is insufficient for pupils to have knowledge of religions that is only good for a pub quiz, the central concepts should be explored and from there the impact of them on the lives of individuals.”
There are many things that can be done to ensure that learning is assessed effectively and I feel that this test does not do it. Consider this satirised version of the Sermon on the Mount:
Then Jesus took his disciples up the mountain and gathered them around. He taught them saying:
Blessed are the poor in spirit for theirs is the kingdom of God.
Blessed are the meek.
Blessed are they that mourn.
Blessed are the merciful.
Blessed are they who thirst for justice.
Blessed are you when you are persecuted.
Blessed are you when you suffer.
Be glad and rejoice for your reward is great in Heaven.
And Simon Peter said, ‘wilt we be having a test on this?’
And Phillip said, ‘I don’t have any paper.’
And Bartholemew said, ‘Does it matter about my spelling?’
And Mark said, ‘Do we have to hand this in?’
And John said, ‘The other disciples didn’t have to learn this.’
And Matthew said, ‘Can I go to the toilet?’
Then one of the Pharisees who was present asked to see Jesus’ lesson plan and enquired of Jesus, ‘Where are your learning and assessment objectives?’
Another asked, ‘What range of teaching strategies did you draw from, and do you have differentiated provision?’
A third Pharisee asked to see a cross-section of work.
And Jesus wept (source unknown).
I would be loathe to tell the student who ‘failed’ that he failed because in every area he has succeeded and achieved really well according to his context. This is in contrast to another young man in the same class who did not really engage with Seminary but has 15 years of a Church upbringing to draw on to make sure he passes the test. Surely application is far more important.
One person suggested to me that the purpose of the assessment for the assessment is to get (some) teachers away from having lessons on folk doctrine (“The Seven Classes of Angels”) or faith-promoting rumours (“An Hour-Long Version of the Time the Three Nephites Changed My Tyre”) and so on. I hope this isn’t the purpose. My concern with this is that it is using the assessment of students to respond to shortcomings in teaching. Teacher education is the tool much better suited to that- improving teachers may result but by testing students it seems unfair to the students.
Exploring the reviews of the implementation of the tests online the suggestion is that they have worked because students interest and participation in the classes has increased. Again, this should not be a purpose of assessment- it is not a tool for classroom management. If students are not engaged and participating in class then an exploration of pedagogy and teaching and learning methods is needed rather than a terminal assessment.
If you haven’t noticed I am not world’s biggest fan of terminal written assessments, but if they are to be kept, how could the test be improved? At the moment it is learn more facts- and don’t even get me started on multiple choice questions. There are ways to structure the questions that might help students know how to make the next step.
If I take the theme of the questions above In the New Testament, John the Baptist fulfilled Malachi’s prophecy that a messenger would be sent to prepare the way before the Lord. Evaluate John the Baptist’s role as a messenger. You may want to consider the prophecy’s importance for people at the time of Jesus, and also why it might be important for members of the Church today.
Or maybe: There are many different prophecies about the Messiah in the Old Testament, such as the great King, or the sufferings servant. Evaluate the extent to which Jesus was the Messiah for whom the Jews at the time were waiting.
These are difficult questions, but helps students synthesise knowledge and draw together their learning. If these are used then we could design frames for support, and indeed, could break down the questions and allow students to use their scriptures and class notes. In the exam system within which I work (designed for 16 year olds) the top level descriptor in marking for such a question might be:
Critically deconstructs religious information/issues, leading to coherent and logical chains of reasoning that consider different viewpoints. These are underpinned by a sustained, accurate and thorough understanding of religion and belief. Connections are made among the full range of elements in the question.
Constructs coherent and reasoned judgements of the full range of elements in the question. Judgements are fully supported by the comprehensive appraisal of evidence, leading to a fully justified conclusion.
In the system used in parts of the USA (released time) the training of teachers to function in such an environment and assess in such a way is possible because they are employed and paid to teach and assess. Throughout the rest of the world, with a large number of volunteer teachers this may be inappropriate and would certainly be difficult to standardise. However, when I consider my students I think they may have done enough when they get up at 545 every morning to attend Seminary, learn the scripture mastery passages, read the scriptures the formative assessment through questioning in the lessons is far more valuable a tool in measuring progress.
So returning to the original inspiration for this post, using an American system to assess the learning of students all over the world seems to accept the familiar as the default position. Looking beyond our own context would certainly enrich the experience of members everywhere. It might be that we return to the familiar, but we will know the reasons that we do so and will be stronger and more secure in the approach being suggested. On the other hand, we might recognise that this system works in the USA but that as a global Church sometimes we need to be a little more local in our approach and that more things are able to be adapted to local circumstances.
“Critically deconstructs religious information/issues, leading to coherent and logical chains of reasoning that consider different viewpoints”
This portion, at least to me, may explain part of the issue. Homogenity may actually be the ultimate purpose of some LDS Americans. There is a strong bias among many LDS Americans, particularly from Utah, against public education. Public education is considered by such to be training in socialism. Therefore professional educators become similar to intelligentsia, propagating dangerous concepts that serve to undermine American freedom.
This is quite a deep-seated and serious belief, and its response is to emphasize “facts” over dialogue. Ironically, such thinkers “circle the wagons” and make sure that only approved ideas are discussed.
“Conversation leads to a plurality of thinking, and soon we’re all in the middle of one big hippie love-in.”
“With facts, we stay on topic, on point, and then we can take such facts and beat non-believers with them until they submit.”
Note: The quoted reference is entirely my own fictional construction, and in no way represents the views of actual LDS Americans, living or dead.
“As for living in the mission field, I’m not sure what that means- we have Temples and everything in the UK?”
Until 1975, there were no regular missions in Utah. Even now, the average Utah ward encompasses 240 non-members, while the average UK ward encompasses 185,000 non-members. There may be fields ripe for harvest in the UK, but in Utah we have potted plants or, at most, a backyard garden.
Jerry, I’m not sure that’s true. Some have it as a hobby horse and many just think US education is a complete mess for a slew of reasons ranging from “No Child Left Behind” to the nature of teachers or how special needs like dyslexia are dealt with. But the actual number of people with problems with public education in general seems pretty small.
That said I definitely agree with this post that there are some big problems with seminary. About the best I can say is that it’s better than when I took it. I think a huge issue is early morning seminary which I think is ultimately a waste of time. You’re just too tired to remember anything – especially when there’s a long commute involved. Weekly seminary is much more reasonable. Also as a parent with some dyslexic kids I really worry about that issue as well.
To the issue of facts vs. dialog, I think both are needed. A long term problem with far too much CES instruction, including seminary, is a focus on pointless facts to ensure people did the reading with not enough reflection on meaning. (It probably helps that makes grading easy too) The lessons are then often motivational along a narrow strip without engaging in the real challenges of the students in question.
To the mission field issues, having grown up originally in a district which only got made a Stake when I was around 10, I think I understand what they are saying. There actually is something a bit more meaty about callings in those sorts of wards — although I think we neglect the amount of need and service in Utah wards. The simple reality is that if you are in the mission field you have a much better chance of getting certain callings you might never get in an Utah ward. Although again, the stereotype of Utah wards isn’t quite the reality. I know in our ward getting people to take callings in primary was pretty hard when my wife was Primary President. I’ve (thankfully) never had a leadership calling here, but my Dad back home was frequently in Bishopric/Branch Presidencies and was Branch President twice. Whether that’s a good thing or bad thing depends upon your non-church time commitments I guess.
Thanks for the comments.
Hopefully the post is not seen as a criticism of the US education system- I don’t have enough experience of it to do that- just that it shouldn’t be seen as automatically exportable- or as a one size fits all model. Knowledge is crucial but as are the skills to do something with it.
I’m afraid the mission field comment was a thoughtless throwaway comment- sorry. I knew what they meant- but the way it was used then (and recognise it’s not all the time) was disparagingly- that the Church in the UK is young and we have so much to learn. I’ll tuck that hobby horse back in the cupboard:-)
From a Deseret News report on a court case involving a non-LDS released-time religious class in Virginia: “In addition to more than 84,000 seminary students in Utah, an additional 112,000-plus students are involved in the LDS Church’s seminary programs around the United States. Most participate in early-morning programs, where students meet at their church or at a centrally located home before school for scriptural training and discussion.” So, most seminary, even in the United States, is early-morning seminary. Even though those designing the seminary assessments are oblivious to the distinct qualities of English education, they probably do perform their work with early-morning seminary taught be non-professionals in mind.
John- I would certainly hope so but it is still within the paradigm of US education- which is to be expected because of where they work. But gaining high school credit for release time is not something that happens outside the US and so a different model may be needed in other areas. I’m not disputing it’s validity for certain systems just its applicability for all areas and ages.
I also like early morning as I’m unsure where else it would fit in my and my children’s schedule. I have to say I preferred it as a student than as a parent/teacher but that’s probably because I lead a busier life and like sleep!
Loved your comment, but you lost me at ‘Dallas Cowboys’.
How can anyone be lost with the Cowboys?!
I was probably being too nice in the last comment- I actually am questioning their validity for the stated purpose of S&I- the tests don’t assess progress against the purpose- nor should they. They serve a different purpose and I’m not sure what that is beyond the couple of inappropriate ones I have mentioned in the post. If they meet another requirement for released time then that may be ok in that context.
James, from the article I linked: “None of these programs [LDS released-time and early-morning seminary] involve academic credit, nor are they likely to as a result of the recent judicial ruling [which had no LDS involvement], which church spokesman Scott Trotter said ‘will have little to no impact on the church’s seminary program.'”
John- sorry- should have read the link, rather than assuming what it said. Even less sure of the purpose of tests now though.
I think tests are just there to ensure people did the reading assignment. That’s how it was in far too many BYU religion classes too. From a pedagogical perspective they had almost no other function beyond making it easy to assign grades.
Clark–as an early morning called seminary teacher, I find that things have improved much since my day years ago. The new Doctrinal Mastery program (instead of rote Scriptural Mastery) based on the ASK (acquiring spiritual knowledge) principles engages the students in seeking their own learning and answers. And we make all kinds of assessment testing adaptations for any students who need additional help/time/resources–they can retake it as many times as needed, and are even allowed to take it with parents reading it aloud at home with open scriptures if that helps. Hopefully that’s reassuring for your upcoming students.
American has nothing to do with it. If you don’t think America has pioneered the way to help students with disabilities, you don’t know American education.
But just as in Europe, many places in America fall short. I’ve never seen a worse assessment than in Germany:
Visit with a child. Have them read to you. Talk to them. Do some writing. Do some math. (All great so far). Tell the parent the child is outstanding and well spoken, knowledgeable and well read beyond their years. You’d love to work with them. Then fail them and and explain you reject them because they didn’t pass the math questions.
That’s not Americanism that sometimes puts too much focus on quantitative measurements. That’s the hegemony of quantitative data driven decision making period.
I’m sure the seminary group will definitely be evolving their assessment over time. But I’m also not sure can assess discipleship without it becoming a merit badge checklist; which while probably better than multiple choice questions is also fraught with problems.
Loved this post. I definitely like your ideas of what types of questions should be assessed and (perhaps most importantly) I am a HUGE Dallas Cowboy fan. I also found your comments on “the mission field” to be quite funny and true. As someone who was born in Utah (and spent most of my life there) but currently resides outside of the state I die of embarrassment anytime I hear someone use the phrase. (And anyone who knows me knows I generally think so-called “Utah Mormons” are much better than their stereotypes. But the phrase “the mission field” should be banished from our lexicon forever.)
Now if I may be so bold as to suggest I think the title to your blog post does the material a disservice. In so far as I am reading you correctly (and I have been rightly accused of misreading people online in the past, so please feel free to push back if I am misreading) your main point is you find the one-size fits all approach that the church hands out as not suitable for the needs of students you are teaching. That one size fits all approach just so-happens to come from Utah, which just-so happens to be within the United States of America. So your critique (again, if I am understanding your argument correctly) isn’t really with Americacentrism per se. By that I mean, your problem isn’t that the way the tests are administered are so-unlike anything that might happen in Germany or other countries. And (again if I’m reading you correctly) you are also not saying that the assessments represent how the American education system works. The bigger point is there is a “headquarters” located in America that sends materials that you find constraining and you wish the system was more flexible allowing each location to better adjust to their culture and situation. Am I getting you right? If so I completely agree and think more local flexibility will create better gospel learning and teaching. If it is more of a statement on the type of education America offers, that I have no expertise in at all and no opinion.
I also like how you show that in LDS teachings (for example Elder Perry) expressly state that we need local autonomy. I believe this is the exact type of creative and productive activity that an Elder Perry would or should want to see. I hope moving forward even more autonomy is granted to local teachers and leaders. I know in some ways we are making in roads in that direction. Just in my lifetime many lessons have become less structured. But I certainly hope there is more ahead.
Jason B- Yes! Yes! Yes! In fact- let’s just replace the original post with your comment:)
acw- I think I should also say that that is also how the young man I mention was treated- his Bishop sat down with him with the scriptures and all was good. But the issues with the test remain- sorry I’m a broken record! It’s not to measure discipleship, but if it’s to make sure the content is taught then we need to look at our teacher support- seminary teachers the world over have my immense admiration.
Thank you for raising this subject. You make an excellent point with a solid analysis. It is good topic to talk about. It might be true that assessments are in particular an American approach or it might not. Even if it is, the question is why this would bother anyone.
I research how LDS doctrine comes across in different nations in Europe. And I find that in some nations assessments are liked, loved, and even very much needed. In other cultures assessments on religion are seen as a form of insult, because religion is not about knowing, but about believing; and religion classes are about developing insights and focus on God. Moreover, they take assessments so serious as to see them as some sort of judgment of worthiness from God’s throne.
In my eyes each culture should decide if and how to assess students. Moreover, the new teaching style of the church, questioning, and students participating in the teaching process is the main focus of our didactics. If students are constantly involved in the learning process, why would one want override the teacher’s responsibility by having them do an official centralized prescribed assessment? Are teachers not capable of judging a student’s progress by themselves? Is that not what “teaching by the Spirit” is all about? Moreover, I am finding that in each culture an assessment needs to focus on a different aspect of the gospel. Just as an assessment to a student in Jesus’ time would be different than the assessment of a student in Joseph Smith’s time, as it is in our time, so an assessment needs to be different for people living in Catholic countries, different for people living in Protestant cultures, different for people living in communistic or atheistic or Buddhist cultures or whatever. Mormons living in each of these areas have a different take on the gospel, with a different focal point, and thus the need for a different approach towards the gospel truths. This is also why in each nation there is a different take on education. I would vote for centralized assessments to become a suggestion, rather than obligatory.
That’s a really good point about expectations Hans. I’d not thought about that.
Thanks for this thorough and thought provoking post James.
I agree with you when you say ; “However, when I consider my students I think they may have done enough when they get up at 545 every morning to attend Seminary, learn the scripture mastery passages, read the scriptures the formative assessment through questioning in the lessons is far more valuable a tool in measuring progress.:
I’d be interested to know your thoughts on the value of doing seminary so early in the morning. I applaud the changes that have happened, but feel lots more is necessary to truly equip the young people. However I also feel for the seminary teachers and wonder if the current system is designed to make their jobs easier when it comes to marking the students?
I think a bigger issue for teens is getting enough sleep and early morning seminary makes that worse. The years I did early morning I remembered nothing whereas I at least retained more from doing the weekly meeting and the independent study seminary. My guess is early morning seminary not only isn’t as effective but negatively affects their broader academics.