Setting Exam Questions - Beyond Bloom's Taxonomy

  Most JCs are in the midst of setting their prelim questions and promotional examination questions at this time of the year. This is no different at my college, last Friday a colleague who is new in teaching came and seek me advice on how I usually set questions and he asked me if it was alright for a question to contain many different parts on a story line. So, we had a chit-chat and I shared my experience on setting questions with him, in the midst I realised that it may be possible to pen down some of experiences and hopefully, some new teachers can benefit from it.

Usually each examination will have a specific table of specification (TOS). For each question and topic that I am assigned to I have something similar as well. Usually, I will go through the syllabus and identify all the concepts that I need that students to be able to show mastery in. I usually start off by setting the structured question first, unless I have just come across one very particularly good question in MCQ. For the structured question. as I set each part of question, I check off the concept that has been tested. This avoids over testing of one particular concept - as each mark in the paper is very precious. It also alerts me to which other the concepts that have not been covered in the examination. That allows me to set an appropriate question in MCQ to ensure that concept has been tested

Many schools also use the Bloom's taxonomy to guide them. However, I find that a little difficult to use at times especially in physics were almost every question is an application and applications and extrapolation can be both difficult and easy depending on the question. However, I find it useful to follow the following:

(i) I usually start setting the main scenario of the question. For beginners, a good point is always to pick an interesting question from past questions, or a textbook question. I then first solve the question. Usually textbook questions especially the more challenging ones will need to be solved in parts.

(ii) Then I add in parts to the question for ensure that students are guided to achieve the final answers, start off with the easier calculations and slowly lead the students in. There should not be more than two steps in each part.

(iii) Revisit the question, look at if there are any appropriate qualitative questions that can be added in to get the students to explain a new scenario if something in the questions is changed or if what is calculated can be placed at the end of the question. Sometimes, if there are more than sufficient marks to play with I will also get students to sketch graphs pertaining to scenario, this helps to extend students' thinking and also assess students' ability to translating information to abstract representations.

(iv) When the bulk of the question is set , I then assign the marks for each part. (We will talk more about how I assign marks below) Usually each setter is given a certain of marks to play with for the question to be set. For the filling the rest of the marks, I will now go back and place some regugitative questions like definitions to the front or intermediate parts of the question. For example in a gravitation question on dealing with force approaches that utilises Newton's Law of Gravitation, I would put in a "State Newton's Law of Gravitation" in front. If an energy approach is needed later, I may add in the definition of Gravitation Potential Energy of a 2 mass system. This is particularly essential to guide the students' train of thought in examinations.

4. Throughout the whole process, be clear of the learning objectives that you wish to set, this especially important in awarding of marks. For awarding marks in the question, I make sure I tie the points in my question to a particular concept that I wish to illustrate. So beyond just writing down 1 mark - I usually have a remarks column in my own marking scheme to note down also what this one mark is for e.g. illustrating knowledge of application of Newton's Law of Gravitation.

5. From the description of 3, I also note that beyond just setting a question. I do take note that in all my questions, there should not only be just calculations. In all the questions I set for a topic, I always ensure that there is a little bit of regugitative work (10 - 20% of question), a little bit of calculation (30% -40% should be something familiar), more calculation (10% - 20% more challenging), some graphical work (20%) and some qualitative explanations part (20%) -both of which usually proofs to be more challenging for students.

6. Back to the question on whether it is appropriate to have many different scenarios in a question. It depends on how many marks is allocated and what the objective is for the question, for some time is needed for the candidate to explore and understand each new scenario. For example, in the case if we are looking at a synthesis question on how students understand conservation of energy is applied throughout the various topics in Physics, then each part could be description of a different scenario like describing the energy changes that occurs in a spring undergoing simple harmonic motion, a block sliding down a slope, the excitation and de-excitation of an electron when a photon is incident on it etc. Each scenario is however short and familiar. However, if the scenario could be something not familiar and a lot of description is needed to get the students into the story line, then it makes no sense to give a new scenario every two parts. E.g. in the case where Tarzan swings and picks up Jane and swings back again. The scenario should be sufficient to present itself with many questions the setter can set for. Having a new scenario for each one or two marks is very unfair to the candidate as there is in fact more time needed for candidate to truly understand the scenario.