Generating Practice Questions with AI: Prompts, Pitfalls, and Quality Checks

Question writing is one of the most time-consuming parts of teaching, so it is no surprise that generating questions is among the most popular educational uses of AI. It works — with a caveat that matters enormously: a meaningful fraction of AI-generated questions contain flaws, and the flaws are often invisible until you actually attempt the question. The skill is not getting AI to produce questions. It is getting it to produce good ones, and reliably catching the bad ones.

Prompt patterns that raise quality

Generic prompts produce generic questions. The gap between a vague request and a specific one is larger here than almost anywhere else.

• State the level precisely: 'Secondary 3 Express, O-Level Additional Mathematics syllabus' produces very different questions from 'high school maths'.
• Name the skill, not just the topic: 'questions requiring students to choose between the quotient rule and the chain rule' beats 'differentiation questions'.
• Provide an example question in the style you want — one exemplar improves format, difficulty, and tone more than a paragraph of description.
• Ask for the worked solution and mark scheme alongside every question. This exposes flaws immediately and gives you marking materials.
• For multiple choice, ask for each distractor to represent a named misconception. It forces plausible wrong answers instead of filler.
• Request a range: 'two routine, three standard, one challenging' gives you a ramp instead of a flat set.

The common flaws in AI questions

AI-generated questions fail in predictable ways, which makes checking them much faster once you know the catalogue.

• Wrong answers presented confidently: the stated answer to a maths question is simply incorrect, or the worked solution contains an arithmetic slip halfway through.
• Ambiguity: the question admits two reasonable interpretations, or omits information needed to solve it — common in word problems.
• Ugly numbers: answers that come out to unwieldy decimals when the exam style expects clean values, a small tell that erodes student trust.
• Difficulty drift: questions labelled challenging that are routine, or basic questions with an unannounced twist.
• Syllabus overreach: questions requiring techniques outside the syllabus, phrased as if they belong.
• Distractors that give the game away: three absurd options and one obviously sensible one.

A verification workflow

The non-negotiable step is attempting every question yourself, under something like the conditions your students will face. Reading a question is not checking it; ambiguity and wrong answers reveal themselves only when you actually work the problem. For a set of ten questions this takes perhaps fifteen minutes — far less than writing ten questions from scratch, which is why the workflow still comes out ahead.

1. Generate a batch about half again as large as you need, with solutions.
2. Solve each question independently before reading the provided solution, then compare.
3. Classify each item: use as-is, fix (usually numbers or wording), or discard. Discard freely — generation is cheap.
4. Check the set as a whole: difficulty ramp, syllabus coverage, no accidental repetition of the same underlying structure.
5. Add the survivors to your question bank with tags. Verified questions are reusable; unverified ones are liabilities.

Where the risk concentrates

Multi-step mathematics is the highest-risk category, because a single wrong step corrupts the answer while the working still looks authoritative. Questions involving dates, statistics, or quotations are next — AI will invent all three fluently. Language questions are generally safer, though watch for phrasing that no local exam would use. Whatever the subject, one principle covers everything: an AI question that has not been solved by a human has not been checked, no matter how polished it looks. Held to that standard, AI question generation is one of the clearest time-savers available to teachers today.