Message Boards Message Boards

Exam creation / Testing students with Mathematica

We have a basic Calculus course with high number of students. I would like to crate tests they can solve by hand and type the answer in the computer (e.g. web app.) Some of the answers can be chosen from given ones, but others ask for numbers (Sqrt[2]) or even for simple expressions like [Sin x]. I only want to test their mathematical knowledge, so they do not necessary know anything about syntax of Mathematica. What is the most cost effective way to do this? I have some basic ideas to create notebooks with boxes, and testing the answers, but it would need too many Mathematica license, and I am not sure, they could not regain my answers from the notebook. Is that possible to create some notebooks, put them on web or intranet, students in the computer room can see it and type in, and at the end we can get their results.

Thank for any suggestions or sharing experience, sample tests ...!

POSTED BY: Eva Racz
6 Replies

The most cost-effective solution, including the cost of your time spent in (a) infrastructure development and (b) writing questions is to use an existing web-based homework system. One such that comes with an extensive problem library — and that's free if you run it on your own server — is WeBWork: http://webwork.maa.org.

There are several good commercial systems, which are often available bundled with particular textbooks and have libraries of problems already organized by the textbook's chapters and sections. One such I know well and highly recommend is WebAssign: http://webassign.net

Such systems are particularly helpful for high-enrollment sections, owing to the reporting features, etc.

POSTED BY: Murray Eisenberg

If I remember correctly from the times I was teaching at CUNY, WeBWork is numerically-based (approximately) verification system. And so are the others, I guess, numerically (approximately) or heuristically based. While good initiatives, and a good resources, I still think Wolfram Language (WL) is a tremendous foundation on which such things can be based due to the following advantages:

  • Fundamentally symbolic nature and exact expression manipulation with tools like patterns, string patterns and regular expression processing which opens immensely wider field for automated assignment creation and testing.
  • Large algorithm base (including math and structural expression processing)
  • Large math model base
  • Free form linguistic

I will illustrate last point on example of the function called Interpreter:

Interpreter["MathExpression"]["2sin(3*z^2+1)"]

Note in the expression above there is no strict math-syntax convention, multiplication was correctly interpreted in both $*$ and no-$*$ cases. Below I give some other relevant examples from this discussion.

This app is just a start - a simple working prototype.

Manipulate[Grid[{{"What is the value of this integral?"},
{HoldForm[\[Integral]2 z^2 \[DifferentialD]z]}, {Framed@
      Row[{"the answer is: ", 
        If[FullSimplify[
          If[StringCases[x, "Integrate"] === {}, 
            ToExpression[x, StandardForm] // Quiet, 
            CreateDialog[Style["Please do not cheat!!!", "Title"],
              WindowSize -> {400, 60}];; Null, 
            CreateDialog[Style["Please do not cheat!!!", "Title"], 
              WindowSize -> {400, 60}];; Null] == 2 z^3/3], "CORRECT",
          "WRONG", "WRONG"]}]}}], {{x, "", ""}, 
   InputField[#, String, FieldHint -> "enter answer here..."] &}, 
  AppearanceElements -> None] // TraditionalForm

enter image description here

enter image description here

enter image description here

enter image description here


Next answer is Jens


As I mentioned in my comment to the question, I am quite fascinated by the idea of testing free-form input by numerical means similar to what the AcroTeX bundle has been able to do for a long time. This idea differs quite fundamentally from Vitaiy's suggestion, so I have tried to make it work in Mathematica.

I will suspend discussion of the crucial obstacle that the free CDF format, though being the obvious choice to deploy such tests, does not currently allow InputFields for strings. For the moment, I have to trust in the indirect promise made by Vitaliy that this will change in the future (I realize he doesn't speak for WRI).

Now it turns out that Mathematica has a function that is almost ideal for my suggested method: PossibleZeroQ. Implementing this as a test validator is the main idea of this answer.

The second idea is that with a tool as powerful as Mathematica, making a meaningful testing environment (even for harmless practice tests) requires us to shut off access to symbolic manipulations that can be used to cheat. This problem doesn't arise if we make tests where you only have to enter numbers, as in R.M.'s answer. The question goes much further than that, in that it wants us to validate student input in the form of functions (as in integrals and derivatives).

The main difference between my approach and Vitaliy's in this respect is that I implement a "white-list" instead of a "black-list" approach. That means I want allow only a certain class of functions to be entered. In particular, I will permit only those functions that have NumericFunction in their Attribute list.

The code below tries to implement these main ideas, putting them in the form of a basic dynamic grid where the questions appear next to answer fields. This requires some juggling with Hold, HoldForm, string conversion and replacements inside Dynamic. Strings appear because for security reasons I accept the answer as String input before converting it ton a held expression.

Here is the set of definitions:

Clear[check, quantQuestions, quantAnswers, prepareQuestions];

quantQuestions[
   expressions__] := {ReleaseHold@Map[HoldForm, Hold[expressions]]};
SetAttributes[quantQuestions, HoldAll];

quantAnswers[questions__] := 
  Map[ReleaseHold[(# /. 
        Integrate[f_, x_?AtomQ] :> integralWrapper[f, x]) /. 
      Integrate[f_, {x__}] :> 
       Integrate[f, {x}, GenerateConditions -> False]] &, questions];
SetAttributes[quantAnswers, HoldAll];

check[ans_, reference_] := Module[{ans1, ref1}, 
  If[
   Head[reference] === integralWrapper, 
   ans1 = D[ans, Last[reference]];
   ref1 = First[reference], 
   ans1 = ans; 
   ref1 = reference
  ];
  If[PossibleZeroQ[ans1 - ref1], 1, 2]]
check[ans_String, reference_] := 3;

stringCheck[
  a_] := (Apply[And, 
        Map[MemberQ[Attributes[#], NumericFunction] &, 
         Cases[# /. Hold -> Unevaluated, x_[__] -> x, Infinity]]] /. 
       True :> ReleaseHold[#] /. False -> "no answer") &[
   If[SyntaxQ[a], ToExpression[a, InputForm, Hold]]] /. Null -> ""

prepareQuestions[q_] := 
 DynamicModule[{feedback = "Correct answers: ", maxAttempts = 3, ref, 
   ans, evaluation, attempts, active}, 
  Check[ref = quantAnswers[q], 
   Print["Some questions cannot be answered"]; Abort[]];
  ans = Map["" &, ref];
  evaluation = Map[4 &, ref];
  attempts = Map[0 &, ref];
  active = Map[True &, ref];
  Deploy@Labeled[
    Framed[Grid[
      Append[
       Prepend[
        Array[{
           TraditionalForm[q[[#]]], 
           InputField[Dynamic[ans[[#]], Function[{a},
              If[ans[[#]] != a, ans[[#]] = a;
               If[! (active[[#]] = (attempts[[#]] += 1) < 
                    maxAttempts), 
                FrontEndExecute[{FrontEnd`FrontEndToken[
                   FrontEnd`InputNotebook[], "MoveNextPlaceHolder"]}]];
               ];
              evaluation[[#]] = 
               check[stringCheck[ans[[#]]], ref[[#]]]]], 
            String, 
            FieldSize -> 15, 
            Enabled -> Dynamic[active[[#]]]
           ], 
           Dynamic[{"Correct.", "Wrong.", "No answer", 
              ""}[[evaluation[[#]]]]], 
           Row[{Dynamic[attempts[[#]]], " / ", maxAttempts}]} &, 
         Length[ref]
        ], 
        Map[Style[#, "Subsection"] &, 
         {"Question", "Answer", "Evaluation", "Attempts"}]
       ], 
       {InputField["", String, 
         FieldSize -> 1, ImageSize -> {0, 0}, 
         Appearance -> "Frameless"]
       }
      ]
     ], 
     FrameStyle -> Orange, 
     RoundingRadius -> 5
    ], 
    Row[{feedback, Dynamic[Count[evaluation, 1]]}, 
     Background -> Orange, Frame -> True, FrameStyle -> None, 
     FrameMargins -> 5, RoundingRadius -> 5], Bottom, 
    BaseStyle -> "Subsubsection", 
    LabelStyle -> {"Subsection", LightOrange}, 
    Background -> LightOrange]]

The starting point is quantQuestions which wraps the set of questions in HoldForm so they can be displayed as such. Internally, they are then unwrapped with quantAnswers to get the simplified results with which to compare the user's input later.

The only other user-facing function is prepareQuestions which is then applied like this:

prepareQuestions[
 quantQuestions[
  D[Sin[x + y], y], 
  D[Log[x], x], 
  Integrate[1/x^2, x], 
  Integrate[x, {x, 0, 1}]
 ]
]

This creates the dynamic input fields as shown here:

input checking

Here, I have already entered the answers and received the appropriate feedback.

You can try to enter the answers in different ways. Also try to enter cheats - they are supposed to be caught by the function stringCheck. Its job is to comb the held form of the expression in the input string for non-numerical functions, of which D and Integrate are examples.

Edit

In the updated code above, I added a check for correct Mathematica syntax using SyntaxQ. Other improvements are:

  • The number of attempts at an answer is counted, and the input field is disabled after a maximum number (3 here).
  • The total number of correct answers is displayed at the bottom.
  • If a question has no answer, the error is displayed and no table is created.
  • The problems are displayed using TraditionalForm (whether this is better is a matter of taste).

Edit 2

As pointed out in comments by Andrew and Murray, indefinite integrals have to admit constant offset. This can be done by comparing the derivatives, and I've implemented that for a single variable by substituting a formal wrapper (integralWrapper) into the automatically computed answer (quantAnswers) that tells check to differentiate instead of integrate. It is only needed for indefinite integrals, and the rest of the validation (using PossibleZerQ) works as before.

I'm also suppressing the generation of conditions for definite integrals, so that questions such as Integrate[x,{x,y,z}] with unknown boundaries can be handled too.

POSTED BY: Vitaliy Kaurov

Although much of WeBWorK's answer-checking mechanism is numerically based, there's still a lot you can do with the system.For example, it can check whether the student's answer differs from the "correct" answer by a constant.

WebAssign can definitely handle symbolics with aplomb. In fact, one of the methods you can use to check answers is to call its built-in Mathematica system! Among other things, this means that parametrized problems and answers can be generated dynamically.

As I indicated in my first post, the main issue is whether you want to invest the time and resources in reinventing the wheel or, instead, use an existing, mature and well-tested system in which you select problems from a rich existing library, or even write your own questions.

While I love programming in Mathematica, for a course I'd much rather devote my time to quickly assembling problem sets, then tracking student progress and, outside of class and office hours, interacting with the students via the on-line capabilities of a homework system.

POSTED BY: Murray Eisenberg

I think what would be really interesting and useful, would be if a system actually (perhaps through pattern recognition) could verify the students derivation of the solution and then be able to give partial credit for every correct step toward the solution.

POSTED BY: Kay Herbert

I think what would be really interesting and useful, would be if a system actually (perhaps through pattern recognition) could verify the students derivation of the solution and then be able to give partial credit for every correct step toward the solution.

That's asking for a lot!

The real problem with it is to specify what the "correct" steps are, in the correct order, with allowances for situations where there are different possible legitimate steps or different ordering of steps. E.g., in applying the Chain Rule to differentiate, say, sin(log(x)), one may work from the inside outwards (i.e., treat log first) or, instead, work from the outside inwards.

Having a human instructor specify the correct steps, before describing them in computer-comprehensible form, becomes even more problematic for finding antiderivatives.

For a not-too-large class, there's an easy solution: allow questions where the students just type in their work, and then the instructor reads it on-line. WebAssign, possibly other homework systems, already allows this. I found that I could One read, comment upon, and enter scores for all the students' answers in less time than it would take to collect on-paper solutions read, comment upon, and grade the papers.

POSTED BY: Murray Eisenberg

I would definitely check out the ability to create webforms with FormFunctions. Your students don't need to have a M- license. And you would just need a low tier of the development platform (or Mathematica if you wanted to work offline). You can do free form input field, multiple choice etc.

Here's the page for learning more about the function.

Here's a basic example of a web form that generates the population of a specific country.

POSTED BY: Adriana O'Brien
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract