Over the past weeks and months we have been updating both the R/exams package as well as it’s newest extension exams2forms to resolve known (and new) issues as well as extend the functionality of both packages. I would like to thank everyone involved in the latest additions and improvements!

I, myself, was mainly involved in (i) extending and refining the stresstest functionalilty to automatically stress-testing R/exams exercise questions, as well as (ii) refining and extending the functionality of the R-package exams2forms which now provides additional features for our users developing and testing R/exams exercise questions.

R/exams

Stresstesting exercises

For quite some time, the R/exams package provides automated stress-tests for (one or multiple) exercises using the stresstest_exercise() function (see e.g., our tutorial on Stress Testing Dynamic R/exams Exercises).

Whilst always working perfectly fine, one inconvenience was that the function may ‘stall’ if an exercise was taking an unexpectedly long time to be generated (or ended up in an infinite loop in the worst case).

The new release of R/exams now allows to set a timeout. This useful if one wants to identify exercises which take “too long” to be rendered, or exercises which may end up in infinite loops. It becomes especially useful if one want’s to check or test extensive libraries of exercises (hundreds or thousands) and stress-test them extensively (generate them hundreds or thousand times). stresstest_exercise() now allows to set timeouts which will result in an error if violated. In addition, this is reported back to the user (see below).

Besides a timeout option, stresstest_exercise() now reports all warnings and errors captured during execution. This feature is available no matter if you only test one exercise (you must set stop_on_error = FALSE) or stacks of exercises. This allows to stress-test bulks of exercises and not only get reported where errors occured and how often, but also where warnings have been raised and how often, even if–a the end–the questions was successfully rendered.

num_to_schoice()

The extension of the stresstest-functionality above was somewhat interlocked with the function num_to_schoice() which allows to generate single-choice questions from a numeric solution. If the data generation of an exercise is badly written, it can cause num_to_schoice() to search for a possible (valid) return forever. Besides the new timeout-feature of stresstest_exercise(), num_to_schoice() has also been updated, allowing to specify how often the function is allowed to try to find a valid solution (return) via it’s new maxit argument which can also be set global (via getOption). See ?num_to_schoice for more details.

exams2forms

Development/testing features

Besides exams2forms providing great functionality to use for stand-allone versions of R/exams, its functionalty also helps those writing new exercises or testing existing ones. With the new release, exams2forms() (called by exams2webquiz()) got some new features allowing to (i) display the name of the exercise (show_filename), show (ii) the required tolerance for correct answers of numeric questions (show_tolerance), as well as (iii) features to be used by developers/testers to quickly check if everything is set up correctly (see argument auto).

Check out ?exams2forms for more details (all available via exams2webquiz()).

############################################################################

Over the past weeks and months, we have been updating both the R/exams package and its newest extension, exams2forms, to resolve known (and new) issues and to expand the functionality of both packages. I’d like to thank everyone involved in the latest additions and improvements!

I was primarily involved in:

1. Extending and refining the stresstest functionality to automatically
   stress-test R/exams exercise questions, and
2. Improving and extending the functionality of the R package exams2forms, which now offers additional features for users developing and testing R/exams exercises.

R/exams

Stress-testing Exercises

The R/exams package has long supported automated stress-testing of one or multiple exercises via the stresstest_exercise() function (see the tutorial on
Stress Testing Dynamic R/exams Exercises).

While this functionality has worked reliably and as originally intended, it had one limitation: if an exercise took too long to generate–or entered an infinite loop–the function possibly stalled.

The latest release introduces a timeout option. This is especially helpful when testing large libraries of exercises (hundreds or thousands), allowing you to flag exercises that take too long to render or never complete. If the timeout is exceeded, an error is raised and reported to the user.

In addition, stresstest_exercise() now captures and reports all warnings and errors during execution. This applies whether you’re testing a single exercise (set stop_on_error = FALSE) or a large batch. You’ll get detailed feedback on where and how often issues occurred—even if an exercise was ultimately rendered successfully.

num_to_schoice()

The improvements on stresstest_exercise() were closely related to updates in the num_to_schoice() function, which creates single-choice questions from numeric solutions. Poorly written generation code could previously cause num_to_schoice() to loop indefinitely in search of a valid return.

Alongside the timeout in stresstest_exercise(), num_to_schoice() now accepts a maxit argument, specifying the maximum number of attempts before giving up. This can also be set globally using setOpttion().

See ?num_to_schoice for full details.

exams2forms

Development and Testing Features

In addition to supporting stand-alone versions of R/exams exercises, exams2forms now includes additional features that are especially helpful when developing or testing exercises.

The updated exams2forms() function (typically called via exams2webquiz()) got the following new options:

• show_filename: Displays the name of the exercise file.
• show_tolerance: Shows the tolerance for numeric answers.
• auto: Enables developer/tester-specific features for quick verification that everything is correctly set up.

Check out ?exams2forms or ?exams2webquiz for more details.