OscatsTest

OscatsTest — Computerized Adaptive Test Administration

Synopsis

struct              OscatsTest;
struct              OscatsTestClass;
void                oscats_test_administer              (OscatsTest *test,
                                                         OscatsExaminee *e);
void                oscats_test_set_hint                (OscatsTest *test,
                                                         GBitArray *hint);

Object Hierarchy

  GObject
   +----OscatsTest

Properties

  "id"                       gchar*                : Read / Write / Construct Only
  "itembank"                 OscatsItemBank*       : Read / Write / Construct Only
  "itermax-items"            guint                 : Read / Write / Construct Only
  "itermax-select"           guint                 : Read / Write / Construct Only
  "length-hint"              guint                 : Read / Write

Signals

  "administer"                                     : Action
  "administered"                                   : Action
  "approve"                                        : Action
  "filter"                                         : Action
  "finalize"                                       : Action
  "initialize"                                     : Action
  "select"                                         : Action
  "stopcrit"                                       : Action

Description

Details

struct OscatsTest

struct OscatsTest;


struct OscatsTestClass

struct OscatsTestClass {
  GObjectClass parent_class;
  // Signals
  guint initialize;
  guint filter;
  guint select;
  guint approve;
  guint administer;
  guint administered;
  guint stopcrit;
  guint finalize;
};


oscats_test_administer ()

void                oscats_test_administer              (OscatsTest *test,
                                                         OscatsExaminee *e);

Core CAT administration loop. The particular algorithms used to select and administer items as well as update ability/classification estimates and tabulate statistics are specified by means of signals connected to test.

The test proceeds in the following order:

  1. The examinee's item/response vectors are reset.

  2. The "initialize" signal is emitted.

  3. The item eligibility vector is initialized with the current hinted value (default: all items).

  4. The "filter", "select", and "approve" signals are emitted.

  5. If the approval handler returns TRUE, goto 3 (not more than "itermax_select" times).

  6. The "administer" signal is emitted (which should add the item/response pair to e as necessary).

  7. The "administered" and "stopcrit" signals are emitted.

  8. If the stopping criterion has not been met, goto 3 (not more than "itermax_items" times).

  9. The "finalize" signal is emitted.

Algorithms must be connected to at minimum "select", "administer", and "stopcrit". An initial guess for the examinee's latent IRT ability or classification must be supplied.

test :

the OscatsTest to administer

e :

the OscatsExaminee taking the test

oscats_test_set_hint ()

void                oscats_test_set_hint                (OscatsTest *test,
                                                         GBitArray *hint);

Sets the item eligibility hint. The length of hint must be the same as the number of items in "itembank". This is generally called in handlers connected to "initialize" or "administered".

test :

an OscatsTest

hint :

the initially eligible items

Property Details

The "id" property

  "id"                       gchar*                : Read / Write / Construct Only

A string identifier for the test.

Default value: NULL


The "itembank" property

  "itembank"                 OscatsItemBank*       : Read / Write / Construct Only

The item bank used for the test.


The "itermax-items" property

  "itermax-items"            guint                 : Read / Write / Construct Only

The maximum number of items. If this value is reached (or the item bank is exhausted) before the stoping criterion is triggered, a warning is logged, the "finalize" signal is emitted, and the test is concluded.

Allowed values: >= 1

Default value: 200


The "itermax-select" property

  "itermax-select"           guint                 : Read / Write / Construct Only

The maximum number of iterations in the item-selection loop. If this value is reached without selecting an item, a warning is logged, the "finalize" signal is emitted, and the test is aborted.

Allowed values: >= 1

Default value: 50


The "length-hint" property

  "length-hint"              guint                 : Read / Write

Guess for how long the test will be.

Default value: 0

Signal Details

The "administer" signal

guchar              user_function                      (OscatsTest     *test,
                                                        OscatsExaminee *e,
                                                        OscatsItem     *item,
                                                        gpointer        user_data)      : Action

Item administration. Only one administrator should connect to the "administer" signal. If multiple algorithms attach to "administer", only the response returned from the last administrator will be reported to algorithms connected to "administered". Note: An algorithm connected to this signal should call oscats_examinee_add_item(), as appropriate, to record the response for the examinee so that it can be used in subsequent ability/classification estimation. Whether or not the response is recorded (tests with online calibration, for example, may not record every response), items are still not administered more than once.

test :

an OscatsTest

e :

an OscatsExaminee

item :

the proposed OscatsItem

user_data :

user data set when the signal handler was connected.

Returns :

the guint response of examinee e to item

The "administered" signal

void                user_function                      (OscatsTest     *test,
                                                        OscatsExaminee *e,
                                                        OscatsItem     *item,
                                                        guchar          resp,
                                                        gpointer        user_data)      : Action

The "administered" signal allows algorithms to record statistics concerning items administered and the examinee's responses. In particular, an algorithm to update the examinee's estimated ability/class should be connected to "administered".

test :

an OscatsTest

e :

an OscatsExaminee

item :

the proposed OscatsItem

resp :

the guint response

user_data :

user data set when the signal handler was connected.

The "approve" signal

gboolean            user_function                      (OscatsTest     *test,
                                                        OscatsExaminee *e,
                                                        OscatsItem     *item,
                                                        gpointer        user_data)      : Action

Item approval. Algorithms (e.g. Sympson-Hetter) connecting to "approve" should indicate whether the proposed item is acceptable. If multiple algorithms attach to "approve", each one must approve an item before it will be administered to examinee e. The item may be NULL, in which case handlers must silently return TRUE.

test :

an OscatsTest

e :

an OscatsExaminee

item :

the proposed OscatsItem

user_data :

user data set when the signal handler was connected.

Returns :

FALSE to accept the proposed item, TRUE to reject

The "filter" signal

void                user_function                      (OscatsTest     *test,
                                                        OscatsExaminee *e,
                                                        GBitArray      *eligible,
                                                        gpointer        user_data)      : Action

Item filtration. The list of eligible items is initialized to the test's eligibility hint (default: all items), minus items examinee e has already been administered. Algorithms connecting to "filter" should clear bits in eligible for undesired items. If an algorithm sets bits in eligible, it is the algorithm's responsibility to ensure that examinee e has not already seen the given item.

test :

an OscatsTest

e :

an OscatsExaminee

eligible :

a GBitArray indicating eligible items in the itembank

user_data :

user data set when the signal handler was connected.

The "finalize" signal

void                user_function                      (OscatsTest     *test,
                                                        OscatsExaminee *e,
                                                        gpointer        user_data)      : Action

Test finalization. Algorithms may compute final statistics and clean up internal data structures by connecting to "finalize". It is guaranteed that the test will emit "finalize", even if the test is aborted.

test :

an OscatsTest

e :

an OscatsExaminee

user_data :

user data set when the signal handler was connected.

The "initialize" signal

void                user_function                      (OscatsTest     *test,
                                                        OscatsExaminee *e,
                                                        gpointer        user_data)      : Action

Test initializiation. All algorithms that need to initialize their internal data structures should connect to the "initialize" signal. If the examinee's initial ability/class estimate is not set before oscats_test_administer() is called, it should be set by a handler attached to "initialize".

test :

an OscatsTest

e :

an OscatsExaminee

user_data :

user data set when the signal handler was connected.

The "select" signal

gint                user_function                      (OscatsTest     *test,
                                                        OscatsExaminee *e,
                                                        GBitArray      *eligible,
                                                        gpointer        user_data)      : Action

Item selection. The algorithm connecting to "select" must select a single item from those in the item bank that are eligible, or return -1 on failure. Only one algorithm should connect to "select". If multiple algorithms connect, only the results from the last connected algorithm will be used.

test :

an OscatsTest

e :

an OscatsExaminee

eligible :

a GBitArray indicating eligible items in the itembank

user_data :

user data set when the signal handler was connected.

Returns :

the index of the selected OscatsItem, or -1 on failure

The "stopcrit" signal

gboolean            user_function                      (OscatsTest     *test,
                                                        OscatsExaminee *e,
                                                        gpointer        user_data)      : Action

Stoping criterion. If multiple algorithms attach to "stopcrit", only one is required to end the test. Note, the test will never have more than "itermax_items" items.

test :

an OscatsTest

e :

an OscatsExaminee

user_data :

user data set when the signal handler was connected.

Returns :

TRUE to end the test, FALSE to continue with another item