Psychological Experimental Design

Usability testing is rooted in the psychological experimental design. This is described in Dix et al. [31]:Chap.9. These kind of tests can be of interest if the developers have to know more about details of human perception or human memory functionality. Nevertheless, more recently this paradigm gains a new momentum on account of the growing opportunities to use only asynchronous testing with large group of users over long time periods.

The basioc checklist looks as follows:

Participants:

The test persons should as close as possible be similar to the intended users; there should be at least 10 persons

Hypothesis:

A statement about the expected outcome

Independent Variable (IV):

Those variables whose properties can be manipulated.

Dependent Variable (DV):

Those variables whose different levels are measured depending on independent variables

Design:

Between or within subjects. In he last case the same test persons are used for different settings

Task:

Different tasks which the test persons have to solve.

Measurements/Data:

The observable behavior of the test persons has to be measured. It has to be clarified in advance which properties of the observable behavior are crucial for the measurement.

Analysis:

The measured data have to be interpreted by known models of interpretation.

Figure 4.3: Some Psychological Experimental Paradigms

This general Layout of a psychological experiment can be used for a great variety of cases. Some useful cases are represented by figure 4.3. Here are some examples:

PERCEPTUAL PROPERTIES:

Presenting different kinds of properties as stimuli and measuring, which ones will be sufficiently identified within a time interval which is 'sufficiently' small not to rely on memory.

PERCEPTUAL COMPLEXITY:

Presenting a certain number of properties which have been proven to be able to become identified; there should be a finite maximum above which the number of errors will increase. This will be done within a time interval which is 'sufficiently' small not to rely on memory.

SHORT TERM MEMORY:

Presenting in case (i) a certain number of unknown items with a fixed complexity which have been proven to be able to become identified; there should be a finite maximum above which the number of errors will increase. This will be done within a time interval which is 'sufficiently' large to rely on memory. In case (ii) the number of unknown items is fixed and the complexity increases. There should be a finite maximum above which the number of errors will increase. This will be done within a time interval which is 'sufficiently' large to rely on memory.

LONG TERM MEMORY:

(i) 'Learning' long-term items by repetition ('concrete' items, 'absract' items, 'relations', ...); (ii) checking 'forgetting' by repetition of previously learned items...

:

...

These considerations lead to a general perspective of agent-environment testing as pointed out in figure 4.4.

• In one scenario the agent is a human person and the experimental design provides some interface generating stimuli to the human agent where the human agent has to respond. The intention is to get enough data to be able to construct a minimal model for the perception (PERC), for the evaluating emotions (EMO), for the supporting memory (MEM), as well as for the acting responses (RESP).

• In another scenario the agent as a human person is confronted with the interface of a technical device which functions as the environment function.

• In a third scenario the agent is an artificial agent with an internal structure possessing PERC, EMO, MEM, and RESP. This artificial agent is exposed to the interface of an environment. The environment can be a test environment as in the case of the human person, a technical device or some other environment.

Figure 4.4: Some Psychological Experimental Paradigms