Some Standards

So far we have only discussed what has to be understood as 'measurement' generally. Now we want to illustrate this in the context of some standards. Usually such standards do not cover all aspects which are important for a development tool.

Here we will specifically address the issue of usability by the example of the standard DIN EN ISO 9241 part 11. This standard deals with the design of dialogue systems and mentions three primary goals to be targeted by a good design: effectiveness, efficiency, and user satisfaction. Taken alone, these terms have no clear meaning. To use them for real measurements one has to specify their operational meaning. For this we are referring to the NIST standard NIST-R CISU Common Industry Specification for Usability Requirements (CISU-R), NISTIR 7432 (June 2007)[114]. There you will find the following specific definitions:

Effectiveness:
Effectiveness relates the goals of using the product to the accuracy and completeness with which these goals can be achieved. Common measures of effectiveness include task completion rate, frequency of errors, frequency of assists to the participant from the testers. It does not take account of how the goals were achieved, only the extent to which they were achieved.
Efficiency:
Efficiency relates the level of effectiveness achieved to the quantity of resources expended. Efficiency is generally assessed by the mean time taken to achieve the task. Efficiency may also relate to other resources (e.g. total cost of usage). A common measure of efficiency is time on task, which can be defined as the mean time taken to complete each task, together with the range and standard deviation of times across participants.
Satisfaction:
Satisfaction describes a user's subjective response when using the product. User satisfaction may be related to whether a users wants use a product and may affect performance in some cases. Questionnaires to measure satisfaction and associated attitudes often use Likert or semantic differential scales. A variety of instruments are available for measuring user satisfaction of interactive products, and many organizations create their own. Some examples of standard questionnaires include: ASQ [93], CUSI [94], PSSUQ [94], QUIS [37], SUMI [82], and SUS [107]).

Gerd Doeben-Henisch 2012-12-14