jump to navigation

Preece et al Chapter 12 & 14 June 11, 2009

Posted by toast000 in Uncategorized.

Question Words of Evaluation
The need for evaluation can be answered in

  1. Why?
    Evaluation is necessary to see if a certain design or product is liked or even usable by users. In the business world, evaluation can help designers to improve a product before it is fully released.
  2. What?
    What to evaluate is completely governed by the problem space. It can really be anything from websites to toys to traffic controls.
  3. Where?
    Similar to the “What,” the where to evaluate is dependent on the problem space. Some situations need to be evaluated in a structured, controlled lab environment or a more natural environment. The text uses the example of a web page being better evaluated in a lab setting, while a toy or a phone would be better in a natural setting.
  4. When?
    Once again, when to evaluate depends on “What.” However, there are two general times to evaluate. Summative evaluations are done after the design process and formative evaluations are done during the design process.

Usability Testing

Usability testing is completely controlled by whatever person is performing the evaluation. It is normally used to check for consistency in a product (usually software). Commonly, a user is observed closely and the time it takes to complete a certain task is recorded (quantitative data collection). In addition to performing tasks, users may fill out satisfaction questionnaires or be interviewed for more information (qualitative data collection). Usability testing is especially useful for products that have multiple generations (word processors, etc.). Users are often recorded on camera and microphone so that more than just timing information can be collected.

Usability labs can range in size from an entire facility to the contents of a suitcase. For a real life example, one can be found in Informatics West. Some testers have begun to use remote testing methods as well, eliminating the need for a central usability lab.The final usability section covers how to conduct a usability test and basically equates to an excerpt from a statistics course book. It covers variable choice, control experiments, null and alternative hypotheses, participant designs, and t-tests.

Field Studies

Field studies are more often observed outside the lab setting and are less controlled. The data collected can be used for any number of analyses. Data can also come in many forms (notes, video, interviews, etc) both from the observer and the participant. The amount of time it takes to conduct a field study can be anywhere from a couple of minutes to years. While on site, observers must take into consideration the system or product breaking down and whether or not it can be easily repaired. The results of field studies can be written up in a number of ways, including narratives and vignettes.



1. kh2011 - June 15, 2009

This is a good and concise blog post. I think you covered the important parts well and I especially liked your emphasis on the importance of the problem space. I see understanding and evaluating the problem space as the most important process when attempting to design for any user group. Not understanding the problem space or not taking enough time to evaluate it before jumping into the design process can ruin the entire design.

2. Will - June 15, 2009


You did a great job of summarizing. You also did a great job of describing the difference between field studies and usability tests and why we would want to choose either of them.

I would like for you to engage with the readings a little more reflectively though. I think you have a good start here with the beginning, and you could have used that as a jumping off point for why we should be concerned with usability at all (not just in the the why sense described in the book, but really what is the significance, what is the world like without evaluation, what does it lead to). Also challenge what you read, is it really true?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: