Research for Usability Testing Video

Krug (2006) gives the following advice for facilitating the test subjects:

• Don’t give them hints about what to do
• Use simple instructions - Look around the page and tell me what you think
  everything is and what you would be likely to click on.” “Tell me what you would click on next and what you expect you would see then.” “Try to think out loud as much as possible.”
• explore the responses further without influencing the subject"when a user says, “I like this page” you always want to ask a leading question like “What do you like best about it?” If this produces “Well, I like the layout” then you need to follow with “What appeals to you about the layout?”
  You’re looking for specifics, not because the specifics themselves are necessarily
  important but because eliciting them is the only way you can be sure you
  understand what the user is really reacting to."
• improvise - move on if you have solved the problem with previous subjects
• take notes after each test session

Peyrichoux also (2007) suggests :

• you need to probe deeper into user's statements - ask why?
• make objective observations
• have users identify problems, not solutions
• have users relate to their own experiences

Make sure you are NOT leading your test subject to the answer when writing questions (University of Texas, 2006). http://www.utexas.edu/learn/usability/test.html The university of Texas suggests you give each subject a waiver to sign. Plan to ask entrance questions, task based questions, and exit questions. For each of my test sites I will use the following exit questions based on suggestions from Krug(2006 p.148-155) and the University of Texas site:

Entrance questionnaire

1. Name:
2. Type: Teacher - Student - Parent
3. Grade Level: Elementary Middle School High School College
4. Years Using the Web:
5. Age Group: 18-24 25-34 35-44 45-55 over 55
6. Gender: Female Male
7. How often do you use the internet? Daily Weekly Monthly Occasionally Never
8. What do you usually do on the internet? (e.g., email, use reference materials such as encyclopedias and dictionaries, read news, curriculum activities, games, entertainment etc.)

Web site Feedback (Teachnology in this example)

1. What features of the Teachnology web site were vague or confusing to you, if any?
2. What is your impression about navigating the site? Does it seem easy or difficult? What makes it that way?
3. What else should be included on the Teachnology web site?
4. What did you like best about the site?
5. What did you like the least?
6. Do you think some people would have problems using the Teachnology web site? What kinds of people? What kinds of problems?
7. Would you like to make any other comments about Teachnology?

Task based questions

According to Nielson "it only takes five users to uncover 80 percent of high-level usability problems"(as cited in Gordon, 2000, para. 1). However Faulkner (2004) suggests more subjects are needed. Gordon(2000) recommends using test subjects who could potentially be users of the site. Use repondents of varying levels of internet proficiency, age, and gender. Make the test environment casual and comfortable. I will keep questions open-ended and neutral eg. “What do you mean by that?” or “What did you expect to happen?”

When users identify a problem, we will ask them how they would fix it. I will observe body language and facial expressions as well.

1. Describe the first items you notice on the page
2. Identify which elements on the page are clickable
3. What do you expect to find behind this link?
4. Please describe your experience when trying to complete x task

Gordon(2000) also talks about eye tracking in regards to usability:

1. Users spend a good deal of time initially looking at the top left of the page and upper portion of the page before moving down and right-ward.
2. Normal initial eye movement around the page focuses on the upper left portion of the screen
3. Ads perform better in the left hand column over the right column of a page
4. Smaller type encourages focused viewing behavior
5. Larger type promotes lighter scanning
6. Dominant headlines most often draw the eye first upon entering the page- especially upper left of the page
7. Users only look at a sub headline if it engages them
8. Navigation placed at the top of a homepage performed best
9. People's eyes typically scan lower portions of a page seeking something to grab their attention.
10. Shorter paragraphs performed better than longer ones
11. The standard one-column format performed better in terms of number of eye fixations
12. Ads in the top and left portions of a homepage received the most eye fixations
13. Text ads were viewed mostly intently of all types tested
14. Bigger ads had a better chance of being seen
15. The bigger the image, the more time people took to look at it
16. Clean, clear faces in images attract more eye fixations on homepages
17. New, unfamiliar, conceptual information was more accurately recalled when participants received it in a multimedia graphic format

Faulkner, L. (2004) Beyond the five-user assumption: Benefits of increased sample sizes in usability testing. Retrieved April 14, 2008 from http://www.geocities.com/faulknerusability/Faulkner_BRMIC_Vol35.pdf

Gordon, S. (2000) User testing: setting up the test Retrieved April, 14 2008 fromhttp://articles.techrepublic.com.com/5100-22-5077374.html

Krug, S. (2006) Don't make me think 2nd edition New Riders Publishing

Peyrichoux, I. (2007) When observing users is not enough: 10 guidelines for getting more out of users’ verbal comments Retrieved April 14, 2007 from http://www.uxmatters.com/MT/archives/000183.php

Spiller, F (2008) Demystifying usability. Warning: third - party usability is bad for your health Retrieved April 14, 2008 from http://experiencedynamics.blogs.com/

University of Texas(2006) Develop the Usability Test Documents Retrieved April 14, 2008 from http://www.utexas.edu/learn/usability/test.html