Steph's+User+Test

= = Main Page | Ideas | System Map | Benchmark Analysis | Program Info | Prototypes | User Testing | Refined Prototype

=P3 User Test=

//Determine overall goals of the evaluation//
To determine if a prospective student is able to find out how many credits is required to obtain a Bachelor of Arts Digital Media Program. To find credit requirements for a degree is very important because a student will need to know approximately how many credits to take per year. This will also allow them to keep count and to know whether they are able to graduate in the prescribed time period.

//Actionable Questions//

 * Will a non-student know whether or not to look into the “Undergraduates” section and not just “Admissions”
 * Are credit requirements considered top-level information or will they be overlooked by the user? Because it is located directly in the Undergraduates section, the user may be confused if they think it will be buried very deep within the site.
 * Is the information laid out clear enough in terms of hierarchy and typography?

//Evaluation Paradigm and Technique //
 * Find a test subject who is not a university/college student
 * Have them go to the VISA page
 * Is the information laid out clear enough in terms of hierarchy and typography?
 * One person will inform them of the instructions, subject will be asked to speak aloud their thoughts and observations
 * Investigator will record 1)the exact steps taken, and 2) any reactions or thoughts spoken aloud

A structured task will be used because the questions to be answered are very specific. A test subject who is not a university/college student is required because that way they will most likely not be experienced in school websites and we will know whether they will try to look in “Undergraduates” or “Admissions.” Having the person speak aloud will allow us to hear their thoughts and will provide insight to an outsider’s perception of the site and how they feel about the visuals, typography, and navigation.

//Identify the practical issues that must be addressed//
The tests will be conducted in a typical environment for students browsing websites, i.e. at home, school, or outside location with wireless internet. Specific issues for each environment may alter the test results such as a distracting and loud background resulting in lowered concentration, desktop computers with big screens resulting in a larger browser than anticipated, and poor wireless internet connections resulting in broken links.

Participants will be from 3 different categories: Visual Arts major from York University, any York University student, and an outside student from a different school. Users will need to be unfamiliar with art terminology but used to navigating through school websites, so that navigation categories and sidebars can be effectively judged by their description. This will provide insight in catering to different users and levels of familiarity with Visual Arts to better usability for prospective students.

//Decide how to deal with the ethical issues//
Before the test is conducted, users will be informed about the purpose and methodology of the study. They will also be told that their participation is completely voluntary and have the ability to walk away from the test at any point should they feel uncomfortable. They will have the option to keep their identity and name confidential if they feel uncomfortable about having their names published. To ensure that users know their rights, they will be asked to sign a consent form and be informed about their voluntary status immediately prior to the test.

//Data Analysis//
Data will collected in the form of observational notes and interview responses. After the tests are conducted, the data will be evaluated and interpreted as a research group to find patterns and similar themes from the responses.

**Overall Evaluation Goals**
To determine if a prospective student is able to find out how many credits is required to obtain a Bachelor of Arts Digital Media Program. To find credit requirements for a degree is very important because a student will need to know approximately how many credits to take per year. This will also allow them to keep count and to know whether they are able to graduate in the prescribed time period.

**Test Script**
//“ //You are a high school student looking into Visual Arts programs at York University. You heard about an “Arts Digital Media Program” and would like to know the number of credits required for the degree. Here is the York University Visual Arts website. Please find this information and voice out any of your thoughts and feelings through this process regarding any visuals or the ease at which you find things.”

**Specific Questions //The main questions this test addresses include:// **


 * Will a non-student know whether or not to look into the “Undergraduates” section and not just “Admissions”
 * Are credit requirements considered top-level information or will they be overlooked by the user? Because it is located directly in the Undergraduates section, the user may be confused if they think it will be buried very deep within the site.
 * Is the information laid out clear enough in terms of hierarchy and typography?

**User #1** Profile: Design Student; Age 20 1. Began at Course Information, clicked on Course Map at the side bar 2. Clicked on Studies at the side bar (led to a non-existent page), went back 3. Went to Students under main navigation, almost clicked on Admissions but went to Undergraduates instead 4. Clicked on Student Handbook in the side bar (led to a non-existent page), went back 5. Scrolled down to Bachelor of Arts Digital Media Program and finds information • User said that this information was not confused to be under Admissions because the quick links and blurbs in the Students main page clarifies the kind of information located in Admissions and Undergraduates. This makes descriptions and link names very important. • Credit requirements are moderately important because you may only be glancing at it quickly and not very often • The bars behind some of the links confused the user. Did not notice that they were also links and did not notices that there were words in them in the Students section. Suggested that they should be larger, and maybe take out repeating heading above the blurb. • <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">Mainly the quicklinks and the headlines • <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">Notices the matching background image after • <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">Yes, most of the links are clear and straightforward • <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 13.0pt;">Only issues were “People” being too ambiguous, and the blue bars in Students section not being obvious enough • <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">Very appropriate but asked about money matters such as bursaries and awards • <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">Yes, finds it visually pleasing • <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">Likes the white text on black (“makes it pop”) • <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">Add description to courses in the lightbox • <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 13.0pt;">Bars with links should be made more obvious • <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 13.0pt;">“People” in main navigation should be made clearer • <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 13.0pt;">Quick links can be rethought to include money matters • <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 13.0pt;">The pages: Course Information, Course Map, and Handbook may need renaming to make them clearer so as not to be confused with one another • <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 13.0pt;">Course names should be on the boxes, not necessarily the course code
 * <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 14.0pt;">User Path **
 * <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 14.0pt;">Issues **
 * <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 14.0pt;">General Questions **
 * <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">What do you look at first in these pages? **
 * <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">Do you feel that the links are straightforward and clear enough? **
 * <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">How appropriate are the quick links? Do you think these are the most important information? **
 * <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">Do you find the website visually appealing? **
 * <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">What do you think can be improved upon? **

<span style="font-family: Arial,helvetica,sans-serif; font-size: 11px; line-height: 33px;">**User #2 Profile: Visual Arts Student; Age 18-20 ** <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 14.0pt;">**User Path**
 * 1) Began at Course Map, went to Course Information from the side bar and clicked on Time Based Art
 * 2) Thought for a long time, looking at the For Students section in the side bar
 * 3) Clicked on Research and About in the main navigation but had to go back both times
 * 4) Clicked on Student in main navigation, went to Course Information under the Undergraduates quick links
 * 5) Clicked on Program Information & Options in the side bar, found the information

<span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 14.0pt;">**Issues** • Confused with Course Information and Program Information & Options pages as to what kind of information they provide • Text may be too small to read (especially for parents), but otherwise fine ** <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 14.0pt;">**General Questions** <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">**What do you look at first in these pages?** • Also looked at the quick links black boxes, user said they were confusing at first because so much information there but got used to it ** <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">**Do you feel that the links are straightforward and clear enough?** • Noticed a difference in how links are organized, needs time to get used to it • Confused in Course Information page by the 3 different side bars, only focused first on the first one • Likes how the format of the website goes from general to specific ** <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">**How appropriate are the quick links? Do you think these are the most important information?** <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">**Do you find the website visually appealing?** <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">**What do you think can be improved upon?** • More detail in descriptions and course information • Course map too busy and course names should be provided with course codes • Course map pictures are unrelated to the courses, can be confusing **
 * • Struggled to find the information because user was looking for a .pdf
 * • In the main page, looks at For Students section because she is a student
 * • Most links are clear and straightforward, but some can be confusing, such as “People”
 * • Did not know they were quick links **
 * • Yes, likes how they show the Visual Arts facilities and gives the feeling of York University **
 * • Organization of the links

<span style="font-family: Arial,helvetica,sans-serif; font-size: 11px; line-height: 33px;">**User #3** <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 13.0pt;">Profile: Grade 12 High school Student; Age 17 1. <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 13.0pt;">Began at Undergraduates page, clicked on Course Information in the side bar 2. <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 13.0pt;">Clicked on Course Map in the side bar 3. <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 13.0pt;">Went to Admissions under Students in the main navigation (led to a non-existent page), went back 4. <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 13.0pt;">Clicked on Students in the main navigation 5. <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 13.0pt;">Clicked on Undergraduates, found the information • <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 13.0pt;">It was not obvious enough that the information would be in the Undergraduates section as opposed to Admissions • <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 13.0pt;">Program credit requirements are important but might not necessarily be top-level information • <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 13.0pt;">Page names can be confusing • <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">Pink bars in home page and About page catches the user’s attention more than the main navigation • <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">Looks at quick links next and skims through it, which is useful • <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">Main navigation is straightforward but gets more abstract with the specific pages, for example Course Maps and Course Information • <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">Course Map was confused to be “site map” • <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">The quick links are good and very useful • <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">Yes, but the pink bars in the home page in combination with the background picture bothers the user alot • <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">Less ambiguity in page names • <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">Less white in the Course Information page <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 13.0pt;"> (and more information-based pages) • <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 13.0pt;">Would be nice if there was an option to go back a page easily (breadcrumbs) • <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 13.0pt;">Streams in Course Map should be alphabetized • <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 13.0pt;">Should have more information on one page instead of many different pages (liked the current site when shown to them because all the course information was in one place)
 * <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 14.0pt;">User Path **
 * <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 14.0pt;">Issues **
 * <span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 14.0pt;">General Questions **
 * <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">What do you look at first in these pages? **
 * <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">Do you feel that the links are straightforward and clear enough? **
 * <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">How appropriate are the quick links? Do you think these are the most important information? **
 * <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">Do you find the website visually appealing? **
 * <span style="font-family: Helvetica; font-size: 8.0pt; mso-bidi-font-family: Helvetica; mso-bidi-font-size: 12.0pt;">What do you think can be improved upon? **

Analysis
<span style="font-family: Arial; font-size: 8.0pt; mso-bidi-font-family: Arial; mso-bidi-font-size: 13.0pt;">This test revealed some very valuable information pertaining to the organization and naming of pages. All of the users could not immediately find the credit requirements for the Arts Digital Media Program. Instead of finding the information directly in the Undergraduates section, all users tried to look for the information buried somewhere deeper. This means that this may not be considered top-level information and placing this somewhere deeper within the site should be considered. As well, the naming of certain links were confusing to users, mainly Course Information, Course Map, Student Handbook, and Admissions. Something to improve on this may be to consolidate information and have it all in one page under a more general heading. Otherwise, the names may need to be rethought. An important realization was that Undergraduates and Program Information & Options are the same pages, yet they are inconsistently named in different pages. In the main navigation, it is simply called “Undergraduates.” However, in the sidebars within the Undergraduates pages, it is titled “Program Information & Options.” Although the users in the test didn’t directly raise this issue, it seems that it is a contributing factor to their confusion. The bars used for some of the links were also overlooked as not being links and in the instance of multiple sidebars, this caused the user to focus mainly on the first one and not look at the other ones. This test has definitely shown new perspectives on the prototype and has determined that naming links and placement of information is crucial to a user’s experience of a website.