Yvonne's+User+Test

Main Page | Ideas | System Map | Benchmark Analysis | Program Info | Prototypes | User Testing | Refined Prototype

=P3 User Test= flat

//Determine overall goals of the evaluation//
The purpose of this test is to see how the user comprehends the website, and how the placement of links and navigation can be improved to create an intuitive way-finding system for the website. The test specifically requires the user to find the requirements needed to apply for an Independent Studies credit for Practicum and Art History. This evaluation is important because students need to know how to navigate the Course Information site for courses that are available to them. The Independent Studies credit is especially important to be easy to find, as it is a unique course that is not open to all students, and it has a deadline for applications.

//Actionable Questions//

 * Do users understand the categories of the main navigation?
 * Are users able to make the connections between main navigation and content?
 * Is the information located too "deep" and difficult to locate?
 * Are the links placed in easily accessible places?
 * Does the information for course descriptions provide enough detail?

//Evaluation Paradigm and Technique//
Usability testing is the main method of evaluation as the site is set up with a predetermined path in the prototype. However, there is a level of unstructured exploration in this test because there is no specific link or quick link to the Independent Studies section. Users will have to understand the organization of the website categories to find this particular piece of information. Testers will also be asked to find the information without instruction and to narrate their actions and thought processes as they navigate the website.

//Identify the practical issues that must be addressed//
The tests will be conducted in a typical environment for students browsing websites, i.e. at home, school, or outside location with wireless internet. Specific issues for each environment may alter the test results such as a distracting and loud background resulting in lowered concentration, desktop computers with big screens resulting in a larger browser than anticipated, and poor wireless internet connections resulting in broken links.

Participants will be from 2 different categories: non-Visual Arts major from York University and outside student from a different school. Users will need to be unfamiliar with art terminology but used to navigating through school websites, so that navigation categories and sidebars can be effectively judged by their description. This will provide insight in catering to different users and levels of familiarity with Visual Arts to better usability for prospective students.

//Decide how to deal with the ethical issues//
Before the test is conducted, users will be informed about the purpose and methodology of the study. They will also be told that their participation is completely voluntary and have the ability to walk away from the test at any point should they feel uncomfortable. They will have the option to keep their identity and name confidential if they feel uncomfortable about having their names published. To ensure that users know their rights, they will be asked to sign a consent form and be informed about their voluntary status immediately prior to the test.

//Data Analysis//
Data will collected in the form of observational notes and interview responses. After the tests are conducted, the data will be evaluated and interpreted as a research group to find patterns and similar themes from the responses.

Project Brief
We are researchers from the Bachelor of Design (Honours) Program at York University, and we are working on developing a new Visual Arts Website for current students and prospective students. The main scope of this website is to help current students better organize their course schedules, provide easy access to course information, and to provide a more dynamic and visual way of comprehending the courses that are available to students. This user study is conducted to better our understanding of our target users. We are going to show you the screen and ask you to perform some tasks, we are checking our work to see if it can actually help you. In no way are we testing your abilities but rather testing our website and looking for ways to improve it’s capabilities.


 * Other Questions**
 * 1) What do you look first in these pages?
 * 2) Do you feel that the links are straightforward and clear enough?
 * 3) How appropriate are the quick links? Do you think that they show the most important information?
 * 4) Do you find the website visually appealing?
 * 5) What do you think can be improved on?

Test Script
“You are a second year Visual Arts student and you have heard about an Independent Studies Course for Practicum or Art History available at York University. You are interested but have heard that there are specific requirements needed to apply to the course. Now, go to the computer and start looking for the requirements or restrictions that apply to the Independent Studies Course for Practicum or Art History.”

=Results=

User #1
Profile: Non-Visual Arts major at York University; Age 18-25

**//Overall Evaluation Goals//** //The purpose of this test is to see how the user comprehends the website, and how the placement of links and navigation can be improved to create an intuitive way-finding system for the website. The test specifically requires the user to find the requirements needed to apply for an Independent Studies credit for Practicum and Art History. This evaluation is important because students need to know how to navigate the Course Information site for courses that are available to them. The Independent Studies credit is especially important to be easy to find, as it is a unique course that is not open to all students, and it has a deadline for applications.// //“You are a second year Visual Arts student and you have heard about an Independent Studies Course for Practicum or Art History available at York University. You are interested but have heard that there are specific requirements needed to apply to the course. Now, go to the computer and start looking for the requirements or restrictions that apply to the Independent Studies Course for Practicum or Art History.”// //The main questions this test addresses include:// //1.// //Do users understand the categories of the main navigation?// //2.// //Are the links placed in easily accessible places?// //3.// //Does the information for course descriptions provide enough detail?//
 * //Test Script//**
 * //Specific Questions//**

User Path

 * 1) Clicked "Undergraduate" in the global navigation sub-menu.
 * 2) Arrived at the Undergraduate homepage
 * 3) Selected Course Information
 * 4) Select link for Independent Studies and found information very quickly

[Shows that she has become familiar with the general system navigation]

Issues

 * None. Very fast and efficient
 * Notes that the quicklinks in the sidebar make it easy to navigate through the pages

Test Summary
**What do you look at first in these pages?** **Do you feel that the links are straightforward and clear enough?** **How appropriate are the quick links? Do you think these are the most important information?** **Do you find the website visually appealing?** **What do you think can be improved upon?**
 * Looks first at the Tabs and the categories, how are things organized? And what do I fit under?
 * Colours/Identity systems—must indicate some form of hierarchy/organization
 * Mouse over links—shows that certain pages are linked and can be clicked
 * Descriptions—short and concise for easy browsing
 * Categories—Easy to understand and organize
 * “Art Lingo” and Terminology—Not a Visual Arts student so certain terms are confusing ie. Time-Based. Not sure if it is a course name or a description. Could possibly explain these terms better in the homepage
 * Header in Course Maps—FA/VISA description is confusing and too similar, needs distinguishing
 * VISA (logo)—Confused about what VISA stands for and did not associate it with the home page. [possibly because the homepage does not state itself as “VISA” or a close variation of that.
 * Most of the links are relevant, although the student gallery is questionable. As a non-visual arts student, she would not call the student gallery an important feature of the website
 * Would like more rollover differentiation ie. Underline or colour change
 * Likes the visuals and concept
 * Good use of photography but the image on the homepage shows a broken lightbulb! Distracting
 * Colour system is very nice and shows differentiation
 * Likes the concise and summarized nature of the pages, easy to browse and find information
 * Links that don’t work
 * The “—more>>” labeling, finds it unattractive and unnecessary
 * VISA logo and homepage link
 * Background--wants the opacity of the black box to be increased because it makes the image distracting

=Results=

User #2
Profile: Non-York University student; Age 18-25

**//Overall Evaluation Goals//** //The purpose of this test is to see how the user comprehends the website, and how the placement of links and navigation can be improved to create an intuitive way-finding system for the website. The test specifically requires the user to find the requirements needed to apply for an Independent Studies credit for Practicum and Art History. This evaluation is important because students need to know how to navigate the Course Information site for courses that are available to them. The Independent Studies credit is especially important to be easy to find, as it is a unique course that is not open to all students, and it has a deadline for applications.// //“You are a second year Visual Arts student and you have heard about an Independent Studies Course for Practicum or Art History available at York University. You are interested but have heard that there are specific requirements needed to apply to the course. Now, go to the computer and start looking for the requirements or restrictions that apply to the Independent Studies Course for Practicum or Art History.”// //The main questions this test addresses include:// //1.// //Do users understand the categories of the main navigation?// //2.// //Are the links placed in easily accessible places?// //3.// //Does the information for course descriptions provide enough detail?//
 * //Test Script//**
 * //Specific Questions//**

**User Path**

 * 1) Go to global navigation, drop down menu for Undergraduates
 * 2) Arrive at Undergraduate homepage and select "Course Information"
 * 3) Select link for Independent Studies and finds information

Issues

 * Wants more information about Independent Studies ie. What is it? Other opinions?
 * Tries to click all bolded text, perhaps needs more differentiation for rollover links

Test Summary
**What do you look at first in these pages?**
 * Images and links—concise description of what’s available in this site
 * Scrolls through the tabs quickly to find out the general organization of information
 * Likes to click around a lot, and is frustrated by the lack of working links
 * Do you feel that the links are straightforward and clear enough?**

//[Note: User did not realize that the actual headings of the global navigation could be clicked until after completing the tests and the tester told her that there were individual pages for “About”, “Students”, “People”, “Facilities”, and “Resources”.]// **How appropriate are the quick links? Do you think these are the most important information?** **Do you find the website visually appealing?** **What do you think can be improved upon?**
 * The quicklinks and subheadings are easy to understand
 * Confused by the fact that the headings could be clicked because generally they lead to nowhere on other websites. Suggests a different form of rollover to indicate the link, ie. Underlining or Colour change.
 * Links are relevant
 * Likes the use of colours and photography
 * Appreciates the attention to detail and concise summary of each category
 * Good global navigation other than the fact that headers could be clicked
 * Thinks that the photographs on each “main site” are appropriate to the topic, but thinks the photo for students is blurry
 * VISA logo—Did not understand what it stood for and did not realize it was a logo
 * Centering of the background—Finds it odd that when window is expanded, the black box is no longer centered
 * Spacing and alignment of some of the quicklinks are off
 * Labeling of the course map—wants more description
 * Images in the course map should correspond to course
 * Text size is too small

=Results=

User #3
Profile: Design Major, York University student, Age 20

**//Overall Evaluation Goals//** //The purpose of this test is to see how the user comprehends the website, and how the placement of links and navigation can be improved to create an intuitive way-finding system for the website. The test specifically requires the user to find the requirements needed to apply for an Independent Studies credit for Practicum and Art History. This evaluation is important because students need to know how to navigate the Course Information site for courses that are available to them. The Independent Studies credit is especially important to be easy to find, as it is a unique course that is not open to all students, and it has a deadline for applications.// //“You are a second year Visual Arts student and you have heard about an Independent Studies Course for Practicum or Art History available at York University. You are interested but have heard that there are specific requirements needed to apply to the course. Now, go to the computer and start looking for the requirements or restrictions that apply to the Independent Studies Course for Practicum or Art History.”// //The main questions this test addresses include:// //1.// //Do users understand the categories of the main navigation?// //2.// //Are the links placed in easily accessible places?// //3.// //Does the information for course descriptions provide enough detail?//
 * //Test Script//**
 * //Specific Questions//**

**User Path**

 * 1) Began at home page, clicked on Course Information in quick links
 * 2) Clicked Visual Arts Program & Options in the side bar
 * 3) Clicked on Handbook link in sidebar, empty page so went back
 * 4) Clicked on Course Map “The course names should be on the boxes.”
 * 5) Gave up, could not find the information

Issues

 * Fairly straight forward, route was specific
 * Could use more information in the description ie. Credits

Specific Questions

 * “People” is a little vague, try using another word
 * Coloured text make things look like a link but the bars make it look less like a link
 * There are no paragraphs to describe the courses, even though sometimes found to be useless and too vague

Test Summary
**What do you look at first in these pages?**
 * Tabs and Sidebars
 * Images: how do they fit in the page and how it relates
 * Do you feel that the links are straightforward and clear enough?**

//[Note: User did not realize that the actual headings of the global navigation could be clicked until after completing the tests and the tester told her that there were individual pages for “About”, “Students”, “People”, “Facilities”, and “Resources”.]//
 * Confused by the global navigation, did not realize they were links!
 * Side bars were useful but some labeling could be more descriptive ie. Course Map (what is it?)
 * How appropriate are the quick links? Do you think these are the most important information?**
 * Links are relevant
 * Do you find the website visually appealing?**
 * Love the colours!
 * Enjoys the use of photography, but thinks that some photos could be more relevant to subject of the page
 * Good global navigation other than the fact that headers could be clicked
 * What do you think can be improved upon?**
 * More organization in the Course Map ie. colour coding or icons
 * Spacing and alignment of text
 * More description in course information
 * Centre the black box when the window is expanded

=Analysis=

The user tests have raised some important issues that need to be dealt with in the next section of this project. Most of the issues deal with the location of content, labeling, and differentiating information. The test for finding the “Independent Studies” requirement showed that this information was easy to access and learn as the user familiarized themselves quickly with the navigation. However, through the “quick and dirty” method, I have observed certain issues that almost all users struggled with. The naming of the VISA logo as the homepage was confusing for many people, because it was not immediately obvious what “VISA” stood for and it did not have a colour differentiation from the global navigation. The course map was also confusing to many people because the images do not correspond to the subject and many of the images repeat in different variations. One thing that can be improved is the use of icons or colours to distinguish the different courses. Although it was not raised as a concern, many quick links were confusing because I found many of the users attempting to click at almost all the bolded text. This may be a result of the inconsistent rollover effects for links on various pages, sometimes it changes colour, and other times it has an underline. Finally, the last major issue raised by the tests dealt with the global navigation where some users did not realize that the first button could be clicked to bring them to a sub main menu for students. It was stated that, “most websites do not have main menus where those buttons work”.

If I were to continue the user tests, I would make sure that I inform the user that some links do not work immediately because it is a prototype. I would also go back and change the links so that it doesn’t go to an “error” page making the user think that there is a problem with the website. The tests have been very valuable in revealing logistical and navigational problems in the website that we might have missed or overlooked.