Alex's+User+Test

Main Page | Ideas | System Map | Benchmark Analysis | Program Info | Prototypes | User Testing | Refined Prototype

=**P3 User Test**= flat

//Determine overall goals of the evaluation//
This test has been created to test the effectiveness of the sorting mechanism of the website, that has been created to help students create their course timetables each year. This test specifically concentrates on the long term degree planning using the site as it asks the user to find out what he or she might take if said individual wanted to eventually take 4090H, Advanced Explorations. This provides the user with the option of several methods to find this course and the information sought in the test.

//Actionable Questions//

 * Do users understand how to use the sorting method made available?
 * Does this sorting method make it more efficient for users to find information?
 * Do users think they would use this site to plan their degrees?

//Evaluation Paradigm and Technique//
Usability testing will be the method main used in this test, as the site is set up in such a way that there are predetermined paths in the prototype. The loose structure will allow participants experience the site as though it were simply exploration as the task is not so strict that needs to be done in one particular way, but there are many ways to find the information. By giving a loose overall goal, the participants have options, but are also provided with a main end point to work towards.

//Identify the practical issues that must be addressed//
Ideally the tests would take place in the location where a student use the website. This could really be in any location with Internet access. Common places would include: the home, like a computer room, the school library, or school labs. The only required item would be a computer with Internet access.

Our site prototype concentrates on current undergraduate students and their needs in terms of course information. It is not so specific that future students actually need to be disregarded, however. Ideally the majority of our participants would be current students, but not necessarily in the Visual Arts program as other York students have the potential to take Visual Arts courses. We are also considering other test subjects, such as those from older age groups, so we can get feedback from their perspectives. A finished VISA site would not just be for students in the common age group of 18-22, so it is advantageous to complete such testing earlier on in the prototyping process.

//Decide how to deal with the ethical issues//
Issues that might arise in this testing process would include issues of confidentiality. Our consent forms outline the intentions of the tests and note that the user can stop at any time and will be anonymous in all results. As testers, we will protect the results and treat the participants’ information will due care, preventing any personal information from becoming publicly available. We will also protect that trust placed in us by stating our results in way that is mindful of the participants’ rights and feelings.

//Data Analysis//
We will take care to observe the results to determine the effectiveness of all aspects of the prototype. The analysis of data will help us locate problem areas of the site and determine how we might improve the overall user experience. We hope to run tests many times to make sure that results and subsequent decisions are made with greater emphasis on improving the site for the target audience.

**Project Brief**
We are researchers from the Bachelor of Design (Honours) Program at York University, and we are working on developing a new Visual Arts Website for current students and prospective students. The main scope of this website is to help current students better organize their course schedules, provide easy access to course information, and to provide a more dynamic and visual way of comprehending the courses that are available to students. This user study is conducted to better our understanding of our target users. We are going to show you the screen and ask you to perform some tasks, we are checking our work to see if it can actually help you. In no way are we testing your abilities but rather testing our website and looking for ways to improve it’s capabilities.


 * Other Questions**
 * 1) What do you look first in these pages?
 * 2) Do you feel that the links are straightforward and clear enough?
 * 3) How appropriate are the quick links? Do you think that they show the most important information?
 * 4) Do you find the website visually appealing?
 * 5) What do you think can be improved on?

**Test Script**
“You are in your second year in the Visual Arts Degree Program at York University. Having adjusted to university life in your first year, you want to approach your second year’s choice of courses with the next few years in mind. You are interested in taking VISA 4090H, Advanced Explorations, in your fourth year. Using the new Visual Arts website, find the information to help you achieve this goal.”

=**Results**=

**User #1**
Mature Part-Time College Student

**//Overall Evaluation Goals//** //This test has been created to test the effectiveness of the sorting mechanism of the website, that has been created to help students create their course timetables each year. This test specifically concentrates on the long term degree planning using the site as it asks the user to find out what he or she might take if said individual wanted to eventually take 4090H, Advanced Explorations. This provides the user with the option of several methods to find this course and the information sought in the test.// //“You are in your second year in the Visual Arts Degree Program at York University. Having adjusted to university life in your first year, you want to approach your second year’s choice of courses with the next few years in mind. You are interested in taking VISA 4090H, Advanced Explorations, in your fourth year. Using the new Visual Arts website, find the information to help you achieve this goal.” // //The main questions this test addresses include:// // 1. Do users understand how to use the sorting method made available? 2. Does this sorting method make it more efficient for users to find information? 3. Do users think they would use this site to plan their degrees? //
 * //Test Script//**
 * //Specific Questions//**

**User Path**
 1. Uses Course Information link near the top of the main box on the index page 2. Uses Course Map menu's "Sort by Course" option on the top left 3. Scrolls down to 4090H and clicks to view information

**Issues**

 * Does not understand what VISA is
 * Not intuitive as to how to get back to home page
 * No link to contact department, did not understand what department
 * Small issues with information wording for prerequisites, (did get it correct, however)
 * Never notices main navigation to get to Undergraduates, even with further exploration

Test Summary
**What do you look at first in these pages?** >  **How appropriate are the quick links? Do you think these are the most important information?**
 * Top page titles like 'York University Visual Arts'
 * Background and Images
 * Main quick links at top of page
 * Do you feel that the links are straightforward and clear enough?**
 *  Links about financial issues and payments?
 * Used main navigation links to get to main pages
 * Did not use heading links; never found undergraduates page
 * Would there be a map of school on student page?
 * Found that things were 'where they should be'
 * Went through all links, reading aloud, and found them suitable
 * Was interested in whether there would be information on housing and residence at York
 * Never used quick links below the images, only on tops of pages
 * Do you find the website visually appealing?**
 * Liked the images
 * Found it site welcoming
 * Found text easy to read
 * Liked visual organization of main pages
 * What do you think can be improved upon?**
 * VISA logo—Did not understand what it stood for and did not realize it was a logo
 * Undergraduates main page confusing
 * Visual Program & Options not obvious or easily accessible
 * Course Map sort not really applied, but sorting not always required to find info

=**Results**=

**User #2**
News Website Editor

//**Overall Evaluation Goals**// //This test has been created to test the effectiveness of the sorting mechanism of the website, that has been created to help students create their course timetables each year. This test specifically concentrates on the long term degree planning using the site as it asks the user to find out what he or she might take if said individual wanted to eventually take 4090H, Advanced Explorations. This provides the user with the option of several methods to find this course and the information sought in the test.// //**Test Script**// //“You are in your second year in the Visual Arts Degree Program at York University. Having adjusted to university life in your first year, you want to approach your second year’s choice of courses with the next few years in mind. You are interested in taking VISA 4090H, Advanced Explorations, in your fourth year. Using the new Visual Arts website, find the information to help you achieve this goal.”// //**Specific Questions** The main questions this test addresses include:// //1. Do users understand how to use the sorting method made available? 2. Does this sorting method make it more efficient for users to find information? 3. Do users think they would use this site to plan their degrees?//

**User Path**

 * 1) Looks for 'Courses' or 'Curriculum'
 * 2) Checks in main navigation
 * 3) <span style="font-family: Helvetica,helvetica,sans-serif;">Goes to Course Information from quick links at top of black box
 * 4) <span style="font-family: Helvetica,helvetica,sans-serif;">Goes right to Course Map, sorting by Level
 * 5) <span style="font-family: Helvetica,helvetica,sans-serif;">Sees Level 4000, but wants to scroll and look (also finds pictures interesting)
 * 6) <span style="font-family: Helvetica,helvetica,sans-serif;">Holds mouse over and sees information but HTML tags distracting
 * 7) <span style="font-family: Helvetica,helvetica,sans-serif;">Finds relevant information

**Issues**

 * Type a little small on main pages
 * Title/Subject of the site not that obvious
 * 'About' seems like the main page, home page link not obvious
 * Seems like main navigation is repeated in the top of the black box quick links
 * No course summaries available
 * Photo for the courses seem a little irrelevant, though visually appealing

**General Questions**
**What do you look at first in these pages?**
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Main navigation bar noticed first
 * Moves on to background and images
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Search bar could say 'site search' to indicate what it searches
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Likes section at top of black box with smaller links
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Facilities in main navigation a good section to have
 * Do you feel that the links are straightforward and clear enough?**

//[Note: User did not realize that the actual headings of the global navigation could be clicked until after completing the tests and the tester told her that there were individual pages for “About”, “Students”, “People”, “Facilities”, and “Resources”.]// <span style="font-family: Helvetica,helvetica,sans-serif;">
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Thinks that on People page, Administration more important than Alumni (could be 2nd column instead)
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Thought General FAQ could be at beginning in 'About' section
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Thought News and Events could be last on 'About' Page
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Did not know about main section pages (eg: Students, Facilities, etc)
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Title of 'Students' could be different; is a little broad (could be Programs? For Students?)
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">'Students' sounds like faculty is talking about the students
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Found 'People' and 'Facilities' very useful
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">'Visual Arts Program & Options' doesn't seem to be effective naming
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Some issues seeing 'Visual Arts Program & Options' teal links on white
 * How appropriate are the quick links? Do you think these are the most important information?**
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">finds them to be useful links; of interest to students
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Upcoming events could be a little repetitive with 'News and Events' already there on 'About' page
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Might put 'volunteering and jobs' on the home page
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Could have hyperlinks in text
 * Do you find the website visually appealing?**
 * Finds it neat and bright
 * Good global navigation other than the fact that headers could be clicked
 * Thinks that the photographs on each “main site” are appropriate to the topic, but thinks the photo for students is blurry
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Liked the teal on black
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Could move 'more --->' part, seems unecessary
 * What do you think can be improved upon?**
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Text seems small
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Didn't know what background was supposed to be
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Found layout odd when window was enlarged (maybe box should move?)
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;"> Found smaller images on each page a bit irrelevant, make the page unnecessarily busy because of type of shots
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">the consistency in size of the main page columns does not really provide hierarchy as to what is most important
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Did not understand VISA, or the logo
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Thought there could be a general explanation about the main pages and about programs
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Might be missing: FAQs on specific points and general overview of Visual Arts
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">What is the priority for each page?
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Thickness of coloured bars needs to be standardized on all pages
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Helps realize error in 'About' page navigation, some are different, needs to be rechecked
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Spelling of "Studies Course", needs to be "Studies Courses"
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Could cut image banners on "Text" pages in half, too big

=**Results**=

**User #3**
Retired Teacher

//**Overall Evaluation Goals**// //This test has been created to test the effectiveness of the sorting mechanism of the website, that has been created to help students create their course timetables each year. This test specifically concentrates on the long term degree planning using the site as it asks the user to find out what he or she might take if said individual wanted to eventually take 4090H, Advanced Explorations. This provides the user with the option of several methods to find this course and the information sought in the test.// //Test Script// <span style="font-family: Helvetica,helvetica,sans-serif;">//“You are in your second year in the Visual Arts Degree Program at York University. Having adjusted to university life in your first year, you want to approach your second year’s choice of courses with the next few years in mind. You are interested in taking VISA 4090H, Advanced Explorations, in your fourth year. Using the new Visual Arts website, find the information to help you achieve this goal.”// //**Specific Questions** The main questions this test addresses include:// 2. Does this sorting method make it more efficient for users to find information? 3. Do users think they would use this site to plan their degrees?// **
 * //1. Do users understand how to use the sorting method made available?

**User Path**

 * 1) <span style="font-family: Helvetica,helvetica,sans-serif;">Uses Course Information link at top of the black box
 * 2) <span style="font-family: Helvetica,helvetica,sans-serif;">Reads top of Course Information page
 * 3) <span style="font-family: Helvetica,helvetica,sans-serif;">Clicks on Photography to go further down the page
 * 4) <span style="font-family: Helvetica,helvetica,sans-serif;">Works out a prerequisite for all Photography courses, is now looking for wrong number
 * 5) <span style="font-family: Helvetica,helvetica,sans-serif;">Goes back to top to go to Course Map sorted by course
 * 6) <span style="font-family: Helvetica,helvetica,sans-serif;">Scrolls to find FA/VISA 2061
 * 7) <span style="font-family: Helvetica,helvetica,sans-serif;">Uses next buttons to look through
 * 8) <span style="font-family: Helvetica,helvetica,sans-serif;">Tries Availability sorting
 * 9) <span style="font-family: Helvetica,helvetica,sans-serif;">Goes to 3022B
 * 10) <span style="font-family: Helvetica,helvetica,sans-serif;">Prompted to look for 4090H
 * 11) <span style="font-family: Helvetica,helvetica,sans-serif;">Goes to Photography sorting
 * 12) <span style="font-family: Helvetica,helvetica,sans-serif;">Finds required information

**Issues**

 * Never finds Undergraduates page
 * Uses many non-functional links in process of finding it
 * Slightly distracted by text information on 'Course Information'
 * Does note that it was quite 'manageable', not frustrated at process
 * Not obvious as to how to get back to home page
 * VISA is not obviously explained
 * Uses back button to find all former pages (like home page)

**Test Summary**
**What do you look at first in these pages?** //[Note: User did not realize that the actual headings of the global navigation could be clicked until after completing the tests and the tester told her that there were individual pages for “About”, “Students”, “People”, “Facilities”, and “Resources”.]// > <span style="font-family: Helvetica,helvetica,sans-serif;"> =Analysis= There are certain main issues that are coming out of this user test. In general the tested area of the site seems to be working well. The sorting has not always been used, but not because it has not been relevant or functional, but because users are willing to explore that section. It is visually appealing in format and this is enough to distract from using that option, as participants largely do not have specific interests in finding courses accurately and quickly the same way current students might. There were some general issues with the images in the lightbox being irrelevant, but they do seem to add interest for viewers.
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Mentions first: would use General FAQ on About Page
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Noticed CTV photo first
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Thinks course info, student serves, events, are all key information one might look for
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Images stand out
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Deadlines nicely placed at top of pages
 * Do you feel that the links are straightforward and clear enough?**
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Could not find home page at any time
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Main pages hard to find
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Would want to see Events mainly, link could be more obvious
 * <span style="font-family: Helvetica,helvetica,sans-serif;"> Visual Arts Program & Options never found, had to be pointed out
 * How appropriate are the quick links? Do you think these are the most important information?**
 * Links provide good connections (naming-wise)
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Order or hierarchy not always what was looking for first (or what interested in)
 * Do you find the website visually appealing?**
 * <span style="font-family: Helvetica,helvetica,sans-serif;"> Likes images
 * <span style="font-family: Helvetica,helvetica,sans-serif;"> Layout organized
 * What do you think can be improved upon?**
 * <span style="font-family: Helvetica,helvetica,sans-serif; font-size: 12px; line-height: 18px;">Spent much time scrolling, not using quick links
 * <span style="font-family: Helvetica,helvetica,sans-serif;">Type could be bigger but readable
 * <span style="font-family: Helvetica,helvetica,sans-serif;">Didn't have too many problems
 * <span style="font-family: Helvetica,helvetica,sans-serif;">Some pictures not obviously relevant

For these individuals, the point of view of the ‘outsider’ is explored. The participants for this test were intentionally selected from groups other than the obvious group of VISA students to view the site from a different perspective. Each of these individuals had varying degrees of experience with websites and has not been an undergraduate for varying degrees of time. So far they have demonstrated issues with main parts of the site: the global navigation and the undergraduates and student sections.

The main visual identity for the website has created an issue. There has been some confusion as to the point of the site, and some issue with separating ideas of this site and the main York site. Users are not sure exactly what VISA is or even what Visual Arts at York is and why this site exists. General program overviews have also been notes as an issue. Though this information seems obvious to current York students, users who are not general users of York websites represent other users such as first-time users. These might even include an important group –future students. This makes the information presented to us by these individuals all the more important.

Though there is visual consistency and users have generally agreed that the website is visually appealing, there does need to be more attention paid to details. User #2 was much more specific as to how visual consistency and hierarchy need to be reviewed. For example, news items with images on each page might also be labeled right on the pictures as users were sometimes confused as to what they were. Though some users may only sense something wrong, it is key to pay attention to what is and isn’t articulated.

There was general frustration using the undergraduate and program information sections of the website and much of the main navigation goes unnoticed or unused while the quick links at the top of the black box are consistently noticed first. This brings up an important point: it is desirable to have the user explore the site to some extent. It seems that although information is found quickly, this is at the expense of finding much more information, some of this even more important to the experience of the website: such as the main pages for each section. Having quick information is not always as important as getting the right information and following a certain order that can reveal even more information users might not be aware of.