Kitty's+User+Testing+Analysis

KBD Index | PACT Analysis | Scope | P1 Envision | Moodboard | P2 Process | Discussion | Resources | P3 Process | P3 User Testing

=Kitty's DECIDE FRAMEWORK AND USER TESTING= 3005 DESIGN & SYSTEMS BRITTANY, DASHA AND KITTY

=DECIDE=

** OVERALL GOALS **
We want to find out if our website is use-friendly, and helpful to our scope (undergraduate students and faculty members). Through user-testing, we want to get a better understand of the changes that need to be made in order to improve the website to better cater to the overall user experience.

EXPLORING QUESTIONS
Does the site reflect the nature of the Visual Arts Department at York? Will students see this site as their main resource for information on Visual Arts at York? How easily can users access information on the site? Do students understand the sliding image banner and its navigation? How effective is the visual identity?

EVALUATION PARADIGM AND TECHNIQUES
The evaluation will be conducted using a semi-structured approach. The participant will be asked to read a two part scenerio and complete specified tasks on their own. Meanwhile, the screen will be recorded for future references and any comments made during the explorating will be noted. They will also be asked for feedback after their exploration through general questions about the visual and contextual elements of the site as well as specific questions catered toward the specified tasks completed. This will help us answer bigger question as well as give the user an opportunity to make comments to further help us improve our prototype.

PRACTICAL ISSUES TO BE ADDRESSED
The testing environment will be anywhere with a device that can access the internet. We will need to approach students within our target range (current students) and inform them of their rights before conducting our research under confidentiality. For my particular test, students do not necessarily need to be visual arts students. For this research users will be conducting the test on our own personal laptop. We may also want to test it on palm devices like blackberrys and iPhones to see any accessibility problems that may arise.

ETHICAL ISSUES
Ethical issues will include respecting the privacy of the users and thoroughly informing them before conducting the test. We will present them with a consent form in which the user will sign only when they have fully agreed and understood the conditions, purpose, and implications of the experiment, under no pressure by us. We will also acknowledge that they can choose to quite the exercise at any point they wish without any discrimination. Also, we will need to be aware of the things said to the user during the testing, so not to obscure the results and avoid bias.

DATA EVALUATION
Analysis and interpret the collected data to identify patterns or themes that emerge

PROJECT BRIEF
We are third-year Graphic Design students at York University currently developing a website prototype for the York visual arts department. The purpose of our research is to develop a better understanding of the user experience for this site. To gain this information we will be asking you to navigate through our website prototype and complete specific tasks to help us identify any potential issues. We are testing our prototype to determine whether the content and navigation are user-friendly and easily accessible. We will also be asking you a series of questions related to your experience both contextually and visually. In no way are we testing your abilities but rather testing our website and looking for ways to improve its capabilities.

TEST SCRIPT

 * a)** You are a 3rd year Design student finishing up your summer term. Today is your Fall/Winter enrollment date and you are picking out your courses for your last year. Spaces fill up quick so you need to find your courses and enroll ASAP! You acknowledge that you still need 6.0 more credits for your fine arts electives in order to graduate on time. In the previous years, you took Drawing: Perception, Proportion & Structure 2081 (3.0), and Drawing: Approaches to Abstract Image Making 2082 (3.0) and really enjoyed it. So you are hoping to enrol in a more advanced drawing course to fill your degree requirement. Look for a drawing course that interests you and write down the course number.


 * b)** As you go to add the course, the system shows up an error message that displays: “This course cannot be added. For further information please contact the visual arts department.” You REALLY want this course and must now find this information ASAP before the spots are taken.

General Interview Questions (Contextual and Visual)

 * what is the identity/mood of the site?
 * how is the quality of our images?
 * how would you rate our site in terms of being visually appealing out of 5?
 * is background working for the overall look?
 * what would you like to see more of / less of?
 * did you find the information easy to read?
 * how do you find the hotlinks toolbar at the bottom? confusing/useful?
 * what did you like least about this site?
 * what do you like most?

Specific Interview Questions

 * did you find the course descriptions easy to read?
 * did you get the information you needed quickly?
 * was the information laid out in a clear and easy to read manner?
 * if you were in this situation would you use this VA site as your primary source to get information?
 * did you find any part of your experience on the site confusing?
 * comments on how to improve?
 * would you think of using the image bar to navigate to the drawing page?

__User 1__
This user completed both tasks with ease. For Part A, the path taken was straight forward and intuitive without any hesitation: Students>Undergraduate>Courses>Drawing. For Part B, the user intuitively went under 'About' but then hesitated between the sub-navigation, specifically the 'Locate Us' and 'Directory' links. While looking under the About tab, she narrates: " I guess I'll see what's in Locate Us" only to find it is not a working link. She then moves onto 'Directory' and once on that page, the information was readily accessible and straight forward.

POSITIVE FEEDBACK:
 * The information was laid out very well, concise and clear, and the font used is “clear, attractive, serious but pretty.”
 * Navigating to the courses pages was very straight forward and would definitely use this site for this type of information over the York site.
 * The mood and background images are fitting of the Visual Arts Department, recieving a 5/5 for visual appeal
 * Like the main image scroller the most because it showcases all the different areas.

NEEDS IMPROVEMENT:
 * There should be a link to YorkU main site
 * Instead of ‘Directoy’user looked specifically for a ‘Contact Us’page
 * Suggested that there be shorter body text/descriptions
 * Suggested different picture for photography in the main image bar

**__User 2__**
This user had much difficulty determining where to navigate within the main navigation. She looked under every other heading before finding the right path under ‘Students’. There were a couple of instances that she hovered over Students and moved on to another heading. Her initial reaction was to go under ‘Admissions>Programs’ but quickly found that they were not working links. Then she went under ‘About’ and also found nothing. She went back and forth the main headings and was almost stumped before realizing that she should look under ‘Undergraduate’. When she realized that she exclaimed, “I should have looked under here in the first place, that’s what I would have normally done”. After clicking on Undergraduate, she found the ‘Courses’ link on the side navigation quickly and the process was smooth and easy from there. For Part B, this user also had trouble locating the contact information because she intuitively looked for a ‘Contact Us’ link. When she couldn’t find it at the top, she scrolled to the bottom and expected the information to be at the bottom. She was looking in particular for a line of text but as it was not there, she went back to the top and then back down to try the hotlinks. She hovered over the ‘Directory’ icon and said “I think I could find it in here” and clicked on it. Once on that page, information was straight forward.

POSITIVE FEEDBACK:
 * overall very good legibility and descriptions were very well laid out, easy to read.
 * said that our with is “way easier to get information” compared to the current site.
 * described site as “warm and playful and very expressive”.
 * quality of images were very good and the backgrounds were thought to really add to the site.
 * overall rating of ~5/5 in terms of visual appeal.
 * really liked the main image bar/slider

NEEDS IMPROVEMENT:
 * ** main navigation bar is too sensitive and moves too much when trying to access the sub-heads. It was described as “frustrating”. **
 * ** needs a home button at the top of side of the page, and not just at the bottom. This user thought that ‘About’ served as the ‘Homepage’. **
 * ** did not like how there was no “Contact Us” page and thinks there should be contact information at the bottom of every page in text. **
 * ** the hotlinks toolbar was not very noticeable and was not very useful to this user. **

__User 3__
This user completed both tasks with ease. Like User 01, this user took a very straight forward and intuitive path to the drawing courses page without any hesitation: Students>Undergraduate>Courses>Drawing. For Part B, the user intuitively went under ‘About’ but then hesitated between the sub-navigation, specifically the ‘Locate Us’ and ‘Directory’ links. She decided not to try any of those links and moved to the Hotlinks toolbar at the bottom. Starting at the ‘Home’ link, she hovered over every icon until she reached ‘Directory’ and says “ I think I could go to Directory for this information”. She finds the infomation easily after navigating onto the Directory page.

POSITIVE FEEDBACK:
 * ** course descriptions were very legible and especially enjoyed the indication of credits right within the description. **
 * ** information was easy to get to and the process was very quick. **
 * ** thought this site to be a quicker resource over the YorkU site. **
 * ** overall look is “colorful, vibrant, and very catchy” and liked how it showcases various student works as the background and within the image bar. **
 * ** rated the site a 4.5/5 for visual appeal and thought the image quality to be perfect. **
 * ** favorite aspects of the site included its straight-forwardness, the image slider, and the easy to read navigation system. **

NEEDS IMPROVEMENT:
 * main navigation menu is too sensitive and should maybe drop down verticle versus horizontal to give it more hover space.
 * did not like how the main navigation menu goes on top of the VA logo.
 * was not use to ‘Directory’ when looking for contact information.
 * thought some backgrounds to be a bit too busy, in particular the ones with specific imagery like the goldfish. She believed the simpler backgrounds were less imposing.
 * would like to see more variety in images, perhaps even a gallery tab directly as an option within the main menu and definitely images of the facilities, buildings etc.
 * didn’t like how the Hotlinks toolbar sat right at the bottom and thought there should be a bigger space between the end of the browser and the Hotlinks.
 * Hotlinks icons don’t really suit the overall look of the site and should maybe be just words, or have both words and redesigned icons.
 * Logo shuld be clickable and bring user back to the home page.