Brittany's+User+Testing+Analysis

=** P3 USER TESTING **= 3005 DESIGN & SYSTEMS BRITTANY

DECIDE Framework
1. Determine overall goals of the evaluation

The goals of the evaluation of the visual arts website prototype is to improve the system based on results of several user tests. These tests will be performed to gauge the understandability and intuitiveness of various aspects of the site as observed through potential users. The main goal is to determine the weaknesses and strengths of navigation and content layout, through testing the users ability to locate specific content on the site. The purpose of the website is to provide easy access to important information and the user’s ability to navigate to that content is vital to its success. These evaluations are important to the development of a successful website, which is not only aesthetically pleasing but also user-friendly in its navigation and content.

2. Actionable Questions

Do users recognize the various levels of navigation and their different purposes? Are users able to make connections between navigation and the potential content?
 * (ie: main navigation, sub level navigation, third level navigation and quicklinks)

3. Evaluation paradigm and techniques

The evaluation will be conducted using a semi-structured approach. The test user with be presented with a hypothetical situation and given specific tasks to complete. The user will be free to move about the site without guidance from the tester as they attempt to complete the task. The test user will be observed as they navigate the site to find the required information (noting any exclamations, remarks, issues, etc.) and their screen will be recorded for further analysis. Additional questions will be asked of the user after they complete their tests. These questions will address the users experience of the specific test/task as well as their experience of the site as a whole.

4. Identify the practical issues that must be addressed

The testing will be conducted in an environment that reflects typical locations for browsing a website. This will include one test performed at home and one at school. These two environments are very different from one another, the first is often quiet and comfortable with limited presence of others, making user concentration easy. The second can be busy, noisy and confusing, possibly making a user distracted or pressed for time.

The equipment required for testing purposes is a computer (laptop or desktop) and an internet connection. Users will require a prior knowledge of computer usage and of using internet browsers. Users who are unfamiliar with these two things, will pose issues relating to the usability of the website.

Participants will be from two different categories. The first will be students in the visual arts program in second and/or third year. The second will be students who are in fine arts but not visual arts specifically. This testing pool will give insight from different types of users and indicate how that affects the usability of the website. All participant identities will remain confidential and undisclosed in any documentation or findings.

5. Decide how to deal with the ethical issues

Ethical issues that we may encounter while conducting the user testing include ensuring the confidentiality of all users and their right to disengage from testing at any point. To address this we will provide consent forms to be signed by the participant. Another issue may be the users comfort and honesty. To ensure both of these things throughout the testing process, the participant will be clearly informed that the tests are designed to expose the strengths and weaknesses of the website itself and not of the user.

6. Data Analysis

The data collected will be evaluated and interpreted based on identifiable patterns and emerging themes collected from the various forms of research, which includes: tester notes, recorded video, interview responses.

Project Brief
We are third-year Graphic Design students at York University currently developing a website prototype for the York visual arts department. The purpose of our research is to develop a better understanding of the user experience for this site. To gain this information we will be asking you to navigate through our website prototype and complete specific tasks to help us identify any potential issues. We are testing our prototype to determine whether the content and navigation are user-friendly and easily accessible. We will also be asking you a series of questions related to your experience both contextually and visually. In no way are we testing your abilities but rather testing our website and looking for ways to improve its capabilities.

Test Script
TESTING AREA: NAVIGATION / COURSES & FACULTY BIO CONTENT

SCRIPT: You are a current student in Visual Arts. It is the end of the Fall Term and you are looking for another practicum for the upcoming Winter Term. a) After hearing information about a specific course from a friend, you visit the site looking for more information on that course. You know the course code and name: FA/VISA 2034: Stone Carving. Find this course information on the website. b) After finding the course information, you are interested in enrolling in the course. Unfortunately, it is past the enrollment date for the Winter Term and the course is full. You must contact the professor of the course via email to request enrollment. Find this information.

GENERAL INTERVIEW QUESTIONS:
 * What is the identity/mood of the site?
 * How is the quality of our images?
 * How would you rate our site in terms of being visually appealing out of 5?
 * Is background working for the overall look?
 * What would you like to see more of / less of?
 * Did you find the information easy to read?

SPECIFIC INTERVIEW QUESTIONS:
 * Did you understand the main and sub navigation headings?
 * Students, Undergraduate, Courses, Faculty Bios, etc.
 * What were the obstacles you faced, if any, in finding a specific course?
 * Did you realize there was more than one way to navigate to Faculty Bios?
 * Main Navigation, Hotlinks, Course Description
 * Do you have any recommendations for the organization of content on:
 * Course Information pages?
 * Faculty Bio pages?

Results
TEST SUBJECT 01

Script Test Notes: After test subject 01 was given the test script and began to navigate the website they made general comments acknowledging the attractive nature of the site, "This is nice, very nice!". They quickly and easily navigated from the main navigation menu to Students > Undergraduate > Courses > Sculpture without attempting to click on any other links. They found the specified course very quickly and also noticed links to the teachers, to which they exclaimed, "It even links to the teachers, cool". After being given the second portion of the test, to find the course teacher's email, the test subject immediately clicked on the name of the professor that they noticed earlier at the bottom of the course description. They were taken to the corresponding faculty bio page and located the email immediately.

Specific Interview Responses: The main navigation and sub menus were all where the user expected them to be. The subject noted them as clear and easily visible/locatable. The user did suggest that the links within the body content be identified with another color, so they are not overlooked. While navigating the main menu, the tester noted that the subject did not have any noticeable difficulties and when asked the subject agreed. They felt everything was very straightforward. The test subject did not notice that there were multiple or more than one way to navigate to the Faculty Bio page. However, after being asked about this, the subject easily located the two other possibilities. The subject identified these various access points as useful in the event that you just want to go straight to a specific professor, instead of through course descriptions. Over the test subject felt that there was nothing that needed changing about the content organization on the Faculty Bio or Course Descriptions pages. They described them as straightforward and very organized.

General Interview Responses: Subject 01 described the overall tone/mood of the site as cheerful, professional, fun, and creative. They especially noted the bright colors contrasted with the white spaces and the variations in font. The subject identified the quality of images a good overall, and described them as interesting and fun. They pointed out the sculpting image as one of particular interest, but did not like the painting image because they felt the painting it showcased was not very well done. The backgrounds were well recieved, the subject described them as tasteful, abstract and not distracting from the content. When asked what they would like to see more/less of the subject said that there wasn't anything. They found the news/events on the index page to be very fresh and interesting and easy to keep up to date unlike the current site. The subject described the content as very easy to read. The overall rating of the site given by the subject was 4.5 / 5.

TEST SUBJECT 02

Script Test Notes: After being read the test script test subject 02 quickly chose Students from the main navigation menu, however they did have some difficulty with the submenu when trying to click on Undergraduate. The test subject was unable to remain inside the sub menu and therefore it kept changing. After a couple of tries the subject successfully clicked on Undergraduate, then Courses > Sculpture with no further difficulties. They paused briefly to read the sculpture introduction before locating the specific course further down the page. After being given the second part of the test, the subject hovered over the course director name but did not click on it. Instead the subject went to the main navigation, clicked on Faculty and then first chose Emeriti. After scrolling quickly through the names listed the subject then chose Full-Time from the third level navigation and then clicked on Michael Davey from the list. They were directed to the correct page and found the email quickly after that.

Specific Interview Responses: Subject 02 described the navigation menus as easily understandable and intuitive. However, the subject identified the main navigation menu drop-down as a potential obstacle. They were unable to get the second level of navigation to stay on what they originally highlighted/clicked. Also they did not realise that there were links within the content of the website and as a result it took her longer to find the course director information. The subject thinks these issues need to be addressed. When asked about whether they noticed the multiple ways to navigate to the faculty bio page the subject said that it was not evident to her. After having the various ways pointed out to the subject, they noted that they did not even notice the quicklinks toolbar, however, if they were a frequent visitor they felt they would and that it would be very helpful. The subject did not feel any change was necessary to the Faculty Bio content organization. They mentioned that when they first navigated to the courses page, they did not expect the division between practicum and studies and expected a long list of all the courses. They said this was much better and easier to navigate.

General Interview Responses: The subject described the sites overall appearance as calm with "sweeping" imagery. The subject described the imagery as good, but declares that they are not visiting the website to look at the imagery (unless they specifically visit the gallery). As a result it is not that important to them, however they feel that the imagery does give a good taste of the artwork at york without overpowering the information. *Note: This test subject is a photography major* The subject described the background images as "pretty cool" although, being a photography major they noted that the quality was not always excellent because it was scanned paper/artwork. The subject also noted that the logo, in their opinion, did not match with the look/feel of the background images. When asked what they would like to see more/less of the subject identified the two areas. The first was the possibility of having courses as an option in the main navigation so that it was faster/easier to find. They also suggested that the headings within the content be more stylized/artistic to reflect the artistic nature of the website and of visual arts in general. Subject 02 found the information good overall, easy to read and did not see a need for any change. Subject 02 rates the site 3.5-4 / 5.

TEST SUBJECT 03

Test Script Notes: When read the test script, Subject 03 first tried to click on the "Sculpture" link in the image scroller. Since this link is currently not working, the user found no response and quickly moved on to click Students > Undergraduate from the main navigation menu. They easily navigated through Courses to Sculpture and quickly found the specific course. When given the second part of the test script Subject 03 clicked the name after the course description and after being redirected to the proper faculty bio, found the email.

Specific Interview Responses: Subject 03 felt the navigation was very self explanatory. The subject said they felt they did not encounter any obstacles in completing the tasks. When asked about the three possible ways to navigate the Faculty Bios, the subject said that because the first attempt worked they only tried that one. They did however, comment that the quicklinks toolbar was unnoticed and at first confused with being something specific to the testing computer. The subject did not have any recommendations for the content organization. They explained it as clear and specifically identified the headings as something they liked the style of. They also liked the images.

General Interview Responses: Subject 03 identified the mood of the site as being professional and clean. They enjoyed the use of only white and black for the content and liked the images. The subject further described the images as bright and artsy. However, they did not like the sculpture or painting images. They felt that they were not representative of the visual arts program at York and what is produced there, which they described as being more contemporary and abstract. After being asked about the backgrounds specifically the subject noticed that they changed with each page and identified this as something that made it more interesting. Also they noted that the content floats on top of the background image and scrolls on top of it which they really liked. When asked what they would like to see more/less of, the subject felt that it was very balanced between text and imagery. They compared it to the current York site as being much easier to navigate and to read/understand, as the current one is very confusing. The subject described the information as easy to read, font size was not too big or too small. The content was was very clear and "tidy", not too much to handle. Subject 03 rated the site 4.5 / 5.