DETAILS: 2013 / Coursework in Human-Computer Interaction Master's Program at Iowa State University
2011 / Iowa State World Usability Day Challenge
ROLE: Sole Contributor
Communication and connection between individuals within an organization is an essential component of success. This is even more true in academic organizations, where collaboration between diverse minds is critical for rich research and innovation.
For World Usability Day 2011, Iowa State’s one week Design Challenge was to take the two program sites, hci.iastate.edu and vrac.iastate.edu, and combine them into an improved web experience for prospective students, current students, and faculty and staff. I earned 1st place in the competition with my proposal that included suggestions around Information Architecture, Design, CMS and analytics improvements based on data collected from interviews, surveys and a paper prototype test. The result of that work was a proposal that was presented to the program leadership.
One of the main points of feedback was that the the directory on the site was critical for every target audience, yet it was difficult to effectively search and keep up to date. I used that insight as the basis for a course project. Over the semester, I focused on building a usable, generic academic directory, by conducting a task analysis, designing and paper prototyping an initial solution, conducting a guerilla usabililty test, building a higher fidelity prototype and running an in depth usability test.
Get to Know Audience
Academic directories are used by a wide variety of users, primarily falling into five persona groups: Faculty, Staff, Prospective Students, Current Students, and Alumni. I reviewed interviews provided by the Usability Day challenge and also did deeper interviews with some of the missing, but important audiences, like prospective students and the website administrator. I then built out lightweight goal-oriented personas of each type, and chose to focus primarily on the Staff (who administrate the directory) and the Current/Prospective Students, who had similar needs.
Directories are not a new concept. Most organizations have a directory either on their external site or their intranet. In addition to reviewing multiple examples of company and academic directories, I also wanted to introduce more browse and search functionality based on people's interests. For this I reviewed patterns found on popular social media and e-commerce sites.
ZOOM IN / Light goal-based personas
This product is broken into three main subsystems: the front-end directory page, the user content management and the admin content management. On the directory page, visitors can see, search and filter people’s information. Users can modify their own information and add interests or links (to homepage, Twitter, etc.). All edits must be first approved by the administrator to ensure the quality of information in the directory. The admin content management page will also allow the admin to batch add and edit users (for example, when one class graduates, shift their status to alumni). For now there will be two levels of privileges, user and admin, though in the future faculty may have different privileges than students (like editing classes).
ZOOM IN / Concept subsystems and data flow
Wireframes, Paper Prototype and Informal Test
Numerous whiteboard sketches of each subsystem evolved into the wireframes shown on the right. Those wireframes were used in a series of interviews and informal guerilla user tests, where the wireframes were combined into a paper prototype. The user testing was scenario based, with each user given a realistic task to complete with the prototype.
The administrator identified some missing filters that she had gotten requests for. She also recommended the change approval system, and the importance of bulk edits. Student participants struggled slightly with the forced decision of categories or search. They frequently used the search bar over filtering or browsing. Based on this finding, I decided to remove the categorization, and list all directory profiles in a single list that could be searched or filtered.
ZOOM IN / Wireframes used in the paper prototype
High Fidelity Prototype
The development of the high fidelity prototype was the main focus of this course project. It was built in PHP, with a MySql database, also utilizing Bootstrap and JQuery. The visual design was intentionally highly generic and the prototype built in such a way that a consuming website could easily layer on their brand and styles. This project is no longer hosted online, though you can view a demonstration in the video above, and review the source code on GitHub.
In Depth Usability Testing
In order to compare the effectiveness of the Tassel Directory in comparison to the existing directory, I formed an AB comparison usability study on UserZoom. Participants were asked to complete 10 tasks—2 sets of 4 identical tasks comparing the current HCI directory pages to Tassel directory, and 2 tasks that tested functionality unique to the Tassel Directory. After the tasks, participants were given the System Usablility Scale (Brooke, 1996) survey for each site. UserZoom allowed me to conduct the tests remotely, by sending each participant a link to take the study on their own time, on their own devices.
The 16 participants for this study were recruited from friends and family who had expressed an interest in the project. The majority of these individuals fit the target audience, even if they were not a part of the ISU HCI program. 5 fit the Staff & Faculty personas, and 11 fit the Current & Prospective Student personas.
Tasks were randomized within each set. The two sets were also randomized, so some users did tasks using the HCI site first and some using the Tassel site first. This created some confusion, with a few participants abandoning a task they thought they had already completed. I learned that I need to make it more clear that the same tasks will need to be done twice using the difference sites.
The test was analyzed on multiple quantitative dimensions, including task success, time on task, clicks per tasks, percieved ease of use, percieved time, and System Usability Score. I also analyzed open comments, click paths and heat maps of clicks.
Overall, Tassel was an improvement over the existing site. The only task Tassel performed worse on was the task requiring users to determine which faculty were interested in virtual reality. Since there were no search or filter options for interests at the time, participants grew frustrated with their searches not working and abandoned the task or took longer on it. Interest filtering and searching is a planned feature for Tassel that wasn’t done in time for this test. These are some areas that Tassel significantly improved upon the existing HCI site:
68% higher System Usability Score (SUS),
45% lower average time on task,
38% higher perceived ease of task,
88% chose Tassel as more attractive, and
88% chose Tassel as more functional.
The biggest takeaway from this study is that most users preferred using the search to find directory profiles, and would only use other options when search failed. Participants expected that the search box would also search through company names, departments and interests.
I was also surprised that 25% of participants mentioned wanting to try the advanced options. This will increase the priority of this feature’s implementation.
Another high priority next feature is search and filter by research interests. While it could be the demand for this was triggered by the specific task that required you search that way, I had already considered this as a desired feature for Tassel.
It was mentioned that the list of the departments was a bit overwhelming, and it was frustrating when filtering didn’t yield any results. So narrowing the listed departments down to only those that have profiles associated with them would improve this feature.
Another issue this study brought up was how people not familiar with the program view various aspects. For instance, one participant mentioned looking for HCI on the list of departments. Another mentioned that they simply looked at the people list in the Virtual Reality Application Center when asked to find faculty who were interested in virtual reality. Making this more clear by using terms like “Home Department” instead of just “Department” may help.
There were some expressed concerns of how Tassel would scale, if it would still be as easy to use with a hundred profiles. This is a concern worth investigating further.
While the design style was mentioned by several participants as an area that Tassel is lacking in, I was purposely keeping the design rather generic so that department styling can easily be added on top of it. This project is intended to be flexible enough to be used by multiple departments. Also, I was hoping it would be integrated into the final ISU HCI redesign, which didn't at that point have a final visual design.