Jacquelyn Brioux
Researcher and Service Experience Designer
jb_pivott.png

Pivott — Research & Web UX Design

Pivott

My Role

Lead researcher and designer. 

Expertise Evidenced

Design research, prototyping, usability evaluation, user interface design, qualitative and quantitative research methods, data analysis, academic writing.

Abstract

Individuals choosing a career path today face the difficult challenge of sorting through vast amounts of information to determine the best path forward in achieving their career goals. They must also have the appropriate resources and guidance to draw a clear connection between their career goals and the steps necessary to achieve these goals. While there are numerous academic, financial aid, and career counselling services available, it was hypothesized that it may prove valuable to combine these resources into one comprehensive online service, which facilitates the connection between those who seek education, funding and employment and those who provide it. The purpose of this study was threefold: i) to understand the behavior and preferences of people who search for education, funding and employment opportunities online; ii) to design a data driven prototype of a web application that aims to bring ease and simplicity to the process of searching for education, funding and employment opportunities online; and iii) to validate fundamental user interface design concepts and determine whether the design solution meets user needs and has value in the marketplace. During the design process, several user-centered design methodologies were employed namely, a contextual inquiry, an online survey, a heuristic evaluation, and an unmoderated remote usability study, to better understand the needs of representative users. The contextual inquiry showed that there is no single platform that users turn to in their search for education, funding, and employment online. Instead, a majority of users employ Google Search as their starting point and navigate between many different websites to gather information, a process that is both time consuming and inefficient. Likewise, there is no single platform that users turn to for communication with providers or peers, although the results of the online survey suggest that a majority of users would volunteer to become mentors if a platform existed. The contextual inquiry also demonstrated a need for existing services to perform user testing to improve the experience of searching for opportunities online, a finding that supports the value of a user-centered approach. Online survey participants indicated far more interest in enrolling in skills-based courses over traditional education in the future. This finding supports recent research showing a steady rise in the popularity of skills-based online courses over the past decade. Survey participants also reported that they are not overly satisfied with the current online search options for education, funding, and employment, which validates the demand for a new service to fill this need. The results of the contextual inquiry and online survey informed the design of a low-fidelity prototype for Pivott, a service for individuals who are seeking education, funding and employment, making it easier for them to find the right information and connect with the right people to advance their careers. A heuristic evaluation of the low-fidelity prototype led to a second iteration on the design to create a high-fidelity prototype. An unmoderated remote usability study on the high-fidelity prototype provided valuable user insights that will inform future iterations of this design. Through a user-centered approach to design, Pivott has the ability to narrow the education-jobs gap by making it easier for seekers and providers to find the right information, people and opportunities.

Research Design

Contextual Inquiry: A contextual inquiry was conducted with three participants, which lasted between one and two hours each and contained three parts: to observe the current process of how people search for i) education, ii) funding, and iii) employment. Each participant was asked to conduct their search on the Internet using a 13” MacBook Pro and the Google Chrome browser. Using software called ScreenFlow, the audio (verbal thought process and commentary), video (facial expressions) and screen (keyboard / mouse input) of each participant was recorded, while notes and photos were also taken as participants encountered breakdowns and opportunities in their search.

Online Survey: An online survey was conducted using Zoho Survey, which consisted of 30 questions broken into six sections: i) demographic information, ii) Internet and social media use, iii) education background and experience searching for education opportunities, iv) funding background and experience searching for funding opportunities, v) employment background and experience searching for employment opportunities, and vi) questions to help gauge the participants’ interest in specific functionality and services being considered for inclusion in the web application. Survey questions consisted of multiple choice, semantic differential scale, and open-ended questions (http://bit.ly/pivottsurvey). Instructions were drafted for participants on how to complete the survey, along with introductory text before each section. The general survey instructions explained who was administering the survey and how long it would be available; it emphasized the importance of the survey and the reason for its use; participants were reassured that their answers were confidential and it explained how their information would be handled; and contact information was made available in case any participants needed to get in touch. The individual section instructions explained what type of information the section was asking for and a status bar was configured so that participants could gauge how much of the survey remained. The link to the online survey was shortened using Bitly for easy distribution, but also to view real-time link tracking analytics. Prior to launching the survey, a pretest was conducted to determine if there were any errors, spelling mistakes, and to see roughly how long it should take participants to complete. The survey was launched on November 23, 2013 and it remained open until March 3, 2014, receiving a total of 43 completed survey responses.

Low-Fidelity Prototype: The contextual inquiry identified pain points in the experience of users searching for education, funding and employment opportunities online. These pain points informed the design of a low-fidelity prototype for a web application intended to assist education, funding and employment seekers in discovering suitable opportunities for realizing their career goals. The low-fidelity prototype was designed using Balsamiq, a rapid wireframing desktop application, which offered a low-cost, simple illustration of the design concept. Please visit http://bit.ly/lofipivott to view the low-fidelity prototype. The low-fidelity prototype consisted of 22 linked screens, which were then exported as a single interactive PDF and hosted with Amazon Web Services (AWS) S3 cloud storage for distribution amongst participating usability experts.

Heuristic Evaluation: In reference to Jakob Nielsen’s 1995 publication entitled, “10 Usability Heuristics for User Interface Design”, four usability experts were asked to remotely access the low-fidelity prototype and evaluate the user interface design against the 10 most general principles for interaction design, and then identify areas for improvement (http://bit.ly/pivott_he).

High-Fidelity Prototype: The heuristic evaluations received from participating usability experts informed modifications in the design of the low-fidelity prototype to the high-fidelity prototype. The high-fidelity prototype was designed using Justinmind Prototyper, an authoring tool for software prototypes and high-fidelity website wireframes, which has a much higher learning curve, but offers rich interactions (e.g. show and hide content, change styles, and modal dialogues upon mouseover or mouse click) and visual design components (e.g.
hover and active state effects, opacity and full color images) to simulate the real web application for presentation and user testing. The high-fidelity prototype was exported from Justinmind Prototyper with all of the interactions and conditions defined and functional, to a directory of files suitable for deployment as a standalone website. The directory containing 16.7MB of HTML, JavaScript, CSS, and other supporting files (hereinafter referred to as “application code”) was then uploaded to an FTP server using FileZilla. The application code implemented the user-facing interface of the website, located at: http://jacquelynbrioux.com/pivott/Prototype_Pivott. The link to the website was shortened using Bitly (http://bit.ly/hifipivott) for easy distribution, but also to view real-time link tracking analytics. In this case, ‘Usability Experts’ consisted of peers from my Human-Computer Interaction course, who were involved in conducting a heuristic
evaluation of the low-fidelity prototype.  A unique Google Analytics tracking code was generated and added to the application code to track audience and behavioral data associated with participants visiting the website for remote usability testing. Prior to distributing the high-fidelity prototype to each of the participants, a thorough test was conducted to ensure that each task could be performed effectively.

Unmoderated Remote Usability Testing: A list of 15 tasks was generated for remote usability testing participants to perform as an assessment of the overall usability and perceived value of the web application. The tasks were then divided into two groups (Group A and Group B), with the first three tasks appearing in both groups and the remaining 12 tasks divided evenly into Group A and Group B, for a total of nine tasks in each group. The 12 participants were randomly assigned evenly to Group A and Group B, and participants in both groups were asked via email to complete a pre-test, post-task and post-test questionnaire by navigating to the website and their respective three-part questionnaire via the Bitly links provided. All participants were asked to open the questionnaire in one window and the website in a second window in order to perform the assigned tasks as part of the unmoderated remote usability test. A pre-test section of the questionnaire was prepared using Zoho Survey, which consisted of a consent form and six questions about the basic demographic information of participants, including age, gender, country of residence, ethnicity, education level and employment status. The same pre-test section of the questionnaire was assigned to all 12 participants to be completed prior to conducting the remote usability test. Two post-task sections of the questionnaire were prepared using Zoho Survey, which were assigned to participants in Group A (http://bit.ly/usabilitya) and Group B (http://bit.ly/usabilityb) respectively. Each post-task section consisted of nine tasks to be carried out by participants conducting a remote usability test of the website. After each task, participants were asked to rate how difficult or easy it was to perform each task on a scale of 1 to 7 using the Likert Scale. A post-test section of the questionnaire was compiled using Zoho Survey, which consisted of a System Usability Scale (SUS) that asked participants to rate the level at which they agree or disagree with 10 statements pertaining to the usability of the web application, as well as two open-ended questions about the participants’ level of interest and suggestions for the web application. The same post-test section of the questionnaire was assigned to all 12 participants to be completed after conducting the remote usability test.

Analysis Methods

Contextual Inquiry: In order to identify opportunities for how to design the web application, the information collected during my contextual inquiries was reviewed — paying particular attention to opportunities and breakdowns during participant observations — and ideas were developed for how this web application can better meet the needs, goals and values of its users.

Online Survey: Online survey results were collected in Zoho Survey and exported as a spreadsheet containing a summary of participant response data for each survey question. Using Numbers for iOS, bar graphs, stacked bar graphs, or pie charts were created for each of the relevant survey questions and the trends were examined. Graphs were selected that represented interesting and relevant results pertaining to the objectives of this study.

Heuristic Evaluation: The heuristic evaluations from peer usability experts about my low-fidelity prototype were reviewed, and this feedback was referred to when making changes to the design of the subsequent high-fidelity prototype.

Unmoderated Remote Usability Test: Results from the post-task questionnaire for participants in Group A and Group B were collected in Zoho Survey and exported as separate spreadsheets containing a summary of participant response data for each question. Using Numbers for iOS, bar graphs were created for each question answered by participants in Group A and Group B. Results from the post-test questionnaire for participants in Group A and Group B were collected in Zoho Survey and exported as separate spreadsheets containing a summary of participant response data for each question. The post-test questionnaire consisted of a series of 10 statements, which participants in Group A and Group B were asked to rate on a scale of 1 (strongly disagree) to 5 (strongly agree). A System Usability Scale (SUS) score was calculated for each participant using the methods described by Tullis and Albert. Using Numbers for iOS, frequency distribution charts of the SUS scores were created for each of the two groups and the trends were analyzed.

Results

Please view the Results section (3) of my final thesis (starting on page 22).