Project Details

Research Methods

  • Usability Studies
  • Analysis

Advised By

  • Rob Farrow - Sr. UX Researcher

Time Frame

  • June 2015

Tools

  • Microsoft Office
  • Open Meeting
  • Usability Labs

CloudFormation Lab Study

At the start of my internship at AWS, I shadowed another researcher for a few weeks to get an idea of how my team does research, and how they manage the resources available to them. Once I had done that I was assigned my first solo project, to test a new visual interface for an AWS service. This was a first for AWS and involved coordinating with a 3rd party contractor who was developing the visual tool, as well as working with the project developers for the specific AWS service.


Objective

  • The objective of this study was to discover how users would utilize the visual tool for creating templates in the CloudFormation service.
  • A secondary objective was to discover bugs in the software and coordinate with the development teams to fix them

Process

  • I utilized an internal database of study participants to recruit users who were familiar with the CloudFormation product and were sufficiently advanced users.
  • I then worked together with a designer and developer to put together a research plan that would test various elements of the tool which the developers were keen to get feedback on.
  • 7 lab studies were then organized and administered by myself, for the first several studies the developer was in attendance to help hot-fix significant bugs that were inhibiting the flow of the user study
  • Following the completion of the user study, I analyzed my results and wrote up a detailed report.

Challenges

  • In the very first lab session I ran none of the equipment worked. This resulted in my having to create an ad-hoc work around using a computer in an empty participant room to rig a system that would enable the study to continue. Luckily this worked well and the session went smoothly after that.
  • The software was extremely buggy early on, this required close working with the developers between sessions to get fixes in place or adjust the study protocol as required.

Outcomes

  • The primary outcome of the study was a detailed research report detailing several important results from the study
  • Most important of these results was the discovery that the audience the tool was aimed at was incorrect, and this resulted in a pivot from targeting highly experienced users, to more novice users.