Regression testing enterprise applications, like SAP, is a key element in software development. For the people in charge of testing, the process can be complex and highly technical. This is a story about how I used UX techniques to create a responsive web application to make this process a seamless and deligtful experience.
I conducted this project while studying the UX Design course at General Assembly in 2017. I lead all of the UX Design through the discovery, definition and development phases of the project. I then presented my process, complete with a high-fidelity mockup to the Head of Product and the CTO at Basis Technologies. The aim of this project was to demonstrate the value of UX Design techniques to the business.
Making SAP testing a seamless and delightful experience
Well designed experiences are important in the context of enterprise software because mistakes are extremely costly. If a product isn't designed with the user in mind, it will take them longer to learn and complete tasks as well as make them more prone to making mistakes. These mistakes could cause important elements of a company's digital real estate to grind to a halt, which will affect their bottom line.
“there are fewer examples of good products in this space than any other…customers are frustrated and aren’t anxious to spend yet more money on more disappointing products.”
- Marty Cagan on enterprise software design
For the people in charge of regression testing enterprise software applications, like SAP, it is particularly important that they understand whether new pieces of development will cause such problems before they are released. It just so happens that this process is complex and highly technical. If you have ever used SAP, you may also be aware that the user interface is dull, ugly and dated.
I used Lean UX methods such as sketching, rapid prototyping and constant user feedback in order to get to a high-fidelity prototype within my 10-week deadline.
I was encouraged to go through as many iterative cycles as quickly as possible (build, measure, learn) in order to move my project forward. By always approaching users for feedback on my paper prototype, I was also sparking discussions about the disruptive nature of this method, as it was relatively alien in the world of enterprise software.
SAP is the type of application which puts functionality above all else. According to Stephen Anderson's UX Hierarchy of Needs, in order to create a product that works in a seamless and delightful way, it must first be reliable, usable and convenient. Therefore, I had to first ensure that users could achieve their goals conveniently. It was by paying special attention to their facial expressions and the emotive language they used that highlighted whether I was delighting them or not.
Gaining User Insight
To begin my discovery phase, I conducted market and competitor analysis to explore how other products identify and report where there are defects, and if possible, how these pieces of software auto-produce test case evidence. I also observed a user environment, paying particular attention to their typical office setting and the tools people were using to complete the same tasks.
I conducted face-to-face interviews with a hypothesised audience, who were either Testers or Developers, in order to discover exactly what problems they face when it comes to regression testing.
My research goals were to investigate: What does the current testing process look like? What is the user's experience of automated testing? Where could the testing process be improved? and Where could the user interface be improved?
In order to help me pick out key themes and problem areas I created an affinity map. I wrote down quotes from my research notes on post-its and stuck them on a wall, using different colours for different users. After completing this exercise I was able to separate the post-its into seven themes: User needs, user goals, user furstrations, automation, tools, process and user interface (UI).
Having all of the information in front of me helped me to identify these needs, problems and motivations:
- The test scripting process helps you to create empathy, but is a very time consuming process.
- Automation of this process would save a lot of time, but is difficult to achieve (at the moment).
- The user interfaces on current solutions are cluttered and out of date.
- The user needs to know where defects are occurring in a clear and organised way.
The findings from my research provided the basis for my two personas - a Developer and a Tester. These personas were my original hypothsised audience and now that I had spoken with them, and identified some of their problems, I wanted to solidify these traits in the personas I created.
These were important for me as I wanted these personas to guide my design decisions, help me prioritise features, and ensure I was always empathising with the end user.
Task Analysis and Experience Mapping
I conducted a task analysis in order to visualise how users currently complete their tasks. I got this information from both my observation and user interview research.
From here I was able to create an experience map based on my 'Isaac' persona. Empathising like this meant I could document what he would be thinking, feeling and experiencing at each touch point.
I found that the step "going through the test script" was the area that caused the most pain for this persona. The next step, therefore, was about figuring out how to solve this frustration.
Automated Regression testing in SAP
Based on the research it became clear that time saving was very valuable to users. Therefore, by offering automated testing functionality as well as a clear and concise user interface, the time it takes to regression test would not only speed up, but it would be more accurate.
The Problem (refined)
I refined my primary user goal to fully reflect my research. This would also allow me to maintain a user-centered focus while creating the application. Defining the problem statement was also key to ensure that what I was making was solving this problem. My Hypothesis was to serve as my line in the sand, I had to ensure the product was meeting these user goals.
I believe that by providing easy to use recording and play-back functionality, testers will achieve quicker and more accurate regression testing of SAP systems. I will know this to be true when we see an increase in the accuracy and time it takes to perform a test.
Now, Next, Later
I used the Now, Next, Later method to help me prioritise which features were the most valuable to users. This also helped me explain to stakeholders at Basis Technologies how UX Design would fit into into it's Agile development methods, as well as illustate what the roadmap for this product would look like.
Considering the scale of this design problem, it was really important that I paid close attention to what I heard in my research. I wanted to ensure that the prupose of the product was clear and that it does one thing really well.
I created a user flow for my primary user goal 'Isaac'. It illustrates what would happen as the user makes various choices along the way. The goal of this flow was to allow the user to automatically generate a test script. The user begins by logging into the web app, creating a test plan, recording user actions, viewing user actions and finally generating the test script.
Using this method helped me think about which screens I needed to create and allowed me to iterate quickly. This was also to lay the ground work for my wireframes.
I sketched out the screens based on my user flow and scanned the images into Marvel. This allowed me to test the flow with users quickly and make any changes in a matter of minutes, so that i could get back to gaining more feedback.
This helped me to streamline and evolve my user flow, and it was this lean method that caught people's imagination the most throughout the whole process. It was the speed with which you can test ideas, and the rich discussions they generate that were the most valuable to me as a designer.
I went through a couple of stages of iteration, changing things based on user feedback before moving onto creating wireframes in Sketch. This helped me build up the basis of a design framework which would later help me when it came to creating the high-fidelity mockups.
I used the Marvel plugin to Sketch to sync my designs with my mobile phone, so that I could maintain the speed of testing I had managed with my paper prototypes.
Testing & Iteration
Throughout my tests with the user flow, paper prototype and wireframes I had to make a number of changes, including:
- Taking some elements away from the home screen to avoid clutter and enhance clarity.
- Include a check-list analysis to enhance clarity of feedback to the user.
- The ability to view granular levels of test-scripts generated to better suit my personas needs.
Once I conducted further tests on users using Marvel , I felt comfortable enough with the iteration to move onto creating high-fidelity mockups.
Introducing... Test Express
Test Express is a responsive web application that allows users to test changes in an SAP system. Below are the high-fidelity mockups that were presented to Basis Technologies.
I didn't want to take my design too far away from what they would already be used to with SAP, so I decided to create a flat design for the user interface and used a lot of primary colours. I used clean, bright colours throughout the interface and straight forward language to ensure that It was always clear to the user what was happening.
"Keep doing this, it's disruptive"
A lot of the feedback I had while I was testing was positive, none more so than from the CTO who told me to "keep doing this, it's disruptive". This was in reference to the Lean UX methods that I was using throughout the project and the types of discussions it was yielding throughout the workplace. UX Design has long been an afterthought with enterprise software and by introduing Lean UX methods into this environment I was providing something new, cheap and valuable.
I presented my findings to the QA Lead, Head of Product and CTO at Basis Technologies at the end of the project. This concept for the Test Express web application has been approved by Basis Technologies for further UX exploration. The next steps will be interviewing and testing with more users to further validate ideas, create more user flows for viewing results of tests and re-test usability.
I also presented this project to my class at General Assembly in January 2017. Based on the feedback I received I will now be researching the market and validating the need for UX Design services in companies who make software for SAP.