"With TestFairy, the Problem is Right in Front of You - a 2 Hour Assignment Becomes a 5 Minutes Issue"

- John Ayers, Vice President of Quality Engineering at OV Loop

OVERVIEW ‍

OV loop, based in Boston, Massachusetts, is revolutionizing voice-powered communications and commerce channel for people and businesses to better interact, transact and build relationships. John Ayers is the Vice President of Quality Engineering at OV Loop. He has over 20 years of work experience in Quality Assurance including Samsung Electronics, where he originally started working with TestFairy. t At OV Loop he is responsible for managing QA efforts across software, hardware, and firmware, as well as managing all of the internal applications such as TestFairy, JIRA, Atlassian, Bitbucket, and TestRail. ‍

OV Loop case study

WHAT IS OV LOOP? ‍

“At OV Loop we are doing several different things. We have a social commerce platform that we’re building allowing you to make payments within chats, whether it’s a support chat or a personal chat. We also have a support portal that allows people to use voice, so we’re voice-powered which is different than everybody else. If you were to go to our website right now, you could go to our support section and click the live chat. There you could do the typing by clicking on the microphone button, which will do voice-to-text right through your laptop or phone. Essentially as a customer can do support tickets or requests using your voice. It gets transcribed and creates a ticket automatically in our support portal. Then the agent for the company can pull that ticket and either respond with text or voice. We also send files, whether it’s images or videos.”

THE BUSINESS MODEL IS B2B? ‍

“Yes, but we also have a P2P side to that too. So we have a mobile app that you can get in the Google Play Store or in the App Store, we support both iOS and Android, and that is also voice-powered. So I could message anybody in my contacts with my phone, whether they have OV or not. If they do, it will open the OV app. If they don’t, it will open a webpage that allows you to still chat with me using voice.One of our big customers right now is a company called Binatone and they work with Motorola.”

TestFairy is great for a huge company to bring in because it just makes everybody more efficient, but it’s really key for a startup. Because you don’t have enough resources and you don’t have the money to continue to add bodies to make up for the fact that you’re not debugging your product correctly.”

TELL ME ABOUT YOUR JOURNEY IN QA ‍

“I got into IT management in the 90s and rolled from that into QA. To be honest, there wasn’t really an official QA group in the 90s. Then I went to run the IT for a running shoe company. I was a competitive runner at that time, so it seemed like a dream job,. Then a position opened up for a third-party QA company based out of California, but they were opening a Boston office so I would be one of the first guys in. I interviewed and got the position, and I stayed with them for almost 5 years. When the bubble burst in the early 2000s, I was there. We went from almost 70 full-time QA engineers to about 15, and there were only 2 left in the Boston office. We were lucky enough that the founder of the company signed a deal with Verizon where we did all their field interoperability testing, so I flew around the country testing new phones on the Verizon network. After that I did QA for several different companies until finally finding LoopPay, which was then bought by Samsung. Subsequently, Samsung shut down the lab last year and I was fortunate to contniue my QA journey with Will Graylin who founded LoopPay.By that time he had already started the OV venture and asked if I wanted to join.”

SO LOOPPAY AND SAMSUNG IS WHERE YOU FIRST WORKED WITH TESTFAIRY? ‍

“Exactly. We worked with TestFairy in Samsung, and when I moved on my whole team stayed with me, all 5 of us. It was very fortuitous, we were happy about it.”

TELL ME A LITTLE BIT ABOUT YOUR LIFE AS A QA MANAGER. HOW DO YOU SEE QA TODAY COMPARED TO 20 YEARS AGO? ‍

“When I started, QA wasn’t really a dedicated group within a company, or a dedicated effort. A lot of times developers did their own testing or the sales people did the testing, and so there was no process around it. As a result, a mixed bag of quality would usually get out to the public. I think as software took off and handheld phones became smartphones, all of those apps needed to be sent through a more rigorous and defined process to ensure quality. You only get one chance to impress somebody, and a good QA group makes sure the experience the first time through is solid. So QA is more mature now, it’s seen as a legitimate and necessary part of a company.”

HOW DO YOU ALLOCATE RESPONSIBILITIES WITHIN YOUR TEAM? ‍

“A good QA group has to have flexibility. With that being said, I think engineers all have certain parts in their job that they’re best at. I have somebody that in particular manages the mobile side, same with the automation side and the support portal dashboard. Everyone has a specialty on my team, but everybody can do most types of testing. I mix and match when I need to. If I need somebody to jump from testing some service side changes over to a new mobile version, they can do that. If the regular portal tester is busy on something and I have another portal project come in, I can move somebody without a problem. You need to cross-train.”

WHAT INTERESTING TOOLS DO YOU USE FOR YOUR DEVELOPMENT? ‍

“For our test case management, we use TestRail, which is really one of the best test case management tools out there. It integrates with most other tool sites. We’re using the entire Atlassian suite for software development, life cycle, and documentation. So we use Confluence for our document management, in collaboration. We use JIRA for our ticketing and Bitbucket for our Git. And of course last but not least, TestFairy.”

‍‍

WHAT IS THE VALUE YOU GET FROM USING TESTFAIRY IN YOUR DEVELOPMENT AND TESTING PROCESS? ‍

“It provides the details behind anything that’s going on when there’s a problem with the mobile app. So if you get a crash, we can go in and see a video of what the user was doing. We don’t have to go through logs that you can’t decipher. We can watch and see, exactly what the user was doing at that point. And some things aren’t even correctly logged, so you couldn’t figure out what was going on without the video or the screen captures because maybe the developers aren’t registering the actions of the user correctly. Between the video, the logging, and the screen capture, it makes it easier to reproduce the issue and isolate the code quicker. All in all, the team is much more efficient because of the features that TestFairy has.”

‍‍

HOW WOULD YOU MEASURE THAT VALUE OF EFFICIENCY FOR YOUR TEAM AND DEVELOPMENT? ‍

“That’s a good question. I know that the developers are much happier to move from what they were using to TestFairy. There is a much greater likelihood of a timely fix for any issue that you find when you have TestFairy. One of the problems with not having TestFairy or having a different tool is that a lot of times the time to investigate, isolate, and fix the problem is so long that you can’t get it into that build or that release. Some things will just sit. And with TestFairy, the problem is right in front of you. The builds are much more complete, you’re not deferring issues because you don’t have time to go through endless logs. Or you don’t sit there trying to reproduce a hard-to-reproduce problem, because you have the video right in front of you. Without the tool, I’ve got one of my engineers sitting there for 2 hours because they weren’t able to grab all the information the first time it happened. With TestFairy, bang, we’ve got the information we need right then.”

‍‍

FROM WHAT YOU SAID THE EQUATION IS 2 HOURS WITHOUT TESTFAIRY AND 5 MINUTES WITH TESTFAIRY, THAT IS MEANINGFUL TIME SAVING VALUE. ‍

“Absolutely! As soon as something happens, you can either manually get feedback or you can go in and grab the log or look at the session right then. Collaboratively login with the developer, say this is exactly what I was doing at that point and here’s the screen I was on. And away they go, they’re off and fixing, or investigating.”

‍‍

WHICH COMPANIES COULD GET THE MOST VALUE FROM USING TESTFAIRY? ‍

“To be honest, any company. Enterprises might think they can just throw bodies at something, but any good manager knows if you’re just ramping up people to try and increase your output, then you’re a bad manager. What you really need to do is take the team you have and make them more efficient, make them work together in a more collaborative way, and TestFairy is a great platform for mobile companies to do that. So it’s great for a huge company to bring this in because it just makes everybody more efficient, but it’s really key for a startup. Because you don’t have enough resources and you don’t have the money to continue to add bodies to make up for the fact that you’re not debugging your product correctly.”

"Between the video, the logging, and the screen capture, it makes it so much easier to reproduce the issue and isolate the code quicker. All in all, the team is much more efficient because of the features that TestFairy has."
- John Ayers, Vice President of Quality Engineering at OV Loop