With over 100,000 mobile developers trusting TestFairy as their platform of choice for OTA app distribution, crash reporting and customer feedback, I can say we have seen a lot.
Seeing a company work is really understanding what problems they faced during development and how they solved them.
We learned habits and tricks from each developer and compiled a list of the 7 most important bits.
Here is what I’m going to talk about:
- Eating your own dog food
- Reduce the effort to report problems
- Automated release process
- Staging and production apps on the same device
- Rapid app release
- Not all issues are created equal
- Being comfortable with your tools
1. Eating Your Own Dog Food
Or in its more common name, dogfooding, is a practice companies take when company employees use their own company’s products. After all, how can you sell it to a customer if you yourself won’t use it.
A company we worked with to improve their workflow, wanted to get more beta testers for their beta versions.
Instead of relying on a small QA team to manually test the new features, they prepare a nightly build and have it distributed to all company employees using TestFairy’s Over The Air (OTA) app distribution with automatic update.
Here is how they incentivized over 5,000 employees in taking part:
An employee using the nightly build automatically receives a 5% discount on every order made through the app.
It doesn’t always have to be about money. Recognition works just as well. Employee of the week with most submitted bugs to the nightly builds, or a specific thank-you from the CEO.
2. Reduce The Effort to Report Problems
On a call with one of our customer, a large US mobile carrier, we heard this mind-blowing claim:
If we have a bug in our mobile app, out of 100 people who have experienced it, only 1 will bother to submit a bug report.
It would take 4 back and forth emails for us to fully understand a bug.
Out of 10 emails to customers, only 1 would reply to our requests and help us debug and reproduce.
As a result, we may need 10,000 people to experience a fault just to be able to know about it and fix it.
Yes. The Bystander Effect is real. It could be laziness, but whatever it is, your users are not enjoying using your app.
This is a common problem among apps with large traffic of users, or when the output gained from this bug fix is lower than the effort to submit a bug report.
In this specific case, the mobile carrier changed their Submit Feedback button to do something completely different.
Instead of sending out an email, the button now asks you if you could please reproduce the bug while having screencast video recording using TestFairy Insights.
The results were incredible! The development team completely cut the need for a second email, and can now fix bugs rapidly. Talk about 100x improvement!
3. Automated Release Process
What really differentiates the pros from the noobs, is having processes in place.
You just can’t rely on an app that is being compiled on an employees private laptop. Xcode versions change, OS patches applied, git branch being compiled, were tags used?
All of these may affect the output IPA or APK, and suddenly what you tested last week is a completely different app from what you have now.
Thanks to many Continuous Integration (CI) services that are available today, it is so easy to get started. Check out Jenkins, CloudBees, CircleCI, Travis CI and many others.
Once the build is not happening on an individual’s laptop anymore, you open the doors to end-to-end continuous testing, nightly builds, and automatic app distribution.
TestFairy’s Upload API makes it easy to hook up CI compilation with distribution to testers or company employees and make your testing continuous. A simple curl, that’s all.
4. Staging and production apps on the same device
Nightly builds are not stable. They aspire to be, but we’re having nightly builds to detect errors as close as possible to their development.
Both Android and iOS allow a single instance of a specific app to be installed at any given time. So for My Application (com.example.app), I can have only one icon on my phone.
Companies with a stable app in the app store cannot ask their beta testers (or dogfooding employees) to uninstall their stable version of the app every time they want to try the nightly build. It’s very unproductive.
The solution? Having a second flavor (scheme) of the app, com.example.app.nightly that is being used by the non-app-store version.
With this trick, an employee can now hold both production and non-production versions of the app installed at the same time.
Developers go even further and change the icon of the nightly build app with a special banner so everybody knows which version they are using.
5. Rapid App Release
An interesting behavior we see among large software houses is that they constantly release new versions. Aside from the nightly builds that they distribute daily to their employees (see dogfooding) they also release versions at least once every 2 weeks.
Two weeks is a magic number.
On one hand, you have made so many small incremental changes to the code, that you are confident nothing will break when you hit Publish.
And on the other hand, an update every 2 weeks is not a burden on your users.
Flushing the changelog between your master branch and the version on the app store reduces instability problems and breaking configuration changes, optimizing for both customer satisfaction, and the developer’s peace of mind.
6. Not all issues are created equal
Priorities. Priorities. Priorities.
We are often asked to share “how others are handling the workload?”. The amount of code that needs to be written, bugs that must be fixed today, edge-cases to reproduce, is so high, that developers might sink in and do nothing productive but talk about problems.
Let’s clear the mess:
- Reports coming from an end user: a user is telling you about a problem she’s having with your app. It might be a button that’s not working or a page is not readable. These are important to go through. Feedbacks here are one step before submitting a negative rating on the app store. The key is to reduce friction as much as possible, and make it super-easy for an end-user to report problems.
- Crash reports: looking at an aggregated crash report, you can understand which bugs are the more urgent to fix. For example, 10,000 crashes by 1 user is definitely less important to fix than 10,0000 crashes by 10,000 different users. Be sure to know that all apps are shipped with known bugs: it’s more important to serve a wide audience with a great product.
- Analytics: analytics can tell you a lot about how people use the app, or the new features you have just released. Before moving to the next feature, they spend some time investigating if everything works as expected. The debt that you collect by improving a feature that was poorly designed in the first place grows and would reduce productivity to a halt.
7. Being comfortable with your tools
The tools you use define you, as a developer and as an organization.
Most of the enterprises we work with have Single Sign-on configured, some on OKTA, others on Ping Identity or Azure Active Directory.
Some of our customers use JIRA, others handle tasks in Trello or in Monday.com.
There are many reasons for preferring one tool over the other, but in the end, it’s how they all play together.
The tools are here to serve you, and if you are feeling that a tool is slowing you down, maybe it’s time to find out what you can do about it.
I will make a note here and say that while some teams within a company might choose JIRA, while other teams may use Zendesk. Different issues go to different places, as Elaine Benes famously once said.
For some teams Zendesk may be more appropriate for feedback collected from end users, while for others JIRA is more fit for keeping records of crashes.
A feedback sent to Zendesk will usually have somebody taking care of it, who will talk to the customer, a human, see how they can resolve this issue together and make sure the customer is satisfied.
An issue on JIRA is usually aggregated, and then put in a board, where it is prioritized in a meeting of the entire team.
All the data recorded by TestFairy can be exported by our REST API to any platform. We understand enterprises like yours. We have the capability to adapt our tools to your needs.
Final words
The key to a better development cycle is to constantly improve it.
Finding the weak spots in your workflow, that’s your job. At TestFairy we have helped mobile development teams improve their efficiency by 5x!
Learn from the pros.
Want to hear more?
Want to see how we can help you cut down costs, improve app ratings, optimize for customer satisfaction, and boost developer morale?
Come talk to us, we are TestFairy
