Thankfully, my first TestFlight submission was rejected

Last week I had a pleasant Zoom call with a UX researcher who found my Medium post about prototyping accessible apps with Protopie and wanted to know more about my experience. Protopie, unlike virtually all other wireframing and prototyping tools, provides access to a mobile device’s internal sensors, text-to-speech engine, and haptics, making it possible to design and test an app for people who are blind or have low vision.

But once you’ve designed and implemented your accessible app, how do you test it?

For a small company or a lone developer, it’s a burden and a risk to meet your testers in person during the pandemic. This difficulty in testing was a major reason my company pivoted in late 2020 from hardware development to app development: an app can be deployed, tested, and updated remotely, without once having to meet someone in person.

Apple’s TestFlight is intended for beta testing, but I needed to reach testers earlier in the development process, closer to the proof of concept stage. Before the pandemic, a proof of concept demo might only require a few days of coding. Then I would stuff my laptop and other hardware in a backpack and meet an accessibility expert for a demo. Could I submit a proof of concept to TestFlight? Or would my app get rejected for being underdeveloped?

Thankfully, my first TestFlight submission was rejected. The earliest version of the app was designed to demonstrate accessibility using a mockup of a real-world object. Apple TestFlight engineers didn’t have a copy of the mockup. They asked me to create a video demonstrating the app in use.

The first 10-minute video took nearly a week to create. It mixed narration, live screen recording, and video clips shot using an over-the-shoulder camera to prove I was interacting with the app in real time. Shortly after I provided the unpublished YouTube link to the TestFlight team my app was approved.

Now I make a video for every beta release. And I’m thankful because the beta release videos require me to provide accessible documentation.

Since that first beta release development has followed this process:

1. With my team, discuss the few app changes for the next software development sprint.
2. Code.
3. Write a script to explain the new changes.
4. Record narration using the MetaRecorder app and a Yelloway lavalier microphone plugged into my iPhone.
5. Use the iPhone’s built in screen recording feature. To record with VoiceOver, the microphone must be enabled during the recording, although you may not hear VoiceOver during the recording.
6. Use iMovie to splice, dice, mash, and squeeze the new material into a copy of the video project from the last release. A semblance of the script survives.
7. End up with two minutes of new material in the beginning of the video, followed by older clips about features that haven’t changed since the previous version.
8. Post the video on YouTube. Publish using an unlisted link so that only the TestFlight team and other people with the link can find the video.
9. Update the Test Information section in AppStoreConnect to provide the YouTube link and explain the changes.
10. Complete the TestFlight submission.

This process has worked for four beta release so far. For the second and later videos I needed less time to create a new video since only a quarter to a third is new, but the process didn’t seem to be easier. Just today I realized my process was backwards.

I’m a true believer in top-down design methodology. I want to create a wireframe or a functional prototype before a single low-level “algorithm” is written. Design the interface first. Make it magical. Delight the user. Get the interface right, then go make the software real.

And so I realized I should write the script before coding. Sure, I have lists of features and bug tickets planned for each release, but those don’t amount to a narrative.

A script tells a coherent story for the user. What is the unifying theme for this beta release? What hypothesis is being tested? What will the tester remember days or weeks later when you summarize the test results?

What you’re scripting isn’t a documentary that looks backward. It’s a utopian dream. Your video gives a sense how your app will make their life incrementally better. Aren’t you itching to see this beta become a real app?

If you got something from this post, and if one day you win an Oscar for Best Adapted Screenplay for a Beta Release Video, please mention me in your thank you speech before they rush you off stage.