Calm's App Intents integration: how (and how not to) work with new Apple developer technologies

By Alaina Kafkes

This blog post has been adapted from a presentation that I gave at SwiftConf 2023.

Though the iOS engineering community is home to a diversity of dabblers, devotees, and debutants, we all have at least one thing in common: when June hits, we tune in to Apple's annual Worldwide Developers Conference, lovingly dubbed WWDC.

WWDC is chock-full of anticipation, usually positive, so iOS folks have bestowed it with yet another pet name: developer Christmas, birthday, or any other gift-receiving event of their choosing. And Tim and Craig give generously to us underlings. They shower us in Swift language improvements, new features, new frameworks, new platforms, and increasingly specialized processing chips to power them all. But unbox these toys, play around with them, and you may start to see the cracks. Sometimes Apple releases its novelties before they've finished building and documenting them, and, unfortunately, their developer gifts don't come with a return policy or satisfaction guarantee.

Last year, I learned this the hard way. As iOS 16 rolled out to the masses, I hopped on a project to metamorphosize Calm's SiriKit intents into App Intents, a Dub Dub darling of the year twenty-two.

Why and why now

Adopting the App Intents framework seemed like a favorable investment for Calm. From an engineering perspective, our SiriKit intents were crumbling under the weight of ancient bugs that the present-day iOS lineup lacked the context to fix. Migrating the Calm app to use the App Intents framework would also offer our iOS engineers a chance to try out async/await and @MainActor in relatively isolated components. Best of all, App Intents would automagically expose all intents to Siri, Shortcuts, and Spotlight as soon as a user downloaded the Calm app, which would obsolesce the cumbersome "Add to Siri" buttons that we buried deep within our in-app settings.

As luck would have it, business conditions augmented these engineering "why"s into roadmappable "why not now"s. Calm has a history of attracting users who download the app in a time of crisis, some of whom discontinue using it after only a few days. My team was tasked with re-engaging these users, and we hypothesized that a better intent implementation could help streamline re-entry into the Calm app. Learning about the App Intents framework and its promise of automagic system integration piqued my team's interest.

We slated an App Intents project mere months after WWDC 2022 because we believed in a connection between bite-sized applets and better user engagement. Such a project would be exploratory, yes, but small and seemingly low risk / high reward, given the brokenness of our existing SiriKit intents, so we Tetrised it into our Gantt chart.

In retrospect, I wish we would've waited. If waiting hadn't been an option, I wish we would've better prepared. But how does one prepare to work with a technology so new and ill-understood?

Here, I've tried to write the blog post that I would've wanted to read before wading into any new Apple framework. I'll share the struggles that I faced and overcame while applying the App Intents framework to the Calm iOS app, and distill general guidance from my harrowing developer experience that I hope will come in handy for anyone productionizing new WWDC offerings.

But first, code!

I'll start by grounding my soon-to-be-shared struggles in App Intents code. An awareness of the shape of an app intent will free readers up to focus on the general points of advice that I've dovetailed with each struggle.

Here's a pared-down code snippet of an app intent that plays the Daily Calm, Calm’s most iconic daily meditation.

struct PlayDailyCalm: CustomIntentMigratedAppIntent {
   static let intentClassName = "PlayDailyCalmIntent"
   static var title = "Play the Daily Calm"

   func perform() async throws -> some IntentResult {
      // insert logic to play the daily calm here
      return .result()
   }
}

Its key players are:

• CustomIntentMigratedAppIntent, a protocol that inherits from the basic AppIntent protocol and conveys to the system that this app intent has been migrated from a prior SiriKit intent;
• intentClassName, the unique identifier for this app intent;
• title, which contains the phrase that a user can speak to Siri to invoke this app intent; and
• perform(), the method that executes the app intent's main functionality. In this case, perform() will play today's Daily Calm.

In the second code snippet (immediately below), I've generalized the initial PlayDailyCalm app intent to turn it into PlayDailyProgram, which plays the latest guide from one of Calm's daily meditation programs: the Daily Calm, Jay, Trip, or Move.

struct PlayDailyProgram: AppIntent {
   static let intentClassName = "PlayDailyXIntent"
   static var title = "Play a daily program on Calm"

   @Parameter(title: "Program type")
   var programType: PlayGenericDailyProgramType

   static var parameterSummary: some ParameterSummary {
        Summary("Play the \(\.$programType) on Calm")
    }  

   func perform() async throws -> some IntentResult {
       // insert logic to play a daily program here
       return .result()
   }
}

The programType IntentParameter powers this flexibility. The @Parameter property wrapper instantiates this IntentParameter, and parameterSummary provides a string interpolation that Siri and Shortcuts use to correctly interpret and execute user invocations of this app intent. Thanks to this parameter, the user can ask Siri to run this app intent by saying "Hey Siri, play the Daily Trip on Calm" or "Hey Siri, play the Daily Jay on Calm."

Finally, this CalmShortcutsProvider code snippet shows how an app can automagically integrate its app intents (as AppShortcuts) with the Shortcuts app.

struct CalmShortcutsProvider: AppShortcutsProvider {
   static var appShortcuts: [AppShortcut] {
      AppShortcut(
         intent: PlayDailyProgram(),
         phrases: [
            "Play the \(\.$programType) on \(.applicationName)”,
            // insert more invocation phrases here
         ]
      )
   }
}

Enough toy code snippets: let's hop on the real-world struggle bus!

Struggle #1: Disappearing intents

I embarked upon this App Intents migration project by creating a bare-bones app intent not unlike the first toy example shared above. I was able to validate this skeletal app intent by observing it within the Shortcuts app, but, as soon as I tried to flesh it out, it disappeared on me. When I attempted to troubleshoot by creating another minimally viable app intent, that intent didn't show up either. Attempting to add business logic to an app intent effectively erased Calm from the Shortcuts app.

To solve this mystery, I flipped through (okay, clicked around) the usually unflappable Apple Developer docs. If Apple writes the rules, then it must have the answers, right?

What I hadn't accounted for was that Apple dropped the App Intents framework well before it finished documenting it. Four months after its launch at WWDC 2022, Apple Developer documentation for App Intents was sparse at best. Nor were the Apple Developer Forums any use: because the framework was so new, there were only questions, not answers. As of the time of this writing, many such questions remain unanswered.

When the source of truth offers nothing but silence, where should you turn? Trial and error is certainly an option, but I had one more card to play: Twitter dot com. I trawled through FKA Twitter, searching for queries like "app shortcuts error" until I found this veritable gift of a tweet.

Apple stuffs helpful App Intents compilation information in the build logs in a step named "Extract app intents metadata." Like tweet author Michael Tigas, my app intent implementation contained "at least one halting error" caused by a faulty AppEnum that I had created while adding business logic. (An IntentParameter with discrete known values conforms to the AppEnum protocol. For example, PlayGenericDailyProgramType, as seen in the PlayDailyProgram toy example above, is an AppEnum with values trip for the Daily Trip, calm for the Daily Calm, move for the Daily Move, and jay for the Daily Jay.) The AppEnum protocol required me to work with new types such as DisplayRepresentation and LocalizedStringResource, both of which I had bungled. (More on the spicy LocalizedStringResource type later!)

After dredging up this tweet, I was able to quickly fix my app intent implementation, et voilà! The "Extract app intents metadata" build step rewarded me with a green checkmark and Calm's app intents sauntered their way back into the Shortcuts app.

Without this tweet, it would've taken me much longer to look for the the "Extract app intents metadata" step in the build logs and debug from there. At the risk of stating the obvious, I want to emphasize how helpful it can be to seek answers from the wider iOS community, not just Apple-sanctioned sources. Searching through tweets or Slack messages is not so different from turning to a coworker to ask a quick question: you may have eventually arrived at an answer yourself, but asking for advice will get you there faster, and with less strife.

Here are a few iOS communities that I turn to when I can't get answers from Apple:

• iOS Folks Slack
• Accessible Mobile Apps Slack
• Twitter/Github/etc. follows and searches
• Former coworkers
• Fellow iOS conference attendees

I encourage you to join and grow these (and other) iOS communities, which will result in more questions, more perspectives, more answers, and ultimately, more learning for all.

Struggle #2: Opaque method signatures

Remember the perform() method that I mentioned in the toy examples section? Though it is the most important method in an app intent declaration, it was challenging to figure out its return value.

I read in the Apple Developer documentation that the successful return type of perform() should be a PerformResult, which inherits from the IntentResult protocol. When I tracked down the IntentResult docs, its twenty-eight possible method signatures overwhelmed me.

The Apple Developer documentation for IntentParameter aka @Parameter further swamped me. The convenience initializers looked indistinguishable, and I struggled to draw the connection between hefty initializers like

convenience init<Spec, OptionsProvider>(title: LocalizedStringResource, description: LocalizedStringResource?, inputOptions: String.IntentInputOptions?, requestValueDialog: IntentDialog?, inputConnectionBehavior: InputConnectionBehavior, optionsProvider: OptionsProvider, resolvers: () -> Spec)

and the snappy property wrapper declaration syntax of

@Parameter(title: "Program type")

The @Parameter declaration shown above is valid, though it doesn't include all of the seemingly required method arguments shown in the convenience init. Odd!

Fortunately, this struggle was more straightforward to overcome. I found that these convoluted method signatures were easier to break down in Xcode's built-in docs than in the online Apple Developer ones. Though both the Xcode and online documentation come from Apple and thus should say the same thing, I noticed a few discrepancies.

Xcode's right-click then jump-to-definition feature surfaces the default values of method arguments much sooner. In Xcode, it takes fewer clicks to realize, for example, that the hefty IntentParameter convenience init shown above equips most of its arguments with default values. (Bless!) You have to dig deeper into the online docs to find this information.

Upon learning that most IntentParameter initializer arguments defaulted to nil, I was able to figure out how to zhuzh up the PlayDailyProgram app intent's @Parameter with an interactive dialog.

@Parameter(
   title: "Program type",
   requestValueDialog: "Please select..."
)
var programType: PlayGenericDailyProgramType

I found Xcode documentation to not only be faster, but also more specific than its online counterpart. Recall my perform() method gripe from a few paragraphs ago? Xcode empowered me to right-click on the dialog argument in result(dialog:) - the IntentResult returned by the perform() method I was working on - and drill down into its requirements in order to validate my code. It is not possible to dive into dialog and the other individual arguments of result() on the online documentation.

Admittedly, I haven't upheld my stated preference for Xcode over online docs outside of my work with the App Intents framework. For new and under-documented Apple frameworks, I would counsel trying both the Apple Developer online and Xcode documentation options and picking your favorite for that framework.

Struggle #3: Haunted string localization

The App Intents framework required a new type for declaring localizable strings, and this new type did not play nicely with the traditional NSLocalizedString plus Localizable.strings files localization strategy employed by Calm.

Meet my enemy - er, the new localization type - LocalizedStringResource. As much as it vexed me, Apple had its reasons for requiring it in the App Intents framework. Per the Apple Developer docs, "LocalizedStringResource [is used] to perform a late resolution of localized strings. This allows the Siri UI to potentially use different localization preferences than the app providing the intent." (Emphasis added by me.)

Sounds innocuous, and maybe even useful for some apps. So why did this type make waves in the Calm codebase?

Let's step back and chat about how an iOS engineer at Calm adds a new localized string for the next version of the app.

First, the engineer declares a new NSLocalizedString in the main app target or an ancillary package. In Swift, this might look like

NSLocalizedString(
   "Strings.Skip",
   bundle: Bundle.main,
   value: "Skip",
   comment: "Title for skip action"
)

A week before we plan to release the next version of the app, an automated script extracts all NSLocalizedString keys (e.g., "Strings.Skip") and values (e.g., "Skip") from the Calm codebase and uploads them to a third-party localization service.

This service generates a localized value in every language supported by the Calm app for each new key, and compiles these translations into Localizable.strings files (one per supported language). In the French Localizable.strings file, "Strings.Skip" might be translated as

/* Title for skip action */
“Strings.Skip” = "Ignorer";

Another automated script downloads these Localizable.strings files and overwrites the existing ones in the Calm codebase. These Localizable.strings files get bundled into the next App Store submission build of the Calm app, which enables the next app version to swap in the appropriate translation of that new string based on its key.

Because it seemed unnecessarily complex to write new scripts for uploading LocalizedStringResources and downloading their translations, I attempted to shoehorn NSLocalizedStrings into LocalizedStringResource initializers.

LocalizedStringResource(
   stringLiteral: NSLocalizedString(
      "PlayDailyProgram.ParameterTitle",
      value: "Program type"
   )
)

Treating LocalizedStringResource as a wrapper for NSLocalizedStrings would allow me to work within Calm's existing localization workflows.

Though the compiler did not complain when I initalized a new LocalizedStringResource with an NSLocalizedString with the key "PlayDailyProgram.ParameterTitle", the text stayed stubbornly in English no matter how many times I switched device languages.

To add insult to injury, I couldn't write NSLocalizedStrings to supply invocation phrases for an AppShortcut due to the latter's unusual keyword interpolation syntax (e.g., $programType in the CalmShortcutsProvider toy example from a few sections back).

How was I supposed to get LocalizedStringResource working without disrupting Calm's existing localization workflow? Once again, Apple's documentation offered no guidance, and the folks in the Apple Developer Forums were just as confused about LocalizedStringResource as I was.

My false starts prompted voracious Googling, which led me to this beautiful blog post by Arnaud Joubay. Tears may actually have fallen from my eyes upon discovering it because Joubay answered not one, but all of my questions. I wondered: how did this non-Apple employee create the definitive source for AppIntent and AppShortcut localization?

In short, he tweeted at a Shortcuts engineer and got critical localization guidance that to this day does not exist in the Apple Developer docs.

Arnaud's aplomb reminded me that Apple isn't a black box, as much as it may try to appear that way to outsiders. Plenty of Apple engineers dwell in the same iOS community spaces as you and me, and might be willing to field a polite inquiry. Should you work for a corporate app, your company's Apple developer representative would also be happy to bring your question to the right Apple engineers.

This game-changing blog post has sparked within me the aplomb to turn this hot tip into a request: if you manage to get valuable, non-proprietary intel from Apple, please share it within your iOS communities! Your blog post or conference talk may have the power to unblock someone else going through the same struggles as you. Pay it forward, as Joubay did and as I'm doing now.

Struggle #4: Intent identity crisis

After surmounting the three aforementioned struggles, I (mostly) sailed through the rest of my rewrite of Calm's SiriKit intents as app intents. Unfortunately, at the eleventh hour, I learned from Calm's Apple representative that the App Intents framework wasn't meant to supersede all SiriKit intents. SiriKit is still the best tool for creating intents that play media, which, awkwardly, is what nine out of the ten Calm app intents do. SiriKit's INPlayMediaIntent would've suited Calm's needs better than the App Intents framework, which is most appropriate for building custom intents not already supported by SiriKit.

My dignity deflated. I realized that my team had gotten too caught up in the promise of app intents at WWDC 2022. We were susceptible to this rosy outlook because our old SiriKit intent implementation - which, unsurprisingly, did not use INPlayMediaIntent - had so many intractable bugs. We assumed it best to raze and rebuild.

Though that gut reaction may have been valid, we should've grounded it in evidence at the outset of this project. I wish that I had compared the performance of one Calm SiriKit intent re-implemented as an INPlayMediaIntent against that same intent re-implemented as an AppIntent before deciding whether adopt the App Intents framework. I've since learned of the steel-threaded approach to rearchitecting software, and its principles corroborate my would've-could've-should've retrospection.

I entreat other teams to let data drive the decision to migrate or not migrate a component, no matter how small.

App Intents: The Good Parts

I will concede that a few good things did come from Calm's early App Intents adoption.

Calm's app intents _do_ seem to propel habit formation within the app. While only 8% of Calm's SiriKit intent users invoked more than one intent per day, that percentage has spiked to 30% since we shipped Calm's app intents. App intents users actually do engage more with the Calm app, which is a promising outcome that I like to cling to after all this toil and trouble.

Calm has also enjoyed a fruitful collaboration with Apple throughout this project that has fostered improvements to the App Intents framework. Fixes for the bugs that I spotted were escalated and shipped quickly in minor and patch versions between iOS 16.2 and 16.4.

And, lastly, I must confess that it has been a pleasure to turn my chaos into counsel for future pioneering iOS developers.

Alighting the struggle bus

Writing this blog post a year after working on Calm's rocky App Intents integration, I feel more aware of the strategies that I can employ to puzzle through new and sparsely documented Apple frameworks. I hope that, by sharing my specific struggles and the general learnings that I have synthesized from those struggles, readers can skip the struggle bus altogether and simply relish in playing with the fun new toys that Apple gives us year after year.

Before I bounce from the metaphorical struggle bus stop, I would like to thank my teammates Jamie and Nate, as well as Calm's Apple partners Ennis, Ziqing, and Eric, for their support while I forged my way through this project.