Asynchronous Xcode UI Testing

As I continue to delve into Xcode UI tests, I’m starting to discover some of the lesser advertised benefits and drawbacks of the tool. One specific nuance that has caught my eye is how to deal with asynchronous Xcode UI testing. Xcode UI tests are asynchronous in that they simulate end user interaction with your application. End user interaction triggers animation (and in some cases waiting on remote services), and animation doesn’t happen instantaneously. There is a disconnect between the speed at which the lines of code in your tests can be executed, and whether the simulator (where the app under test is located and running) can keep up or not. In most cases, the simulator can’t keep up. As a result, you need to explicitly instruct your tests to wait for the app to catch up in the simulator. There are two ways to do this that I’ve been using.

Smart Waiting


When performing asynchronous Xcode UI testing, it’s often really useful to be able to verify something has appeared on screen. This way you know that the application is a certain state, and can continue executing a test script. For example, imagine an application with a tab bar, selecting a different tab should show a different view controller. If you were writing a test that relied on features within the second tab of the tab bar, it’s important have some assurances that when you instruct your test to tap the second tab, the tab actually appears. Furthermore, the act of waiting for something on screen can also act as a test in itself. If tapping the second tab doesn’t actually show what you expect, something probably went wrong and the test should fail.

Inspired from Joe Masilotti’s Cheat Sheet for Xcode UI testing, here’s how you can leverage waitForExpectationsWithTimeout(_:handler:) to verify that something appeared on screen:

// 1
let goLabel =["Go!"]
let exists = NSPredicate(format: "exists == true")
expectationForPredicate(exists, evaluatedWithObject: goLabel, handler: nil)

// 2 
app.buttons["Ready, set..."].tap()

// 3
waitForExpectationsWithTimeout(5, handler: nil)

Here’s an explanation for this code:

  1. Define a NSPredicate to be used in a XCTestExpectation to verify that the Go! button exists in the app.
  2. Tap the Ready, set… button.
  3. Wait for the Go! button to appear.

waitForExpectationsWithTimeout(_:handler:) will repeatedly look for the NSPredicate to be true within the timeout provided, in this case 5 seconds.

This approach to verifying that something exists on the screen is incredibly useful. It’s very much like my old favorite KIF API waitForViewWithAccessibilityLabel. The thing is, it isn’t perfect.

Not Perfect

I’ve observed a number of cases when using this approach has led to false positives that an element “exists” on the screen. Essentially the XCTest UI framework observes that the element exists, but isn’t actually tappable, or in a state that a human-end-user would consider “on the screen.” If seen this most often happen when this technique is used in combination with animations happening in conjunction with the test action. For example, if you have an application that taps a button, which triggers a navigation controller to push a new view controller onto the stack, there’s a relatively slow animation that happens. With the predicate approach laid out above, asynchronous Xcode UI testing will actually identify incoming screen elements as “existing” before they are in their “final resting spot” at the end of the animation. So if you gate further steps of your test on the presence of something on the screen as determined by the XCTest UI framework, you’ll get false positives that the element is on the screen and ready for further interaction. Essentially, Xcode UI tests will tell you, “Yes, that button exists on the screen,” but in reality it isn’t yet “on the screen.” Don’t worry though, there’s a solution, and it’s incredibly dumb.

Dumb Waiting


In those cases when you need to a little extra pause in your asynchronous Xcode UI testing in response to an asynchronous event like an animation, I’ve found that just using a sleep(1) goes a long way. Essentially this will block your tests from continuing for the specified duration of time, while the application under test is allowed to continue. To update the example code above, I add the sleep(1) in conjunction with the NSPredicate technique as follows:

let goLabel =["Go!"]
let exists = NSPredicate(format: "exists == true")
expectationForPredicate(exists, evaluatedWithObject: goLabel, handler: nil)

app.buttons["Ready, set..."].tap()

waitForExpectationsWithTimeout(5, handler: nil)

Notice that the sleep(1) happens right after the NSPredicate is determined to be true. This follows my discovery that Xcode UI tests identify things as “existing” before they are actually on screen. Using the NSPredicate technique with the sleep(1) let’s you verify the presence of specify UI elements, while also giving the app time to complete long running tasks like animations before continuing.

What’s funnier is that this sample code shown on screen at WWDC 2016 caught my eye:

asynchronous Xcode UI testing

Turns out, Apple uses the same technique!

Assertions in Closures

Another useful technique in asynchronous Xcode UI testing is to perform assertions on user-interface elements in response to closure execution. Often, indication of completion of long running calls like web service retrievals will be implemented with closures. I’ve written Xcode UI tests that explicitly call the web service separate from the application in order to cross-check data returned. Basically, I’ll ensure that the data the raw web service returns matches what is in the application. Whether this is your use case or not, you can use this pattern to make assertions in the closure on callback completion. The only requirement is that Xcode UI API calls need to happen on the main thread:

// 1
let expectation = expectationWithDescription("API Request Complete")
var response: Response?

// 2
RequestUpManager().get(id: "3688840") {
  (response: Response?, error: NSError?) -> () in
  if let response = response {
    response = response

    // 3
      // Verify there is something on screen that matches what the API provided.
      let cell =
      XCTAssertEqual(cell.staticTexts[response!.position].identifier, "Position")
  else {
    // 4
    XCTFail("Unexpected object returned from API")
// 5
waitForExpectationsWithTimeout(WAIT_TIMEOUT, handler: nil)
  1. Use expectations to keep the test running while the request loads.
  2. Synchronously load data from the API.
  3. Tell the main thread to verify what’s on screen. (This will crash if not done on the main thread).
  4. Explicitly fail the test if unwrapping the optional data returned from the API fails.
  5. Using XCTestExpectations, wait for the asynchronous code to complete.

Wrap Up

These are just a couple little tricks I’ve learned for asynchronous Xcode UI testing when taking my first dive into the API. It’s always nice to get in and try out a tool beyond what’s shown in documentation or videos, you get a much better handle for nuances of what’s available. Have you come up with any solutions for these challenges in asynchronous Xcode UI testing?

Happy cleaning.

Xcode UI Tests Review

After writing Xcode UI tests for the first time today, immediately some things formed my initial impression. I wanted to pass along my Xcode UI tests review here on in effort to help educate others. For at least the pass year, if not longer, I’ve been using KIF for writing my functional UI tests. I love KIF. KIF is a third party, open source tool. That brings both benefits and drawbacks. Right off the bat, due to changes in the iOS 10/Xcode 8 Accessibility Inspector, KIF was broken in the beta versions of our tools. It was quickly fixed, and the impact was low, but it does make you wonder about what’s in store for the future when there are less people available to maintain it? This wasn’t the only thing that drove me towards Xcode UI tests for this project, but it was certainly a concern. I still love KIF and will continue to use it on other projects for the foreseeable future.


Xcode UI tests build on an older tool from Apple called UI Automation. UI Automation enabled the iOS developer to write UI tests for their apps, and it was officially supported by Apple. “Officially supported” is certainly a loaded term, especially with UI Automation. The documentation was pretty bad. While there was some light guides on developing UI Automation tests, the actual API documentation had very little information. Combine that with the fact that the tests were written in JavaScript and run with Instruments (outside of Xcode), I was never happy with the tool.

Then last year, in WWDC 2015, Apple announced a reimagined solution for writing UI tests, Xcode UI tests. Here’s my Xcode UI tests review.


Officially Supported by Apple

This shouldn’t be underplayed. I’d live through a lot of crappy documentation just to use an officially supported tool. Apple officially supports Xcode UI tests, and that means a lot to me. It means you get WWDC videos on the subject. You get subtle integration with Xcode that only Apple could have access to. It means that your tests are less likely to break in future versions of iOS and Xcode.

Tests Written In Swift

Unlike UI Automation tests, Xcode UI tests can be written in either Objective-C or Swift. This could be the single best improvement. This means that you get everything from code completion, to compile-time syntax checking, to full Xcode integration.

Integration with Xcode

It was really annoying with UI Automation that you needed to switch to Instruments just to run your UI tests. Furthermore, JavaScript support in Xcode is pretty bad too, so one was left searching for a good editor experience when writing tests. Xcode UI tests change this. Now, the tests are first-class citizens right in the same IDE that we are using to write our actual application code. There’s even a “New File template” for Xcode UI tests.

Xcode UI Tests Review

XCTestCase subclasses

Buliding on the integration with Xcode, Xcode UI tests subclass XCTestCase just like unit tests. This means the full API of XCTest is available. Methods like XCTestAssertEqual and XCTestAssertTrue can be leveraged for writing your tests. Additionally, things like Scheme integration into your Test action enable execution of your Xcode UI tests when running tests, right from Xcode. Also, all those others niceties of running unit tests and the wonderful keyboard shortcuts can also be used with your Xcode UI tests.

Xcode UI Tests Review

UI Recording

UI Recording for Xcode UI tests is pretty cool. You can click a “Record” button in Xcode, and your app will launch in the simulator. Then, as you tap and navigate through your app, actual code will be generated in a test reflecting the path through your app that you took. It’s not perfect though, and more importantly, it doesn’t actually generate any assertions. Also, to prevent your test from being really brittle, you do need to consider breaking your tests up into cohesive chunks, and not one giant test.


Xcode UI tests aren’t without their flaws.

Documentation Is Still Light

The documentation for Xcode UI tests is still light. It’s not as bad as UI Automation was, but it’s still not perfect. KIF has been around for years. There are tons of questions and answers for it, the GitHub repository is mature, and you’ll find plenty of tutorials. I still can’t find an official piece of web-hosted API documentation for Xcode UI tests.

See the end of the article for a list of resources that I’ve been using as reference for Xcode UI testing.


Obviously all UI tests are going to be slow, regardless of whatever platform and tools you are using. Something uniquely slow jumps out at me about Xcode UI tests though. As a developer, you are required to call XCUIApplication().launch() from your tests when you want the app to launch. Each test must call this. The common pattern from what I’ve seen is to place the call to XCUIApplication().launch() in setUp(). setUp() is an overridden method from the XCTestCase base class, and is executed by the XCTest framework before each test. The thing is, XCUIApplication().launch() is REALLY SLOW. It triggers a fresh launch of your app. Now I guess this is useful from the context of resetting state in your app, but if you aren’t careful about structuring your tests, you could inadvertently introduce a lot of overhead in the speed of your tests.

iOS9 and Up

One of the main advantages of a suite of automated tests, regardless of the platform or type of tests, is to be able to run those tests in all places that your code base is supported. If you are writing a web application, this might be different browsers at different screen resolutions and window sizes. In iOS development, you can exponential benefit from running the same suite of tests across different devices of different form factors and different iOS versions. Unfortunately, Xcode UI tests put a major hamstring in this, in that they will only run on devices with iOS 9 and greater. To me, this is a big deal. We don’t all have the luxury of requiring the minimum iOS version to be the latest on the market. At the time of this post, iOS 9 is the latest iOS version, so that means I can’t even run my automated UI test suite on one version back, iOS 8. There have been so many times in my testing career in which UI tests identify and/or verify a bug that is present on one iOS version and not another. I’m really disappointed that Xcode UI tests don’t work on anything older than iOS 9. I guess I just have to hope for my applications, we’ll soon be able to require iOS 9 and greater.

XCUIElement != UIView

One of the most awesome things about KIF is the overlap between the KIF APIs and UIKit. When writing KIF tests, there are many methods like:

func waitForViewWithAccessibilityLabel(label: String!) -> UIView!

It’s so useful to have a UIView returned. This can be cast to more specialized subclasses that actually represent the object returned. And once you do that, you have full access to all the methods available on that object as defined in the actual code. Code like this is possible:

let saveButton = tester().waitForViewWithAccessibilityLabel("Save") as! UIButton
XCTAssertEqual(, saveButton.currentTitleColor)

This code is really useful because once you have a handle on the UIView, you can access so many APIs to verify behavior.

Unfortunately, there is no such intersection in Xcode UI tests. In Xcode UI tests, the object returned from similar method calls is a XCUIElement, and there’s no way to translate this to UIView (that I’ve found). As such, you are limited to the limited API available through Xcode UI tests and XCUIElement – none of which translates to a UIKit equivalent.

Final Thoughts

Despite the drawbacks, I’m still excited to use Xcode UI tests and I hope this Xcode UI tests review conveys that. It’s always fun to learn something new, and I think there are plenty of positives to using the tool. Have you used Xcode UI tests? What’s your Xcode UI tests review?

Happy cleaning.

Useful Resources