Separate Schemes For Better iOS Code Coverage

As you continue to construct the test pyramid for your iOS apps to Martin Fowler’s specification, you’ll find the desire to separately measure your code coverage for both unit tests and functional tests. And if you don’t have that desire, then you should know that measuring a combined code coverage for both unit and functional tests is not an accurate representation of how your production code is truly covered. For one, functional tests will generally have a higher coverage, while also including less assertions. This is because of how much code gets executed while broadly navigating through your app as functional testing does. This data will then be glom’d on to the code coverage metrics for your unit tests, making it really hard to identify where you can refine the coverage of your unit tests. And the opposite can also be true, where you have unit test coverage and no functional test coverage, but since you are only evaluating a single number, you can’t figure out where one ends and one begins. To fix this, use separate schemes for better iOS code coverage. By moving each group of tests into their own scheme, you’ll be able to measure code coverage independently.

The Fix – Separate Schemes

To fix this, you are going to create separate schemes for better iOS code coverage. To start, you are going to duplicate the main scheme for your app, and then specify a different set of tests for each scheme. I’ll walk you through creating a separate scheme for your unit and functional tests. For this example, we’ll be using KIF as the tool for the functional tests (Yes, soon enough I’m going to take a deep dive into Xcode’s new UI tests). For this walkthrough, you’re going to continue with the project from Wednesday where you setup code coverage for unit tests. You can find the project on GitHub if you just want to start there. https://github.com/obuseme/CodeCoverage.

To start, open the Scheme Editor and duplicate the one and only scheme.

Separate Schemes For Better IOS Code Coverage

The new, duplicate, scheme that you created will be used for executing your functional tests, while the initial scheme will be used for your unit tests. The benefit of this is that you’ll be able to separately calculate code coverage for each set of tests. The drawback is that you’ll need to manually switch the scheme to execute each set of tests.

After you’ve duplicated the scheme, rename the new scheme to CodeCoverageFunctionalTests.

Next, create a new “iOS Unit Testing Bundle” target named CodeCoverageFunctionalTests. This target will contain your KIF tests.

Next, install KIF, preferably with CocoaPods. For detailed instructions on doing this, follow my previous post here. Ensure that the target you specify for where KIF should be installed is CodeCoverageFunctionalTests. And of course, after first configuring CocoaPods with a project, after you install pods for the first time, close CodeCoverage.xcodeproj and open CodeCoverage.xcworkspace instead.

And for the final step, edit each scheme to selectively use one of the two test targets for its Test step. First, edit the scheme for CodeCoverage and set the test suite for the Test step to be CodeCoverage. Make sure Gather code coverage is selected as well in each scheme.

Separate Schemes For Better IOS Code Coverage

Then, edit the scheme for CodeCoverageFunctionalTests and set the test suite for the Test step to be CodeCoverageFunctionalTests.

Separate Schemes For Better IOS Code Coverage

BOOM! That’s it. Now you can explicitly change schemes, and use Command-u to run tests for each scheme. Depending on the scheme selected, a different set of tests will be run – either the unit tests or the functional tests, and as a result, different code coverage metrics will be generated.

Wrap Up

This post concludes a week of code coverage information. I hope you enjoyed it, and I hope your tests are better covered as a result separate schemes for better iOS code coverage. I’d love to hear how you’re using code coverage. Please post a comment and let me know!

Happy cleaning!

Broken Code Coverage in Xcode: How To Fix It

When using metrics to make decisions about your code, it’s fundamentally important that those metrics are 100% correct. You need to have absolute faith in the reported numbers. If this is not the case, you risk making decisions and taking action through inaccurate data, and risk making incorrect decisions. By default, there’s a critical flaw in how code coverage is measured by Xcode for iOS apps. From the moment you setup unit tests for a project, Xcode will automatically identify code as “covered” for anything that is triggered through the normal application launch sequence, such as your application delegate. This means that your code coverage numbers will be artificially inflated! And broken code coverage in Xcode means you won’t fully understand how well your app is tested.

Let me say this one more time to it sinks in, your iOS code coverage numbers are not correct unless you take specific action to fix them.

An Example Of Broken Code Coverage in Xcode

First, I’m going to demonstrate to you the broken-ness of code coverage in Xcode. Then, I’m going to show you how to fix it.

To observe the broken code coverage, you are going to perform these steps:

1) Create a new empty project, including unit tests 2) Leave the unit tests empty 3) Turn on code coverage 4) Run the tests, and review coverage

Let’s do it. First, create a new Swift iOS project called CodeCoverage. Be sure to check Include Unit Tests.

broken code coverage in xcode

Open CodeCoverageTests.swift. You aren’t going to make any changes to this file, but notice how there are two empty test implementations testExample() and testPerformanceExample(). These tests will run and pass, but should generate 0% coverage of the application.

Now, turn on Code Coverage. Open the Scheme Editor and check Gather coverage data.

broken code coverage in xcode

Finally, run the tests. Command-U (you only get a keyboard shortcut today :). Open the Code Coverage results from the Report Navigator.

broken code coverage in xcode

Uhhh, what’s wrong with that picture? It should be obvious, code coverage is being shown for ViewController and AppDelegate despite there being absolutely no legitimate tests in the project.

Why There Is Broken Code Coverage in Xcode

Well, I wouldn’t blame it all on Xcode. Xcode is measuring the code that is executing when your tests execute. And technically, since you app is starting up and showing the first view controller, that code has executed, so it’s reported as covered. The thing is, by the definition of how you want to measure code coverage, that code isn’t actually “covered.” There’s a really easy way to correct this.

How To Fix Broken Code Coverage in Xcode

Jon Reid’s article on How to Switch Your App Delegate for Fast Tests inspired me to figure out how to fix this. You are going to create a separate app delegate that is used by your tests. This app delegate will be entirely empty, so it totally intercepts the app launch sequence. This way, no code in your real app delegate will be executed unless explicitly done so from a test, and ditto for any view controller that it would have otherwise instantiated.

Note: I want to give full attribution to Jon Reid on this code. I just figured out that it also fixes broken code coverage in Xcode.

To fix this, first open AppDelegate.swift and delete this line:

@UIApplicationMain

Create a new Swift file named TestingAppDelegate.swift, and replace it’s contents with:

import UIKit

class TestingAppDelegate: UIResponder {
}

This is the meat of the fix. It’s an empty implementation of an app delegate that will be used rather than your “real” app delegate.

Create a new Swift file named main.swift, and replace its contents with:

import UIKit

let isRunningTests = NSClassFromString("XCTestCase") != nil
let appDelegateClass : AnyClass = isRunningTests ? TestingAppDelegate.self : AppDelegate.self
UIApplicationMain(Process.argc, Process.unsafeArgv, nil, NSStringFromClass(appDelegateClass))

This is the first code that executes on app launch. It first checks whether XCTestCase is an available class to determine whether the app is being launched from tests or not. Depending on the result, a decision is made as to which app delegate should be used – the real one, or the empty one.

That’s it. Now re-run your tests and open your coverage report.

Note: You may need to Clean for a successful build.

broken code coverage in xcode

Woohoo! 0% coverage. Ya, that’s the only time you’ll ever be happy about 0% coverage, but in our case, we have no legitimate tests, so it’s what we want! Yay, we fixed our code coverage.

Side Benefit: Faster Tests

A side benefit of this fix for correcting broken code coverage in Xcode is that your tests will run faster. By alleviating the simulator of bootstrapping a significant portion of app startup, you’ll save that time each test run. I just compared the before state and after state of this fix on one of my current projects where we have about 500 unit tests. Before the fix, the tests ran in 21 seconds. After the fix the tests ran in 19 seconds. That’s about a 5% speed increase. Multiple 2 seconds over the large number of times that the tests will be run, and that’s a lot of time.

Looking forward

I added the final project to GitHub at https://github.com/obuseme/CodeCoverage. I hope that you find use in this approach. Just remember, you want 100% confidence in your code metrics. For me, if I notice something wrong with one of my code metrics, I stop using it until I get to the bottom of the false data.

Tomorrow, I want to show you how you can gather separate code coverage metrics for your different types of tests. Hint, it involves some crafty Scheme creation.

Happy cleaning.

Swift Code Coverage: How to measure it

Yesterday, I talked about the merits of how code coverage can be used as a metric in your software development process. Measuring Swift code coverage in Xcode has never been easier. Apple provides a great overview in their WWDC 2015 session, “Continuous Integration and Code Coverage in Xcode”. Xcode 7 provides a integrated experience for tracking the coverage of your tests. You can literally start measuring the code coverage of your tests by clicking a single checkbox. And good news, the coverage also works for your KIF tests.

How To Turn On Code Coverage Measurement

Open your scheme editor by selecting Product -> Scheme -> Edit Scheme, or the keyboard shortcut Command-Shift-Comma.

Select Test in the left hand pane, and then check the box for Gather code coverage.

swift code coverage

That’s it! On your next test run, Xcode will measure the Swift code coverage of your tests.

Viewing Swift Code Coverage Results

Viewing the results of how your tests fare with code coverage is just as easy. Run your tests, and then open the Report Navigator. You can open this by either selecting the thing that looks like a chat bubble in the left hand pane of Xcode, or select the menu Product->Scheme->Edit Scheme…, or use the keyboard shortcut Command-8. Then select your most recent Test run in the list.

swift code coverage

From there, in the center pane of Xcode, look for the Coverage tab.

swift code coverage

On the Coverage tab, you’ll see a list of your classes, and their methods, with bar charts indicating how much of their code is covered from your tests.

You can then even jump right to the corresponding code from the coverage viewer. Just double click either the class or method name. And what’s even cooler, is that Xcode will show you the number of times the line of code was executed in the right hand gutter of the editor.

Overlapping Test Types

Keep in mind, that depending on how your test targets are configured in the scheme, that the results you are looking at may be an aggregate of more than one type of test. For example, if you’ve created both unit and UI tests, and they each live in their own targets, but both targets are included in the Test action for the scheme, that the coverage numbers will be an aggregate of both types of tests. I’ll write a post later this week with a proposal on how you can separate these metrics (hint: it involves creating separate schemes for each type of tests).

Wrap Up

Measuring your Swift code coverage really is that easy. Normally code coverage is also tracked in actual numbers, and reviewed for trends over time. With Xcode alone, it doesn’t provide this. Xcode Server helps remediate the problem while providing specific measurement numbers and also allowing you to compare coverage numbers across different devices. Have fun with code coverage. When working in a team, it’s fun to watch code coverage change over time, hopefully for the better. I suggest that even as you code review your peers’ work, peak at how the code coverage for a given piece of code changes with their change. Does it go up? Does it go down? Remember, code coverage is just a metric that indicates whether a line of code was executed or not. It doesn’t speak at all to the quality of the test. Please be a professional, and professionals don’t write code without using test driven development.

Happy cleaning.

Code Coverage Is A Silver Bullet

Have you ever measured code coverage on any of your projects? How did it work out for you? What problems did it solve? Did it present any new problems? Code coverage is a silver bullet, a silver bullet for understanding how well your tests cover your code. Beyond that, it’s really what you make of it.

What Is Code Coverage

Code coverage is a metric that measures how much of your “production” codebase is being tested by your automated tests, usually unit tests. It’s usually measured at the granularity of a line of code, and sometimes measures methods and classes in aggregate. It’s nothing more than a black and white measurement, how much production code was executed when a given test suite was run. There’s no judgements made to the quality of the tests, simply, how much code did they execute.

Where Code Coverage Gets Tricky

Just to reiterate, code coverage as a metric makes no claims about the quality of tests. If you write a “test” (intentionally in quotes), that simply calls a method and does nothing with the result, the method it calls will have a high level of coverage. That being said, the “test” has done nothing to actually verify an outcome, or craftily provide edge case inducing input. It simply calls the method and discards the result. Voilah, high coverage, crappy test. For example:

class Adder {
  func add(x: Int, y: Int) -> Int {
    return x + y
  }
}

Here’s a corresponding test:

func testAdd() {
  let toTest = Adder()
  toTest.add(2, y: 3)   
}

Adder right now has 100% unit test code coverage! Yay!

Wait a minute, slow down hoss. There’s not a single assertion made in that test. While the test generates a high code coverage metric, it doesn’t validate squat.

code coverage is a silver bullet

TDD FTW

This is where following your test driven development cycle of “Red, Green, Refactor” you’ll never get into this state of test crappiness, and it can truly ensure that your code coverage is a silver bullet. The “red” step of that cycle is critically important, as it ensures that your test actually verifies something. Without knowing that your test can actually fail, you never know that your test actually does anything. And likewise, that your code coverage even means anything.

Be a Professional

The biggest counter argument you will hear about measuring code coverage, is that it can be cheated. Of course it can be cheated! Software “professionals” don’t cheat though. Craftsmen don’t take shortcuts. My life was changed when I read “The Clean Coder” by Bob Martin. He talks about what it takes to be a “professional software engineer.” (He also advocates for 100% code coverage despite all other costs, but I have other opinions on that). It’s a sense of taking pride in your work. With this post, I just want to put this call to action out there, be a professional. Code coverage measurement is just another tool in your toolbox. Code coverage is a silver bullet, but only one that returns to you what you put into it. Crap in, crap out. Use it appropriately, know it’s shortcomings, and know where it shines. When properly used in conjunction with TDD it can powerfully help you continually improve over time. I’d love for you to try it out, and let me know how it works for you.

I was inspired to write this article from listening to episode 67 of “This Agile Life” podcast, where they reference this article, “Is Code Coverage a Silver Bullet?

Happy cleaning.

KIF Tips and Tricks

Now that you’ve written your first KIF test or two, there’s a couple more KIF tips and tricks I wanted to share with you. Nothing too fancy, just a couple nice touches I’ve developed while writing KIF tests.

Don’t Forget About XCTestCase

KIFTestCase is a subclass of XCTestCase. This means that all the goodness of the XCTest framework is available to you in KIF tests. This makes for some really nice KIF tips and tricks.

XCTAssertEqual, XCTAssertTrue and related methods

These are all methods that would be familiar to anyone writing unit tests. These are the meat of how you make assertions about outcomes and expectations when writing unit tests. You can do the same thing in KIF tests. It’s especially powerful when combined with tester().waitForViewWithAccessibilityLabel(String) since that method returns a UIView. You can cast that view to a UIView subclass, and then access any custom properties on it, and then make assertions.

For example, suppose you have a view that should change colors in response to a button being pressed. You could write this KIF test:

func testViewChangesColor_WhenButtonPressed() {
  tester().tapViewWithAccessibilityLabel("Some View")
  let redView = tester().waitForViewWithAccessibilityLabel("the supposed red view")
  XCTAssertEqual(redView.backgroundColor, UIColor.redColor())
}

In this test, you programmatically tap a view with a given accessibility label, presumably the button. Then, you get a reference to the view that should have changed colors, and make an assertion on its background color.

setUp(), tearDown(), beforeAll(), and afterAll()

setUp(), beforeAll(), and tearDown() are powerful methods that help you do common legwork before or after tests run. They help to stabilize state between tests, and remove redundant code by providing a single place for it to be executed. setUp() and tearDown() run before and after each test method in the test class. These are really useful if each test needs to assume some sort of initial state. Imagine you are testing a view that represents a form. Before each test, you want that form to be in a clean state. These methods can enable you to clean up, or set some intial state before each test runs.

beforeAll() and afterAll() run before or after all tests in a given test class. These are useful when a given test class contains tests for a certain view in the app, that isn’t the initial view of the app. Say you are trying to test the third view controller deep in a navigation stack. It would be appropriate in beforeAll() to navigate down the stack to the view to test, and then in afterAll() to pop back up to the root view for other tests to run.

This leads me to the next item in my KIF tips and tricks, some suggestions on how to break up your test classes.

Segmenting Your Tests

The key to maintainable KIF tests is good segmentation of what you’re testing, across different tests and test suites. A “test” refers to a single function in a KIFTestCase subclass. A “test suite” refers to an entire KIFTestCase subclass, and all the tests within it. I don’t have any hard and fast rules on how I break up my KIF tests. Thinking through my KIF tips and tricks, I would phrase my suggested best practice as, group tests of related functionality into a single test suite, while keeping your tests themselves standalone and cohesive. As much as you can avoid it, avoid any interdependcies between tests. If later you go back and delete tests, or add tests, you don’t want failures to crop up just because the order of execution changes based on assumptions of state you made from test to test. I might have a test class/suite called “EditModeTests” that goes through all the verification necessary for “Edit Mode” of the thing I’m building. Remember, at the end of the day, KIF tests are slow, so you don’t want a lot of redundancy between tests in terms of execution steps. So if you have the opportunity to perform verification and assertions on related items in a test, do it, as long as you aren’t totally sacrificing decoupling of that test from other tests. I know what you’re thinking, I’m proposing contradictory best practices. It’s all balance. You’ll feel it out as you go, I just wanted to bring up a couple things to be aware of. Remember, when a test fails, the best thing you can do to help yourself is to do everything possible to reduce the amount of time it takes to figure out why it failed. I see two easy ways to do this: ensure your tests don’t fail, and ensure that when your tests fail the context of why the test failed is clear.

KIFUITestActor Extension

KIF Tips and Tricks

KIFUITestActor is the class of the tester() available in KIFTestCases. It’s what you use to perform the navigation through your app. Don’t forget about extensions, they are a great way to add behavior to KIFUITestActor, especially common pieces of code for repetitive navigatoin tasks. For example, one of my apps conditionally shows an onboarding flow depending if the user is launching the app for the first time or not. I added two methods to a KIFUITestActor extension – one to check if the onboarding view was showing, and one method to close the onboarding flow if it was showing. This way, in all my KIF tests, I can reuse this code and have the confidence in the repeatability of the test. It’s KIF tips and tricks like this that make me really enjoy iOS functional testing.

Verify Something Is NOT On The Screen

KIF makes it really easy to verify that something IS on the screen, but there’s no obvious API for verifying something isn’t on the screen. You can use Swift’s do/try/catch to achieve this.

Consider this test:

func testPreviewIsNotAvailable() {
  do {
    try tester().tryFindingViewWithAccessibilityLabel("Preview")
    XCTFail("Preview should not be found.")
  } catch {
    // Nothing to do here - a throw here is a success.
  }
}

This test verifies that “Preview” is not available on the screen. KIF will throw an exception when it can’t find a view with the matching accessibility label after a 10 second timeout. That exception will be caught by the catch handler, at which point nothing is done, and the test will pass. In the case that a view with a matching accessibility label IS found, the test is explicitly told to fail. If you use this pattern, I suggest a good comment in the empty catch block so you help your future self and others understand what’s happening.

Wrap Up

I hope you find use of these KIF tips and tricks, and I hope that you are setup well to have success with your journey into iOS functional testing with KIF. This wraps up the week of KIF. I’d love to hear how you it works for you.

Happy cleaning.

Writing Your First KIF Test

Earlier this week I introduced you to the concept of functional testing on iOS. Yesterday, I showed you how to install KIF, a tool for functional testing on iOS. Today, I’m going to walk you through writing your first KIF test.

If you haven’t done so, please start by downloading the demo application, and installing KIF from yesterday’s post, . This article will continue where it left off. If you want to skip that, and just start with a fully configured demo app with KIF installed, you can download that here.

Demo Of The Demo App

From the root directory of the Demo app, KIFDemoApp, open the workspace file KIFDemoApp.xcworkspace in Xcode.

Build and run the app, and give it a try.

Writing Your First KIF Test

You’ll notice a view with a text field, and two buttons. Each button presents a new view. The difference is that one button will replay back the text that you specified in the field, and the other button just shows the next view with static text.

Step 0 – Add An Extension For KIF’s tester

Disclaimer: This should probably move to yesterday’s post on installing CocoaPods.

Before writing your first KIF test, there’s a step that must be done. The critical mechanism that the KIF API provides for enabling you to navigate through your app is called tester(). It’s implemented as a C preprocessor macro that isn’t naturally available in Swift. You’ll need to manually configure this via an extension that you create.

Create an empty new Swift file in the group and target KIFDemoAppFunctionalTests called KIF+Swift.swift. Replace it’s contents with:

extension XCTestCase {
    func tester(file : String = #file, _ line : Int = #line) -> KIFUITestActor {
        return KIFUITestActor(inFile: file, atLine: line, delegate: self)
    }
}

extension KIFTestActor {
    func tester(file : String = #file, _ line : Int = #line) -> KIFUITestActor {
        return KIFUITestActor(inFile: file, atLine: line, delegate: self)
    }
}

You will then use tester() in writing your first KIF test case. Now you need to create that new test file.

Step 1 – Create a Unit Test

You might be wondering, “Why am I creating a unit test case class? I want to create a functional test!” The answer is that it’s convenient to use Xcode’s template for unit tests, you’ll see how to then convert it to a functional KIF test next.

To create the unit test, back in Xcode, in the group KIFDemoAppFunctionalTests create a new Unit Test Case Class by clicking File -> New -> File… and then in the resulting prompt, iOS -> Source -> Unit Test Case Class. Yes, I recognize this is a little weird.

Name the new file, “MyFirstKIFTest” and make sure the Language is Swift.

Writing Your First KIF Test

On the next screen, make sure that the new file will only be added to one target, the KIFDemoAppFunctionalTests target, and click Create.

Writing Your First KIF Test

Xcode just created a unit test for you, a good ole subclass of XCTestCase. Now we’re going to convert that to a KIFTestCase.

Step 2 – Convert The Unit Test to a KIF Test

In MyFirstKIFTest.swift, change the superclass from XCTestCase to KIFTestCase. Also, delete all code within the class.

MyFirstKIFTest.swift should now look like:

import XCTest

class MyFirstKIFTest: KIFTestCase {
}

Build the app, and verify everything still compiles. Congratulations on writing your first KIF test!

Step 3 – Write A Functional Test For a Button

For writing your first KIF test, you are going to write a test that taps the Continue without saving button in the demo app, and verifies that the subsequent view is shown.

Add this method to MyFirstKIFTest

func testContinueWithoutSaving_ShowsNextScreen_WithCorrectText() {
    tester().tapViewWithAccessibilityLabel("Continue, without saving")  // 1
    tester().waitForViewWithAccessibilityLabel("Welcome Player 1")      // 2
    tester().tapViewWithAccessibilityLabel("Start Over")                // 3
}

This method is simple, yet very powerful. In 1, the tester() is told to tap something on screen with the accessibility label “Continue, without saving.” If that thing isn’t there, the test will fail. So right off the bat, this line alone provides verification that there is something on the screen with that label, AND that it can be tapped.

In 2, the tester() is told to wait for a view to appear on the screen with a accessibility label of “Welcome Player 1”. If you played around with the demo app, you would have realized that “Player 1” is a static name for players who don’t provide a name. This is the real thing that this test is verifying- that whey the user continues without providing a name, a new screen is shown with a default name.

In 3, you need to reset the app to a “default” state. This isn’t technically required, but I suggest it as a best practice. Future tests you write will automatically continue in the app wherever the prior test left off. For good decoupling of your tests, I recommend “undoing” anything that your test did in the app, so that future tests can start assuming a “fresh” app.

Run your tests by selecting Product -> Test, or use the keyboard shortcut Command-U. Watch how the iOS Simulator launches your app, and magically navigates through the screens. This part never gets old for me, I love watching my apps get automatically verified.

It’s always best to verify that tests fail as well. That way you aren’t lulled into a false sense of security, when the test was just giving false positives to begin with. You should do this all the time, not just when writing your first KIF test. Change either of the accessibility labels in the test, and re-run. You’ll see that the test fails.

Test Suite 'Selected tests' started at 2016-04-27 20:14:17.801
Test Suite 'MyFirstKIFTest' started at 2016-04-27 20:14:17.801
Test Case '-[KIFDemoAppFunctionalTests.MyFirstKIFTest testContinueWithoutSaving_ShowsNextScreen_WithCorrectText]' started.
/Users/andyo/Documents/MyGitHubProjects/KIFDemoApp/KIFDemoAppFunctionalTests/MyFirstKIFTest.swift:14: error: -[KIFDemoAppFunctionalTests.MyFirstKIFTest testContinueWithoutSaving_ShowsNextScreen_WithCorrectText] : The step timed out after 10.00 seconds: Failed to find accessibility element with the label "Continue, without savingg"
Test Case '-[KIFDemoAppFunctionalTests.MyFirstKIFTest testContinueWithoutSaving_ShowsNextScreen_WithCorrectText]' failed (10.064 seconds).
Test Suite 'MyFirstKIFTest' failed at 2016-04-27 20:14:28.022.
     Executed 1 test, with 1 failure (1 unexpected) in 10.064 (10.221) seconds
Test Suite 'Selected tests' failed at 2016-04-27 20:14:28.023.
     Executed 1 test, with 1 failure (1 unexpected) in 10.064 (10.222) seconds

In this output, also notice the message “The step timed out after 10.00 seconds.” That’s an important nuance. By default, KIF allows a 10 second grace period for a given “thing” to be found by the API. This allows for slow user interface transitions to complete without falsely failing a test.

Let me also pause to clarify and reinforce that KIF’s magic is totally driven off of accessibility labels and identifiers (and luckily this is in common with other functional tests in case you ever want to switch test tools later, as well as best practice for enabling people with impaired vision to use your app). As you create your app, and write your KIF tests, it’s much easier if you specify the accessibility information along the way.

Step 4 – Write A Functional Test For a Text Field

Now that your whistle is wet, let me further satiate your desire to write functional tests. This time, you’re going to raise the stakes by inputting text, and based on this input, verify an expected output on the second view.

Add this second test method to MyFirstKIFTest:

func testSaveAndContinueWithAName_ShowsNameOnNextScreen_WithMatchingText() {
    let name = "Clean Swifter"                                                   // 1
    tester().enterText(name, intoViewWithAccessibilityLabel: "Name field")       // 2
    tester().tapViewWithAccessibilityLabel("Save, and continue")                 // 3
    tester().waitForViewWithAccessibilityLabel("Welcome \(name)")                // 4
}
  1. Create a constant string representing the name that will be entered and verified.
  2. Enter the name into the text field.
  3. Tap the Save, and continue button
  4. Wait for the next view, reusing the name constant to verify the output string.

Run your tests. What happens?

The tests won’t pass.

Failed to find accessibility element with the label "Name field"

You never specified an accessibility label on the text field. Open Main.storyboard and set the Accessibility label on the text field. This screenshot should help you figure out how to find and do that:

Writing Your First KIF Test

Run your tests, again. All should be good!

Wrap Up

I hope you enjoyed writing your first KIF test. Well, it was a bonus because you really wrote two! With these basics, it really opens up a world of opportunity for functional iOS testing. For further reference, take a review of the methods available on KIFUITestActor. Those methods provide the comprehensive list for navigating through your app. There’s a lot of available, try it out, and let me know what creativity you come up with after writing your first KIF test! You can find the demo app with the KIF tests added in my GitHub repository.

Happy cleaning.

Installing KIF with CocoaPods for Swift Apps

If I haven’t driven it home yet this week, I love KIF for writing functional iOS tests. I want to cultivate that love in you as well, so here’s a walkthrough on installing KIF with CocoaPods for Swift Apps. Installation of KIF isn’t totally straightforward for Swift apps, I walk you through some of the finer points of installation with CocoaPods basics for getting up and running with KIF.

To help you get started with KIF, I created a demo project with the basic application code for which you’ll be able to write the functional tests. You can find it on GitHub at: https://github.com/obuseme/KIFDemoApp or download in a zip file here. We’ll use this project for the rest of the week as I go deeper and deeper into the capabilities of KIF with the next couple posts.

Installation

I recommend installing KIF with CocoaPods for Swift apps. Ensure you have CocoaPods 1.0 installed, I specifically used 1.0.0.beta.8 when writing this up. First start by creating a new target that will be dedicated for your functional tests. Open KIFDemoApp.xcodeproj in Xcode. Then, in Xcode, select File -> New -> Target…. Select iOS -> Test -> iOS Unit Testing Bundle. Name it “KIFDemoAppFunctionalTests”.

Next, create a new text file in the root of the demo project called Podfile. Add this text to it:

source 'https://github.com/CocoaPods/Specs.git'
platform :ios, '9.0'

project 'KIFDemoApp'

target :KIFDemoAppFunctionalTests do
        pod 'KIF'
end

This podfile sets up KIF to ONLY be included in the dedicated target, KIFDemoAppFunctionalTests. This is incredibly important because KIF uses private APIs, and if you accidentally include KIF in your main app target that you ship to Apple for review for the App Store, you’ll be rejected.

Then, from the command line, in the root of the project run:

pod install

and you’ll see this output:

KIFDemoApp|master⚡ ⇒ pod install
Analyzing dependencies
Downloading dependencies
Installing KIF (3.4.2)
Generating Pods project
Integrating client project

[!] Please close any current Xcode sessions and use `KIFDemoApp.xcworkspace` for this project from now on.
Sending stats
Pod installation complete! There is 1 dependency from the Podfile and 1 total pod installed.
KIFDemoApp|master⚡ ⇒ 

Follow the instructions, and close KIFDemoApp.xcodeproj and open KIFDemoApp.xcworkspace. Now KIF is installed in the workspace.

We will be writing our KIF tests in Swift. Since KIF is written in Objective-C, you must include KIF’s header file from a bridging header file in the KIFDemoAppFunctionalTests target.

To do this, right-click the group KIFDemoAppFunctionalTests and select New File.

Installing KIF with CocoaPods for Swift Apps

Then select iOS -> Source -> Cocoa Touch Class, Next. Name it anything, literally anything you want but ensure that the Language is specified as Objective-C, we’ll delete it right afterwards, click Next. On the next screen, ensure that only the KIFDemoAppFunctionalTests target is selected:

Installing KIF with CocoaPods for Swift Apps

And click Create.

Select the option in the resulting prompt to “Create Bridging Header.”

Installing KIF with CocoaPods for Swift Apps

You’ll have two Objective-C files created, as well as new file called KIFDemoAppFunctionalTests-Bridging-Header. Delete the two Objective-C files – their only purpose served to help create the bridging header.

Installing KIF with CocoaPods for Swift Apps

You can verify that everything is working now by selecting Product -> Test. Everything should compile, and an empty implementation of KIFDemoAppFunctionalTests should run (and pass).

After installing KIF with CocoaPods for Swift apps, you are all set to create your first KIF test case!

Wrap Up

I hope you found this walkthrough of installing KIF with CocoaPods for Swift apps useful. In case something didn’t work out for you, you can find the final demo project at https://github.com/obuseme/KIFDemoApp/tree/kif-installed or here. Tomorrow, we’ll go through the basics of actually writing some functional tests for the demo app. And Friday, I’m going to share a couple advanced tips and tricks to leverage when writing your KIF test cases. Be sure to check back.

Happy KIF’ing!

Three Reasons KIF is Awesome

As I introduced yesterday, KIF is a tool that enables functional testing of iOS applications. I wanted to go into a little more detail today on KIF’s capabilities by giving you three reasons KIF is awesome. Tomorrow, I’ll present an actual walkthrough of writing a KIF test case or two, so be sure to check back then.

What is KIF?

KIF stands for “keep it functional.” The name does it justice, it’s a test framework totally focused on providing a mechanism to write functional tests for your app. KIF tests hook into your app by interacting with the user interface. Similar to all automated tests of any sort, KIF enables you to write repeatable steps of verification to ensure that certain inputs determine certain outputs. Unlike a unit test where you might instantiate a model object, and then directly call a method, and verify the outcome, in KIF tests you instead write code to literally navigates through your app’s user interface and looks for things on the screen. You’ll never call viewDidLoad() explicitly from a KIF test, but you will instead simulate the tap of a button. In order to find things on the screen, KIF leverages accessibility labels and identifiers on your user interface elements.

Three Reasons KIF is Awesome

There are three main reasons why I like KIF:

  1. Tests can be written in Swift or Objective-C. For anyone that has experience with the first iteration of UI Automation test cases, THIS IS HUGE. UI Automation was Apple’s original provided tool for developers to write functional tests, and it was lame because you had to write the tests in JavaScript with a poorly documented API. Three Reasons KIF is Awesome
  2. Tests are executed as part of a normal old test target. Similarly replacing UI Automation JavaScript testing, THIS IS EVEN HUGER! Being able to execute your tests as a normal target in Xcode means first class citizenship with your build process, and thus more easily enabling repeatability and integration with continuous integration tools. With a single Command-U (the keyboard shortcut to run all tests), you can run both your unit tests and your KIF tests.
  3. KIFTestCases, the class that you subclass to write your tests, are subclasses of XCTestCase. This means that the same familiar API from writing verification in unit tests applies to KIF tests, no need to learn a new API for making assertions in your KIF tests.

Can KIF Test Swift Code?

For sure! In fact, this is where I got my first introduction to writing Swift code as part of a “real” project. The first Swift code that we started adding into our repositories on my team was in KIF tests. If you’re looking for a way to start introducing Swift code into your Objective-C project, give it a try in your tests.

Wrap Up

I’ve really enjoyed using KIF for my functional tests, and I hope I helped you by explaining why. Tomorrow, I’m excited to walk you through creating your first KIF test.

Happy cleaning.

Blending Cultures: The Best of Functional, Protocol-Oriented, and Object-Oriented Programming

realm.io recently posted a video that really resonated with me. The video was of a presentation by Daniel Steinberg at the 2016 try! Swift conference. His presentation was titled “Blending Cultures: The Best of Functional, Protocol-Oriented, and Object-Oriented Programming.”

What I Liked

I’ve been having some self doubt lately, specifically around what feels like an overwhelming mountain to climb to re-program my mind to think in a functional way.

blending cultures

I’ve certainly made progress, and as I’ve written before, one of the main reasons I started this blog was to capture my learnings along the way in that area. A year ago, reading Functional Swift by Chris Eidhof, Florian Kugler, and Wouter Swierstra I kind of had my world turned upside down. It was really what first opened my eyes to this new way of programming. Fast forward to today, as these new functional techniques gain more and more appeal in my mind, I start to wrestle with the question, “Well what about all that object oriented stuff I learned and have so much experience with?” I spent many many years in Object Oriented languages (Java and Objective-C). It’s where a lot of my go to tools in my toolbox are still based. So, are those all just obsolete in this new way of programming? While no one is saying so specifically, I do hear a lot of things like “Try to write code in the Swift way.” Or I see sample Swift code that is heavily functional. And I’m still unsure what a monad is, and forget about a monoid. It quickly becomes over my head.

It was very refreshing to hear someone I respect, Daniel Steinberg, present a compelling argument that these programming paradigms not only can live together, but also lead to a more compelling system design. Each tool should be appropriately used.

When all you have is a hammer, everything looks like a nail. And maybe that’s where, before really making the functional (or even protocol-oriented) shift, you see a problem and immediately think: composition! Or, inheritance! Watching this video, now I feel reassured that there’s room and even good purpose for some of those older trusty tools (object oriented programming).

I Owe A Lot To Daniel

Back in 2008, when I was just diving into iOS programming, I attended a course by the Pragmatic Studio taught by Daniel Steinberg and Bill Dudney. It provided a critical foundation of learning, and really introduced me to the community of iOS development. I loosely maintained a relationship with Daniel too. About a year later, I hired him through Pragmatic Studio to come teach a similar course on iOS engineering at my employer for others to learn as part of our efforts to kickstart mobile development. Then fast-forward about five years, and we crossed paths again at CocoaConf 2014 in Boston where he put on a spectacular day long introduction to Swift, and then gave an incredibly passioned keynote during the conference. Thank you Daniel for everything, if it wasn’t for you, I wouldn’t be where I’m at right now. I continue to learn from you.

Happy cleaning.

Your First iOS Unit Test

I recently had the question posed to me, “how do I get started with unit testing on iOS?” I have a suggestion for your first iOS unit test, look for a piece of code that performs an operation on a String. Write your first iOS unit test for that method. Strings and their related manipulations are easy to unit test. They don’t have deep integration with other frameworks like UIKit or CoreData, and the behavior is usually easily understandable just by reading the code, thus making it easy to reverse engineer what “should” happen, in order to translate it into a unit test. Let’s work through an example.

A “Unit” Defined

The first thing to understand about “unit testing” is what a “unit” even is.

Your First iOS Unit Test

No, not a storage “unit.”

It’s a vague term for sure. I’d describe it as, “the smallest piece of code that actually does something useful.” It’s kind of like art, you know it when you see it. I’ve written dedicated unit tests for single lines of code before. I’ve written unit tests for entire methods before. Bigger than that, and you start to take on too much scope for the test. You want the test to be straightforward. You want the test to be easily troubleshootable when it fails. And you want to minimize the reasons it could fail, to only a few reasons. This keeps your tests well factored, and easily maintainable.

My suggestion for your first test is to look for a standalone method, aka a “unit” that performs an operation on a string. Here’s an example for your first iOS unit test:

The Code To Test

I wanted to come up with a straight forward example to help you visualize a piece of code that is easily verified with a unit test. Consider the following struct that represents a person with a name:

struct Person {

  let firstName:String
  let lastName:String
  let suffix:String?

  init(firstName: String, lastName: String) {
    self.firstName = firstName
    self.lastName = lastName
    self.suffix = nil
  }

  init(firstName: String, lastName: String, suffix: String) {
    self.firstName = firstName
    self.lastName = lastName
    self.suffix = suffix
  }

  func formattedName() -> String {
    if let suffix = suffix {
      return "\(firstName) \(lastName), \(suffix)"
    } else {
      return "\(firstName) \(lastName)"
    }
  }
}

The struct sets up a Person as a thing that is represented by three Strings, a firstName, a lastName, and an optional suffix.

formattedName jumps out as a prime candidate for a unit test. A standalone method that does some isolated logic based on some inputs, and returns a a new value based on those inputs.

Add An XCTestCase Your Project

Now that you know what you want to test, in order to test it, you need to leverage the XCTest framework to test it. Xcode makes this easy.

From your project, select File -> New -> File… and pick Unit Test Case Class.

Your First iOS Unit Test

Give it an appropriate name, usually ending in the convention of: Tests, so in my case, PersonTests.

That’s it, a new file will be created containing a class that is a subclass of XCTestCase. Add your tests here. Anything function whose name begins with “test” will be executed when you run unit tests.

Writing The Unit Tests

Reviewing formattedName() in Person there are two flows through the code, when the Person has a suffix and when they don’t. Two flows through the “unit” means two tests – keep your tests small. Test one thing at a time.

Consider this test to verify that a Person‘s formattedName() is correct when they don’t have a suffix:

func testFormattedName_ProperlyFormats_WhenNoSuffix() {
  let theCleanSwifter = Person(firstName: "Andy", lastName: "Obusek")
  let expectedFormattedName = "Andy Obusek"
  XCTAssertEqual(expectedFormattedName, theCleanSwifter.formattedName())
}

First, a Person is setup without a suffix, then the expected output is defined, and finally XCTest is used to verify that the output matches the expectation.

Here’s a test for the opposite condition, when a Person does have a suffix:

func testFormattedName_ProperlyFormats_WhenSuffixPresent() {
  let theCleanSwifter = Person(firstName: "Andy", lastName: "Obusek", suffix: "Jr.")
  let expectedFormattedName = "Andy Obusek, Jr."
  XCTAssertEqual(expectedFormattedName, theCleanSwifter.formattedName())
}

This test is very similar to the prior test. The only differences are that a suffix is provided for the Person and the expectedFormattedName is different.

Running The Tests

IMPORTANT: For your tests to compile, any files that you reference in the main target (eg. the code under test) need to be included in the test target. You specify this in the File Inspector:

Your First iOS Unit Test

To run the tests, use the keyboard shortcut Command-U (or if you want to be slow, select the menu item Product -> Test. The iOS Simulator will launch, the tests will run, and you’ll see the results in the Console. They look like this:

22:11:52.771 PersonApp[12486:9452629] _XCT_testBundleReadyWithProtocolVersion:minimumVersion: reply received
22:11:52.777 PersonApp[12486:9452629] _IDE_startExecutingTestPlanWithProtocolVersion:16
Test Suite 'Selected tests' started at 2016-04-19 22:11:52.785
Test Suite 'PersonTests' started at 2016-04-19 22:11:52.787
Test Case '-[PersonAppTests.PersonTests testFormattedName_ProperlyFormats_WhenNoSuffix]' started.
Test Case '-[PersonAppTests.PersonTests testFormattedName_ProperlyFormats_WhenNoSuffix]' passed (0.021 seconds).
Test Case '-[PersonAppTests.PersonTests testFormattedName_ProperlyFormats_WhenSuffixPresent]' started.
Test Case '-[PersonAppTests.PersonTests testFormattedName_ProperlyFormats_WhenSuffixPresent]' passed (0.000 seconds).


Test session log:
    /Users/andyo/Dropbox (Personal)/Clean Swifter/PersonApp/DerivedData/PersonApp/Logs/Test/78B021DC-E467-4F0E-8D95-EE44156AB2DE/Session-2016-04-19_22:11:43-ZVrKpY.log

Test Suite 'PersonTests' passed at 2016-04-19 22:11:52.810.
     Executed 2 tests, with 0 failures (0 unexpected) in 0.021 (0.023) seconds
Test Suite 'Selected tests' passed at 2016-04-19 22:11:52.811.
     Executed 2 tests, with 0 failures (0 unexpected) in 0.021 (0.025) seconds

Congratulations, You Did It!

BOOM! That’s it, you just wrote your first iOS unit test! Well, two of them. How does it feel? Trivial? Powerful? I can relate to both of those feelings. Starting out, you might be thinking, “That’s so simple, how does it even provide value?” Well, ya, if you write the test and never execute it again, chances are you’ll never get any value out of it. The power of tests comes with repeated execution of those tests. Anytime you make a change to your project, run the tests! (and add new tests too!) I guarantee you that at some point, your tests will eventually catch a bug that you didn’t otherwise notice. Then you’ll be sold on automated testing. You’ll never get there if you don’t start somewhere.

Thanks for reading, happy cleaning.

PS – Continuous integration tools like Xcode Server or Jenkins further magnify the power of your first iOS unit test by enabling automatic execution of builds and tests in response to events like source code repository commits. I’ll tackle this topic in a future post.