fbsnapshottestcase Archives - cleanswifter.com

Proposed iOS Testing Pyramid

The Original Testing Pyramid

Before I go into my proposed iOS testing pyramid, I know you’ve heard me mention Martin Fowler’s test pyramid before. In case you weren’t reading, or don’t remember, here it is:

Fowler advocates for three levels of testing: unit, service, and UI. If you’d like the full description, you can read about it here.

Proposed Revision

I disagree with this pyramid for iOS apps. I think we can do better. Here’s my **proposed revised iOS testing pyramid **:

The changes include:

Unit Tests Remain – If you don’t write any other automated tests, please write unit tests. In the least, unit tests open the door for things like test driven development and refactoring.
UI Tests Remain – With the vast variety of device sizes and OS versions, UI tests’ value can multiply with the more devices and OS’s you run them on.
No Service Layer – I think with most iOS apps, there’s no need for the Service layer. I think a comprehensive suite of unit tests will take care of anything that the service layer would have otherwise taken care of.
Add Snapshot Testing – Snapshot testing can really help “lock in” your user interface to ensure it looks pixel perfect and preventing any future code changes from introducing slop.
Add Manual Testing – There are simply bugs that no amount of automated testing will find. There was one bug we came across where a hidden `UIView` was covering a button in a very specific edge case such that the button was not tappable. This was found by a human during manual QA. It never would have been found by automated tests alone. TestFlight makes this so easy these days, don’t skimp on it.

Suggested Tools

Manual Testing – You and your friends’ own two fingers.
Snapshot Testing – As far as I can tell FBSnapshotTestCase is the only game in town.
UI Tests – Lots of options here: KIF, Xcode UI Tests, Frank, Calabash.
Unit Tests – XCTest FTW.

Writing Your First FBSnapshotTestCase

I’m in the thick of preparing for my talk at Philly CocoaHeads this week, but I wanted to get a quick post out that shows you how easy writing your first FBSnapshotTestCase is. Yesterday, I showed you how to setup FBSnapshotTestCase with Carthage. I’m going to assume you’ve done that already. I’ve create a sample project for my talk on Thursday that I’m going to use for this walkthrough on writing your first FBSnapshotTestCase. You can download that on GitHub here.

What You’ll Test

Open the application, and Build and Run.

You’ll see it’s a simple app, one that I’ve even used before in other posts. There’s two flows forward from this first screen, either with the Save, and continue button or the Continue, without saving button. If the user chooses, they may enter their name and tap Save, and continue to access the Welcome view where their name is shown to them.

Simple enough. In writing your first FBSnapshotTestCase, you are going to verify that the name specified shows up correctly on the Welcome view.

Take The Baseline Snapshot

To create the test, right-click the SnapshotTest group, and select New File:

Select Unit Test Case Class and Next:

Name the test WelcomeSnapshotTests and specify it as a Subclass of FBSnapshotTestCase. Click Next:

Click Create on the subsequent screen.

If you are prompted to create a bridging header, select Don’t create.

Now, Xcode will create the source file for you and drop you in it. The first thing to do is correctly import FBSnapshotTestCase.

Replace this:

import XCTest

with this:

import FBSnapshotTestCase
@testable import CocoaHeadsTestingPresentation

Importing CocoaHeadsTestingPresentation is necessary to access classes from that module so we can create views specific to the app. Now, you should be able to build with Command-B.

Delete everything within the WelcomeSnapshotTests class, and add this:

override func setUp() {
  super.setUp()
  recordMode = true
}

This setUp() method tells FBSnapshotTestCase that when recordMode is true, new snapshots will be taken. This requires the application to be in a “known good state.” That means that the view that you are going to “snapshot” looks just as you want it to look, because all future test runs will compare against this view.

Next, add this test method:

func testWelcomeView_WithName() {
  let welcomeVC = UIStoryboard(name: "Main", bundle: nil).instantiateViewControllerWithIdentifier("WelcomeViewController") as! WelcomeViewController
  welcomeVC.name = "Andy Obusek"
  FBSnapshotVerifyView(welcomeVC.view)
  FBSnapshotVerifyLayer(welcomeVC.view.layer)
}

This test creates a WelcomeViewController from a storyboard, specifies the name to be shown, and then verifies the view and layer.

Run this test. Just as usual, I suggest the keyboard shortcut Command-U. You’ll actually see the test fail:

But looking closer at the message:

Test ran in record mode. Reference image is now saved. Disable record mode to perform an actual snapshot comparison!

Nothing is actually wrong. FBSnapshotTestCase is just telling you that since it’s in recordMode, it will take new snapshots, but not let the test pass.

To see the snapshot, open the directory /SnapshotTests/ReferenceImages_64/SnapshotTests.SnapshotTests/. Inside that directory, you should see a png file that is the snapshotted view! So cool!

Turn Off Record Mode

Now that the baseline snapshot has been taken, turn recordMode off.

override func setUp() {
  super.setUp()
  recordMode = false
}

Now, rerun the test with Command-U. And bingo bango, the test passes! Light is green, trap is clean!.

Make Sure It Fails When It Needs To

It’s hard, if not impossible, to practice test driven development when writing your first FBSnapshotTestCase, or really any snapshot test at all. Since it requires the known “good state” to be snapshotted, some amount of real development has to happen first. That being said, you should still make sure that the test fails when it should. To do that, we’ll hack a bug into WelcomeViewController. Open WelcomeViewController.swift and add a few “eeee” to how the welcome message is set:

override func viewDidLoad() {
  super.viewDidLoad()
  if let name = name {
    welcomeLabel.text = "Welcomeeeeeeeee \(name)"
  } else {
    welcomeLabel.text = "Welcome Player 1"
  }
}

Re-run the test. It will fail! Whew, now we know that it will actually fail when it should.

Wrap Up

See, wasn’t it easy writing your first FBSnapshotTestCase? I hope snapshot testing helps you out. I’d love to hear how it helps you, or what you think of this approach. Please leave a comment!

Happy cleaning.

FBSnapshotTestCase Installation with Carthage

FBSnapshotTestCase installation failed with CocoaPods 1.0.0.rc.2 while in preparation for an upcoming presentation to Philadelphia CocoaHeads. I gave Carthage a try, and it worked! I wanted to write it up and share it with you. Now I know that switching to Carthage may not work for everyone just to use a test framework, but maybe there’s a hybrid solution that you could come up with?

What is FBSnapshotTestCase

FBSnapshotTestCase is a testing framework that was originally written at Facebook by Jonathan Dann with significant contributions from Todd Krabach. As a testing framework, it allows you to test the user interface of your iOS app by diff’ing screenshots. Yep, you heard me write, you literally take a source screenshot, mark it as “correct” and then all future runs of the test suite use this as the basis for determining if the test passes or not.

As my preferred channel, and as the README suggested, I wanted to install FBSnapshotTestCase with CocoaPods, but this issue prevented me from doing so in a Swift project. Instead, I tried using Carthage and was successful.

FBSnapshotTestCase Installation with Carthage

Step 1: Download Carthage

Carthage is an alternate dependency management framework, one that is more lightweight than CocoaPods (and doesn’t require Ruby! YEY). If you don’t have Carthage installed, download the latest .pkg file from here. I used 0.16.2 for this tutorial. FBSnapshotTestCase Installation was really easy with Carthage.

Step 2: Create a Cartfile

In the root of your project, create a new file called Cartfile. Add this to it:

github "facebook/ios-snapshot-test-case"

The Cartfile contains your dependencies for the project. While you can specify versions of your dependencies, I was content just picking the latest release, and thus didn’t specify a version.

Step 3: Install the Dependencies

Now that you have a Cartfile, the next thing to do is install the dependencies with Carthage. To do this, from a shell, run:

carthage update --platform iOS

You’ll see output like:

*** Fetching ios-snapshot-test-case
*** Checking out ios-snapshot-test-case at "2.1.0"
*** xcodebuild output can be found in /var/folders/mp/k1jy2r2d3gg9bzkz0v9y5jxm00024f/T/carthage-xcodebuild.GOHIjS.log
*** Building scheme "FBSnapshotTestCase iOS" in FBSnapshotTestCase.xcworkspace

A new Carthage/ directory will be created with your dependencies. Carthage is different from CocoaPods, in that, you now need to manually configure the libraries within your Xcode project.

Step 4: Add Dependencies to Your Project

Open a Finder window for the root folder of your project, and then navigate down the hierarchy to Carthage/Build/iOS. You should see the framework for FBSnapshotTestCase.

Now, in Xcode, open the target settings for your test target, in my case it’s called SnapshotExampleTests, and then select the Build Phases tab, and then expand Link Binary With Libraries. Drag the framework in there:

It will then look like:

Step 5: Add a FBSnapshotTestCase

Create a new unit test (File -> New -> File):

And specify it as a subclass of FBSnapshotTestCase

At the top of the file, replace:

import XCTest

with

import FBSnapshotTestCase

At this point, you can try running your new FBSnapshotTestCase (Command-U). Everything should compile, but the test will fail.

Step 6: Copy-Frameworks

I’ll be honest, I’m not really sure why this final step is necessary, but without it, the tests would not pass. Carthage’s README indicates it’s necessary for an “App Store submission bug” but I’m not even archiving here, just running tests.

Do this (copied right from Carthage’s [README]):

On your application targets’ “Build Phases” settings tab, click the “+” icon and choose “New Run Script Phase”. Create a Run Script in which you specify your shell (ex: bin/sh), add the following contents to the script area below the shell:

/usr/local/bin/carthage copy-frameworks

and add the paths to the frameworks you want to use under “Input Files”, e.g.:

$(SRCROOT)/Carthage/Build/iOS/FBSnapshotTestCase.framework

It should now look like this:

Now, try to run your FBSnapshotTestCase again with Command-U. It should compile and pass your test!

Wrap Up

See, isn’t FBSnapshotTestCase installation easy? Now you’re free to go ahead and use FBSnapshotTestCase to your heart’s content. I plan to write another post that will help you through creating your first FBSnapshotTestCase. If you’re a long time CocoaPods user, I know this isn’t optimal, but hey, look at it this way, at least you have an opportunity to try out Carthage if you’ve never looked at it before.

I got a log of inspiration and ideas for installing FBSnapshotTestCase with Carthage from this article on <raywenderlich.com>.

Happy cleaning!