Cheaper Than A Dollar A Day
I came across this article today, called “Continuous Integration on a Dollar a Day” since Jon Reid of qualitycoding.org mentioned it on his Facebook page. I first want to recognize that this article was originally written back in 2006, so no need to repeat the obvious in case you discover this fact yourself. A lot has changed in 10 years, and a lot remains the same. For one, I absolutely recommend using continuous integration. The author also clarifies that the article is intended for people new to continuous integration, so some of the principles are introductory. While I recognize that, I think there are better introductory solutions for your continuous integration workflow For iOS.
To me, if you were to ask me on the street, off the top of my head, continuous integration ensures that: – Code that’s committed to your repository always compiles – Code always has tests passing
Having lived with it for so long, it’s not fathomable to consider anything else. To think that I could potentially check out what’s in the source code repository and it not compile is unthinkable. If clean code is your desired end state, then continuous integration can help you achieve it.
A Proposed Continuous Integration Workflow For iOS
I am in love with Github pull requests. A past version of me scoffed at thinking that my code must go through code review before it could be considered “done” or merged. I remember using Subversion back in 2010-ish and committing directly to the main trunk of code, with no one review it, and no automated builds. YIKES! Today though, I feel that a combination of test driven development, code review, and continuous integration truly remove the need for dedicated testers on the product team. That’s not to see we don’t manually test, I’m just saying that it removes the need for something dedicated in that role.
I don’t want to git into (pun intended) the pros and cons of other version control systems, I know there are plenty. I use GitHub for all my projects, and I love it. I think you’ll love it too.
To sum up pull requests, you work locally in a branch of code. When you’re ready to share it with others, you “push” the branch to the remote repository, and open a “pull request.” An open pull request is basically a proposed change set. Other team members can review the changes, chat about them, suggest improvements. Ultimately, the proposed change set can be accepted, and then is merged into the destination branch where you intended the code to end up.
If you want more detail, this is a great writeup directly from GitHub describing the process of using pull requests in your workflow to enable pre-merge code review.
So where does continuous integration fit into this? I propose that anytime code is about to change somewhere, continuous integration jobs should verify that the incoming change will compile AND pass tests. The thing is, as you move up the iOS testing pyramid, tests will run slower and slower. Your wait time can really grow if these gate parts of your development.
Setting Up Your Jobs
Continuous integration for iOS projects is a delicate balance between enough test coverage across OS versions and devices, running all your tests, and the putting the checks in just the right places. Here’s how I recommend striking this balance:
- Run your full suite of tests on one device/OS combination each time code is pushed to the repo – I prefer to do this on the lowest iOS version that the app supports. I also usually do this on an iPhone for my Universal apps (no real reason, and I’ve gotten bit by iPad only bugs, so probably worth mentioning that some reasonable amount of thought is worth it here for your situation).
- Run your full suite again on a different device/OS combination each time code merges on the result of the merge.
- Each night, run the full suite of tests on every possible combination of devices and OS that you support that is possible within your means.
Let’s play this over the GitHub pull request model:
- I push a branch to the repository and open a pull request.
- CI server detects the push, runs all tests (unit, UI, and snapshot) on an iOS8 iPhone 5.
- CI build either fails or passes.
- CI machine puts a flag in the pull request based on the result of the build. If the build does not pass, it should not be merged.
- If build is successful, once code merges, a new build is triggered on the destination branch using the result of the merge to run the full suite of tests. If this build fails, manual intervention is necessary to understand why, and be addressed.
- Starting around 11pm, a sequence of dependent jobs kick off running the full suite of tests across every device that I own – iPhones, iPads, iOS7 (if supported), iOS8, iOS9, etc.
- Results from “nightlies” are reviewed in the morning.
Anytime a build fails, it should be immediately addressed. Failing builds tend to build up over time, especially with the nightlies. If a nightly fails, make fixing it the first thing you do in the morning when you arrive. It usually represents a bug in your code.
Room For Improvement
There is definitely some manual intervention needed in this workflow, it’s not full proof. Ideally, continuous integration works along side you and prevents you from doing anything stupid. Removing any manual steps from your workflow is paramount, and something I’m still working on. I’d love to hear from you, have you tried a continuous integration workflow for iOS? How has it worked out? Or if not, why not?
Happy cleaning.