Buddybuild self describes itself as a system that “ties together and automates building, deploying and gathering feedback for mobile apps.” I’m always on the lookout for something that makes the process of developing my apps easier. I wanted to give it a try and write a buddybuild review. I know fastlane is really popular now for the automation it provides, but I wanted to peek at something at a higher level where some of the legwork is taken care of for me.
Lately, I’ve been bouncing between TestFlight and Fabric for some of this functionality, but those solutions fall short in one specific area: automatic builds with tests (whenever I say “build” in this buddybuild review, I’m talking about both compilation and test execution). Since I’ve been on a continuous integration kick lately with the posts here on cleanswifter.com, I figured I’d give you a full rundown with a buddybuild review.
A couple specific things I was looking for in my buddybuild review:
- A hosted continuous integration solution – In case you didn’t see my post from yesterday, I’m tired of trying to maintain a physical build server. The idea of hosted continuous integration is very attractive to me.
- Integration With GitHub – I’m all in on GitHub, both github.com and the hosted enterprise instance. I want deep integration with GitHub from my automation solution. I rely on pull requests, and I need my build automation system to integrate with that workflow.
- Solid Documentation – In my opinion, when you invest your time in using a 3rd party product, you don’t want to waste your time with poor documentation.
- Any Other Bonuses – If there are any other niceties I discovered, I wanted to report out on those as well.
My Approach For My Buddybuild Review
Whenever trying out new supporting infrastructure for software that you write, never use your “real” code base to begin with. I suggest creating a fake project to use so you can feel out how to use the tool before then applying it to your real code base. As such, I setup two apps in buddybuild: BuddyBuildSampleProject which started with a single test target for unit tests, and my CocoaHeadsTestingPresentation project which has three test targets: one for unit tests, one for KIF tests, and one for FBSnapshotTestCases.
This probably isn’t comprehensive, but it’s what jumped out at me during my buddybuild review:
- Automatic builds, triggered by pushes to code repositories
- Tests can run as part of builds, YAY
- Wide variety of source control systems and repositories supported. Full list here
- Integration with Apple Developer Portal and Google Play Developer Console for automatically uploading builds, synchronize provisioning profiles, and run App Store readiness tests (wow, I didn’t even know this was possible)
- Support for both iOS and Android applications (I only tried iOS)
- Manual control over which Xcode version is used for building (I wouldn’t use this, but I guess I could see a use case)
- Status badges to show on your source code repository to indicate build status (I wish there was a way to show build result on an actual Pull Request)
- Beta distribution of your app including analytics, and ability for testers to annotate and submit screenshots of issues
- Crash reporting (Who doesn’t offer this these days?)
- REST API for programmatically triggering a build (Only one endpoint so far, but a good start)
- Team supported, invite others to collaborate on your apps and builds (This is done really well)
- Lots of integrations: Automatically create JIRA and GitHub issues, post notifications to Slack or HipChat (It’s a nice list)
Beautiful and Easy To Use
Initial impressions are always important, and buddybuild exceeded the bar for me. It’s an entirely web-based experience, and it’s beautifully designed. Not only is it visually appealing, but I had no problem figuring out where to find different options to do accomplish tasks. Also, there was a really nice first user experience that helps walk you through all the features of buddybuild. They even built a sample app that you actually download, just like a real beta tester so you can understand features that the buddybuild SDK can enable your beta testers to use.
The buddybuild documentation is out of this world and crazy good. It’s comprehensive, well-written, and even goes out of its way to teach you about best practices about things like CocoaPods, Git Submodules, and Code Signing 101.
As I worked through a couple problems getting my builds up and running, I had near immediate chat access with support personnel. They were very helpful in getting my issues fixed. Additionally, they were even open minded and sought feedback on how to make things better. Shout out to Dennis, Sid, and Rashin!
There Will Always Be A Free Tier
Despite not having specific pricing information available at the moment (which I see as a bad thing), they guarantee that they “will always offer a free tier.” That’s awesome.
Flexible Beta Tester Deployments
I’ve tried both other hosted build automation solutions as well as having rolled my own, and I really like how buddybuild allows you to create different groups of app testers. And then for each group, you can specify the frequency that they will receive a build: Every build, nightly, or manually. This is really cool especially if you want to segment your tester base between developers, other internal stakeholders, and external public testers. You can verify the right people get the right builds at the right time.
The Improvement Areas
Full Read and Write Access On Your Repo
In setting up buddybuild, one thing that concerned me is that it asked for a lot of access to my source code repositories. Whether I was connecting GitHub or Bitbucket, buddybuild required full read and write access to my repositories with no explanation as to why.
(I would have included a GitHub screenshot, but after having initially connected my account, I couldn’t find anywhere to disconnect it).
No Pricing Information Yet
Ya, I see this as a bad thing. No good thing in life is free. If I’m going to spend time building out a critical piece of my development and distribution workflow on something, I want to know that there’s a formal business relationship in place. I’m not looking to pay thousands of dollars, but rather just make sure that things are on the up and up, and that I’m not going to be left out to dry because buddybuild disappeared overnight, or that I can’t get support at some point.
Right now, the builds on buddybuild are really slow. Using my vanilla Xcode project that literally had nothing else in it but the template provided code and unit tests, it took several minutes to build on buddybuild, where locally it was done in less than 10 seconds. My project with three test targets took five minutes. I can’t imagine how long these builds would take with a more substantial project with hundreds of tests.
Open Your Firewall If You Use GitHub Enterprise
I don’t really have a good solution here, besides selling a instance of buddybuild that you could host internally in your own datacenter, but that starts to become a lot of work. If you are using hosted enterprise GitHub, you need to open a hole in your firewall so buddybuild has access to your code repository. This scares me, again, I don’t have a good solution here.
No Test Device Configuration
There is no ability to specify which iOS version, or which device your tests will run on. This is crucial functionality for me in my continuous integration workflow. For a universal app that supports iOS8 and higher, I want to at least be able to run my tests on iOS8 iPhone, iOS9 iPhone, iOS8 iPad, and iOS8 iPad. Furthermore, running on device is even better than simulator. There’s no control for this right now exposed to me, the end user. Luckily, support was very responsive in helping me change the configuration behind the scenes. Ideally I don’t want to rely on them though. Furthermore, I’ll forget what I had him set this to, and there’s no place in the buddybuild UI for me to check.
Other Suggestions From My Buddybuild Review
With Twitter acquiring Crashlytics and fastlane it’s going to be hard to catch up in a couple areas. I do really like the idea of having buddybuild abstract another layer of automation on top of tools like this though where I can just quickly click a GUI to configure things. My suggestion would be to go through fastlane’s suite of tools and add a similar feature into buddybuild. I think fastlane’s maturity is a good representation of exactly what developers are looking for in automating their workflow of creating applications.
I’d also love to see code coverage reports, and static code analysis, and even other code-health metrics integrated with buddybuild’s dashboard.
After my buddybuild review, as cool as buddybuild is, until they resolve some of my improvement areas, I can’t recommend it for use in a business critical application. The lack of pricing is a major red flag to me. As polished as their site is, and how well it works, the fact that there isn’t official pricing to establish a business to business relationship makes me feel like they are operating out of a garage with the possibility that they will shutdown at any moment (disclaimer: I got stuck when something similar happened with ship.io disappeared overnight – another hosted CI solution). I also don’t want to wait around for my builds to happen. I also really want to be able to configure which devices, simulators, and iOS versions that my tests run on- this is important in any CI stack. I need that to verify that my code works well, everywhere.
The good news out of my buddybuild review is that buddybuild is working on all of these improvement areas so the outlook is good. I’ll keep an eye out for developments in this area and report back in the future.
I hope you found this buddybuild review useful, if you want to try out buddybuild and either make your own impression, or use it for something non-mission-critical, I think you’ll be impressed with what they have to offer.