I’ve constantly met with problems while running application tests from command line during development of Hosted CI. Apple doesn’t seem to care much about usability of it’s command line tools.
So given that iOS developers generally don’t unit test, we’ve decided that a radically different approach is needed.
Machine would never test your app better than a human person can. So starting from today we provide an option (on Premium and Max plans) to enable human testing of each otherwise successful Hosted CI build. It is made possible thanks to availability of human intelligence API from Amazon Mechanical Turk service, which we use through modified version of Jenkins Xcode Plugin.
As distributing your iOS apps to random people is quite a hassle (having to manage all those UDIDs) – we take care of that for you. We sign your app with our own distribution certificate and use own provisioning profile, so that you don’t have to deal with it.
Note that it means that apps using In-App Purchase and Push Notifications won’t work without modifications, please contact our support if you need to use aforementioned functionality.
P.S. I plan to push appropriate updates to my fork of Jenkins Xcode Plugin to allow you to use same approach on your own Jenkins server. Stay tuned.