What to do when Apple is messing with you ?

So you care about app quality. Thus you need to test your app. So you have a lot of tests for several parts of your application. Thus you want to run them as often as possible. And you want to keep track of all those runs. Thus you want to automate all this and record the results. Yes, indeed, I'm talking about the continuous integration we all know about.

But how do you do it ? With Xcode Server, because it's what Apple says you should use ? Or Jenkins because your company has already one master with severals slaves set up and you just have to create a job ? Oh, look at all those plugins jenkins has to monitor code coverage and to easily notify you wherever cough cough slack/irm/jabber cough cough you want.

Let's try with Jenkins (or any other CI which isn't Xcode Server). So whatever the tool you are using (TravisCI, xctool wrapped in a shell script inside a gitlab-ci job etc etc...) you won't be able to do without xcodebuild. And this is where it gets less funny (since Xcode 6).

Because after searching for hours, maybe days, blaming you first before trying to reject the fault on anybody else, you will find that when xcodebuild is asked to test a scheme from a headless shell (i.e. ssh, i.e. no GUI), it will time out. Without any meaningful informations.

So you may try whatever you want but you'll always come back to xcodebuild. You will use any tests frameworks that look exotic enough so that you can have a glimpse of hope that it won't be related too much to xcodebuild. But they all are.

Thus your 170+ unit tests and you 90+ UI tests will be as useful as a saddle-less bicycle. For days/weeks/monthts. Whatever. Not Apple problem. You can open a radar. You may get an answer. But whatever.

And then, one day (let's say with the Yosemite public release), everything is back into order. And you are required to be grateful. Got that. Be grateful.