I recently decided to start writing User Interface tests for Tusker, my iOS app for Mastodon and Pleroma. But I couldn’t just write tests that interacted with an account on any real instance, as that would be far too unpredictable and mean my tests could have an impact on other people. The solution to this problem is, of course, mocking. The core idea is that instead of interacting with external things, your program interacts with mock versions of them, which appear to be their real counterparts, but don’t actually perform any of the operations they claim to. This allows for very tight control over what data the application receives, making it much more amenable to testing.

Unfortunately, if you search around, some of the most common solutions on the internet recommend using the environment variables (one of the only ways of sending data directly from the test to the application under test) to insert the mocked response into the app. Meaning, for every API response you need to mock, you would have an environment variable that contains the response data. This isn’t a great solution, because it leaves whole code paths untested (everything after the request URL would be generated). It would also mean that there’s no way of testing things like what requests are actually made by your app.

The solution to this problem is to actually run a local HTTP server that functions as the API server. This way, the app can communicate with the web server exactly as it would in the normal operating environment, but still have strictly controlled data. Of course, actually doing this isn’t quite so straightforward.

There are a couple of things to think about when looking for a solution: First, in order to meet the requirement of being able to test what API calls are made, the web server needs to be accessible to the test process. Second, we want to avoid modifications to the app if at all possible. There should be as few differences as possible in the app between the testing and production environments. The more differences, the more code that goes untested and the more potential edge-cases.

For the first requirement, there are a pair of handy open-source libraries that we can use to take care of the grunt work of responding to HTTP requests and serving responses. First there’s Embassy, which acts as an asynchronous HTTP server. It handles actually listening for connections on a port and receiving and sending data to them. The other part of this stack is Ambassador which handles routing incoming requests and responding to them in a nicer fashion than just sending strings out.

(Note: both of these libraries are made by a company called Envoy, to which I have no relation.)

So, we need to get these libraries into our project, with the condition that we only want them to be present at test-time, not for release builds (as that would mean shipping unnecessary, extra code). If you’re using CocoaPods, it can take care of this for you. But if you’re not using a package manager, it’s (unsurprisingly) more complicated.

First, the Xcode projects for the two libraries need to be included into the workspace for your app^[1]. Next, our app needs to be configured to actually compile against the two frameworks so we can use them in our tests. This is done in the app target of the Xcode project. Both frameworks should be added to the “Frameworks, Libraries, and Embedded Content” section of the General tab for the app target. Once added, the libraries should be set as “Do Not Embed”. We don’t want Xcode to embed the frameworks, because we’ll handle that ourself, conditioned on the current configuration.

To handle that, add a new Run Script phase to the Build Phases section of the target. It’ll need input and output files configured for each of our projects:

Input files:

• ${BUILT_PRODUCTS_DIR}/Embassy.framework
• ${BUILT_PRODUCTS_DIR}/Ambassador.framework

Output files:

• ${BUILT_PRODUCTS_DIR}/${FRAMEWORKS_FOLDER_PATH}/Embassy.framework
• ${BUILT_PRODUCTS_DIR}/${FRAMEWORKS_FOLDER_PATH}/Ambassador.framework

For reference, the BUILT_PRODUCTS_DIR environment variable refers to the location in Xcode’s DerivedData folder where, as the name suggests, the compiled outputs of the project’s targets live. FRAMEWORKS_FOLDER_PATH refers to the Frameworks folder inside of our app bundle (e.g. Tusker.app/Frameworks).

Configuring input and output files for the Run Script build phase instead of just hard-coding paths in the script itself has the advantage that Xcode will know to re-run our script if the input files change or the output files are missing.

The script our build phase will run is the following:

if [ "${CONFIGURATION}" == "Debug" ]; then
    echo "Embedding ${SCRIPT_INPUT_FILE_0}"
    cp -R $SCRIPT_INPUT_FILE_0 $SCRIPT_OUTPUT_FILE_0
    codesign --force --verbose --sign $EXPANDED_CODE_SIGN_IDENTITY $SCRIPT_OUTPUT_FILE_0
    
    echo "Embedding ${SCRIPT_INPUT_FILE_1}"
    cp -R $SCRIPT_INPUT_FILE_1 $SCRIPT_OUTPUT_FILE_1
    codesign --force --verbose --sign $EXPANDED_CODE_SIGN_IDENTITY $SCRIPT_OUTPUT_FILE_1
else
    echo "Skipping embedding debug frameworks"
fi

If the product configuration is anything other than Debug, the script will simply log a message and do nothing else. If it is in Debug configuration, then we’ll take a couple actions for each framework: First, we simply copy the built framework into the app’s Frameworks folder. Then, we re-codesign the frameworks in the app so that they’re signed with the same identity as our app is. In the script, the input/output file environment variables refer to exactly what they say, the input and output files configured for the build phase in Xcode. The EXPANDED_CODE_SIGN_IDENTITY environment variable gives the code signing identity^[2] that was used to sign the app, which is the same one we want to sign the frameworks with. We also need to give codesign the --force option, so that it will overwrite any existing signature on the framework.

Now, if we build our app for debugging and take a look at the frameworks folder inside the app bundle, we can see both Embassy.framework and Ambassador.framework are present. Switching to the release build and again looking at the product in Finder, neither of those two frameworks are present.

One thing to be aware of is that this setup only excludes the frameworks from being copied in release configurations. They’ll still be available at compile-time, so if you’re not paying attention, you could accidentally import one of them and start using it, only to encounter a crash in production due to a missing framework. As far as I could tell, there’s no way within Xcode of specifying that a framework only be compiled against in certain configurations. If I’ve missed something and this is indeed possible, please let me know.

With that finally done, we can start setting up the web server so we can test our app. This simplest way to do this is to create a base class for all of our test cases which inherits from XCTestCase that will handling setting up the web server. We’ll have a method called setUpWebServer which is called from XCTestCase’s setUp method. (In our actual test case, we’ll need to be sure to call super.setUp() if we override the setUp method.)

var eventLoop: EventLoop!
var router: Router!
var eventLoopThreadCondition: NSCondition!
var eventLoopThread: Thread!

private func setUpWebServer() {
	eventLoop = try! SelectorEventLoop(selector: try! KqueueSelector())
	router = Router()
	server = DefaultHTTPServer(eventLoop: eventLoop, port: 8080, app: router.app)
	try! server.start()

	eventLoopThreadCondition() = NSCondition()
	eventLoopThread = Thread(block: {
		self.eventLoop.runForever()
		self.eventLoopThreadCondition.lock()
		self.eventLoopThreadCondition.signal()
		self.eventLoopThreadCondition.unlock()
	})
	eventLoopThread.start()
}

In that method, we’ll create an event loop which handles performing asynchronous operations for the web server. We’ll also need a Router object which will handle HTTP requests and delegate them to other handlers based on the request’s path. Next, we’ll need the web server itself, an instance of DefaultHTTPServer. Once we’ve got that, we can add our routes to the router and start the web server. Finally, we’ll create a separate thread that runs in the background for actually processing web requests. We also override the tearDown method and stop the web server, waiting for it to gracefully shut down.

override func tearDown() {
	server.stopAndWait()
	eventLoopThreadCondition.lock()
	eventLoop.stop()
	while eventLoop.running {
		if !eventLoopThreadCondition.wait(until: Date(timeIntervalSinceNow: 10)) {
			fatalError("Join eventLoopThread timeout")
		}
	}
}

Now that we’ve finally gotten everything set up, we can test out the web server and make sure everything’s working. In a new test case class, we’ll extend the base class we created instead of XCTestCase, and we’ll override the setUp method to add a new route:

override func setUp() {
	super.setUp()
	router["/hello"] = JSONResponse(handler: { (_) in
		return ["Hello": "World"]
	})
}

To actually test it out, a simple test method that just sleeps for a long time will do the trick:

func testWebServer() {
	sleep(10000000)
}

Once we run the test and the simulator’s up and runing, we can visit http://localhost:8080/hello in a web browser and see the JSON response we defined. Now, actually using the mock web server from the app is a simple matter of adding an environment variable the override the default API host.

One caveat to note with this set up is that, because the web server is running in the same process as the test code (just in a different thread), when the debugger pauses in a test (not in the app itself), any web requests we make to the mock server won’t complete until the process is resumed.

---

1.

I tried to use Xcode 11’s new Swift Package Manager support for this, but as far as I can tell, it doesn’t provide direct access to the built frameworks produced by the libraries, so this technique isn’t possible. ↩

2.

“Expanded” refers to the format of the identifier. If you look at the codesign manpage, it describes several formats that can be used with the --sign operation. The expanded format is the forty hexadecimal digit SHA-1 hash of the identity. ↩