Kitura/iOS, software engineering

In my previous post about Kitura/iOS I described possible use cases for running a web server on iOS and introduced a hello world Kitura/iOS app. In brief, I explained how server-side Swift web frameworks allow developers to use the same web serving code on the server (macOS, Linux) and on the client, embedded in an iOS app. In this post I will describe the software engineering aspects of developing Kitura/iOS applications and provide some related gory technical details. I will conclude with the steps required to create your own Kitura/iOS application.

The code examples and the infrastructure for running Kitura/iOS apps described in this blog post were developed together with my former colleague Roded Zats.

Software Development Principles of Kitura/iOS Sample Apps

We applied the following general software engineering principles in development of Kitura/iOS sample apps:

In the next sections, I will explain how each principle is applied.

DRY – Don’t Repeat Yourself

In our sample apps, we tried to prevent duplication of application code and build automation code (scripts, makefiles) as much as possible. Whenever duplication is required, it is performed automatically during the build process, making sure that the application code and build automation code are not duplicated in version control. In the code repositories of the Kitura/iOS sample apps, we separate the server-side and the client-side parts. We want the server-side part to be developed and to evolve independently from the iOS applications it is embedded in. We also want to reuse the client-side part (the view controller that runs the web server) for multiple Kitura/iOS applications.

Server Side

To reuse the same server-side code for multiple Kitura/iOS applications and to enable independent development of the server-side code, that code must reside in its own repository. We used plain server-side Kitura sample apps, Kitura-HelloWorld and Kitura-Sample AS IS, and embedded them in our Kitura/iOS sample apps: Kitura-HelloWorld-iOS and Kitura-Sample-iOS. The “embedding” is performed automatically during the build. The server-side code only resides in the original repositories and is not duplicated in the version control system of the Kitura/iOS applications.

The Schism in Swift Build and Dependency Management Tools: Client Side vs. Server Side

The caveat with Swift today is the schism between the Swift build and dependency management tools on the client side and those on the server side. On the client side, the apps are developed using xcodebuild, Xcode and CocoaPods/Carthage. The unit of code sharing between applications is Framework. On the server side, the primary build and dependency management tool is Swift Package Manager (SPM). The unit of code sharing between the applications is Package.

Bridging the Schism by Xcode Project Generation

We are not aware of a way to directly import an SPM package into an Xcode project. The SPM packages must be transformed into Frameworks and the resulting Frameworks can then be imported into the Xcode projects of iOS apps. SPM can generate an Xcode Project from a Package. The generated Xcode Project will contain a Framework per each module of the Package. The resulting Frameworks can then be imported into an iOS app.

Note that the generated Frameworks will not be synchronized with the contents of the Package. Therefore, once the Package changes, for example when new files are added, the Xcode project with the Frameworks must be regenerated.

Since there is no way to instruct Xcode to fetch an SPM code from a remote repository and embed it as a Framework, we had to implement this embedding ourselves, as part of the build process.

Using Git Submodules to Combine Multiple Repositories

We used the git submodule command to handle fetching the server-side code into the directories of the Kitura/iOS apps. An alternative would be to clone the server-side repository inside the app directory. However, we wanted to use an “official” git/GitHub mechanism for multi-repository projects. Using git submodules makes the dependency on the server-side repository explicit to git and GitHub, and is supported by the both tools.

The git command for fetching submodules (git submodule update) is invoked automatically during our build. Then our build invokes the swift package generate-xcodeproj command and generates an Xcode project with the server-side code. In the generated Xcode project, each SPM module is defined as an Xcode Framework. The Xcode project generation is performed each time during the build to prevent the generated server-side Frameworks from getting out of sync with the original server-side code.

We call the submodule with the server-side code, well, ServerSide.

Client Side

We developed a common client-side part for our Kitura/iOS sample apps, namely Kitura-Mobile-Server. It is an Xcode project that contains UI code to start and stop Kitura and to display Kitura log messages. Kitura-Mobile-Server provides common functionality that can be used to run any Kitura server-side code. To allow reusing Kitura-Mobile-Server in multiple Kitura/iOS applications, we put it in a separate GitHub repository. We included it as a submodule in the repositories of the sample Kitura/iOS applications, similar to the server-side part.

We used an Xcode workspace to combine the generated server-side Xcode project and the Xcode project of the client-side part. Since the server-side Xcode project is generated, we generate the Xcode workspace automatically as well. We used the Ruby Xcodeproj package in our scripts to generate the Xcode workspace, and for other Xcode Project manipulations described below.

We call the submodule with the client-side code, surprise, surprise, ClientSide.

Gluing Together ServerSide and ClientSide

In our design, ClientSide is independent of the ServerSide it invokes. To connect the two parts, we use an additional Xcode project. This project is called SharedServerClient and it contains a single module with the same name. ClientSide uses this module to create the main Kitura Router of the ServerSide app.

SharedServerClient is specific per server-side part. It knows how to create the Kitura Router of the ServerSide. SharedServerClient exposes this functionality through its RouterCreator.create() public method.

The following diagram illustrates our proposed structure for the repository of a Kitura/iOS application. The Builder submodule is described in the next section.

Kitura/iOS Application Project Structure
Kitura/iOS Application Project Structure

Build Automation Code

To reuse the build automation code, i.e., makefile and build scripts, we put them in yet another repository, namely Kitura-Builder-iOS. We define Kitura-Builder-iOS as the Builder submodule in our Kitura/iOS application repositories.

I will explain why we used make as the build tool later; for now, the relevant feature is the ability to compose makefiles using the include directive. This way you can put the common makefile targets in a makefile, shared between multiple repositories, and have a custom makefile per repository. Each custom makefile will include the shared makefile.

We implemented “bootstrapping” of the Builder submodule by the custom makefile. The trick is to include the makefile from the submodule in the custom makefile, and, at the same time, to specify the included makefile as a target of the custom makefile. The action of the target will fetch the submodule with the included makefile. See a snippet from our custom makefile:

-include Builder/Makefile
Builder/Makefile:
        @echo --- Fetching submodules
        git submodule init
        git submodule update --remote --merge

The included makefile is in the Builder submodule. Note that the dash before the include directive prevents an initial error message, which is emitted when Builder submodule is not yet fetched.

To summarize the code organization of the sample Kitura/iOS applications: we use three git submodules to reuse application and build automation code. The submodules are ServerSide for the server-side code, ClientSide for the client-side code and Builder for the build automation code. We fetch and combine the submodules automatically, using the makefile include directive and Xcode Workspace.

The following diagram shows the Kitura/iOS sample applications and their dependencies, including the generic Kitura/iOS components.

Kitura/iOS Related Repositories and Dependencies Between Them
Kitura/iOS Related Repositories and Dependencies Between Them

Build Automation

We fully automated building and testing of our Kitura/iOS applications. You only need to clone your application repository and run make openXcode (in the command line). It fetches the submodules, creates a Workspace with the server-side and client-side parts and even opens the Workspace in an Xcode editor for you. Then you only need to push the Run button in the opened Xcode editor, and voila, the app builds and starts in your simulator.

Well, there is one caveat. We did not want to download and install the required libraries for you due to possible licensing/security issues, so you will have to do it yourself using a script we provide. To understand how the required libraries are used, read the Libraries section in Gory Technical Details below. The libraries will be downloaded to iOSStaticLibraries directory.

The make command will check whether the required libraries are missing and will kindly ask you to install them. The good news is that you have to install the static libraries only once and then just copy or soft-link them between multiple Kitura/iOS applications.

We also automated performing unit tests using the xcodebuild test command. To run the Kitura/iOS application’s unit tests on an iPhone 8 iOS 11.1 simulator, we run the following command (by the makefile test target).

xcodebuild test -workspace EndToEnd.xcworkspace -scheme ClientSide -destination 'platform=iOS Simulator,OS=11.1,name=iPhone 8'

Rationale for Using Makefile for Build Automation

Now, after mentioning makefile several times already in this post, I would like to explain why we used the good old build automation tool from the seventies.

Build Automation and the Swift tools

Build automation, in a broad sense, usually involves multiple tasks: managing dependencies, i.e., fetching the packages required by the project being built, verifying build prerequisites, installation of the required libraries, code and documentation generation, code compilation and linkage, packaging and deployment. While there are a lot of shiny, modern build automation and dependency management tools, like Ant, Maven, SPM, npm, Rake, Gradle and others, none of them can usually implement all the required tasks, in a convenient way.

In our case, SPM does not support running code generation scripts and does not handle resource packaging. Moreover, due to the build tool schism in the Swift world between SPM on one side, and Xcode and xcodebuild on the other side, we have to use at least two build tools.

In my opinion it is usually better to use multiple build tools, and not only in our case. You can choose the best tool for each particular task. Therefore, you need one top-level build tool to orchestrate all the others.

Make as the Build Orchestrator

make looks like the best choice for such a build orchestration tool due to its availability on Unix-like systems, simplicity, popularity, and its easy integration with other command-line tools. make can invoke all other build tools, in a simple way, just like any other command-line tool or script. In our case, we use make to fetch the required git submodules; run SPM; generate an Xcode project for the server-side code; generate a workspace to contain the client-side and server-side code; fix Xcode project settings; and run xcodebuild to perform unit tests in the command line.

To see an example of a popular modern project that uses a similar approach, take a look at Kubernetes, a popular Container Scheduling and Management framework. Their makefile invokes bash scripts, the Bazel build tool, and the go compiler. Here, as in our case, several build tools (Bazel, makefile, bash scripts) are used, with makefile orchestrating all the others.

Daily Build and Continuous Integration

The automated build allows us to perform daily build and continuous integration. We use Travis CI, the same continuous integration system Kitura projects use (see this blog post for more information). Travis CI is integrated with GitHub and performs a build and runs tests per each commit in a GitHub repository.

Running the build per each commit allows us to catch breaking changes in the Kitura/iOS sample apps repositories. However, since there are multiple projects and dependencies involved in Kitura/iOS applications (Kitura, the application’s client side and server side, and their dependencies, including C libraries), it is not feasible to run a continuous integration build on each possible change in a sub-project or a dependency. In addition to running the build per each commit in the Kitura/iOS repositories, we run the automated build daily, per each Kitura/iOS sample app repository. Travis CI includes the Cron Jobs feature that allows us to schedule the daily build automatically. This way we catch any issues that could happen due to a change in any of the dependencies, within 24 hours.

In addition to testing Kitura-HelloWorld-iOS and Kitura-Sample-iOS, and their corresponding server-side parts, Kitura-HelloWorld and Kitura-Sample, we perform a daily iOS build of Kitura itself, including running Kitura tests on iOS. This allows us to catch any breaks in the Kitura build or tests on iOS (the Kitura project itself is tested on Linux and macOS).

Automated Code Quality Analysis

We believe in writing high quality code and are big fans of automated code quality analysis tools. We use the Codebeat tool for the Swift code and for our build automation scripts written in Ruby, and also the Code Climate tool for Ruby. Both of the tools are integrated with GitHub, and perform code quality analysis automatically, per each commit.

Automated Code Coverage Analysis

Since our unit tests basically check that all the unit tests of the server-side part run correctly on iOS, the code coverage is the same as for the server-side part. The automated code coverage analysis of the server-side parts are performed in their corresponding projects, using Codecov. See this blog post for more details.

Gory Technical Details

I am glad you survived reading this post until now. Now it is time to toss in some technical issues that we had to handle to enable server-side Kitura code to run on iOS.

Libraries

Server-side Swift apps can be linked with various legacy libraries, written in C. In my opinion, this is a major advantage of the Swift programming language on the server, compared to the bytecode-compiled languages like Java or Javascript. These libraries are either part of the standard OS distribution on macOS or Linux, or they can be installed, using various package managers like Homebrew or apt-get. For iOS, however, the libraries that are not part of the SDK, must be compiled from source.

In our case, Kitura depends transitively on the libcurl library. To enable Kitura to run on iOS, we compile libcurl from source, using a bash script. Then we add the libcurl source directory to Header Search Paths in Xcode Build Settings, and the compiled libcurl library to Library Search Paths, also in Xcode Build Settings. We also add the “-lz” flag to Other Linker Flags in Xcode Build Settings, to link with libz library used by libcurl. We perform all these manipulations by Ruby scripts in Kitura-Builder-iOS, which use the Ruby Xcodeproj package.

Resources

Another issue we had to handle is the lack of support for resources in SPM. It would be nice if you could define some files/directories as resources and SPM would handle their copying to locations accessible by the compiled code, on Linux and macOS, and also on iOS, through the SPM-generated Xcode project. Tough luck. SPM does not have this functionality.

Kitura applications access static resources in the public directory and templates in the Views directory (those are the default names, the resource locations can be customized programmatically in Kitura). Since SPM lacks the support for resources, we had to copy them ourselves. For that, we added Copy Resources Build Phases to our automatically generated Xcode workspace. These Build Phases copy the resource directories into the iOS application bundle during Xcode build. As with all other Xcode manipulations, we added these Build Phases automatically, by Ruby scripts in Kitura-Builder-iOS, which use the Ruby Xcodeproj package.

Unit Tests

To run the ServerSide part unit tests on iOS, we must add them to the ClientSide part. The tests in Kitura-HelloWorld and Kitura-Sample inherit from the KituraTest class. We rewrote that class to perform the tests on iOS, using KituraTableViewController. The copied tests inherit from the iOS version of the KituraTest class and run on iOS as part of the ClientSide unit tests. We copy the test files by makefile from the ServerSide part to a subdirectory in the application’s main directory. Then we add the copied tests to the ClientSide Xcode Project by Ruby scripts in Kitura-Builder-iOS, which use the Ruby Xcodeproj package.

It is possible to run unit tests either from an Xcode editor or using the xcodebuild test command.

Creating Your Own Kitura/iOS App

To create your own Kitura/iOS application, you can perform the following steps:

  1. Recreate the project directory structure as in Kitura-HelloWorld-iOS and Kitura-Sample-iOS – create ClientSide, ServerSide and Builder git submodules. Use your own server-side application as ServerSide.
  2. Create the SharedServerClient project with a single target SharedServerClient with a single class RouterCreator. This class must have a single function – create(). The only purpose of this function, and of the entire SharedServerClient project, is to create and return the main Kitura Router of your Server Side application.
  3. Write a makefile. This makefile should include the makefile in the Kitura-Builder-iOS repository and have a target that updates and fetches the git submodules.
  4. Create a directory named ClientSideTests. Program your makefile to copy the unit tests from ServerSide into ClientSideTests. Your unit tests must inherit from the KituraTest class, see an example in Kitura-Sample. Copy the iOS version of KituraTest to your ClientSideTests. Alternatively, write your own version of a base test class that will use KituraTableViewController from Kitura-Mobile-Server.
  5. Add an xcodebuild test command to the makefile to run the unit tests from the command line.

The simplest way though would be to apply the Monkey See, Monkey Do principle, described in the Contributing to Eclipse: Principles, Patterns, and Plug-ins book by Erich Gamma and Kent Beck. This is the last software engineering principle for this blog post. In our case, it means just to fork Kitura-Sample-iOS and to change the relevant parts. If further customization is required, you can fork and customize Kitura-Mobile-Server and Kitura-Builder-iOS.

Conclusion

In this blog post, I described the technical issues and software engineering principles involved in developing Kitura/iOS applications. Future work could be to run Kitura on tvOS.

Note that in the sample applications described in this blog post, we used Kitura as a mobile server, providing a mobile backend for other applications. An additional future work item could be to explore opportunities for using Kitura as an embedded backend for the iOS application itself, i.e., for the application that embeds and runs Kitura. This embedded backend can be used for offline demos or for implementing the offline mode of the application, see my previous blog post about Kitura/iOS for a description of these use cases.


Star

Leave a Reply