Digital Developer Conference: Cloud Security 2021 – Build the skills to secure your cloud and data Register free

Convert legacy Java APIs into remote services with no extra coding

Introduction

A typical large, Java-based system, such as an analytics pipeline, contains many useful plain Java APIs (of utilities or tools) that are rarely reused because of the following key reasons:

  • The implementation of the API is resource-intensive (often memory) and cannot be called “in-process” from within the caller’s JVM. The caller might be a simple web service that can’t fit the entire analytics pipeline into its memory in order to call the API.

  • The APIs might have a lot of unwanted classpath dependencies that the caller cannot handle.

The preferred solution is to make such APIs as remote services. Frameworks such as Spring Boot and Dropwizard simplify building an application as a set of microservices from scratch. However, a framework comes with baggage.

Wikipedia defines a framework as:

… an abstraction in which software providing generic functionality can be selectively changed by additional user-written code, thus providing application-specific software. It provides a standard way to build and deploy applications and is a universal, reusable software environment that provides particular functionality as part of a larger software platform to facilitate the development of software applications, products and solutions.

alt

Figure 1. Application built using a framework and an application built using libraries

Contrast this to a library), which Wikipedia defines as:

… a collection of non-volatile resources used by computer programs, often for software development. These may include configuration data, documentation, help data, message templates, pre-written code and subroutines, classes, values or type specifications. In IBM’s OS/360 and its successors they are referred to as partitioned data sets.

The key difference between a library and a framework is “Inversion of Control.” When you call a method from a library, you are in control. But with a framework, the control is inverted: The framework calls you.

alt

Figure 2. The difference between a framework and a library

When compared to libraries, frameworks are highly opinionated and require the API owner to implement framework-specific boilerplate. Frameworks also dictate how the code is organized, which may be unacceptable to a legacy application. They often bring in additional classpath dependencies. Although frameworks are generally more powerful, it is hard to use them in a legacy codebase because these frameworks are meant to be used as the backbone or glue-code of the entire system, instead of as a simple library for remoting. They are more appropriate to be used when beginning a new project than to be integrated into an existing codebase.

To transform an existing plain Java API into a remote service using a framework, a non-trivial amount of work is required, which imposes a certain degree of inertia. But what if we could make use of a library to turn our Java APIs into remote services?

This tutorial introduces a new light-weight library that addresses these concerns and satisfies the following design goals:

  • The library should make it really easy to transform any existing API into a remote service. The API owner should not need to implement any additional interfaces to turn their API into a remote service other than to structure their API into “interface” and “implementation” classes.

  • Both the API owner and the client should not need to worry about the implementation details of the remoting other than configuration information on the client to locate the remote instances (say, using host, port combinations).

The library architecture is based on Java Dynamic Proxies and Akka actors.

Example

As an example of how the library might work, consider the following toy API (source code available in GitHub) that we want to expose as a remote service:

public interface ExampleApi {
    public String echo();
    public int add (int a, int b);
    public int subtract (int a, int b);
    public int multiply (int a, int b);
    public int divide (int a, int b);
    public ComplexObject getComplexObject(int i);
}

And let us assume that there is a trivial implementation:

public class ExampleImpl implements ExampleApi {
    public String echo() {
        return "Echo";
    }
    public int add (int a, int b){
        return a + b;
    }
    public int subtract (int a, int b) {
        return a - b;
    }
    public int multiply (int a, int b) {
        return a * b;
    }
    public int divide (int a, int b) {
        return 1.0 * a / b;
    }
    public ComplexObject getComplexObject(int i) {
        if (i == 99) {
            throw new RuntimeException("Sample");
        }
        return new ComplexObject(i + 1);
    }
}

If the implementation is accessed locally, the normal client-side code would look like the following:

ExampleApi api = new ExampleImpl();
api.add (5, 6);

Using the library, a service of the implementation could be started in a remote machine with a command line similar to the following:

java -Dthreads=$NUM_THREADS server.impl.ServerLauncher ExampleImpl $PORT

In a production setup, service JVMs could be launched from Docker instances that are managed by Kubernetes.

But with the framework in place, the client code will look something like the following:

ExampleApi api = ClientFactory.getInstance().getClient(Protocol.REMOTE_ACTOR, props).get(ExampleApi.class)
api.add(5, 6);

The argument props is a java.util.Properties object containing the connection information, such as the host and the port of the server. Further, load-balancing between the multiple service instances is handled transparently by the library. This can be configured by specifying multiple host, port combinations in the props.

Note that the API implementation code itself hasn’t undergone any change to become a remote service. Now that we have seen an example, let’s discuss the architecture of the library.

Architecture

The library architecture consists of two layers — client-side and server-side. The client-side portion of the library intercepts calls made to the target API and translates them to calls to a remote service. The server-side component receives these calls and invokes the actual implementation. This mechanism is mostly transparent to both sides — the client code calling the API as well as the API implementation itself. Both the client-side and server-side layers make use of Java Dynamic Proxies. The architecture is shown in the following figure. The original Java API and its implementation remain unchanged even as the API is turned into a remote service.

alt

Figure 3. Architecture of the remoting library

The architecture is modular with the following three components:

  • Client-side Invocation Handler – Uses the Java Dynamic Proxy mechanism to create a proxy that intercepts the calls made to the API. The proxy passes the API method and parameters information to the Payload and Protocol Handler.

  • Payload and Protocol Handler – Responsible for managing the client-server communication, including payload marshalling/unmarshalling, and wire-protocol management. It marshals the method and parameter information into a protocol-specific payload and invokes the remote service. On the service side, once the call is received, the Payload and Protocol Handler unmarshals the payload into method and parameters, and passes the information to the Remote Invocation Handler.

  • Server-side Invocation Handler – Invokes the method on the local implementation object with the parameters. When the invocation results are available, the control goes back to the Payload and Protocol Hander, which marshals the response (either a valid result or an exception) and sends it back to the client side of the framework, where after unmarshalling, the response is given to the caller.

Modular architecture of the library, which allows for supporting different protocols between the client and the server side of the framework

Figure 4. Modular architecture of the library

Currently, Akka actors are used for the client-server communication, but the library can be extended to support different implementations of Payload and Protocol Handler, as shown in Figure 4. A variety of transport protocols (such as HTTP, TCP Sockets, and more) and payloads (such as Java objects, JSON, and more) could be supported through the extension mechanism, as shown in Table 1. The modular design that makes such extensions possible is described in the next section.

alt

Table 1. The library is capable of supporting different protocols, such as plain sockets, HTTP, and more, between the client and the server side through extensions

Design

The class diagram of the library is shown in Figure 5. The framework makes use of an abstract factory pattern to provide different implementations of the server- and client-side components, AbstractServer and AbstractClient respectively.

alt

Figure 5. Class diagram of the library

View a larger version of Figure 5

Control flow – client side

The control flow within the framework on the client side is shown in Figure 6. The main component of the system is Invoker, which intercepts the method call made to the API. It extends the InvocationHandler interface and implements the mandatory invoke() method. Upon intercepting the method call, it extracts the information about the target API method being invoked and the method parameters using Reflection. It constructs a Payload with this information. Finally, it invokes the call() method of the AbstractClient with the Payload object. Note that this is a synchronous operation. While this is not generally preferable for scalability reasons, it is perfectly suitable in this context: the client code cannot make progress without a response from the API. At this point, the control passes to the a specific implementation of the client, such as ReactiveClient, which implements the call() method. Once the response is received, the doCall() method of the AbstractClient is invoked.

alt

Figure 6. Control flow on the client side for the method invocation add (2+3)

View a larger version of Figure 6

The communication between the client and server side could be asynchronous, depending on the choice of protocol implementation. This means that:

  • The AbstractClient needs to keep track of the caller and pass the response back to the appropriate caller once it is received from the server side.
  • The communication with the server side needs to take place in a separate thread in order to not block other callers of the API.
  • The actual caller needs to wait until a response is received from the server.

To achieve these three goals, the AbstractClient maintains a map (handlers) of ResponseHandler objects. Each object is identified by a unique request id. The ResponseHandler class implements the Callable interface to be called from a thread. The thread pool is implemented through an ExecutorService maintained by the AbstractClient. Each ResponseHandler object maintains a CountDownLatch to wait for the response from the server. Once the response is received, the handle() method of corresponding ResponseHandler object is invoked by the AbstractClient to trigger the shutdown of the CountDownLatch.

Control flow – server side

The control flow within the framework on the server side is relatively simple, as shown in Figure 7. The AbstractServer instantiates the actual API implementation during the start up using reflection. The request is received by the AbstractServer after going through the protocol-specific server-side layers. It unpacks the Payload object and extracts the method and parameters. Then it invokes the actual implementation of the API, again using reflection.

alt

Figure 7. Control flow on the server side for the method invocation add (2+3)

View a larger version of Figure 7

Running the code

The source code for the library is available in GitHub. To build the code, Gradle, Version 5.1 or later, is required. Run gradle clean build to compile the code and run the unit tests. Provided tests demonstrate the capabilities of the library by starting up a temporary server instance with the ExampleImpl and invoking the ExampleApi from within the tests.

Benchmarking

The runtime telemetric about API usage is also exposed through JMX. For example, for the toy service used in this tutorial, we can inspect the statistics about the performance of various methods of the API, as shown in Figure 8.

alt

Figure 8. Real-time telemetry about the usage of different API methods can be obtained using JMX

The performance benchmarking was done by running the toy API as a service, with a million invocation of each method from a multi-threaded remote client running within a gigabit local network. Testing indicates the performance overhead due to the library layers is around 0.01 millisecond including the network transport time.

Conclusion

Frameworks are considered anti-patterns in software development. In the case of legacy applications, using a new framework just to expose an existing functionality as a remote API could be very difficult, as the new framework may increase the project complexity and dependencies. They are also hard to learn well-enough to use effectively in a short time. A library, on the other hand, does not have these shortcomings. In this tutorial, we introduced a new library that helps turn plain Java APIs into remote services with very little effort. Unlike many microservice frameworks, the library is light-weight, performant, and, most importantly, non-intrusive.