How do we make the future serverless?

The first post of this two-part series explored why we believe serverless computing is the future of cloud computing. We examined the evolution of cloud computing and observed the current use cases that are already moving to a serverless model. In this post, we describe how to achieve this serverless future and some of the challenges that await. Finally, we conclude this series by summarizing the argument and future vision.

Challenges

Moving to a more serverless world would solve many of the issues that are encountered when running modern workloads, but many challenges remain. To explore the challenges with running serverless workloads, we divide them into four broad categories: cloud optimization, service design, functions and workflows, and security. In each category, we list important specific issues that remain to be fully solved. While not comprehensive, we believe that these challenges can serve as stepping stones that serverless communities can use to plan their roadmaps.

Summary of challenges

Cloud optimization

Serverless fits the allocated compute resources to the service demand. To achieve this, serverless allocates new compute resources as demand grows, and releases compute resources as demand shrinks. The overhead of allocating and releasing compute resources is probably one of the most important issues that comes up after serverless application deployment. Serverless services (whether functions, containers, or even groups of containers that operate as a workflow) incur a cost associated with scaling down and back up. This performance cost can be significant since it comprises various aspects of the lifecycle of a serverless service.

Most cloud technologies, including Kubernetes, were designed with pre-provisioning of service resources in mind. But with a serverless model, they must be optimized for more dynamic use cases by reducing the overhead of adding and removing a running service instance. For example, adding a pod replica to a Kubernetes deployment must be optimized to minimize overhead, which also minimizes cost and energy consumption for each change in service demand.

Another aspect of the need to add resources when demand grows is service latencies. The time that it takes to add resources can result in increased service latency while you wait for additional resources to be allocated. In some extreme cases, when a first request arrives to a service that has no allocated resources, the service might be affected by significant cold start times as discussed in Reducing cold start times in Knative. In the worst case scenario, the service startup can be multiple seconds, whereas a normal service request-response sequence is typically just tens of milliseconds. Therefore, the cold start penalty is significant and must be accounted for when you create serverless solutions that consist of multiple parts that scale independently. Further improvement of cold start issues requires more optimizations in cloud technologies.

Last, optimization of Kubernetes and Knative is needed to fully take advantage of the potential for energy and financial savings. Pod allocation and cluster protocols should evolve to support dynamically condensing (defragmentation) services to a subset of worker nodes such that remaining worker nodes can be suspended and later resumed when overall demand for cluster resources increases.

Service design

One of the main challenges when you create serverless applications and solutions is to ensure that the application and services that comprise it are designed with an event-driven mindset. That is, with an understanding of the events that drive the various parts of the applications and overall solutions. Sometimes these events come from your natural understanding of the application's domain. However, sometimes they must be explicitly extracted.

For instance, in an application that represents a business process, say a vacation reservation system, there are clear events that drive the entire usage of the application. As customers plan a vacation, they must select and book a series of choices in sequence, such as location, flights, hotels, car rental, meals, and activities. The entire process can be decomposed into services and events that flow between them. This kind of decomposition and design is well known as enterprises make the transition to designing their applications as microservices.

The serverless, event-driven architecture simply adds that the different services are triggered with events (external or internal) and can be scaled down to zero if they are not processing any requests. To take full advantage of this architecture, you must externalize the events that flow across the services, such that these events trigger the processing flow in the serverless components mesh. Because services can scale dynamically, a new concern, as previously discussed, is accounting for the potential cold start consequences, while still displaying a good, fast user interface.

Event-driven architecture has its own set of challenges that must be also addressed. Much can be dealt with by choosing the correct event broker, triggers, event model, and such. However, parts of your application might need to be made aware of the pitfalls. In particular, event storms and the possibility of events dropping or becoming irrelevant after some amount of time. All of this is what we term Designing for Serverless.

Functions and workflows

When AWS introduced its Lambda service in 2014, the first model was a purely functional programming model. Services executed as functions defined in various languages and libraries. Over time, projects like Knative proposed a more generic model that centers around containers, which like Lambda functions, execute on events and scale up or down as the demand (requests) arrives for the service. In time, Knative proposed its own functional programming model: Knative Functions.

While the existence of different serverless technologies is great for innovation, the result is that they are not compatible as a standard doesn't exist. Yet the compatibility challenge is addressed at the boundaries. First, the container image formats and execution are being defined by the Kubernetes community. Most importantly, the event model in many of these serverless platforms is the CloudEvents standard. It's important that such standards exist to allow cooperation and collaboration, and one day execution, of these workloads across clouds. Kubernetes provides the common execution platform, so what we need are common definitions for the serverless bits on top.

A final challenge is having a common representation of flows between serverless services. Creating a nontrivial serverless solution not only requires triggers, sources, and sinks of events to the different services, but also coordination between the services. Partial solutions exist in various flow languages. The Tekton pipeline language is a step in that direction, but it's not designed for arbitrary complex graphs with synchronization points. A complete workflow language might be needed. This language might be an evolution of Tekton or at least a superset.

Security

With every new technology, new cybersecurity challenges emerge. Serverless changes the threat landscape as more control moves from users to cloud automation and specifically the control over where and how many compute resources are used to meet the demand. As a result, network boundaries fade away and with them many of the associated security controls.

Further, with the lower costs and simplification that are offered by serverless, we expect many workloads to divide into more granular autonomous services. With more fragments that compose a workload, offenders see a greater attack surface. See Chapter 5: Security Threat Model of Serverless of the Cloud Security Alliance (CSA) paper, How to Design a Secure Serverless Architecture.

The Knative community addresses the new cybersecurity challenges in several ways:

• Increasing the efficiency and speed of rolling out updates to deployed services. Knative provides primitives to help with revision management and the ability to split traffic across service revisions. Services deployed with Knative offer a smooth and risk-free transition between revisions, allowing more frequent updates to mitigate evolving threats and newly discovered vulnerabilities.

• Avoiding service configuration drift Knative automation ensures that all Kubernetes resources necessary for deploying a service are automatically derived from a single service resource. Manual changes to any of the resources, intentional or unintentional, by an offender or insider, are overwritten by Knative automation while avoiding configuration drift.

• Avoiding infrastructure configuration drift Knative uses Kubernetes operators to help manage the underlying Knative components themselves and to ensure that no manual changes are added to the deployed serverless infrastructure.

• Security-Behavior Monitoring and Controls The Knative community is introducing new technologies to monitor the behavior of deployed services and the behavior of events sent to such services. Security Guard is a per-service, tailor-made, runtime security that protects vulnerable services against both zero-day and known vulnerabilities. The technology also provides you with mechanisms to cope with cases in which a service is exploited.

Conclusion

As cloud computing continues to grow, it becomes a significant player in worldwide energy consumption, estimated by the International Energy Agency at 1-1.5% from global energy consumption in 2021 and with 1% of global CO2 emissions. Projects like Kubernetes standardized the unit of workload deployment and how it is managed. Knative goes a step further by making it serverless and efficient. This efficiency contributes to greener cloud computing and to significant cost savings for both cloud providers and cloud users. We argue that this serverless model should apply to all workloads. Being more efficient is not only about individual workloads but also about using underlying cloud resources and consuming less energy per workload execution.

On the one hand, beyond efficiency, this new serverless-first world offers additional benefits to cloud users, including simplification, automation, revision control, and improved cybersecurity. Yet, it is not without its challenges. Kubernetes needs to evolve and optimize better for the dynamic nature of auto-scaling services based on actual load. Additional work is still to come to make serverless more generic and encompass additional use cases. Users must advance their workload and deployment strategies to use a serverless model, which, like any change, takes time to evolve.

As we move to a serverless-first world, new workloads and use cases need to be accounted for. The AI, data, and machine learning (ML) communities have a head start in understanding the value of this on-demand computing model. Projects like Kubeflow and Tekton are great examples of how to design and execute AI pipelines of data-intensive models and workloads in a serverless fashion. Other future workloads and use cases include quantum serverless, which enables on-demand use and execution of quantum algorithms on quantum computers. It mixes a serverless execution of the classical parts of the quantum solution with quantum circuits that execute on quantum computers as needed, for the necessary timeframe.

A serverless-first world is one where costs and resources are economically rationed because the applications are designed for it and the platform is efficient and economical for running serverless solutions. This serverless-first world makes sense for all parties. As a bonus, it is great for the environment.

Acknowledgments

Thank you to Paul Schweigert and Lionel Villard for their feedback and comments. Also, we owe a huge debt of gratitude to the entire Knative community for their dedication and hard work to create the serverless platform for Kubernetes. In less than two years, the community released over a dozen releases of the project, improving all aspects, staying compatible with Kubernetes, and allowing various innovations as extensions and experiments.