The OSGi demystified article series addresses common OSGi issues in CICS — offering insight into OSGi, discussing best practices, and providing setup and configuration advice. This second article addresses a number of common Java and OSGi class loader problems that you might encounter in CICS OSGi and Liberty JVM servers.
This section looks at:
- The difference between a
- How to solve
- OSGi dependency resolution and class loading
- Gaining performance with stricter OSGi compliance
Whether you are developing a new OSGi application, migrating an existing Java application to OSGi, or simply using third-party components that aren’t playing nicely with OSGi — chances are good that you’ll eventually hit a class loading issue and curse CICS for inflicting OSGi upon you.
Once you have finished turning the air blue … take a deep breath and relax. Here at CICS Java HQ, we believe the benefits of OSGi far outweigh the initial learning curve. Whether it’s the mature component model, the power of versioning, the improved class loading performance, the dynamic service architecture, or the install time dependency resolution — there are many reasons why OSGi is a good choice. That said, you are probably not here because your OSGi journey has been smooth. You have most likely fallen afoul of OSGi’s sharp edges and are looking for enlightenment. So let’s get straight to it.
ClassNotFoundException and NoClassDefFoundError
Take a careful look at the failure. Did you get a
ClassNotFoundException or a
conditions are easily confused. They mean very different things despite
the seemingly common problem of the class failing to load. Let’s start
with the obvious differences.
ClassNotFoundException is an exception while
NoClassDefFoundError is an error. You probably realised that already, but it’s a
good way to tell them apart. Be advised there are no other variants of
either the exception or the error despite numerous typing errors out on
the wild web!
NoClassDefFoundError is a result of the JVM failing to load a
class that was present during compile time. That means the class was
available to your development environment and the application compiled
— but for some as yet unknown reason, the class is not available to
A ClassNotFoundException, on the other hand, is a result of your code
(or third-party code) — not the JVM — dynamically attempting to load a
class and failing. The code has probably used
loadClass() with a textual class name. That class name might have been
taken from a properties file, a configuration file, or it was just plain
hard-coded. Alas, the compiler didn’t know about this class when your
application was compiled. The exception indicates that the class you
want to dynamically load is not visible to your OSGi bundle’s class
Let’s begin by analysing the
NoClassDefFoundError. Why did your
application fail to load the class if it was found at compile time? We
stressed the word load because this error can occur either as a result
of the class not being found, or if the class was found but failed to
load. It is quite plausible that something else may have prevented the
class from fully loading — perhaps an exception from a static
initialiser? Take a good look at the exception stack trace as the clues
to load failures are usually there.
If a class really is not found and you are in plain Java (non OSGi), it is extremely likely that you have omitted a JAR from your application class path. Go find it, add it to the class path, and sit on the naughty step for a while.
But we’re dealing with OSGi here… so that can’t happen, can it? Newsflash:
Yes it can! Each bundle has a class path (the
configured in the
MANIFEST.MF of the JAR), and your bundle can contain
JARs. So if you are
abusing exploiting OSGi in this way and making
it look like a class path application, then you should expect all the
associated problems of a class path. Remedies include:
- Ensuring that all the required JARs are deployed within your bundle and listed on the
- Taking the dependent JARs, turning them into bundles, and deploying them as first class OSGi bundles. Doing so will give you the full power of
OSGi’s dependency resolution, versioning, dynamic updates, and so on.
Don’t forget to update your original bundle with an appropriate
This section looks at:
NoClassDefFoundErrorissues beyond class path clumsiness
- Boot delegation — the act of delegating class loading to OSGi’s boot class loader
- Boot delegation in CICS
NoClassDefFoundError: Beyond minor class path errors
Looking beyond class path clumsiness, a
generally indicate that you failed to declare a dependency on another
package. Your bundle does not go looking for classes from other bundles
unless you tell it to with an
Import-Package statement in your
MANIFEST.MF. Maybe your class depends on a super class or has a member
from another package that is not available within the same bundle? If so,
you might hope that your development tooling will spot the
external dependency. Usually it does, but unfortunately not always. Some
development environments such as Eclipse (and by inference CICS
Explorer) can be too tolerant and don’t always perform the strict OSGi
compliance that’s required to flag up missing dependencies.
For example, let’s say your application uses Java extensions. At coding
time, if you use an object from an extension package, you’ll want to add
import for that particular
javax.\ package to your
source code. Failure to add the
import is obvious from the red-X.
Clicking the “I don’t know what I’m doing — fix it for me” suggestion
import to your Java source. However, the same is not true
for OSGi dependencies: Eclipse won’t notice if you fail to add the
javax.\*package to your bundle’s
Import-Package statement. That
particular omission is mysteriously tolerated, as explained next…
The mystery explained with boot delegation
This situation arises because the OSGi framework running in your Eclipse
development environment has been configured, by default, to delegate all
javax.\* package requests to the parent class loader of the OSGi
environment. In OSGi terms, the parent class loader is called the boot
class loader. Don’t confuse it with Java’s bootstrap class
loader, which is just the OSGi name for whichever
class loader loaded the OSGi framework. The act of delegating class
loading to OSGi’s boot class loader is referred to as boot
Now, because the OSGi framework’s parent class loader is usually Java’s application (or system) class loader, delegating class load requests to it will break out of OSGi’s network of peer (OSGi bundle) class loaders, and back to standard Java class loading rules. If you weren’t already aware, Java performs parent-first class loading. So the application class loader delegates to the extensions class loader, and the extensions class loader delegates to the bootstrap class loader, and the bootstrap class loader is connected to the elbow… er, well you get the idea.
Typically, core JRE classes are on the class path of the bootstrap class
javax.\* packages are on the class path of the
extensions class loader. Each request has to go all the way up the
hierarchy before rippling back down until the load request can be
fulfilled. Keep that parent-first search pattern in mind.
In a strict OSGi runtime, boot delegation of
javax.\* does not
occur because the act of boot delegation precludes the
existence of an alternate implementation or a newer version of a
package. OSGi looks directly to the boot class loader hierarchy,
and due to the parent-first class loading it will likely find the
(original) class on the extensions class loader and ignore the bundle
where your preferred and versioned implementation of an extension
package resides. Boot delegation does this even if you are a good
citizen and declare your bundle’s dependency on the newer version.
So in a stricter OSGi environment like CICS, you don’t boot delegate the
javax.\* packages. If you don’t declare your
dependencies and your development environment doesn’t highlight the
omissions, expect to be served with a
yourself a favour and declare all your package dependencies on the
Import-Package statement in the
MANIFEST.MF of your OSGi bundle.
As a general guideline, only the core
java.\* packages and any
packages internal to the bundle can be omitted from your
Import-Package statement. The core
java.\* packages are always
delegated to the bootstrap class loader of the JRE, for obvious security
3. OSGi resolution
This section includes:
- Definitions for class path and class loader
- The problems around
- The Four Scopes of OSGi resolution
Let’s take a moment to review some essential definitions.
A class path is a list of files, typically JARs on the file-system where a class loader looks for classes. A class loader uses its class path to read the classes from the file-system and returns instances of classes to those parts of the application that require them.
As mentioned earlier, a
ClassNotFoundException is most likely a result
of your code actively loading a class using
Class.forName(). To rub
salt in the wound, it probably could have worked outside of OSGi. This can easily be
explained because when everything is on one big class path, and
presuming the necessary JARs exist on the class path, the class will
So what is the problem with Class.forName() in OSGi?
Recall that OSGi is a component-based system that allows multiple versions of a package to exist within the same runtime. Each component is a bundle. Each bundle has its own class loader. Each bundle’s class loader loads only the classes contained in that bundle. By allowing different versions of a class to be loaded by different bundles, multiple versions of the same class can be available in the runtime. This is something that cannot easily be achieved using Java’s standard parent-first delegation model.
In contrast to the one big class path approach, OSGi has a peer
network of class loaders. External packages required by your bundle are
declared as dependencies (i.e.
Import-Package) with a package/version
combination. When the OSGi framework resolves your bundle, it determines
the best fit and “wires” your bundle to those bundles that are providing the
dependencies. Thus OSGi becomes a web of bundle class loaders that
delegate to each other. If a class is not on your bundle’s class loader,
then wiring is used to ask another bundle to load the class on your
behalf and return a class instance.
Therein lies a problem.
Class.forName() can be used to dynamically
load arbitrary classes at runtime. An example of this might be
applications that load the IBM Data Server Driver for JDBC and SQLJ.
Although no longer a requirement for Db2, the technique is still
prevalent. Tooling isn’t able to spot dependencies hidden in your
source code in this way. So unless you know in advance what the
dynamically loaded packages will be, and can specify those dependencies
MANIFEST.MF and ensure that a provider bundle exports the
relevant package, then the resolution process (wiring) will not occur.
To put it simply, if the class you want to load is from a package
outside of your OSGi bundle, and you are using “back-door” class loading
techniques, then it’s up to you declare the dependency!
More often than not, failing to declare dependencies on classes that are
dynamically loaded causes
ClassNotFoundExceptions. If you want to
understand more about dynamic class loading, how to workaround the
problems it introduces, or (better still) how to avoid it entirely, sit
tight for Part 3 of the OSGi demystified series which focuses on dynamic class
Before we move on, I ought to mention that there are other, less common
causes of a
ClassNotFoundException. Taking one particular scenario: A
dependency is specified, the package exists, and it is exported by
another bundle. So what’s up? Well, it is feasible that the specific
class from the declared package is not actually contained within the
bundle. Granted, one might consider this a development error, but OSGi
is effectively tricked into resolving your bundle against a package
that is a banana short of a bunch. You don’t need me to tell you not to
point that gun at your feet.
The magic wand of OSGi resolution
(Spoiler — there isn’t one.) There may not be a magic wand, but there is a substitute technique that can help you resolve class loading problems. I like to call it the Four Scopes of OSGi resolution, coined here for the first (and probably last!) time to help fix all those failures of the Class Not Found genus.
It’s a pretty simple concept. Progressively larger scopes are used to perform dependency resolution until all classes are found. A Liberty JVM server is a little more involved. Java EE applications are packaged into WAR, EJB, or EBA files which are loaded through isolated OSGi sub-systems, where additional class loading is provided through shared libraries — so for this article, let’s stick to the pure OSGi picture.
Your bundle is the first
scope (shown with “1” on the diagram); anything located directly within your bundle can be loaded without further ado.
If your bundle includes or wraps other JARs, just remember to put those
JARs on the
Bundle-ClassPath to get at their contents. If you want
to make the packages from those JARs available to other bundles, you can
do so with an
But before you do, hold up — there is a curveball. If a package within
your bundle (or on your
Bundle-ClassPath) is also explicitly imported
by your bundle, the imported package will take precedence. That’s
mildly perplexing — until someone explains to you that the exact same
classes loaded from a different class loader are not considered the same
to Java (read “actively incompatible”). So to reduce the likelihood of
the same class being loaded by your bundle and by another bundle, an
imported package takes precedence over your internal package. The
result is that class loading is delegated to the same class loader
(bundle) and everyone gets the same ball. Technically this is known as
ensuring class space consistency.
The second (“2”) scope, as we’ve just discussed, expands to include all the other bundles
that are installed into the OSGi framework. If a class isn’t in your bundle, maybe
it can be found from another? For that, you need to declare an
Import-Package dependency. Naturally, a bundle must also exist that
provides that class and exposes the package with a corresponding
Still not found? Then the third (“3”) scope
encompasses all bundles and the inner lining of the OSGi framework, more
technically known as the system bundle or bundle zero. The system
bundle provides all the dependencies that are defined on OSGi’s system
packages parameter. To your bundle, it is just another bundle.
Frameworks such as Equinox (used by CICS and Liberty) have a
default list of packages that typically include
org.w3c.\* packages. This is OSGi’s way of exposing classes from
the external boot class hierarchy to bundles within the framework
without resorting to the undesirable parent-first delegation that
spoils your version party. CICS allows you to extend this list of
packages, if necessary, using the following syntax in your JVM profile:
# Extend the System Packages list to expose the following boot class-loader packages through the OSGi system bundle -Dorg.osgi.framework.system.packages.extra=com.ibm.xylem.types, com.ibm.xylem, com.ibm.xtq
The fourth (“4”) scope includes packages outside of the OSGi framework that aren’t
exposed by the system bundle, but are exposed through boot delegation.
Boot delegated packages should be considered internal to Java, like
com.sun.\*. They are not considered suitable for replacement
implementations. Boot delegation is the worm-hole that leads straight to
the outermost scope with no stops for sightseeing and no chance to
change your mind. Again, CICS allows you to specify your own list of boot-delegated
packages, but you should only modify it as a last resort — there are very few
situations where this is the right choice.
4. Boot compatibility
This section includes:
- Closing words on boot compatibility and how it works within CICS TS V5.3+
- The effects of switching boot compatibility off
Addendum — boot compatibility
From CICS TS V5.3, the OSGi runtime compensates for the lack of strict
development tool dependency checking by performing a type of class
loader delegation that’s affectionately referred to as boot compatibility.
Boot compatibility instructs the Equinox framework to perform a last
gasp delegation to the boot class loader. If the class is not found
through the explicit dependency declarations (i.e.
then and only then will delegation to
the boot class loader happen. Consider it a “get out of jail for a small
fee” option. The small fee is one of performance and of tolerance to
improperly specified dependencies.
That said, if you have the freedom to make application changes, I
recommend that you you turn off this mollycoddling behaviour by adding
-Dosgi.compatibility.bootdelegation=false to the JVM profile of your JVM server:
# Switch off OSGi boot compatibility so that it highlights when Import-Package dependencies have been omitted -Dosgi.compatibility.bootdelegation=false
What happens with boot compatibility off?
1) If nothing different happens, put your halo on. Either your code
doesn’t import any of the boot delegated packages, or you already
added those dependencies to the
Import-Package statement in your
MANIFEST.MF. Consider yourself optimised.
2) If your bundle(s) fail to resolve, that’s a good thing — really! Now
you can explicitly add the dependencies (indicated by
NoClassDefFoundErrors) to your bundle
MANIFEST.MF until it once again
resolves and starts. Sit back, feel smug. Class loading will be more
efficient and your bundle’s interface will be well-defined and more
portable. The CICS OSGi default settings are no longer required to
protect your application.
Of course there is a quicker eyeballing method. If you can view the
import statements in the Java source, go ahead and determine whether
all the packages provided from outside of your bundle have a
Import-Package statement in the
While we’re still on the
Import-Package soap-box (and please don’t
Require-bundle — it is a legacy header that ties you to a
specific bundle implementation, sacrificing flexibility and package
granularity), be sure to pay attention to the version information on your
- Never omit the version qualifier, because you’ll get anything.
- Never specify a single version, because it is interpreted by OSGi as a minimum, and when an incompatible interface change comes along you’ll walk right into it.
- Always specify a version range up to, but not including, the next likely interface breakage.
If you aren’t already familiar with OSGi semantic versioning, please read about it to learn how it can be used to tolerate bug fixes (micro version changes) and API additions (minor version changes) while protecting you against major version changes (interface breakages).
A good example of how to give flexibility to your application is
Import-Package: com.ibm.cics.server;version="[1.0.0,2.0.0)". A
version range like this tolerates newer versions of JCICS, while providing a heads
up if IBM makes an incompatible change. In this syntax, version 1.0.0
and upwards are included — denoted by the square bracket, while
version 2.0.0 is excluded — denoted by a round bracket.
This article highlighted the differences between
ClassNotFoundException. It suggested
probable causes of each failure condition, and provided the Four Scopes
of OSGi resolution as a rule of thumb for fixing them. It provided
guidance on how to achieve stricter OSGi compliance and take
performance advantages by explicitly declaring all of your bundles
dependencies. Finally, I strongly advised you to avoid
dynamic class loading — there are better ways.
Now you can put your newfound knowledge to work by practicing on an existing OSGi failure. Or try differentiating the types of issues described here. Be sure to check out the other articles in the OSGi demystified series. Stay tuned for Part 3, which will show you how to avoid dynamic class loading.