IBM Cloud Satellite: Build faster. Securely. Anywhere. Read more

OSGi demystified, Part 2: Unraveling Java and OSGi class loader problems

The OSGi demystified article series addresses common OSGi issues in CICS — offering insight into OSGi, discussing best practices, and providing setup and configuration advice. This second article addresses a number of common Java and OSGi class loader problems that you might encounter in CICS OSGi and Liberty JVM servers.

OSG-sigh!

This section looks at:

  • The difference between a ClassNotFoundException and a NoClassDefFoundError
  • How to solve ClassNotFoundExceptions and NoClassDefFoundErrors
  • OSGi dependency resolution and class loading
  • Gaining performance with stricter OSGi compliance

Whether you are developing a new OSGi application, migrating an existing Java application to OSGi, or simply using third-party components that aren’t playing nicely with OSGi — chances are good that you’ll eventually hit a class loading issue and curse CICS for inflicting OSGi upon you.

Once you have finished turning the air blue … take a deep breath and relax. Here at CICS Java HQ, we believe the benefits of OSGi far outweigh the initial learning curve. Whether it’s the mature component model, the power of versioning, the improved class loading performance, the dynamic service architecture, or the install time dependency resolution — there are many reasons why OSGi is a good choice. That said, you are probably not here because your OSGi journey has been smooth. You have most likely fallen afoul of OSGi’s sharp edges and are looking for enlightenment. So let’s get straight to it.

ClassNotFoundException and NoClassDefFoundError

Take a careful look at the failure. Did you get a ClassNotFoundException or a NoClassDefFoundError? These conditions are easily confused. They mean very different things despite the seemingly common problem of the class failing to load. Let’s start with the obvious differences. ClassNotFoundException is an exception while NoClassDefFoundError is an error. You probably realised that already, but it’s a good way to tell them apart. Be advised there are no other variants of either the exception or the error despite numerous typing errors out on the wild web!

A NoClassDefFoundError is a result of the JVM failing to load a class that was present during compile time. That means the class was available to your development environment and the application compiled — but for some as yet unknown reason, the class is not available to your runtime.

A ClassNotFoundException, on the other hand, is a result of your code (or third-party code) — not the JVM — dynamically attempting to load a class and failing. The code has probably used Class.forName() or loadClass() with a textual class name. That class name might have been taken from a properties file, a configuration file, or it was just plain hard-coded. Alas, the compiler didn’t know about this class when your application was compiled. The exception indicates that the class you want to dynamically load is not visible to your OSGi bundle’s class loader.

NoClassDefFoundError

Let’s begin by analysing the NoClassDefFoundError. Why did your application fail to load the class if it was found at compile time? We stressed the word load because this error can occur either as a result of the class not being found, or if the class was found but failed to load. It is quite plausible that something else may have prevented the class from fully loading — perhaps an exception from a static initialiser? Take a good look at the exception stack trace as the clues to load failures are usually there.

If a class really is not found and you are in plain Java (non OSGi), it is extremely likely that you have omitted a JAR from your application class path. Go find it, add it to the class path, and sit on the naughty step for a while.

But we’re dealing with OSGi here… so that can’t happen, can it? Newsflash: Yes it can! Each bundle has a class path (the Bundle-ClassPath header configured in the MANIFEST.MF of the JAR), and your bundle can contain JARs. So if you are abusing exploiting OSGi in this way and making it look like a class path application, then you should expect all the associated problems of a class path. Remedies include:

  • Ensuring that all the required JARs are deployed within your bundle and listed on the Bundle-ClassPath
  • Taking the dependent JARs, turning them into bundles, and deploying them as first class OSGi bundles. Doing so will give you the full power of OSGi’s dependency resolution, versioning, dynamic updates, and so on. Don’t forget to update your original bundle with an appropriate Import-Package statement.

Boot delegation

This section looks at:

  • NoClassDefFoundError issues beyond class path clumsiness
  • Boot delegation — the act of delegating class loading to OSGi’s boot class loader
  • Boot delegation in CICS

NoClassDefFoundError: Beyond minor class path errors

Looking beyond class path clumsiness, a NoClassDefFoundError will generally indicate that you failed to declare a dependency on another package. Your bundle does not go looking for classes from other bundles unless you tell it to with an Import-Package statement in your MANIFEST.MF. Maybe your class depends on a super class or has a member from another package that is not available within the same bundle? If so, you might hope that your development tooling will spot the external dependency. Usually it does, but unfortunately not always. Some development environments such as Eclipse (and by inference CICS Explorer) can be too tolerant and don’t always perform the strict OSGi compliance that’s required to flag up missing dependencies.

For example, let’s say your application uses Java extensions. At coding time, if you use an object from an extension package, you’ll want to add an import for that particular javax.\ package to your source code. Failure to add the import is obvious from the red-X. Clicking the “I don’t know what I’m doing — fix it for me” suggestion adds the import to your Java source. However, the same is not true for OSGi dependencies: Eclipse won’t notice if you fail to add the javax.\*package to your bundle’s Import-Package statement. That particular omission is mysteriously tolerated, as explained next…

The mystery explained with boot delegation

This situation arises because the OSGi framework running in your Eclipse development environment has been configured, by default, to delegate all javax.\* package requests to the parent class loader of the OSGi environment. In OSGi terms, the parent class loader is called the boot class loader. Don’t confuse it with Java’s bootstrap class loader, which is just the OSGi name for whichever class loader loaded the OSGi framework. The act of delegating class loading to OSGi’s boot class loader is referred to as boot delegation.

Now, because the OSGi framework’s parent class loader is usually Java’s application (or system) class loader, delegating class load requests to it will break out of OSGi’s network of peer (OSGi bundle) class loaders, and back to standard Java class loading rules. If you weren’t already aware, Java performs parent-first class loading. So the application class loader delegates to the extensions class loader, and the extensions class loader delegates to the bootstrap class loader, and the bootstrap class loader is connected to the elbow… er, well you get the idea.

Typically, core JRE classes are on the class path of the bootstrap class loader, and javax.\* packages are on the class path of the extensions class loader. Each request has to go all the way up the hierarchy before rippling back down until the load request can be fulfilled. Keep that parent-first search pattern in mind.

In a strict OSGi runtime, boot delegation of javax.\* does not occur because the act of boot delegation precludes the existence of an alternate implementation or a newer version of a package. OSGi looks directly to the boot class loader hierarchy, and due to the parent-first class loading it will likely find the (original) class on the extensions class loader and ignore the bundle where your preferred and versioned implementation of an extension package resides. Boot delegation does this even if you are a good citizen and declare your bundle’s dependency on the newer version.

So in a stricter OSGi environment like CICS, you don’t boot delegate the javax.\* packages. If you don’t declare your javax.\* dependencies and your development environment doesn’t highlight the omissions, expect to be served with a NoClassDefFoundError. Do yourself a favour and declare all your package dependencies on the Import-Package statement in the MANIFEST.MF of your OSGi bundle.

As a general guideline, only the core java.\* packages and any packages internal to the bundle can be omitted from your Import-Package statement. The core java.\* packages are always delegated to the bootstrap class loader of the JRE, for obvious security reasons.

3. OSGi resolution

This section includes:

  • Definitions for class path and class loader
  • The problems around Class.forName() in OSGi
  • The Four Scopes of OSGi resolution

ClassNotFoundException

Let’s take a moment to review some essential definitions.

A class path is a list of files, typically JARs on the file-system where a class loader looks for classes. A class loader uses its class path to read the classes from the file-system and returns instances of classes to those parts of the application that require them.

As mentioned earlier, a ClassNotFoundException is most likely a result of your code actively loading a class using Class.forName(). To rub salt in the wound, it probably could have worked outside of OSGi. This can easily be explained because when everything is on one big class path, and presuming the necessary JARs exist on the class path, the class will be found.

So what is the problem with Class.forName() in OSGi?

Recall that OSGi is a component-based system that allows multiple versions of a package to exist within the same runtime. Each component is a bundle. Each bundle has its own class loader. Each bundle’s class loader loads only the classes contained in that bundle. By allowing different versions of a class to be loaded by different bundles, multiple versions of the same class can be available in the runtime. This is something that cannot easily be achieved using Java’s standard parent-first delegation model.

In contrast to the one big class path approach, OSGi has a peer network of class loaders. External packages required by your bundle are declared as dependencies (i.e. Import-Package) with a package/version combination. When the OSGi framework resolves your bundle, it determines the best fit and “wires” your bundle to those bundles that are providing the dependencies. Thus OSGi becomes a web of bundle class loaders that delegate to each other. If a class is not on your bundle’s class loader, then wiring is used to ask another bundle to load the class on your behalf and return a class instance.

Therein lies a problem. Class.forName() can be used to dynamically load arbitrary classes at runtime. An example of this might be applications that load the IBM Data Server Driver for JDBC and SQLJ. Although no longer a requirement for Db2, the technique is still prevalent. Tooling isn’t able to spot dependencies hidden in your source code in this way. So unless you know in advance what the dynamically loaded packages will be, and can specify those dependencies in your MANIFEST.MF and ensure that a provider bundle exports the relevant package, then the resolution process (wiring) will not occur. To put it simply, if the class you want to load is from a package outside of your OSGi bundle, and you are using “back-door” class loading techniques, then it’s up to you declare the dependency!

More often than not, failing to declare dependencies on classes that are dynamically loaded causes ClassNotFoundExceptions. If you want to understand more about dynamic class loading, how to workaround the problems it introduces, or (better still) how to avoid it entirely, sit tight for Part 3 of the OSGi demystified series which focuses on dynamic class loading.

Before we move on, I ought to mention that there are other, less common causes of a ClassNotFoundException. Taking one particular scenario: A dependency is specified, the package exists, and it is exported by another bundle. So what’s up? Well, it is feasible that the specific class from the declared package is not actually contained within the bundle. Granted, one might consider this a development error, but OSGi is effectively tricked into resolving your bundle against a package that is a banana short of a bunch. You don’t need me to tell you not to point that gun at your feet.

The magic wand of OSGi resolution

(Spoiler — there isn’t one.) There may not be a magic wand, but there is a substitute technique that can help you resolve class loading problems. I like to call it the Four Scopes of OSGi resolution, coined here for the first (and probably last!) time to help fix all those failures of the Class Not Found genus.

OSGi dependency resolution tiers

It’s a pretty simple concept. Progressively larger scopes are used to perform dependency resolution until all classes are found. A Liberty JVM server is a little more involved. Java EE applications are packaged into WAR, EJB, or EBA files which are loaded through isolated OSGi sub-systems, where additional class loading is provided through shared libraries — so for this article, let’s stick to the pure OSGi picture.

Your bundle is the first scope (shown with “1” on the diagram); anything located directly within your bundle can be loaded without further ado. If your bundle includes or wraps other JARs, just remember to put those JARs on the Bundle-ClassPath to get at their contents. If you want to make the packages from those JARs available to other bundles, you can do so with an Export-Package statement.

But before you do, hold up — there is a curveball. If a package within your bundle (or on your Bundle-ClassPath) is also explicitly imported by your bundle, the imported package will take precedence. That’s mildly perplexing — until someone explains to you that the exact same classes loaded from a different class loader are not considered the same to Java (read “actively incompatible”). So to reduce the likelihood of the same class being loaded by your bundle and by another bundle, an imported package takes precedence over your internal package. The result is that class loading is delegated to the same class loader (bundle) and everyone gets the same ball. Technically this is known as ensuring class space consistency.

The second (“2”) scope, as we’ve just discussed, expands to include all the other bundles that are installed into the OSGi framework. If a class isn’t in your bundle, maybe it can be found from another? For that, you need to declare an Import-Package dependency. Naturally, a bundle must also exist that provides that class and exposes the package with a corresponding Export-Package statement.

Still not found? Then the third (“3”) scope encompasses all bundles and the inner lining of the OSGi framework, more technically known as the system bundle or bundle zero. The system bundle provides all the dependencies that are defined on OSGi’s system packages parameter. To your bundle, it is just another bundle. Frameworks such as Equinox (used by CICS and Liberty) have a default list of packages that typically include javax.\* and org.w3c.\* packages. This is OSGi’s way of exposing classes from the external boot class hierarchy to bundles within the framework without resorting to the undesirable parent-first delegation that spoils your version party. CICS allows you to extend this list of packages, if necessary, using the following syntax in your JVM profile:

    # Extend the System Packages list to expose the following boot class-loader packages through the OSGi system bundle
    -Dorg.osgi.framework.system.packages.extra=com.ibm.xylem.types, com.ibm.xylem, com.ibm.xtq

The fourth (“4”) scope includes packages outside of the OSGi framework that aren’t exposed by the system bundle, but are exposed through boot delegation. Boot delegated packages should be considered internal to Java, like sun.\* and com.sun.\*. They are not considered suitable for replacement implementations. Boot delegation is the worm-hole that leads straight to the outermost scope with no stops for sightseeing and no chance to change your mind. Again, CICS allows you to specify your own list of boot-delegated packages, but you should only modify it as a last resort — there are very few situations where this is the right choice.

4. Boot compatibility

This section includes:

  • Closing words on boot compatibility and how it works within CICS TS V5.3+
  • The effects of switching boot compatibility off

Addendum — boot compatibility

From CICS TS V5.3, the OSGi runtime compensates for the lack of strict development tool dependency checking by performing a type of class loader delegation that’s affectionately referred to as boot compatibility. Boot compatibility instructs the Equinox framework to perform a last gasp delegation to the boot class loader. If the class is not found through the explicit dependency declarations (i.e. Import-Package statements), then and only then will delegation to the boot class loader happen. Consider it a “get out of jail for a small fee” option. The small fee is one of performance and of tolerance to improperly specified dependencies.

That said, if you have the freedom to make application changes, I recommend that you you turn off this mollycoddling behaviour by adding -Dosgi.compatibility.bootdelegation=false to the JVM profile of your JVM server:

    # Switch off OSGi boot compatibility so that it highlights when Import-Package dependencies have been omitted
    -Dosgi.compatibility.bootdelegation=false

What happens with boot compatibility off?

1) If nothing different happens, put your halo on. Either your code doesn’t import any of the boot delegated packages, or you already added those dependencies to the Import-Package statement in your MANIFEST.MF. Consider yourself optimised.

2) If your bundle(s) fail to resolve, that’s a good thing — really! Now you can explicitly add the dependencies (indicated by NoClassDefFoundErrors) to your bundle MANIFEST.MF until it once again resolves and starts. Sit back, feel smug. Class loading will be more efficient and your bundle’s interface will be well-defined and more portable. The CICS OSGi default settings are no longer required to protect your application.

Of course there is a quicker eyeballing method. If you can view the import statements in the Java source, go ahead and determine whether all the packages provided from outside of your bundle have a corresponding Import-Package statement in the MANIFEST.MF.

Final thoughts

While we’re still on the Import-Package soap-box (and please don’t ever use Require-bundle — it is a legacy header that ties you to a specific bundle implementation, sacrificing flexibility and package granularity), be sure to pay attention to the version information on your Import-Package statement:

  • Never omit the version qualifier, because you’ll get anything.
  • Never specify a single version, because it is interpreted by OSGi as a minimum, and when an incompatible interface change comes along you’ll walk right into it.
  • Always specify a version range up to, but not including, the next likely interface breakage.

If you aren’t already familiar with OSGi semantic versioning, please read about it to learn how it can be used to tolerate bug fixes (micro version changes) and API additions (minor version changes) while protecting you against major version changes (interface breakages).

A good example of how to give flexibility to your application is Import-Package: com.ibm.cics.server;version="[1.0.0,2.0.0)". A version range like this tolerates newer versions of JCICS, while providing a heads up if IBM makes an incompatible change. In this syntax, version 1.0.0 and upwards are included — denoted by the square bracket, while version 2.0.0 is excluded — denoted by a round bracket.

Summary

This article highlighted the differences between NoClassDefFoundError the ClassNotFoundException. It suggested probable causes of each failure condition, and provided the Four Scopes of OSGi resolution as a rule of thumb for fixing them. It provided guidance on how to achieve stricter OSGi compliance and take performance advantages by explicitly declaring all of your bundles dependencies. Finally, I strongly advised you to avoid dynamic class loading — there are better ways.

Now you can put your newfound knowledge to work by practicing on an existing OSGi failure. Or try differentiating the types of issues described here. Be sure to check out the other articles in the OSGi demystified series. Stay tuned for Part 3, which will show you how to avoid dynamic class loading.