Testing JASPIC 1.1 on IBM Liberty EE 7 beta

In this article we take a look at the latest April 2015 beta version of IBM's Liberty server, and specifically look at how well it implements the Java EE authentication standard JASPIC.

The initial version of Liberty implemented only a seemingly random assortment of Java EE APIs, but the second version that we looked at last year officially implemented the (Java EE 6) web profile. This year however the third incarnation is well on target to implement the full profile of Java EE 7.

This means IBM's newer and much lighter Liberty (abbreviated WLP), will be a true alternative for the older and incredibly obese WebSphere (abbreviated WAS) where it purely concerns the Java EE standard APIs. From having by far the most heavyweight server on the market (weighing in at well over 2GB), IBM can now offer a server that's as light and small as various offerings from its competition.

For this article we'll be specifically looking at how well JASPIC works on Liberty. Please take into account that the EE 7 version of Liberty is still a beta, so this only concerns an early look. Bugs and missing functionality are basically expected.

We started by downloading Liberty from the beta download page. The download page initially looked a little confusing, but it's constantly improving and by the time that this article was written it was already a lot clearer. Just like the GlassFish download page, IBM now offers a very straightforward Java EE Web profile download and a Java EE full profile one.

For old time WebSphere users who were used to installers that were themselves 200GB in size and only run on specific operating systems, and then happily downloaded 2GB of data that represented the actual server, it beggars belief that Liberty is now just an archive that you unzip. While the last release of Liberty already greatly improved matters by having an executable jar as download, effectively a self-extracting archive, nothing beats the ultimate simplicity of an "install" that solely consists of an archive that you unzip. This represents the pure zen of installing, shaving every non-essential component off it and leaving just the bare essentials. GlassFish has an unzip install, JBoss has it, TomEE and Tomcat has it, even the JDK has it these days, and now finally IBM has one too :)

We downloaded the Java EE 7 archive, wlp-beta-javaee7-2015.4.0.0.zip, weighing in at a very reasonable 100MB, which is about the same size as the latest beta of JBoss (WildFly 9.0 beta2). Like last year there is no required registration or anything. A license has to be accepted (just like e.g. the JDK), but that's it. The experience up to this point is as perfect as can be.

A small disappointment is that the download page lists a weird extra step that supposedly needs to be performed. It says something called a "server" needs to be created after the unzip, but luckily it appeared this is not the case. After unzipping Liberty can be started directly on OS X by pointing Eclipse to the directory where Liberty was extracted, or by typing the command "./server start" from the "./bin" directory where Liberty was extracted. Why this unnecessary step is listed is not clear. Hopefully it's just a remainder of some early alpha version. On Linux (we tried Ubuntu 14.10) there's an extra bug. The file permissions of the unzipped archive are wrong, and a "chmod +x ./bin/server" is needed to get Liberty to start using either Eclipse or the commandline.

(UPDATE: IBM responded right away by removing the redundant step mentioned by the download page)

A bigger disappointment is that the Java EE full profile archive is by default configured to only be a JSP/Servlet container. Java EE 7 has to be "activated" by manually editing a vendor specific XML file called "server.xml" and finding out that in its "featureManager" section one needs to type <feature>javaee-7.0</feature>. For some reason or the other this doesn't include JASPIC and JACC. Even though they really are part of Java EE (7), they have to be activated separately. In the case of JASPIC this means adding the following as well: <feature>jaspic-1.1</feature>. Hopefully these two issues are just packaging errors and will be resolved in the next beta or at least in the final version.

On to trying out JASPIC, we unfortunately learned that by default JASPIC doesn't really work as it should. Liberty inherited a spec compliance issue from WebSphere 8.x where the runtime insists that usernames and groups that an auth module wishes to set as the authenticated identity also exist in an IBM specific server internal identity store that IBM calls "user registry". This is however not the intend of JASPIC, and existing JASPIC modules will not take this somewhat strange requirement into account which means they will therefor not work on WebSphere and now Liberty. We'll be looking at a hack to work around this below.

Another issue is that Liberty still mandates so called group to role mapping, even when such mapping is not needed. Unlike some other servers that also mandate this by default there's currently no option to switch this requirement off, but there's an open issue for this in IBM's tracker. Another problem is that the group to role mapping file can only be supplied by the application when using an EAR archive. With lighter weight applications a war archive is often the initial choice, but when security is needed and you don't want or can't pollute the server itself with (meaningless) application specific data, then the current beta of Liberty forces the EAR archive upon you. Here too however there's already an issue filed to remedy this.

One way to work around the spec compliance issue mentioned above is by implementing a custom user registry that effectively does nothing. IBM has some documentation on how to do this, but unfortunately it's not giving exact instructions but merely outlines the process. The structure is also not entirely logical.

For instance, step 1 says "Implement the custom user registry (FileRegistrysample.java)". But in what kind of project? Where should the dependencies come from? Then step 2 says: "Creating an OSGi bundle with Bundle Activation. [...] Import the FileRegistrysample.java file". Why not create the bundle project right away and then create the mentioned file inside that bundle project? Step 4 says "Register the services", but gives no information on how to do this. Which services are we even talking about, and should they be put in an XML file or so and if so which one and what syntax? Step 3.4 asks to install the feature into Liberty using Eclipse (this works very nicely), but then step 4 and 5 are totally redundant, since they explain another more manually method to install the feature.

Even though it's outdated, IBM's general documentation on how to create a Liberty feature is much clearer. With those two articles side by side and cross checking it with the source code of the example used in the first article, I was able to build a working NOOP user registry. I had to Google for the example's source code though as the link in the article resulted in a 404. A good thing to realize is that the .esa file that's contained in the example .jar is also an archive that once unzipped contains the actual source code. Probably a trivial bit of knowledge for OSGi users, but myself being an OSGi n00b completely overlooked this and spent quite some time looking for the .java files.

The source code of the actual user registry is as follows:

package noopregistrybundle;

import static java.util.Collections.emptyList;

import java.rmi.RemoteException;
import java.security.cert.X509Certificate;
import java.util.ArrayList;
import java.util.List;
import java.util.Properties;

import javax.naming.InvalidNameException;
import javax.naming.ldap.LdapName;
import javax.naming.ldap.Rdn;

import com.ibm.websphere.security.CertificateMapFailedException;
import com.ibm.websphere.security.CertificateMapNotSupportedException;
import com.ibm.websphere.security.CustomRegistryException;
import com.ibm.websphere.security.EntryNotFoundException;
import com.ibm.websphere.security.NotImplementedException;
import com.ibm.websphere.security.PasswordCheckFailedException;
import com.ibm.websphere.security.Result;
import com.ibm.websphere.security.UserRegistry;
import com.ibm.websphere.security.cred.WSCredential;

public class NoopUserRegistry implements UserRegistry {

    @Override
    public void initialize(Properties props) throws CustomRegistryException, RemoteException {
    }

    @Override
    public String checkPassword(String userSecurityName, String password) throws PasswordCheckFailedException, CustomRegistryException, RemoteException {
        return userSecurityName;
    }

    @Override
    public String mapCertificate(X509Certificate[] certs) throws CertificateMapNotSupportedException, CertificateMapFailedException, CustomRegistryException, RemoteException {
        try {
            for (X509Certificate cert : certs) {
                for (Rdn rdn : new LdapName(cert.getSubjectX500Principal().getName()).getRdns()) {
                    if (rdn.getType().equalsIgnoreCase("CN")) {
                        return rdn.getValue().toString();
                    }
                }
            }
        } catch (InvalidNameException e) {
        }

        throw new CertificateMapFailedException("No valid CN in any certificate");
    }

    @Override
    public String getRealm() throws CustomRegistryException, RemoteException {
        return "customRealm"; // documentation says can be null, but should really be non-null!
    }

    @Override
    public Result getUsers(String pattern, int limit) throws CustomRegistryException, RemoteException {
        return emptyResult();
    }

    @Override
    public String getUserDisplayName(String userSecurityName) throws EntryNotFoundException, CustomRegistryException, RemoteException {
        return userSecurityName;
    }

    @Override
    public String getUniqueUserId(String userSecurityName) throws EntryNotFoundException, CustomRegistryException, RemoteException {
        return userSecurityName;
    }

    @Override
    public String getUserSecurityName(String uniqueUserId) throws EntryNotFoundException, CustomRegistryException, RemoteException {
        return uniqueUserId;
    }

    @Override
    public boolean isValidUser(String userSecurityName) throws CustomRegistryException, RemoteException {
        return true;
    }

    @Override
    public Result getGroups(String pattern, int limit) throws CustomRegistryException, RemoteException {
        return emptyResult();
    }

    @Override
    public String getGroupDisplayName(String groupSecurityName) throws EntryNotFoundException, CustomRegistryException, RemoteException {
        return groupSecurityName;
    }

    @Override
    public String getUniqueGroupId(String groupSecurityName) throws EntryNotFoundException, CustomRegistryException, RemoteException {
        return groupSecurityName;
    }

    @Override
    public List<String> getUniqueGroupIds(String uniqueUserId) throws EntryNotFoundException, CustomRegistryException, RemoteException {
        return new ArrayList<>(); // Apparently needs to be mutable
    }

    @Override
    public String getGroupSecurityName(String uniqueGroupId) throws EntryNotFoundException, CustomRegistryException, RemoteException {
        return uniqueGroupId;
    }

    @Override
    public boolean isValidGroup(String groupSecurityName) throws CustomRegistryException, RemoteException {
        return true;
    }

    @Override
    public List<String> getGroupsForUser(String groupSecurityName) throws EntryNotFoundException, CustomRegistryException, RemoteException {
        return emptyList();
    }

    @Override
    public Result getUsersForGroup(String paramString, int paramInt) throws NotImplementedException, EntryNotFoundException, CustomRegistryException, RemoteException {
        return emptyResult();
    }

    @Override
    public WSCredential createCredential(String userSecurityName) throws NotImplementedException, EntryNotFoundException, CustomRegistryException, RemoteException {
        return null;
    }
    
    private Result emptyResult() {
        Result result = new Result();
        result.setList(emptyList());
        return result;
    }
}

There were two small caveats here. The first is that the documentation for getRealm says it may return null and that "customRealm" will be used as the default then. But when you actually return null authentication will fail with many null pointer exceptions appearing in the log. The second is that getUniqueGroupIds() has to return a mutable collection. If Collections#emptyList is returned it will throw an exception that no element can be inserted. Likely IBM merges the list of groups this method returns with those that are being provided by the JASPIC auth module, and directly uses this collection for that merging.

The Activator class that's mentioned in the article referenced above looks as follows:

package noopregistrybundle;

import static org.osgi.framework.Constants.SERVICE_PID;

import java.util.Dictionary;
import java.util.Hashtable;

import org.osgi.framework.BundleActivator;
import org.osgi.framework.BundleContext;
import org.osgi.framework.ServiceRegistration;
import org.osgi.service.cm.ConfigurationException;
import org.osgi.service.cm.ManagedService;

import com.ibm.websphere.security.UserRegistry;

public class Activator extends NoopUserRegistry implements BundleActivator, ManagedService {

    private static final String CONFIG_PID = "noopUserRegistry";
    
    private ServiceRegistration<ManagedService> managedServiceRegistration;
    private ServiceRegistration<UserRegistry> userRegistryRegistration;

    @SuppressWarnings({ "rawtypes", "unchecked" })
    Hashtable getDefaults() {
        Hashtable defaults = new Hashtable();
        defaults.put(SERVICE_PID, CONFIG_PID);
        return defaults;
    }

    @SuppressWarnings("unchecked")
    public void start(BundleContext context) throws Exception {
        managedServiceRegistration = context.registerService(ManagedService.class, this, getDefaults());
        userRegistryRegistration = context.registerService(UserRegistry.class, this, getDefaults());
    }
    
    @Override
    public void updated(Dictionary<String, ?> properties) throws ConfigurationException {

    }

    public void stop(BundleContext context) throws Exception {
        if (managedServiceRegistration != null) {
            managedServiceRegistration.unregister();
            managedServiceRegistration = null;
        }
        if (userRegistryRegistration != null) {
            userRegistryRegistration.unregister();
            userRegistryRegistration = null;
        }
    }
}

Here we learned what that cryptic "Register the services" instruction from the article meant; it are the two calls to context.registerService here. Surely something that's easy to guess, or isn't it?

Finally a MANIFEST.FM file had to be created. The Eclipse tooling should normally help here, but it our case it worked badly. The "Analyze code and add dependencies to the MANIFEST.MF" command in the manifest editor (under the Dependencies tab) didn't work at all, and "org.osgi.service.cm" couldn't be chosen from the Imported Packages -> Add dialog. Since this import is actually used (and OSGi requires you to list each and every import used by your code) I added this manually. The completed file looks as follows:

Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: NoopRegistryBundle
Bundle-SymbolicName: NoopRegistryBundle
Bundle-Version: 1.0.0.qualifier
Bundle-Activator: noopregistrybundle.Activator
Import-Package: com.ibm.websphere.security;version="1.1.0",
 javax.naming,
 javax.naming.ldap,
 org.osgi.service.cm,
 org.osgi.framework
Bundle-RequiredExecutionEnvironment: JavaSE-1.7
Export-Package: noopregistrybundle

Creating yet another project for the so-called feature, importing this OSGi bundle there and installing the build feature into Liberty was all pretty straightforward when following the above mentioned articles.

The final step consisted of adding the noop user registry to Liberty's server.xml, which looked as follows:

<?xml version="1.0" encoding="UTF-8"?>
<server description="new server">

    <featureManager>
        <feature>javaee-7.0</feature>
        <feature>jaspic-1.1</feature>
        <feature>localConnector-1.0</feature>
        <feature>usr:NoopRegistryFeature</feature>
    </featureManager>

    <httpEndpoint httpPort="9080" httpsPort="9443" id="defaultHttpEndpoint"/>

    <noopUserRegistry/>
</server>

With this in place, JASPIC indeed worked on Liberty, which is absolutely great! To do some more thorough testing of how compatible Liberty exactly is we used the JASPIC tests that I contributed to the Java EE 7 samples project. These tests have been used by various other server vendors already and give a basic impression of what things work and do not work.

The tests had to be adjusted for Liberty because of its requirement to add an EAR wrapper that hosts the mandated group to role mapping.

After running the tests, the following failures were reported:

Test	Class	Comment
testPublicPageNotRememberLogin	org.javaee7.jaspic.basicauthentication.BasicAuthenticationPublicTest
testPublicPageLoggedin	org.javaee7.jaspic.basicauthentication.BasicAuthenticationPublicTest
testProtectedAccessIsStateless	org.javaee7.jaspic.basicauthentication.BasicAuthenticationStatelessTest
testPublicServletWithLoginCallingEJB	org.javaee7.jaspic.ejbpropagation.ProtectedEJBPropagationTest
testProtectedServletWithLoginCallingEJB	org.javaee7.jaspic.ejbpropagation.PublicEJBPropagationLogoutTest
testProtectedServletWithLoginCallingEJB	org.javaee7.jaspic.ejbpropagation.PublicEJBPropagationTest
testLogout	org.javaee7.jaspic.lifecycle.AuthModuleMethodInvocationTest	SAM method cleanSubject not called, but should have been
testJoinSessionIsOptional	org.javaee7.jaspic.registersession.RegisterSessionTest
testRemembersSession	org.javaee7.jaspic.registersession.RegisterSessionTest
testResponseWrapping	org.javaee7.jaspic.wrapping.WrappingTest	Response wrapped by SAM did not arrive in Servlet
testRequestWrapping	org.javaee7.jaspic.wrapping.WrappingTest	Request wrapped by SAM did not arrive in Servlet

Specifically the EJB, "logout calls cleanSubject" & register session (both new JASPIC 1.1 features) and request/response wrapper tests failed.

Two of those are new JASPIC 1.1 features and likely IBM just hasn't implemented those yet for the beta. Request/response wrapper failures is a known problem from JASPIC 1.0 times. Although most servers implement it now curiously not a single JASPIC implementation did so back in the Java EE 6 time frame (even though it was a required feature by the spec).

First Java EE 7 production ready server?

At the time of writing, which is 694 days (1 year, ~10 months) after the Java EE 7 spec was finalized, there are 3 certified Java EE servers but none of them is deemed by their vendor as "production ready". With the implementation cycle of Java EE 6 we saw that IBM was the first vendor to release a production ready server after 559 days (1 year, 6 months), with Oracle following suit at 721 days (1 year, 11 months).

Oracle (perhaps unfortunately) doesn't do public beta releases and is a little tight lipped about their up coming Java EE 7 WebLogic 12.2.1 release, but it's not difficult to guess that they are working hard on it (I have it on good authority that they indeed are). Meanwhile IBM has just released a beta that starts to look very complete. Looking at the amount of time it took both vendors last time around it might be a tight race between the two for releasing the first production ready Java EE 7 server. Although JBoss' WildFly 8.x is certified, a production ready and supported release is likely still at least a full year ahead when looking at the current state of the WildFly branch and if history is anything to go by (it took JBoss 923 days (2 years, 6 months) last time).

Conclusion

Despite a few bugs in the packaging of the full and web profile servers, IBM's latest beta shows incredible promise. The continued effort in making its application server yet again simpler to install for developers is nothing but applaudable. IBM clearly meant it when they started the Liberty project a few years ago and told their mission was to optimize the developer experience.

There are a few small bugs and one somewhat larger violation in its JASPIC implementation, but we have to realize it's just a beta. In fact, IBM engineers are already looking at the JASPIC issues.

To summarize the good and not so good points:

Good

• Runs on all operating systems (no special IBM JDK required)
• Monthly betas of EE 7 server
• Liberty to support Java EE 7 full profile
• Possibly on its way to become the first production ready EE 7 server
• Public download page without required registration
• Very good file size for full profile (100MB)
• Extremely easy "download - unzip - ./server start" experience

Not (yet) so good

• Download page lists totally unnecessary step asking to "create a server" (update: now fixed by IBM)
• Wrong file permissions in archive for usage on Linux; executable attribute missing on bin/server (update: now fixed by IBM)
• Wrong configuration of server.xml; both web and full profile by default configured as JSP/Servlet only
• "javaee-7.0" feature in server.xml doesn't imply JASPIC and JACC, while both are part of Java EE (update: now fixed by IBM)
• JASPIC runtime tries to validate usernames/groups in internal identity store (violation of JASPIC spec)
• Mandatory group to role mapping, even when this is not needed
• Mandatory usage of EAR archive when group to role mapping has to be provided by the application
• Not all JASPIC features implemented yet (but remember that we looked at a beta version)

Arjan Tijms