Wednesday, April 22, 2015

Testing JASPIC 1.1 on IBM Liberty EE 7 beta

In this article we take a look at the latest April 2015 beta version of IBM's Liberty server, and specifically look at how well it implements the Java EE authentication standard JASPIC.

The initial version of Liberty implemented only a seemingly random assortment of Java EE APIs, but the second version that we looked at last year officially implemented the (Java EE 6) web profile. This year however the third incarnation is well on target to implement the full profile of Java EE 7.

This means IBM's newer and much lighter Liberty (abbreviated WLP), will be a true alternative for the older and incredibly obese WebSphere (abbreviated WAS) where it purely concerns the Java EE standard APIs. From having by far the most heavyweight server on the market (weighing in at well over 2GB), IBM can now offer a server that's as light and small as various offerings from its competition.

For this article we'll be specifically looking at how well JASPIC works on Liberty. Please take into account that the EE 7 version of Liberty is still a beta, so this only concerns an early look. Bugs and missing functionality are basically expected.

We started by downloading Liberty from the beta download page. The download page initially looked a little confusing, but it's constantly improving and by the time that this article was written it was already a lot clearer. Just like the GlassFish download page, IBM now offers a very straightforward Java EE Web profile download and a Java EE full profile one.

For old time WebSphere users who were used to installers that were themselves 200GB in size and only run on specific operating systems, and then happily downloaded 2GB of data that represented the actual server, it beggars belief that Liberty is now just an archive that you unzip. While the last release of Liberty already greatly improved matters by having an executable jar as download, effectively a self-extracting archive, nothing beats the ultimate simplicity of an "install" that solely consists of an archive that you unzip. This represents the pure zen of installing, shaving every non-essential component off it and leaving just the bare essentials. GlassFish has an unzip install, JBoss has it, TomEE and Tomcat has it, even the JDK has it these days, and now finally IBM has one too :)

We downloaded the Java EE 7 archive, wlp-beta-javaee7-2015.4.0.0.zip, weighing in at a very reasonable 100MB, which is about the same size as the latest beta of JBoss (WildFly 9.0 beta2). Like last year there is no required registration or anything. A license has to be accepted (just like e.g. the JDK), but that's it. The experience up to this point is as perfect as can be.

A small disappointment is that the download page lists a weird extra step that supposedly needs to be performed. It says something called a "server" needs to be created after the unzip, but luckily it appeared this is not the case. After unzipping Liberty can be started directly on OS X by pointing Eclipse to the directory where Liberty was extracted, or by typing the command "./server start" from the "./bin" directory where Liberty was extracted. Why this unnecessary step is listed is not clear. Hopefully it's just a remainder of some early alpha version. On Linux (we tried Ubuntu 14.10) there's an extra bug. The file permissions of the unzipped archive are wrong, and a "chmod +x ./bin/server" is needed to get Liberty to start using either Eclipse or the commandline.

(UPDATE: IBM responded right away by removing the redundant step mentioned by the download page)

A bigger disappointment is that the Java EE full profile archive is by default configured to only be a JSP/Servlet container. Java EE 7 has to be "activated" by manually editing a vendor specific XML file called "server.xml" and finding out that in its "featureManager" section one needs to type <feature>javaee-7.0</feature>. For some reason or the other this doesn't include JASPIC and JACC. Even though they really are part of Java EE (7), they have to be activated separately. In the case of JASPIC this means adding the following as well: <feature>jaspic-1.1</feature>. Hopefully these two issues are just packaging errors and will be resolved in the next beta or at least in the final version.

On to trying out JASPIC, we unfortunately learned that by default JASPIC doesn't really work as it should. Liberty inherited a spec compliance issue from WebSphere 8.x where the runtime insists that usernames and groups that an auth module wishes to set as the authenticated identity also exist in an IBM specific server internal identity store that IBM calls "user registry". This is however not the intend of JASPIC, and existing JASPIC modules will not take this somewhat strange requirement into account which means they will therefor not work on WebSphere and now Liberty. We'll be looking at a hack to work around this below.

Another issue is that Liberty still mandates so called group to role mapping, even when such mapping is not needed. Unlike some other servers that also mandate this by default there's currently no option to switch this requirement off, but there's an open issue for this in IBM's tracker. Another problem is that the group to role mapping file can only be supplied by the application when using an EAR archive. With lighter weight applications a war archive is often the initial choice, but when security is needed and you don't want or can't pollute the server itself with (meaningless) application specific data, then the current beta of Liberty forces the EAR archive upon you. Here too however there's already an issue filed to remedy this.

One way to work around the spec compliance issue mentioned above is by implementing a custom user registry that effectively does nothing. IBM has some documentation on how to do this, but unfortunately it's not giving exact instructions but merely outlines the process. The structure is also not entirely logical.

For instance, step 1 says "Implement the custom user registry (FileRegistrysample.java)". But in what kind of project? Where should the dependencies come from? Then step 2 says: "Creating an OSGi bundle with Bundle Activation. [...] Import the FileRegistrysample.java file". Why not create the bundle project right away and then create the mentioned file inside that bundle project? Step 4 says "Register the services", but gives no information on how to do this. Which services are we even talking about, and should they be put in an XML file or so and if so which one and what syntax? Step 3.4 asks to install the feature into Liberty using Eclipse (this works very nicely), but then step 4 and 5 are totally redundant, since they explain another more manually method to install the feature.

Even though it's outdated, IBM's general documentation on how to create a Liberty feature is much clearer. With those two articles side by side and cross checking it with the source code of the example used in the first article, I was able to build a working NOOP user registry. I had to Google for the example's source code though as the link in the article resulted in a 404. A good thing to realize is that the .esa file that's contained in the example .jar is also an archive that once unzipped contains the actual source code. Probably a trivial bit of knowledge for OSGi users, but myself being an OSGi n00b completely overlooked this and spent quite some time looking for the .java files.

The source code of the actual user registry is as follows:

package noopregistrybundle;

import static java.util.Collections.emptyList;

import java.rmi.RemoteException;
import java.security.cert.X509Certificate;
import java.util.ArrayList;
import java.util.List;
import java.util.Properties;

import javax.naming.InvalidNameException;
import javax.naming.ldap.LdapName;
import javax.naming.ldap.Rdn;

import com.ibm.websphere.security.CertificateMapFailedException;
import com.ibm.websphere.security.CertificateMapNotSupportedException;
import com.ibm.websphere.security.CustomRegistryException;
import com.ibm.websphere.security.EntryNotFoundException;
import com.ibm.websphere.security.NotImplementedException;
import com.ibm.websphere.security.PasswordCheckFailedException;
import com.ibm.websphere.security.Result;
import com.ibm.websphere.security.UserRegistry;
import com.ibm.websphere.security.cred.WSCredential;

public class NoopUserRegistry implements UserRegistry {

    @Override
    public void initialize(Properties props) throws CustomRegistryException, RemoteException {
    }

    @Override
    public String checkPassword(String userSecurityName, String password) throws PasswordCheckFailedException, CustomRegistryException, RemoteException {
        return userSecurityName;
    }

    @Override
    public String mapCertificate(X509Certificate[] certs) throws CertificateMapNotSupportedException, CertificateMapFailedException, CustomRegistryException, RemoteException {
        try {
            for (X509Certificate cert : certs) {
                for (Rdn rdn : new LdapName(cert.getSubjectX500Principal().getName()).getRdns()) {
                    if (rdn.getType().equalsIgnoreCase("CN")) {
                        return rdn.getValue().toString();
                    }
                }
            }
        } catch (InvalidNameException e) {
        }

        throw new CertificateMapFailedException("No valid CN in any certificate");
    }

    @Override
    public String getRealm() throws CustomRegistryException, RemoteException {
        return "customRealm"; // documentation says can be null, but should really be non-null!
    }

    @Override
    public Result getUsers(String pattern, int limit) throws CustomRegistryException, RemoteException {
        return emptyResult();
    }

    @Override
    public String getUserDisplayName(String userSecurityName) throws EntryNotFoundException, CustomRegistryException, RemoteException {
        return userSecurityName;
    }

    @Override
    public String getUniqueUserId(String userSecurityName) throws EntryNotFoundException, CustomRegistryException, RemoteException {
        return userSecurityName;
    }

    @Override
    public String getUserSecurityName(String uniqueUserId) throws EntryNotFoundException, CustomRegistryException, RemoteException {
        return uniqueUserId;
    }

    @Override
    public boolean isValidUser(String userSecurityName) throws CustomRegistryException, RemoteException {
        return true;
    }

    @Override
    public Result getGroups(String pattern, int limit) throws CustomRegistryException, RemoteException {
        return emptyResult();
    }

    @Override
    public String getGroupDisplayName(String groupSecurityName) throws EntryNotFoundException, CustomRegistryException, RemoteException {
        return groupSecurityName;
    }

    @Override
    public String getUniqueGroupId(String groupSecurityName) throws EntryNotFoundException, CustomRegistryException, RemoteException {
        return groupSecurityName;
    }

    @Override
    public List<String> getUniqueGroupIds(String uniqueUserId) throws EntryNotFoundException, CustomRegistryException, RemoteException {
        return new ArrayList<>(); // Apparently needs to be mutable
    }

    @Override
    public String getGroupSecurityName(String uniqueGroupId) throws EntryNotFoundException, CustomRegistryException, RemoteException {
        return uniqueGroupId;
    }

    @Override
    public boolean isValidGroup(String groupSecurityName) throws CustomRegistryException, RemoteException {
        return true;
    }

    @Override
    public List<String> getGroupsForUser(String groupSecurityName) throws EntryNotFoundException, CustomRegistryException, RemoteException {
        return emptyList();
    }

    @Override
    public Result getUsersForGroup(String paramString, int paramInt) throws NotImplementedException, EntryNotFoundException, CustomRegistryException, RemoteException {
        return emptyResult();
    }

    @Override
    public WSCredential createCredential(String userSecurityName) throws NotImplementedException, EntryNotFoundException, CustomRegistryException, RemoteException {
        return null;
    }
    
    private Result emptyResult() {
        Result result = new Result();
        result.setList(emptyList());
        return result;
    }
}

There were two small caveats here. The first is that the documentation for getRealm says it may return null and that "customRealm" will be used as the default then. But when you actually return null authentication will fail with many null pointer exceptions appearing in the log. The second is that getUniqueGroupIds() has to return a mutable collection. If Collections#emptyList is returned it will throw an exception that no element can be inserted. Likely IBM merges the list of groups this method returns with those that are being provided by the JASPIC auth module, and directly uses this collection for that merging.

The Activator class that's mentioned in the article referenced above looks as follows:

package noopregistrybundle;

import static org.osgi.framework.Constants.SERVICE_PID;

import java.util.Dictionary;
import java.util.Hashtable;

import org.osgi.framework.BundleActivator;
import org.osgi.framework.BundleContext;
import org.osgi.framework.ServiceRegistration;
import org.osgi.service.cm.ConfigurationException;
import org.osgi.service.cm.ManagedService;

import com.ibm.websphere.security.UserRegistry;

public class Activator extends NoopUserRegistry implements BundleActivator, ManagedService {

    private static final String CONFIG_PID = "noopUserRegistry";
    
    private ServiceRegistration<ManagedService> managedServiceRegistration;
    private ServiceRegistration<UserRegistry> userRegistryRegistration;

    @SuppressWarnings({ "rawtypes", "unchecked" })
    Hashtable getDefaults() {
        Hashtable defaults = new Hashtable();
        defaults.put(SERVICE_PID, CONFIG_PID);
        return defaults;
    }

    @SuppressWarnings("unchecked")
    public void start(BundleContext context) throws Exception {
        managedServiceRegistration = context.registerService(ManagedService.class, this, getDefaults());
        userRegistryRegistration = context.registerService(UserRegistry.class, this, getDefaults());
    }
    
    @Override
    public void updated(Dictionary<String, ?> properties) throws ConfigurationException {

    }

    public void stop(BundleContext context) throws Exception {
        if (managedServiceRegistration != null) {
            managedServiceRegistration.unregister();
            managedServiceRegistration = null;
        }
        if (userRegistryRegistration != null) {
            userRegistryRegistration.unregister();
            userRegistryRegistration = null;
        }
    }
}

Here we learned what that cryptic "Register the services" instruction from the article meant; it are the two calls to context.registerService here. Surely something that's easy to guess, or isn't it?

Finally a MANIFEST.FM file had to be created. The Eclipse tooling should normally help here, but it our case it worked badly. The "Analyze code and add dependencies to the MANIFEST.MF" command in the manifest editor (under the Dependencies tab) didn't work at all, and "org.osgi.service.cm" couldn't be chosen from the Imported Packages -> Add dialog. Since this import is actually used (and OSGi requires you to list each and every import used by your code) I added this manually. The completed file looks as follows:

Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: NoopRegistryBundle
Bundle-SymbolicName: NoopRegistryBundle
Bundle-Version: 1.0.0.qualifier
Bundle-Activator: noopregistrybundle.Activator
Import-Package: com.ibm.websphere.security;version="1.1.0",
 javax.naming,
 javax.naming.ldap,
 org.osgi.service.cm,
 org.osgi.framework
Bundle-RequiredExecutionEnvironment: JavaSE-1.7
Export-Package: noopregistrybundle

Creating yet another project for the so-called feature, importing this OSGi bundle there and installing the build feature into Liberty was all pretty straightforward when following the above mentioned articles.

The final step consisted of adding the noop user registry to Liberty's server.xml, which looked as follows:

<?xml version="1.0" encoding="UTF-8"?>
<server description="new server">

    <featureManager>
        <feature>javaee-7.0</feature>
        <feature>jaspic-1.1</feature>
        <feature>localConnector-1.0</feature>
        <feature>usr:NoopRegistryFeature</feature>
    </featureManager>

    <httpEndpoint httpPort="9080" httpsPort="9443" id="defaultHttpEndpoint"/>

    <noopUserRegistry/>
</server>

With this in place, JASPIC indeed worked on Liberty, which is absolutely great! To do some more thorough testing of how compatible Liberty exactly is we used the JASPIC tests that I contributed to the Java EE 7 samples project. These tests have been used by various other server vendors already and give a basic impression of what things work and do not work.

The tests had to be adjusted for Liberty because of its requirement to add an EAR wrapper that hosts the mandated group to role mapping.

After running the tests, the following failures were reported:

Test Class Comment
testPublicPageNotRememberLogin org.javaee7.jaspic.basicauthentication.BasicAuthenticationPublicTest
testPublicPageLoggedin org.javaee7.jaspic.basicauthentication.BasicAuthenticationPublicTest
testProtectedAccessIsStateless org.javaee7.jaspic.basicauthentication.BasicAuthenticationStatelessTest
testPublicServletWithLoginCallingEJB org.javaee7.jaspic.ejbpropagation.ProtectedEJBPropagationTest
testProtectedServletWithLoginCallingEJB org.javaee7.jaspic.ejbpropagation.PublicEJBPropagationLogoutTest
testProtectedServletWithLoginCallingEJB org.javaee7.jaspic.ejbpropagation.PublicEJBPropagationTest
testLogout org.javaee7.jaspic.lifecycle.AuthModuleMethodInvocationTest SAM method cleanSubject not called, but should have been
testJoinSessionIsOptional org.javaee7.jaspic.registersession.RegisterSessionTest
testRemembersSession org.javaee7.jaspic.registersession.RegisterSessionTest
testResponseWrapping org.javaee7.jaspic.wrapping.WrappingTest Response wrapped by SAM did not arrive in Servlet
testRequestWrapping org.javaee7.jaspic.wrapping.WrappingTest Request wrapped by SAM did not arrive in Servlet

Specifically the EJB, "logout calls cleanSubject" & register session (both new JASPIC 1.1 features) and request/response wrapper tests failed.

Two of those are new JASPIC 1.1 features and likely IBM just hasn't implemented those yet for the beta. Request/response wrapper failures is a known problem from JASPIC 1.0 times. Although most servers implement it now curiously not a single JASPIC implementation did so back in the Java EE 6 time frame (even though it was a required feature by the spec).

First Java EE 7 production ready server?

At the time of writing, which is 694 days (1 year, ~10 months) after the Java EE 7 spec was finalized, there are 3 certified Java EE servers but none of them is deemed by their vendor as "production ready". With the implementation cycle of Java EE 6 we saw that IBM was the first vendor to release a production ready server after 559 days (1 year, 6 months), with Oracle following suit at 721 days (1 year, 11 months).

Oracle (perhaps unfortunately) doesn't do public beta releases and is a little tight lipped about their up coming Java EE 7 WebLogic 12.2.1 release, but it's not difficult to guess that they are working hard on it (I have it on good authority that they indeed are). Meanwhile IBM has just released a beta that starts to look very complete. Looking at the amount of time it took both vendors last time around it might be a tight race between the two for releasing the first production ready Java EE 7 server. Although JBoss' WildFly 8.x is certified, a production ready and supported release is likely still at least a full year ahead when looking at the current state of the WildFly branch and if history is anything to go by (it took JBoss 923 days (2 years, 6 months) last time).

Conclusion

Despite a few bugs in the packaging of the full and web profile servers, IBM's latest beta shows incredible promise. The continued effort in making its application server yet again simpler to install for developers is nothing but applaudable. IBM clearly meant it when they started the Liberty project a few years ago and told their mission was to optimize the developer experience.

There are a few small bugs and one somewhat larger violation in its JASPIC implementation, but we have to realize it's just a beta. In fact, IBM engineers are already looking at the JASPIC issues.

To summarize the good and not so good points:

Good

  • Runs on all operating systems (no special IBM JDK required)
  • Monthly betas of EE 7 server
  • Liberty to support Java EE 7 full profile
  • Possibly on its way to become the first production ready EE 7 server
  • Public download page without required registration
  • Very good file size for full profile (100MB)
  • Extremely easy "download - unzip - ./server start" experience

Not (yet) so good

  • Download page lists totally unnecessary step asking to "create a server" (update: now fixed by IBM)
  • Wrong file permissions in archive for usage on Linux; executable attribute missing on bin/server (update: now fixed by IBM)
  • Wrong configuration of server.xml; both web and full profile by default configured as JSP/Servlet only
  • "javaee-7.0" feature in server.xml doesn't imply JASPIC and JACC, while both are part of Java EE (update: now fixed by IBM)
  • JASPIC runtime tries to validate usernames/groups in internal identity store (violation of JASPIC spec)
  • Mandatory group to role mapping, even when this is not needed
  • Mandatory usage of EAR archive when group to role mapping has to be provided by the application
  • Not all JASPIC features implemented yet (but remember that we looked at a beta version)

Arjan Tijms

45 comments:

  1. You may want to consider replacing Require-Bundle with Import-Package. That way you're not dependent on how IBM chooses to package and name their bundles.

    Also if Liberty Profile supports Dynamic Services you can replace the activator with a service.xml.

    ReplyDelete
    Replies
    1. Thanks for the suggestions, maybe something to look at indeed ;)

      Delete
  2. I have updated our download pages to remove the step for creating a server. For the simplest out of the box experience you are right that server start works and that is what the download page should include.

    If starting a server with a specified name you do need to call create first, we didn't want a typo in the server name to end up creating and starting new things with no error returned.

    ReplyDelete
    Replies
    1. >I have updated our download pages to remove the step for creating a server. For the simplest out of the box experience you are right that server start works and that is what the download page should include.

      Wow, that's an amazingly fast response! I'll update the article right away.

      >If starting a server with a specified name you do need to call create first, we didn't want a typo in the server name to end up creating and starting new things with no error returned.

      That sounds like the optimal solution indeed. One default server for the ultimate out of the box experience, and named additional servers that need to be created first using that exact name.

      Delete
  3. The scripts should be executable in the latest beta. Thanks for spotting that.

    ReplyDelete
  4. Minor update

    "javaee-7.0" feature in server.xml doesn't imply JASPIC and JACC, while both are part of Java EE is no longer the case in 8.5.5.6 it at the very least javaee-7.0 implies jaspic-1.1

    ReplyDelete
    Replies
    1. Actually it also includes jacc-1.5

      Delete
    2. Yes, I noticed that got fixed. Still have it on my TODO to retest 8.5.5.6 and the latest 9.0-betas.

      Delete
  5. Do you need to create a wrapper for the bindings? I would think it can be added through the server.xml file as given by the following example:







    ReplyDelete
    Replies
    1. Unfortunately the example seemed to have been filtered out by the blog software, but it's true you can specify group to role mapping there (which for some reason IBM indeed calls "bindings").

      The problem is that server.xml is stored on, well, the server and cannot be bundled with a war. Modifying an installed runtime can be extremely troublesome for some environments.

      In case of the Java EE 7 samples project, it's really inconvenient. The thing is that it runs on many application servers, and those servers often get re-installed. Almost every time that happens the customisations are forgotten and therefor all tests fail.

      Additionally, the mapping (if any would be used) is owned by the tests, not by the server so specifying them at the server level would make very little sense for this specific scenario.

      Delete
    2. I think it's because WebSphere was one of the few application servers that have a "deployment" process. To be honest I haven't seen any other one among Wildfly/Glassfish that do this.

      As part of WAS Classic deployment you have to configure the bindings at deploy time, you don't change the EAR file you get directly instead once you deploy the EAR from the GUI the application server will ask you how to bind things (e.g. which data source, which port, which role group etc) maps to the unbound values.

      In addition this can be scripted using their proprietary WSADMIN scripting tool for CI deployments. Although the lazy approach which I am generally guilty of is to add the bindings files to the package by some batch file or ant script prior to deployment.

      Personally I prefer this for enterprise environments, but woefully annoying for development. What I am happy about is that Liberty makes it fast and easy on Eclipse without ruining the "purity" of the EAR file. For the two other app servers (WildFly and Glassfish) I had to put in their custom "bindings" in the EAR file because they don't have a proper deployer.

      Delete
  6. I think "Mandatory group to role mapping, even when this is not needed" is actually a good thing, in a way user and groups are a deployment artifact rather than an application artifact. Glassfish by default does not do the mapping either you need to enable it via the configuration.

    However, it would be nice to have a simple mapper provided as a feature for CI.

    ReplyDelete
    Replies
    1. >in a way user and groups are a deployment artifact rather than an application artifact

      I don't think this is the case in general.

      There are two main usage scenarios.

      One is where applications have to integrate with an existing operational environment. E.g. in an intranet you have office users who use totally different software from possibly different vendors (say JIRA, Jenkins, and Sonar), and they have to log in to these apps with centrally administrated accounts. Here group mapping is a deployment artefact.

      The other is self-contained applications that do not integrate with an existing environment. These are often public facing Internet applications build and operated by the same party. Almost always there's only one unique installation of that software. For instance in zeef.com external people ("the public") registers with that site. There's no concept of any centrally stored groups. I.e. it's not that because someone is "admin" on Amazon.com (a website we don't own, nor operate in any way), that this person can be automatically "administrator" on zeef.com.

      Group to role mapping really only makes sense for people assigned groups by a single entity (e.g. since they are employed by that entity) and accessing software operated by that entity. It does not make sense for a general public accessing a single unique software product.

      >Glassfish by default does not do the mapping either you need to enable it via the configuration.

      I think that in GlassFish group to role mapping is still mandatory as well. A stock GlassFish mandates a sun-web.xml or glassfish-web.xml. This used to be true for WebLogic as well, but since 12.1.3 this is no longer the case it seems.

      > it would be nice to have a simple mapper provided as a feature for CI.

      This is something I really want to investigate in the security EG. Hope we'll be able to come up with something satisfactory here.

      Delete
    2. That's true, I tend to work with enterprises where there are a multitude of applications. As such an automatic group-to-role mapping may actually be more of a security risk if say a user has been put into an "administrator" group for OS management but shouldn't have gotten the "administrator" role in the application letting them see the data.

      Mind you a simple standard change to allow in would be able to do away with having a vendor specific binding. We already allow for (IMO ugly) things like that in web.xml such as and that sets the values for the resources and servlets.

      WebSphere at least makes it so that these can be changed at deploy time via the GUI or bindings, but Glassfish and Wildfly do not. If it was even possible you need to put in app server specific XML files manually to support this.

      Delete
    3. >As such an automatic group-to-role mapping may actually be more of a security risk if say a user has been put into an "administrator" group for OS management but shouldn't have gotten the "administrator" role in the application letting them see the data.

      This is an important tradeoff. Requiring a user to configure things that are not needed nor a risk for a given usage scenario (i.e. the standalone app with public users) makes the platform feel heavyweight and in this case biased towards enterprises.

      On the other hand, a whole class of security leaks originates from defaults. So we have to choose wisely here. Perhaps all that's needed is a simple setting in web.xml or a property when registering a SAM that says whether automatic group to role mapping should or should not take place.

      >WebSphere at least makes it so that these can be changed at deploy time via the GUI or bindings, but Glassfish and Wildfly do not.

      I'm not entirely sure, but I think GlassFish has a GUI option to that can be used instead of glassfish-web.xml. Overriding application defaults without prying open an ear or war was an important goal of the Java EE Configuration JSR, but unfortunately that one was more or less aborted.

      IMHO there are a few important options here:

      1. Fully self-contained app, all default settings and resources inside app
      2. Fully self-contained app with multiple sets of config settings/resources (choose between them when deploying)
      3. App with unresolved/unbounded dependencies that need to be resolved/bound at deployment time.

      The problem I've seen is that there are different factions who seem to think that only one of these options is the One-and-Only way. I personally think each of these has there place, so Java EE should support them all three.

      Delete
  7. I kind of wish there was a feature I can just get called "NoopUserRegistry" rather than building it myself.

    ReplyDelete
    Replies
    1. IBM mentioned they may provide some solution to this problem. Best thing would be that as soon as a JASPIC module is registered, Liberty totally refrains from calling a UserRegistry in all situations. I very briefly looked at their latest release and I think they may have lessened the dependency, but it seems to be still (have to test this to be sure). JASPIC can only function the way it's intended to function if it's 100% in control and if the Liberty runtime never ever requires a UserRegistry for any of the standardised functionality.

      For the mean time an option may be to ask them to bundle a "NoopUserRegistry" such as provided in this article. You'd still have to configure that, but it would be a step up from the current situation (or at least the one I tested in this article).

      For Java EE 8 I hope to that the JASPIC TCK can be updated to take this specific problem into account.

      Delete
    2. It still requires the userregistry on their current GA

      Delete
    3. Mind you if I do get this NoopUserRegistry working I think Liberty would be my App Server of choice for Java EE 7. Then again I am already a bit biased as I like WAS 8 for Java EE 6 for enterprise deployments. But you'd have to twist my arm a lot if you expect me to say I like WAS classic for any development work.

      Delete
    4. >It still requires the userregistry on their current GA

      Yes, I kinda got that from glancing over a Reddit discussion about this. Still have to try it out, but as far as I understood you need to do a "special move/konami code" kind of thing to de-activate the authentication dependency on the user registry, but then the authorization dependency on the userregistry is still there.

      Being authenticated but not authorized still doesn't give you access. Technically they may have made a huge step, which is absolutely great. However, practically it doesn't seem to mean much since at the end of the day you still need the NoopUserRegistry then.

      Delete
  8. I am trying to build the NoopUserRegistry still having some issues on how to bundle and deploy it. Though I am thinking you can do away with the activator if you had OSGI-INF/blueprint/noop-user-registry.xml like this








    https://github.com/trajano/noop-user-registry/blob/master/src/main/resources/OSGI-INF/blueprint/noop-user-registry.xml

    I'm pretty sure I am just missing some little weird file to get this all working.

    ReplyDelete
    Replies
    1. As I mentioned in the article, it was a bit of a hassle initially to get the NoopUserRegistry to build. Instructions weren't entirely clear, and the IBM Eclipse OSGi beta tooling had some weird bugs as mentioned as well.

      But following both the referenced articles and using the Eclipse tooling to install the feature into Liberty eventually did the trick. The article listed a manual method to install (deploy) the feature to Liberty as well, but as Eclipse worked I didn't try that.

      Delete
    2. Yup trying to grok my head around it. Even their links in there do not work
      https://developer.ibm.com/wasdev/downloads/productsample_customuserregistrysample/

      As you say they have 404s

      Delete
    3. Indeed, that's a shame. The topic is complex enough already, and an article that just hints at things and doesn't have valid links doesn't make it easier. Did you see the link I found via Google in the article? The example code that gives a 404 from the article can be found here: https://developer.ibm.com/wasdev/downloads/#asset/samples-Custom_User_Registry

      Delete
    4. Yup, though their deployment instructions are not working for it. It says it needs "ejblite-3.1" feature installed. Anyway I'll look at it when I have some more free time.

      Delete
    5. Anyway I was able to create a maven project that does the NoopUserRegistry which deploys to the proper spots. So that's one good news, bad news is the JASPIC test case I had won't work on WebSphere :(

      It does not handle SEND_SUCCESS correctly from what I found, the resource needs to exist, it does support SEND_CONTINUE correctly though. In my case I had a "special" endpoint that will send data directly from the module rather than some resource in the application, but I get a 404 error.

      When JASPIC is being used the automatic redirect from non-secure to secure via transport-guarantee does not kick in anymore.

      If I remember correctly, the mapping does work correctly at least so I can map GroupCallback with customGroup to customRole correctly.

      Seems that only GlassFish had worked correctly for my module.

      Delete
    6. >It does not handle SEND_SUCCESS correctly from what I found, the resource needs to exist, it does support SEND_CONTINUE correctly though.

      If you can distill your example to a minimal case and post it to the JASPIC mailing list, then Ron Monzillo (the JASPIC spec lead) may be able to clarify this case.

      From what I understood though, in the servlet container profile SEND_SUCCESS cannot be returned from an invocation of validateRequest prior to the resource being invoked. SEND_SUCESS is primarily intended for secureResponse there.

      >In my case I had a "special" endpoint that will send data directly from the module rather than some resource in the application,

      This should be supported, since that's exactly what Servlet's build-in j_security_check is. Strongly related to this, a SAM *must* be able to use its request object to forward to a resource and afterwards it *cannot* continue to invoke any resource (as per Servlet's rules regarding forwarding).

      This is effectively demonstrated here: https://github.com/javaee-samples/javaee7-samples/blob/master/jaspic/dispatching/src/main/java/org/javaee7/jaspic/dispatching/sam/TestServerAuthModule.java#L59

      A "virtual resource" is now easily created by just putting an if statement in a SAM such as shown above that checks the request URI (I suppose that's what you did as well). Of course forwarding is one way of generating a response "from a SAM". You could indeed write to the response directly.

      Note that the example linked above returns "SEND_CONTINUE" from validateRequest();

      Delete
    7. >> In my case I had a "special" endpoint that will send data directly from the module rather than some resource in the application,

      > This should be supported, since that's exactly what Servlet's build-in j_security_check is. Strongly related to this, a SAM *must* be able to use its request object to forward to a resource and afterwards it *cannot* continue to invoke any resource (as per Servlet's rules regarding forwarding).

      I agree SEND_SUCCESS should be supported. Glassfish does support it as expected. Just WebSphere Liberty didn't support it from what I could gather from debugging. Unless you're claiming that your example actually worked correctly on WebSphere.

      ===

      One correction. Looking at my code again after work today I just checked the server configuration, I was incorrect in saying that group to role mapping worked on WebSphere, it didn't actually work. I had to use a "Special" group called "All Authenticated" and map it to the group I wanted. Mind you for my use case it is actually fine, but it is still bad that I couldn't do the mapping like I could on Glassfish.

      ===

      All in all, WebSphere is still much closer to working the way I would expect more than WildFly when it comes to JASPIC. However, it is a pain to debug because it's not open source and you cannot just trace through something that they had provided easily.

      Delete
    8. Anyway I found out how to tweak the NoopUserRegistry to get the mapping to work properly now. When the "isValidGroup" gets called, I store the group into a set that I pass back when there is a call to getUniqueGroupIds

      I have pushed the code to https://github.com/trajano/noop-user-registry

      One other change I did was use Declarative Services rather than a bundle activator just to reduce some boiler plate code.

      I think it would be a good blog topic on how to create an ESA with Maven for deployment to WAS Liberty when I get the chance.

      Delete
    9. >I agree SEND_SUCCESS should be supported. Glassfish does support it as expected.

      Sorry, I meant that the concept of a "virtual resource" should be supported. I.e. that a SAM listens to a special path (e.g. "/blabla") that does not really exists (no Servlet or JSP mapped to that name). And that a SAM can write to the response directly from its validateRequest method any Servlet needing to be invoked.

      However, in that case the SAM returns "SEND_CONTINUE", not "SEND_SUCCESS".

      But I'm not really sure what the problem is here.

      What does "SEND_SUCCESS" in GlassFish gives you that "SEND_CONTINUE" doesn't?

      Delete
    10. Hmm my last message disappeared. Anyway I do think it would likely work if I change SEND_SUCCESS to SEND_CONTINUE, I'll check it when I get home later. However, per the JavaDoc at http://docs.oracle.com/javaee/7/api/javax/security/auth/message/AuthStatus.html#SEND_CONTINUE

      SEND_CONTINUE > Indicates the message processing by the authentication module is NOT complete, that the module replaced the application message with a security message, and that the runtime is to proceed by sending the security message.

      SEND_SUCCESS > Indicates that the message processing by the authentication module was successful and that the runtime is to proceed by sending a message returned by the authentication module.

      At the point where I am at I am already authenticated and I want to get a virtual resource, so in reality I should be sending SEND_SUCCESS from my interpretation.

      Delete
    11. Confirmed that using SEND_CONTINUE works on both WebSphere Liberty and Glassfish. WildFly 8 and 9 are the only other freely available Java EE 7 servers that I have tried and it fails on initialization due to https://issues.jboss.org/browse/WFLY-5072

      Delete
    12. >so in reality I should be sending SEND_SUCCESS from my interpretation

      Okay, so it seems like it's more a question of that the name "SEND_SUCCESS" just sounds better for what you are doing, but in practice it doesn't do anything better, right?

      The problem with the names and the general JavaDoc is that JASPIC has a general universal meaning (chapter 2 in the spec document as far as I recall), and profile specific meanings (chapter 3 for the Servlet Container Profile).

      As far as I understood, the primary use case for SEND_CONTINUE in the Servlet profile is to display a login screen to a user. This login screen is generated from the SAM and the Servlet chain is not invoked. Note that general JASPIC can not speak about a Servlet chain, since it doesn't know about Servlets.

      This is one of the problems with JASPIC. All the types and JavaDoc only document the general generic case. For a specific profile (Servlet, in our case) you must interpret things in a certain way and do things via certain protocols.

      Now in practice JASPIC hasn't been applied to a ton of different scenarios. E.g. it was never applied to JMS or EJB remoting, just to Servlet and SOAP. SOAP is waning, so practically JASPIC is only used for Servlet. I've been arguing for some time to have a Servlet specific API layered on top of core/generic JASPIC. This will hopefully make the API and named more logical for Servlet users (in other words, to nearly all users).

      >At the point where I am at I am already authenticated and I want to get a virtual resource, so in reality I should be sending SEND_SUCCESS from my interpretation.

      It's likely your specific use case is simply not recognized in the spec. Even though SEND_SUCCESS *sounds* better to you, it's not what the Servlet container profile supports.

      For validateRequest that's called before the service invocation (before the Servlet chain is entered), the module can *only* return SUCCESS. If it was not able to fully complete the authentication it can *only* return SEND_CONTINUE and SEND_FAILURE.

      In the Servlet Container Profile, SEND_SUCCESS is reserved as a return status for the secureResponse method, and perhaps somewhat confusingly for when validateRequest is called *after* the service invocation (after secureResponse is called).

      I'm still somewhat in the dark of what you're *exactly* doing though. A small (but if possible working) code example would help to clarify. If you want you can also provide this sample as PR for the Java EE 7 samples project.

      >At the point where I am at I am already authenticated

      Btw, how can you be authentication at that point? Do you call request.authenticate() explicitly from within validateRequest, such that a nested call to validateRequest happens?



      Delete
    13. >Confirmed that using SEND_CONTINUE works on both WebSphere Liberty and Glassfish. WildFly 8 and 9 are the only other freely available Java EE 7 servers

      Indeed, although you can perhaps also try WebLogic 12.1.3. It's Java EE 6, but the changes between EE 6 and EE 7 for JASPIC weren't that big, and you may still be able to test your specific use case; basically just use a null for the app context ID and don't rely on the semi-automatic session saving.

      Delete
    14. Ya I am trying to create a smaller chunk out of the bigger chunk which is an OpenID Connect provider and JASPIC Module set. Initially I worked on getting it working on GlassFish as it was the reference spec and at the time was the only one out there aside from WildFly for Java EE 7 that I can freely use.

      However, now that WebSphere Liberty is in the picture, I was hoping to switch over to that platform because it has better restart and auto publish capabilities on Eclipse but seeing as it has issues with JASPIC I would need to wait it out again on Glassfish.

      As far as the Glassfish reference implementation, the SEND_SUCCESS works and is valid for my scenario. So the failure is specifically on WebSphere's Servlet container IMO. Well whatever, as SEND_CONTINUE works on both and would do what I need it to do, I'll just use SEND_CONTINUE.

      Also where does it say it is reserved?

      http://download.oracle.com/otn-pub/jcp/jaspic-1_1-mrel2-eval-spec/jaspic-MR1.1Final.pdf
      Table 1-1 on page 5 says SEND_SUCCESS is allowed on validateRequest.

      Table 1-2
      SEND_SUCCESS Processing of a request (secureRequest or validateRequest) or response (secureResponse) message was successful and produced the request (secureRequest) [[[ or response (validateRequest, secureResponse) message to be sent to the peer. ]]]

      Am I missing something in the spec?

      Delete
    15. If you're curious the project I am working on is https://github.com/trajano/openid-connect/

      Delete
  9. >now that WebSphere Liberty is in the picture, I was hoping to switch over to that platform because it has better restart and auto publish capabilities on Eclipse but seeing as it has issues with JASPIC I would need to wait it out again on Glassfish.


    I'm (more or less) in contact with the IBM engineers about the JASPIC implementation of Liberty, so if there's anything really specific other than the requirement for the NOOPUserRegistry, please let me know.

    >As far as the Glassfish reference implementation, the SEND_SUCCESS works

    I'm half suspecting this is a bug in GlassFish (i.e. it should not work), but since I still don't know what you are exactly doing (in a small and condensed example) it's a little hard to tell. Best is to contact Ron Monzillo about this, since it hasn't been the first time that the spec has to be interpreted slightly different from what you think when you casually read it.

    >Am I missing something in the spec?

    Well, the problem is that the sections you quote are written from a theoretical point of view. It's for an abstract JASPIC profile that does not exist (is not really implemented). It's written to be the base for more specific profiles.

    The *real* profile that we're dealing with here is the Servlet Container Profile, which is described in chapter 3. Then in 3.8.3.1 you don't see SEND_SUCCESS listed as an outcome.

    But I'll take a look at your project and see if it's possible to distill the smallest possible SAM from it that still demonstrates the issue. Thanks for the reference.



    ReplyDelete
    Replies
    1. Thanks for checking, the code is a bit long so I am planning to play around with my older JASPIC module project

      However, 3.8.3.1 only talks about *before* in 3.8.3.2 where it talks about *after* we have the following:

      validateRequest After Service Invocation

      When validateRequest is called after the service invocation has completed, the module must return **AuthStatus.SEND_SUCCESS** when the module has successfully
      secured the application response message and made it available through
      messageInfo.getResponseMessage

      Which is what I did.

      Unfortunately trying to did that through my code which I had converted to callbacks would be difficult. However here's one of the errant code

      https://github.com/trajano/openid-connect/blob/master/openid-connect-jaspic-module/src/main/java/net/trajano/openidconnect/jaspic/internal/processors/IdTokenRequestProcessor.java

      Actually there is something specific aside on the way I had implemented NoopUserRegistry. Primarily to deal with groups. Which I think I forgot to type in *git push* looking at my repo. I'll try to do that when I get home

      However here's the blog post regarding it. With the NoopUserRegistry code.
      http://www.trajano.net/?p=863&preview=1&_ppp=c05ce2671e

      Delete
    2. Yes, *after* you can return SEND_SUCCESS, but after means in secureResponse or for the off chance that you call the SAM recursively from within secureResponse.

      But remember that *after* means that the Servlet has been invoked. You said that you're working with virtual resources and that the resource not really exists. But if you are in an *after* situation the Servlet must exists.

      So that's what I meant with not knowing what you're doing ;)

      It would probably help if you created the smallest possible example that demonstrates the problem, but I can also try to see if I can extract it from the larger amount of code you referenced.

      Btw, I think your blog link needs to be this one: http://www.trajano.net/2015/08/developing-add-on-for-websphere-liberty-using-maven ;)

      Delete
    3. The link is scheduled for release on the 17th :) That's the "preview edition".

      Anyway, since SEND_CONTINUE would do the job I think it's fine for now. Perhaps they should make it clear what SEND_SUCCESS does in the validateRequest *BEFORE* service invocation on the next maintenance release and fix that spelling error for *HtppServletResponseWrapper* on the next page.

      Delete
  10. IBM released 8.5.5.7 recently. However the requirement to have the UserRegistry for JASPIC appears to still be there. In addition, the initialize() method does not appear to be called in the custom user registry for some odd reason. So I had to make a change in my implementation of NoopUserRegistry https://github.com/trajano/noop-user-registry/commit/e81846cea43fbe0b39f2735d9ded0c1a6dd712f8 to support it.

    However my session-less jaspic-tester seems to work with it so long as there is a mapping from group to role in place and the work around is put in. https://github.com/trajano/jaspic-tester.

    In addition, I have discovered that the only thing that is needed now is the "group" to be in the basic user registry and that group needs to be mapped to the role. Its a lot simpler than dealing with another ESA file. I think this is new to 8.5.5.7

    ReplyDelete
    Replies
    1. I am starting to look at building a custom JASPIC provider feature in Liberty 8.5.5.7. Essentially the userid, password, and authorization is in a database. Admittedly I am new to extending a feature in Liberty but I thought the above conversation very useful and I wanted to validate my assumptions. I should be able to build a custom JASPIC provider via the jaspic-1.1 feature. To do this however I will need the registry noop functionality implemented as well but now simpler in 8.5.5.7. I do have one question. We are using struts for MVC and I have more research there but I was wondering how it would be possible to have more than 1 custom provider on a server(let's say two different application did authentication and authorization against two different databases. Other than that do you see any reason why a custom jaspic provider with 8.5.5.7 is not viable in Liberty right now?

      Delete
  11. This comment has been removed by the author.

    ReplyDelete
  12. Seems like this is still occurring in WebSphere 9.0.0.3. Have you had any luck with the WebSphere Classic installation?

    ReplyDelete
    Replies
    1. Hmm, I actually somehow got it working. w00t
      Dealing with JSF issues now.

      Delete