Sunday, December 4, 2016

The state of portable authentication in Java EE, end 2016 update

In the beginning and middle of this year we looked at how well modern Java EE servers supported portable authentication (JASPIC) in Java EE. As the end of 2016 approaches we take a third look to see how things are progressing.

Since our last time new versions of all servers have been released. Payara went from 163-beta to 164, WildFly went from 10.0 to 10.1, Liberty beta went from 2016-5 to 2016-11, WebLogic went from 12.2.1 to 12.2.1.2 and TomEE went from 7.0 to 7.0.2. We also added a new server, namely Tomcat. Tomcat was indirectly already tested via TomEE, but given the importance of standalone Tomcat we decided to put this one in explicitly. Do note that Tomcat is not a full or web profile Java EE server, so the integration tests for technologies it doesn't support (like JSF, CDI, etc) are simply omitted.

Tests were added for request.authenticate, an injected CDI request, the servlet path after a forward, isMandatory in a SAM, and finally for a SAM request to be seen by a Filter. For the first time there was also a test removed, namely including a JSF based resource from a SAM. This fails on many servers but the failure appeared to be a general JSF failure unrelated to JASPIC and/or its integration with the Java EE environment.

The results of running the latest series of JASPIC tests are shown below:

Running the Java EE 7 samples JASPIC tests
Module Test Payara 164 WildFly 10.1 Liberty 2016-11 Weblogic 12.2.1.2 TomEE 7.0.2 Tomcat 8.5.6
async-authentication testBasicAsync
Passed
Passed
Passed
Passed
Passed
-
basic-authentication testProtectedPageNotLoggedin
Passed
Passed
Passed
Passed
Passed
Passed
basic-authentication testProtectedPageLoggedin
Passed
Passed
Passed
Passed
Passed
Passed
basic-authentication testPublicPageLoggedin
Passed
Passed
Passed
Passed
Passed
Passed
basic-authentication testPublicPageNotLoggedin
Passed
Passed
Passed
Passed
Passed
Passed
basic-authentication testPublicAccessIsStateless
Passed
Passed
Passed
Passed
Passed
Passed
basic-authentication testProtectedAccessIsStateless
Passed
Passed
Passed
Passed
Passed
Passed
basic-authentication testProtectedAccessIsStateless2
Passed
Passed
Passed
Passed
Passed
Passed
basic-authentication testProtectedThenPublicAccessIsStateless
Passed
Passed
Passed
Passed
Passed
Passed
custom-principal testProtectedPageLoggedin
Passed
Passed
Passed
Failure
Passed
Passed
custom-principal testPublicPageLoggedin
Passed
Passed
Passed
Failure
Passed
Passed
custom-principal testPublicAccessIsStateless
Passed
Passed
Passed
Passed
Passed
Passed
custom-principal testProtectedAccessIsStateless
Passed
Passed
Passed
Passed
Passed
Passed
custom-principal testProtectedAccessIsStateless2
Passed
Passed
Passed
Passed
Passed
Passed
custom-principal testProtectedThenPublicAccessIsStateless
Passed
Passed
Passed
Passed
Passed
Passed
dispatching testBasicForwardViaProtectedResource
Passed
Passed
Passed
Passed
Passed
Passed
dispatching testBasicForwardViaPublicResource
Passed
Passed
Passed
Passed
Passed
Passed
dispatching testBasicIncludeViaPublicResource
Passed
Passed
Passed
Passed
Passed
Passed
dispatching-jsf-cdi testCDIForwardWithRequestProtected
Passed
Passed
Passed
Passed
Passed
-
dispatching-jsf-cdi testCDIForwardWithRequestInjectPublic
Passed
Passed
Passed
Passed
Failure
-
dispatching-jsf-cdi testCDIForwardViaProtectedResource
Passed
Passed
Passed
Passed
Passed
-
dispatching-jsf-cdi testCDIForwardViaPublicResource
Passed
Passed
Passed
Passed
Passed
-
dispatching-jsf-cdi testCDIForwardWithRequestPublic
Passed
Passed
Passed
Passed
Passed
-
dispatching-jsf-cdi testCDIForwardWithRequestInjectProtected
Passed
Passed
Passed
Passed
Failure
-
dispatching-jsf-cdi testCDIIncludeViaPublicResource
Passed
Passed
Passed
Passed
Passed
-
dispatching-jsf-cdi testJSFwithCDIForwardViaPublicResource
Passed
Passed
Passed
Passed
Passed
-
dispatching-jsf-cdi testJSFwithCDIForwardViaProtectedResource
Passed
Passed
Passed
Passed
Passed
-
dispatching-jsf-cdi testJSFForwardViaPublicResource
Passed
Passed
Passed
Passed
Passed
-
dispatching-jsf-cdi testJSFForwardViaProtectedResource
Passed
Passed
Passed
Passed
Passed
-
ejb-propagation publicServletCallingProtectedEJB
Passed
Passed
Passed
Failure
Passed
-
ejb-propagation protectedServletCallingProtectedEJB
Passed
Passed
Passed
Failure
Passed
-
ejb-propagation publicServletCallingPublicEJBThenLogout
Passed
Passed
Passed
Passed
Passed
-
ejb-propagation protectedServletCallingPublicEJB
Passed
Passed
Passed
Passed
Passed
-
ejb-register-session testRemembersSession
Passed
Passed
Passed
Failure
Passed
-
ejb-register-session testRemembersSession
Passed
Passed
Passed
Failure
Passed
-
invoke-ejb-cdi protectedInvokeCDIFromSecureResponse
Passed
Passed
Failure
Failure
Passed
-
invoke-ejb-cdi protectedInvokeCDIFromCleanSubject
Passed
Passed
Passed
Passed
Passed
-
invoke-ejb-cdi protectedInvokeCDIFromValidateRequest
Passed
Passed
Failure
Failure
Passed
-
invoke-ejb-cdi publicInvokeCDIUseInjectedRequestFromValidateRequest
Passed
Passed
Failure
Failure
Passed
-
invoke-ejb-cdi publicInvokeCDIFromSecureResponse
Passed
Passed
Failure
Failure
Passed
-
invoke-ejb-cdi publicInvokeCDIFromValidateRequest
Passed
Passed
Failure
Failure
Passed
-
invoke-ejb-cdi publicInvokeCDIUseInjectedRequestFromCleanSubject
Passed
Passed
Passed
Passed
Passed
-
invoke-ejb-cdi publicInvokeCDIUseInjectedRequestFromSecureResponse
Passed
Passed
Failure
Failure
Passed
-
invoke-ejb-cdi publicInvokeCDIFromCleanSubject
Passed
Passed
Passed
Passed
Passed
-
invoke-ejb-cdi protectedInvokeEJBFromSecureResponse
Passed
Passed
Passed
Passed
Passed
-
invoke-ejb-cdi protectedInvokeEJBFromCleanSubject
Passed
Passed
Passed
Passed
Passed
-
invoke-ejb-cdi protectedInvokeEJBFromValidateRequest
Passed
Passed
Passed
Passed
Passed
-
invoke-ejb-cdi publicInvokeEJBFromSecureResponse
Passed
Passed
Passed
Passed
Passed
-
invoke-ejb-cdi publicInvokeEJBFromValidateRequest
Passed
Passed
Passed
Passed
Passed
-
invoke-ejb-cdi publicInvokeEJBFromCleanSubject
Passed
Passed
Passed
Passed
Passed
-
jacc-propagation callingJACCWhenAuthenticated
Passed
Passed
Failure
Failure
Failure
-
jacc-propagation callingJACCWhenAuthenticated
Passed
Passed
Failure
Failure
Failure
-
jacc-propagation callingJACCWhenNotAuthenticated
Passed
Passed
Failure
Failure
Failure
-
lifecycle testBasicSAMMethodsCalled
Passed
Passed
Passed
Passed
Passed
Passed
lifecycle testLogout
Passed
Passed
Passed
Passed
Passed
Passed
lifecycle testProtectedIsMandatory
Passed
Passed
Passed
Passed
Passed
Passed
lifecycle testPublicIsNonMandatory
Passed
Passed
Passed
Passed
Passed
Passed
programmatic-authentication testAuthenticateFailFirstTwice
Passed
Passed
Passed
Passed
Passed
Passed
programmatic-authentication testAuthenticate
Passed
Passed
Passed
Passed
Passed
Passed
programmatic-authentication testAuthenticateFailFirstOnce
Passed
Passed
Passed
Passed
Passed
Passed
register-session testJoinSessionIsOptional
Passed
Passed
Passed
Failure
Passed
Passed
register-session testRemembersSession
Passed
Passed
Passed
Failure
Passed
Passed
register-session testJoinSessionIsOptional
Passed
Passed
Passed
Failure
Passed
Passed
register-session testRemembersSession
Passed
Passed
Passed
Failure
Passed
Passed
status-codes test404inResponse
Passed
Passed
Passed
Passed
Passed
Passed
status-codes test404inResponse
Passed
Passed
Passed
Passed
Passed
Passed
wrapping testResponseWrapping
Passed
Passed
Passed
Passed
Passed
Passed
wrapping testDeclaredFilterResponseWrapping
Passed
Passed
Failure
Passed
Passed
Passed
wrapping testProgrammaticFilterResponseWrapping
Passed
Passed
Failure
Passed
Passed
Passed
wrapping testDeclaredFilterRequestWrapping
Passed
Passed
Failure
Passed
Passed
Passed
wrapping testProgrammaticFilterRequestWrapping
Passed
Passed
Failure
Passed
Passed
Passed
wrapping testRequestWrapping
Passed
Passed
Passed
Passed
Passed
Passed

As can be see we now have 3 perfectly scoring servers; Payara, WildFly and Tomcat now pass every test being thrown at them. All 3 do this for the first time, and the slightly older versions of each of them contain a variety of (small) bugs that prevents a perfect score.

Liberty still fails the CDI tests as before, but there's a hack available that can let Liberty pass these. Since we test as much as possible out of the box here this hack was not applied. JACC propagation also doesn't work, but this is because there's no JACC provider available by default.

Perhaps the most surprising result of this round is that WebLogic is not longer completely broken when it comes to JASPIC. While WebLogic 12.1.x had decent support for JASPIC, a major regression was introduced in 12.2.1.0 were the basic cases didn't work properly anymore. When the basic cases fail it's almost pointless to continue testing since in practice when an authentication itself doesn't work anymore it doesn't matter so much whether say request wrapping is supported or not. WebLogic does have major problems with the "register session" feature (which likely caused the regression with basic authentication in the first place), doesn't support setting a custom principal (a major feature of JASPIC), and has problems with propagating to EJB. WebLogic fails the same CDI and JACC tests as Liberty does. Contrary to Liberty, WebLogic -does- have a default JACC provider but it's not activated out of the box.

TomEE performs very well, but has a small hick up when an injected request is used in a forwarded resource. It also fails the JACC propagation, but this is because TomEE doesn't implement JACC at all for the Web module (only for EJB).

An important additional aspect not shown in the test table above is whether JASPIC works out of the box (like e.g. a Servlet Filter does), or whether it needs (server specific) activation of some sort.

In this regard, Liberty, WebLogic, TomEE and Tomcat need zero activation. There's a small caveat with Liberty and that's if you use the server specific group to role mapping in combination with a JASPIC SAM, all role checking fails. This is a somewhat peculiar incompatibility. All other servers don't care whether the groups that are set come from a portable JASPIC SAM or from something proprietary when doing the server specific/proprietary group to role mapping.

Payara and WildFly both need an activation of some kind. Payara needs a glassfish-web.xml where default group to role mapping is activated (or alternatively, where groups are actually mapped to roles), while WildFly needs a jboss-web.xml where the "jaspitest" security domain is set. WildFly should eventually lift this mandated activation when the new Elytron security system is introduced, which could possibly happen in WildFly 11. Payara could eventually default to default group to role mapping. WebLogic made this move before, but this is somewhat of a major change that should be done carefully if indeed planned.

Conclusion

Three servers, namely WildFly 10.1, Payara 164 and Tomcat 8.5.6 now pass all tests, but two of them (WildFly and Payara) need some kind of activation so are not perfect-perfect, but still very, very close to it. TomEE performs very well too, with only a minor regression and all the major core authentication functionality working perfectly.

Liberty and WebLogic have a bit more work left to be done, where as core features are concerned Liberty fails the request/response wrapping partially, while WebLogic fails the custom principal, register session and EJB propagation.

In a next article we'll be looking at what things look like after we applied the CDI/Weld hack for Liberty and Weblogic, and if JACC propagation works on them when we install/activate a JACC provider.

Arjan Tijms