Latest versions Payara and WildFly improve Java EE 7 authentication compliance

Two months ago we looked at the state of portable authentication for GlassFish, Payara, JBoss/WildFly, WebLogic and Liberty in Java EE 7. With the exception of WebLogic 12.2.1, most servers performed pretty well, but there were still a number of bugs present.

Since then both Payara and WildFly have seen bug fixes that again reduce the number of bugs present where it concerns portable Java EE authentication. Do note that both updated servers have not had an official (supported) release yet, but pre-release resp. rc/cr builds containing those fixes can be downloaded from the vendors.

In anticipation of the final version of those Java EE 7 servers we already took a look at how they improved. The results are shown in the table below. For reference we show several older versions as well. For Payara we took the GlassFish release upon which Payara based its additional fixes, while for WildFly it's a selection of older builds. (no less than 29 builds were released for WildFly 8,9,10/EAP 7 alpha,beta).

Running the Java EE 7 samples JASPIC tests

Module	Test	Payara 4.1.1.161-pre	GlassFish 4.1.1	WildFly 10rc5	WildFly 10rc4	WildFly 9.0.1	WildFly 8.0.0
async-authentication	testBasicAsync	Passed	Passed	Passed	Passed	Passed	Failed
basic-authentication	testProtectedPageNotLoggedin	Passed	Passed	Passed	Passed	Passed	Passed
basic-authentication	testProtectedPageLoggedin	Passed	Passed	Passed	Passed	Passed	Passed
basic-authentication	testPublicPageLoggedin	Passed	Passed	Passed	Passed	Passed	Passed
basic-authentication	testPublicPageNotLoggedin	Passed	Passed	Passed	Passed	Passed	Passed
basic-authentication	testPublicAccessIsStateless	Passed	Passed	Passed	Passed	Passed	Passed
basic-authentication	testProtectedAccessIsStateless	Passed	Passed	Passed	Passed	Passed	Passed
basic-authentication	testProtectedAccessIsStateless2	Passed	Passed	Passed	Passed	Passed	Passed
basic-authentication	testProtectedThenPublicAccessIsStateless	Passed	Passed	Passed	Passed	Passed	Passed
custom-principal	testProtectedPageLoggedin	Passed	Failure	Passed	Passed	Passed	Passed
custom-principal	testPublicPageLoggedin	Passed	Failure	Passed	Passed	Passed	Passed
custom-principal	testPublicAccessIsStateless	Passed	Passed	Passed	Passed	Passed	Passed
custom-principal	testProtectedAccessIsStateless	Passed	Passed	Passed	Passed	Passed	Passed
custom-principal	testProtectedAccessIsStateless2	Passed	Passed	Passed	Passed	Passed	Passed
custom-principal	testProtectedThenPublicAccessIsStateless	Passed	Passed	Passed	Passed	Passed	Passed
dispatching	testBasicForwardViaProtectedResource	Passed	Passed	Passed	Passed	Passed	Passed
dispatching	testBasicForwardViaPublicResource	Passed	Passed	Passed	Passed	Passed	Passed
dispatching	testBasicIncludeViaPublicResource	Passed	Passed	Passed	Passed	Passed	Failure
dispatching-jsf-cdi	testCDIForwardViaProtectedResource	Passed	Passed	Passed	Passed	Passed	Passed
dispatching-jsf-cdi	testCDIForwardViaPublicResource	Passed	Passed	Passed	Passed	Passed	Passed
dispatching-jsf-cdi	testCDIIncludeViaPublicResource	Passed	Passed	Passed	Passed	Passed	Failure
dispatching-jsf-cdi	testJSFwithCDIForwardViaPublicResource	Passed	Passed	Passed	Passed	Passed	Passed
dispatching-jsf-cdi	testJSFwithCDIForwardViaProtectedResource	Passed	Passed	Passed	Passed	Passed	Passed
dispatching-jsf-cdi	testJSFwithCDIIncludeViaPublicResource	Failure	Failure	Failure	Failure	Failure	Failure
dispatching-jsf-cdi	testJSFForwardViaPublicResource	Passed	Passed	Passed	Passed	Passed	Passed
dispatching-jsf-cdi	testJSFForwardViaProtectedResource	Passed	Passed	Passed	Passed	Passed	Passed
dispatching-jsf-cdi	testJSFIncludeViaPublicResource	Failure	Failure	Failure	Failure	Failure	Failure
ejb-propagation	publicServletCallingProtectedEJB	Passed	Passed	Passed	Passed	Passed	Failure
ejb-propagation	protectedServletCallingProtectedEJB	Passed	Passed	Passed	Passed	Passed	Failure
ejb-propagation	publicServletCallingPublicEJBThenLogout	Passed	Passed	Passed	Passed	Passed	Failure
ejb-propagation	protectedServletCallingPublicEJB	Passed	Passed	Passed	Passed	Passed	Passed
invoke-ejb-cdi	protectedInvokeCDIFromSecureResponse	Passed	Passed	Passed	Passed	Failure	Failure
invoke-ejb-cdi	protectedInvokeCDIFromCleanSubject	Passed	Passed	Passed	Passed	Passed	Passed
invoke-ejb-cdi	protectedInvokeCDIFromValidateRequest	Passed	Passed	Passed	Passed	Passed	Passed
invoke-ejb-cdi	publicInvokeCDIFromSecureResponse	Passed	Passed	Passed	Passed	Failure	Failure
invoke-ejb-cdi	publicInvokeCDIFromValidateRequest	Passed	Passed	Passed	Passed	Passed	Passed
invoke-ejb-cdi	publicInvokeCDIFromCleanSubject	Passed	Passed	Passed	Passed	Passed	Passed
invoke-ejb-cdi	protectedInvokeEJBFromSecureResponse	Passed	Failure	Passed	Passed	Failure	Passed
invoke-ejb-cdi	protectedInvokeEJBFromCleanSubject	Passed	Passed	Passed	Passed	Passed	Passed
invoke-ejb-cdi	protectedInvokeEJBFromValidateRequest	Passed	Failure	Passed	Passed	Passed	Passed
invoke-ejb-cdi	publicInvokeEJBFromSecureResponse	Passed	Failure	Passed	Passed	Failure	Passed
invoke-ejb-cdi	publicInvokeEJBFromValidateRequest	Passed	Failure	Passed	Passed	Passed	Passed
invoke-ejb-cdi	publicInvokeEJBFromCleanSubject	Passed	Passed	Passed	Passed	Passed	Passed
jacc-propagation	callingJACCWhenAuthenticated	Passed	Passed	Failure	Failure	Failure	Failure
jacc-propagation	callingJACCWhenAuthenticated	Passed	Passed	Failure	Failure	Failure	Failure
jacc-propagation	callingJACCWhenNotAuthenticated	Passed	Passed	Passed	Passed	Passed	Passed
lifecycle	testBasicSAMMethodsCalled	Passed	Passed	Passed	Passed	Failure	Passed
lifecycle	testLogout	Passed	Passed	Passed	Passed	Passed	Passed
register-session	testJoinSessionIsOptional	Passed	Passed	Passed	Passed	Passed	Passed
register-session	testRemembersSession	Passed	Passed	Passed	Passed	Passed	Passed
status-codes	test404inResponse	Passed	Passed	Passed	Failure	Failure	Passed
status-codes	test404inResponse	Passed	Passed	Passed	Failure	Failure	Passed
wrapping	testResponseWrapping	Passed	Passed	Passed	Passed	Passed	Passed
wrapping	testRequestWrapping	Passed	Passed	Passed	Passed	Passed	Passed

Not shown in the table, but the absolute greatest improvement since JBoss switched to its new JASPIC implementation all the way back in WildFly 8.0.0.Alpha1 is the fact that JASPIC now finally works without the need of modifying WildFly by putting a dummy fragment in its standalone.xml file. It's not 100% perfect yet as the application archive (.war) still needs what is effectively a marker file to activate JASPIC, but this is much, much preferred over having to modify a server in order to activate a standard Java EE API that should just be there. Kudos to the JBoss team and a special thanks to Jason Greene for finally making this happen!

As can be seen, WildFly has seen many improvements over the years. Along the way a few regressions were introduced, but they were fixed again and now WildFly10rc5 is almost perfect with respect to the known bugs. Role propagation to JACC however still doesn't work. Although the usage of custom JACC providers is not that high, the test in question here uses the default provider for a rather useful query; "Can the authenticated user access a given resource?", e.g. "Can Pete access http://example.com/assets/someresource?".

The top performer as of now is Payarra, which passes all tests except for one of minor importance where a JSF based resource is included by an authentication module. As mentioned in the previous report this likely has to be fixed on the JSF side of things.

If all goes well we'll see a new beta of Liberty 9 this month which should also contain a number of fixes. The most problematic server at this moment is still WebLogic, which introduced a major regression between 12.1.3 and 12.2.1. Hopefully WebLogic will fix this regression soon. We'll repeat this test again when either of those publish their latest version.

Arjan Tijms