The state of portable authentication in Java EE, mid 2016 update
Originally the situation didn't looked that rosy, but since then things have been steadily improving. Since last time new versions of Payara, WildFly and Liberty have been released, while TomEE now also supports JASPIC. No new versions of Oracle's WebLogic and GlassFish were released. New tests have been added for registering a session with a custom principal and EJB propagation after register session.
The results of running the latest series of JASPIC tests are shown below:
Not directly shown in the table, but one of the biggest changes is that Liberty 9 beta 2016.5 no longer needs JASPIC to be activated in a problematic and proprietary way. Historically Liberty was one of the most problematic servers in this area. It originally required all users and groups to be made known to a Liberty specific artefact called a user registry (a kind of identity store). This requirement was later relaxed such that only groups had to be defined, but since group to role mapping was also mandatory, this stil meant defining the same group at least 3 times if no actual group mapping was needed.
In Liberty 9 beta 2016.5 all this overhead is now completely gone, and just like GlassFish and WebLogic, JASPIC just works without any activation whatsoever (with a caveat for GlassFish that by default it does require group to role mapping). One thing to watch out for is that if you do need group to role mapping, you seemingly can't just use Liberty's role mapping file (ibm-application-bnd.xml), but also need to define the above mentioned user registry again. All other servers (that feature a role mapper) allow the mapping of groups set by a JASPIC SAM via only a single XML file. When asked about this, the Liberty lead mentioned that this is due to backwards compatibility, which does make sense. It's also not such a bad thing; if you want to use the proprietary group to role mapping feature it's only a relatively small extra step to also activate a proprietary artefact such as the user registry.
As far as JASPIC functionality goes, Liberty keeps failing the CDI related tests and the JACC ones. The JACC ones are a clear spec violation (confirmed by the JACC and JASPIC spec lead), but since it concerns a requirement that was never enforced (namely, JACC has to be available and used by default) it's likely far from trivial to just fix this. Liberty also fails a test where the authenticated user is changed during the same HTTP session. This however clashes with Liberty's protection against a session fixation attack, which for instance happens when a user logs out and without resetting the http session (specifically changing its ID) a new user logs in again. This protection can be disabled by putting <httpSession securityIntegrationEnabled="false"/> in server.xml.
As mentioned above, TomEE is new to JASPIC scene, but what an entrance it makes! In it's very first release it passed all JASPIC tests, safe the JACC ones. TomEE (as not a full Java EE 7 certified server) is however not required to support JACC at all, let alone out of the box. Since TomEE does actually support JACC it may be possible to have it pass these tests as well. Right from the start TomEE also doesn't require any explicit activation of JASPIC. It just works out of the box.
JBoss/WildFly, while still having a very good JASPIC implementation fell a tiny bit behind. It still doesn't pass all JACC tests, even though JACC is in fact available by default (a bug prevents roles to be propagated correctly). Additionally, a new bug was found that prevents roles being propagated to EJB beans when the remember session feature is used. JBoss also still requires the dummy-like marker file (jboss-web.xml) in an archive to activate JASPIC. Even though such marker file is only a small nuisance compared to having to modify the server itself, it does make JBoss the sole server now that requires such activation at all (but as mentioned above, GlassFish/Payara requires a proprietary group to role mapping file to be present).
Payara also suffered from a newly discovered bug, this one involving the combination of the remember session feature and a custom principal. This was however quickly fixed in the 18.104.22.168 branch and Payara remains one of the strongest JASPIC implementations. There's essentially only one minor failure involving a not too common case of including a JSF resource from within a SAM.
7 years after portable authentication was introduced in Java EE 6 and 3 years after its revision in Java EE 7, JASPIC implementations are now finally getting really compliant. There's a residu of bugs in the various implementations that still need to be fixed, and a few inconveniences to take away in some servers (JBoss' jboss-web.xml and GlassFish/Payara's glassfish-web.xml), but overal things are starting to look very good.
The one big exception remains WebLogic 12.2.1, which currently doesn't work at all after a major bug was introduced in that version that prevents basic authentication to work correctly (effectively a blocker bug that makes running all other tests rather moot)