Thursday, November 20, 2014

OmniFaces 2.0 RC2 available for testing

After an intense debugging session following the release of OmniFaces 2.0, we have decided to release one more release candidate; OmniFaces 2.0 RC2.

For RC2 we mostly focused on TomEE 2.0 compatibility. Even though TomEE 2.0 is only available in a SNAPSHOT release, we're happy to see that it passed almost all of our tests and was able to run our showcase application just fine. The only place where it failed was with the viewParamValidationFailed page, but this appeared to be an issue in MyFaces and unrelated to TomEE itself.

To repeat from the RC1 announcement: OmniFaces 2.0 is the first release that will depend on JSF 2.2 and CDI 1.1 from Java EE 7. Our Servlet dependency is now Servlet 3.0 from Java EE 6 (used to be 2.5, although we optionally used 3.0 features before). The minimal Java SE version is now Java 7.

A full list of what's new and changed is available here.

OmniFaces 2.0 RC2 can be tested by adding the following dependency to your pom.xml:

<dependency>
    <groupId>org.omnifaces</groupId>
    <artifactId>omnifaces</artifactId>
    <version>2.0-RC2</version>
</dependency>

Alternatively the jars files can be downloaded directly.

We're currently investigating one last issue, if that's resolved and no other major bugs appear we'd like to release OmniFaces 2.0 at the end of this week.

Arjan Tijms

Sunday, November 16, 2014

Header based stateless token authentication for JAX-RS

Authentication is a topic that comes up often for web applications. The Java EE spec supports authentication for those via the Servlet and JASPIC specs, but doesn't say too much about how to authenticate for JAX-RS.

Luckily JAX-RS is simply layered on top of Servlets, and one can therefore just use JASPIC's authentication modules for the Servlet Container Profile. There's thus not really a need for a separate REST profile, as there is for SOAP web services.

While using the same basic technologies as authentication modules for web applications, the requirements for modules that are to be used for JAX-RS are a bit different.

JAX-RS is often used to implement an API that is used by scripts. Such scripts typically do not engage into an authentication dialog with the server, i.e. it's rare for an API to redirect to a form asking for credentials, let alone asking to log-in with a social provider.

An even more fundamental difference is that in web apps it's commonplace to establish a session for among others authentication purposes. While possible to do this for JAX-RS as well, it's not exactly a best practice. Restful APIs are supposed to be fully stateless.

To prevent the need for going into an arbitrary authentication dialog with the server, it's typically for scripts to send their credentials upfront with a request. For this BASIC authentication can be used, which does actually initiates a dialog albeit a standardised one. An other option is to provide a token as either a request parameter or as an HTTP header. It should go without saying that in both these case all communication should be done exclusively via https.

Preventing a session to be created can be done in several ways as well. One way is to store the authentication data in an encrypted cookie instead of storing that data in the HTTP session. While this surely works it does feel somewhat weird to "blindly" except the authenticated identity from what the client provides. If the encryption is strong enough it *should* be okayish, but still. Another method is to quite simply authenticate every time over again with each request. This however has its own problem, namely the potential for bad performance. An in-memory user store will likely be very fast to authenticate against, but anything involving an external system like a database or ldap server probably is not.

The performance problem of authenticating with each request can be mitigated though by using an authentication cache. The question is then whether this isn't really the same as creating a session?

While both an (http) session and a cache consume memory at the server, a major difference between the two is that a session is a store for all kinds of data, which includes state, but a cache is only about data locality. A cache is thus by definition never the primary source of data.

What this means is that we can throw data away from a cache at arbitrary times, and the client won't know the difference except for the fact its next request may be somewhat slower. We can't really do that with session data. Setting a hard limit on the size of a cache is thus a lot easier for a cache then it is for a session, and it's not mandatory to replicate a cache across a cluster.

Still, as with many things it's a trade off; having zero data stored at the server, but having a cookie send along with the request and needing to decrypt that every time (which for strong encryption can be computational expensive), or having some data at the server (in a very manageable way), but without the uneasiness of directly accepting an authenticated state from the client.

Here we'll be giving an example for a general stateless auth module that uses header based token authentication and authenticates with each request. This is combined with an application level component that processes the token and maintains a cache. The auth module is implemented using JASPIC, the Java EE standard SPI for authentication. The example uses a utility library that I'm incubating called OmniSecurity. This library is not a security framework itself, but provides several convenience utilities for the existing Java EE security APIs. (like OmniFaces does for JSF and Guava does for Java)

One caveat is that the example assumes CDI is available in an authentication module. In practice this is the case when running on JBoss, but not when running on most other servers. Another caveat is that OmniSecurity is not yet stable or complete. We're working towards an 1.0 version, but the current version 0.6-ALPHA is as the name implies just an alpha version.

The module itself look as follows:

public class TokenAuthModule extends HttpServerAuthModule {
    
    private final static Pattern tokenPattern = compile("OmniLogin\\s+auth\\s*=\\s*(.*)");
    
    @Override
    public AuthStatus validateHttpRequest(HttpServletRequest request, HttpServletResponse response, HttpMsgContext httpMsgContext) throws AuthException {
        
        String token = getToken(request);
        if (!isEmpty(token)) {
            
            // If a token is present, authenticate with it whether this is strictly required or not.
            
            TokenAuthenticator tokenAuthenticator = getReferenceOrNull(TokenAuthenticator.class);
            if (tokenAuthenticator != null) {
                
                if (tokenAuthenticator.authenticate(token)) {
                    return httpMsgContext.notifyContainerAboutLogin(tokenAuthenticator.getUserName(), tokenAuthenticator.getApplicationRoles());
                }                
            }            
        }
        
        if (httpMsgContext.isProtected()) {
            return httpMsgContext.responseNotFound();
        }
        
        return httpMsgContext.doNothing();
    }
    
    private String getToken(HttpServletRequest request) { 
        String authorizationHeader = request.getHeader("Authorization");
        if (!isEmpty(authorizationHeader)) {
            
            Matcher tokenMatcher = tokenPattern.matcher(authorizationHeader);
            if (tokenMatcher.matches()) {
                return tokenMatcher.group(1);
            }
        }
        
        return null;
    }

}
Below is a quick primer on Java EE's authentication modules:
A server auth module (SAM) is not entirely unlike a servlet filter, albeit one that is called before every other filter. Just as a servlet filter it's called with an HttpServletRequest and HttpServletResponse, is capable of including and forwarding to resources, and can wrap both the request and the response. A key difference is that it also receives an object via which it can pass a username and optionally a series of roles to the container. These will then become the authenticated identity, i.e. the username that is passed to the container here will be what HtttpServletRequest.getUserPrincipal().getName() returns. Furthermore, a server auth module doesn't control the continuation of the filter chain by calling or not calling FilterChain.doFilter(), but by returning a status code.

In the example above the authentication module extracts a token from the request. If one is present, it obtains a reference to a TokenAuthenticator, which does the actual authentication of the token and provides a username and roles if the token is valid. It's not strictly necessary to have this separation and the authentication module could just as well contain all required code directly. However, just like the separation of responsibilities in MVC, it's typical in authentication to have a separation between the mechanism and the repository. The first contains the code that does interaction with the environment (aka the authentication dialog, aka authentication messaging), while the latter doesn't know anything about an environment and only keeps a collection of users and roles that are accessed via some set of credentials (e.g. username/password, keys, tokens, etc).

If the token is found to be valid, the authentication module retrieves the username and roles from the authenticator and passes these to the container. Whenever an authentication module does this, it's supposed to return the status "SUCCESS". By using the HttpMsgContext this requirement is largely made invisible; the code just returns whatever HttpMsgContext.notifyContainerAboutLogin returns.

If authentication did not happen for whatever reason, it depends on whether the resource (URL) that was accessed is protected (requires an authenticated user) or is public (does not require an authenticated user). In the first situation we always return a 404 to the client. This is a general security precaution. According to HTTP we should actually return a 403 here, but if we did users can attempt to guess what the protected resources are. For applications where it's already clear what all the protected resources are it would make more sense to indeed return that 403 here. If the resource is a public one, the code "does nothing". Since authentication modules in Java EE need to return something and there's no status code that indicates nothing should happen, in fact doing nothing requires a tiny bit of work. Luckily this work is largely abstracted by HttpMsgContext.doNothing().

Note that the TokenAuthModule as shown above is already implemented in the OmniSecurity library and can be used as is. The TokenAuthenticator however has to be implemented by user code. An example of an implementation is shown below:

@RequestScoped
public class APITokenAuthModule implements TokenAuthenticator {

    @Inject
    private UserService userService;

    @Inject
    private CacheManager cacheManager;
    
    private User user;

    @Override
    public boolean authenticate(String token) {
        try {
            Cache<String, User> usersCache = cacheManager.getDefaultCache();

            User cachedUser = usersCache.get(token);
            if (cachedUser != null) {
                user = cachedUser;
            } else {
                user = userService.getUserByLoginToken(token);
                usersCache.put(token, user);
            }
        } catch (InvalidCredentialsException e) {
            return false;
        }

        return true;
    }

    @Override
    public String getUserName() {
        return user == null ? null : user.getUserName();
    }

    @Override
    public List<String> getApplicationRoles() {
        return user == null ? emptyList() : user.getRoles();
    }

    // (Two empty methods omitted)
}
This TokenAuthenticator implementation is injected with both a service to obtain users from, as well as a cache instance (InfiniSpan was used here). The code simply checks if a User instance associated with a token is already in the cache, and if it's not gets if from the service and puts it in the cache. The User instance is subsequently used to provide a user name and roles.

Installing the authentication module can be done during startup of the container via a Servlet context listener as follows:

@WebListener
public class SamRegistrationListener extends BaseServletContextListener {
 
    @Override
    public void contextInitialized(ServletContextEvent sce) {
        Jaspic.registerServerAuthModule(new TokenAuthModule(), sce.getServletContext());
    }
}
After installing the authentication module as outlined in this article in a JAX-RS application, it can be tested as follows:
curl -vs -H "Authorization: OmniLogin auth=ABCDEFGH123" https://localhost:8080/api/foo 

As shown in this article, adding an authentication module for JAX-RS that's fully stateless and doesn't store an authenticated state on the client is relatively straightforward using Java EE authentication modules. Big caveats are that the most straightforward approach uses CDI which is not always available in authentication modules (in WildFly it's available), and that the example uses the OmniSecurity library to simplify some of JASPIC's arcane native APIs, but OmniSecurity is still only in an alpha status.

Arjan Tijms

Saturday, November 8, 2014

OmniFaces 2.0 RC1 available for testing

We are happy to announce that we have just released OmniFaces 2.0 release candidate 1.

OmniFaces 2.0 is the first release that will depend on JSF 2.2 and CDI 1.1 from Java EE 7. Our Servlet dependency is now Servlet 3.0 from Java EE 6 (used to be 2.5, although we optionally used 3.0 features before). The minimal Java SE version is now Java 7.

A full list of what's new and changed is available here.

OmniFaces 2.0 RC1 can be tested by adding the following dependency to your pom.xml:

<dependency>
    <groupId>org.omnifaces</groupId>
    <artifactId>omnifaces</artifactId>
    <version>2.0-RC1</version>
</dependency>

Alternatively the jars files can be downloaded directly.

If no major bugs surface we hope to release OmniFaces 2.0 final in about one week from now.

Arjan Tijms

Friday, October 31, 2014

Java EE process cycles and server availability

When we normally talk about the Java EE cycle time, we talk about the time it takes between major revisions of the spec. E.g. the time between Java EE 6 and Java EE 7. While this is indeed the leading cycle time, there are two additional cycles that are of major importance:
  1. The time it takes for vendors to release an initial product that implements the new spec revision
  2. The time it takes vendors to stabilize their product (which incidentally is closely tied to the actual user adoption rate)

In this article we'll take a somewhat closer look on the time it takes vendors to release their initial product. But first let's take a quick look at the time between spec releases. The following table lists the Java EE version history and the delta time between versions:

Java EE delta times between releases
Version Start date Release date Days since last release Days spent on spec
1.2 - 12 Dec, 1999 - -
1.3 18 Feb, 2000 24 Sep, 2001 653 days (1 year, 9 months) 584 (1 year, 7 months)
1.4 22 Oct, 2001 12 Nov, 2003 779 days (2 years, 1 month) 751 (2 years)
5 10 May, 2004 11 May, 2006 911 days (2 years, 6 months) 731 (2 years)
6 16 Jul, 2007 10 Dec, 2009 1310 days (3 years, 7 months) 878 (2 years, 4 months)
7 14 Mar, 2011 28 May, 2013 1266 days (3 years, 5 months) 806 (2 years, 2 months)

As can be seen the time between releases has been steadily increasing, but now seems to have been stabilized to approximately three and a half years. The plan is to release Java EE 8 using the same pace, meaning we should expect it around the end of 2016.

It may be worth emphasizing that the time between releases is not fully spend on Java EE. Typically there is what one may call with respect to spec work The Big Void™ between releases. It's a period of time where there is no spec work being done. This void starts right after the spec is released and the various EGs are disbanded. The time is used differently by everyone, but typically it's used for implementation work, cleaning up and refactoring code, project structures, tests and other artifacts.

After some time (~1 year for Java EE 6, ~5 months for Java EE 7) initial discussions start where just some ideas are pitched and the landscape is explored. After that it still takes some time until the work really kicks off for the front runners (~1 year and 5 months for Java EE 6, ~1 year and 3 months for Java EE 7).

Those numbers are however for the front runners, a bunch of sub-specs of Java EE start even later than this, and some of them even finish well before the release date of the main umbrella spec. So while the time between releases seems like a long time, it's important to realize by far not all this time is actually spend on the various specifications. As can be seen in the table above, the time actually spend on the specification has been fairly stable at around 2 years. 1.3 was a bit below that and 6 a bit above it, but it's all fairly close to these two years. What has been increasing is the time taken up by The Void (or uptake as some others call it); less than a month between 1.3 and 1.4 to well over a year between 5 and 6, and 6 and 7.

As mentioned previously, finalizing the spec is only one aspect of the entire process. With the exception of GlassFish, the reference implementation (RI) that is made available at the same time that the new spec revision becomes available, the implementation cycle of Java EE starts right after a spec release.

A small complication in tracking Java EE server products is that various of these products are variations of each other or just different versions taken from the same code line. E.g. WASCE is (was) an intermediate release of Geronimmo. JBoss AS 6 is obviously just an earlier version of JBoss AS 7, which is itself an earlier version of JBoss EAP 6 (although JBoss markets it as a separate product). NetWeaver is said to be a version of TomEE, etc.

Also complicating the certification and first version story is that a number of vendors chose to have beta or technical preview versions certified. In one occasion a vendor even certified a snapshot version. Obviously those versions are not intended for any practical (production) use. It's perhaps somewhat questionable that servers that in the eyes of their own vendors are very far from the stability required by their customers can be certified at all.

The following two tables show how long it took the Java EE 6 Full- and Web Profile to be implemented for each server.

Java EE 6 Full Profile server implementation times
Server Release date Days since spec released
GlassFish 3.0 10 Dec, 2009 0
* JEUS 7 Tech Preview 1 15 Jan, 2010 36
WebSphere 8.0 22 June, 2011 559 (1 year, 6 months)
* Geronimo 3.0 BETA 1 14 November, 2011 704 (1 year, 11 months)
WebLogic 12.1.1 1 Dec, 2011 721 (1 year, 11 months)
Interstage AS 10.1 27 December, 2011 747 (2 years)
* JBoss AS 7.1 17 Feb, 2012 799 (2 years, 2 months)
(u)Cosminexus 9.0 ~16 April, 2012 858 (2 years, 4 months)
JEUS 7.0 ~1 June, 2012 904 (2 years, 5 months)
JBoss EAP 6 20 June, 2012 923 (2 years, 6 months)
Geronimo 3.0 13 July, 2012 946 (2 years, 7 months)
WebOTX AS 9.1 30 May, 2013 1267 (3 years, 5 months)
InforSuite AS 9.1 ~July, 2014 ~1664 (4 years, 6 months)
* denotes a server that's a tech preview, community, developer preview, beta, etc version

Java EE 6 Web Profile server implementation times
Server Release date Days since spec released
* JBoss AS 6.0 28 December, 2010 322 (10 months)
Resin 4.0.17 May, 2011 507 (1 year, 4 months)
* JBoss AS 7.0 12 July, 2011 579 (1 year, 7 months)
* TomEE beta 4 Oct, 2011 663 (1 year, 9 months)
TomEE 1.0 08 May, 2012 880 (2 years, 4 months)
* JOnAS 5.3.0-M8-SNAPSHOT [14 Nov, 2012 ~ 07 Jan 2013] 1070~1124 (~3 years)
Liberty 8.5.5 14 Jun, 2013 1282 (3 years, 6 months)
JOnAS 5.3 04 Oct 2013 1394 (3 years, 9 months)
* denotes a server that's a tech preview, community, developer preview, beta, etc version

As we can see here, excluding GlassFish and the tech preview of JEUS, it took 1 year and 6 months for the first production ready (according to the vendor!) Java EE 6 full profile server to appear on the market, while most other servers appeared after around two and half years.

Do note that "production ready according to the vendor" is a state that can not easily be quantified with respect to quality. What some vendor calls 1.0 Final, may correspond to what another vendor calls 0.5 Beta. From the above table it doesn't mean that say WebLogic 12.1.1 (production ready according to its vendor) is either more or less stable than e.g. JEUS 7 Tech Preview 1 (not production ready according to its vendor).

The Java EE 7 spec was released at 28 May, 2013, which is 522 days (1 year, 5 months) ago at the time of writing. So let's take a look at what the current situation is with respect to available Java EE 7 servers:

Java EE 7 Full Profile server implementation times
Server Release date Days since spec released
GlassFish 4.0 28 May, 2013 0
* JEUS 8 developer preview ~26 Aug, 2013 90 (2 months, 29 days)
* JBoss WildFly 8.0 11 Feb, 2014 259 (8 months, 14 days)
* denotes a server that's a tech preview, community, developer preview, beta, etc version

Although there are just a few entries, those largely follow the same pattern as the Java EE 6 implementation cycle.

GlassFish is by definition the first release, while JEUS is again the second one with a developer preview (a pattern that goes all the way back to J2EE 1.2). There's unfortunately no information available on when JEUS 8 developer preview was exactly released, but a blog posting about this was published at 26 aug, 2003 so I took that date.

For JBoss the situation for Java EE 7 compared to EE 6 is not really that much different either. WildFly 8 was released after 259 days (the plan was 167 days), which is not that different from JBoss AS 6 which was released after 322 days. One big difference here though is that AS 6 was only certified for the web profile, while in fact practically implementing the full profile. The similarities don't end there, as just as with Java EE 6 the eventual production version (JBoss EAP 6) wasn't based on JBoss AS 6.x, but on the major new version JBoss AS 7. This time around, again it strongly looks like JBoss EAP 7 will not be based on JBoss WildFly 8.x, but on the major new version JBoss WildFly 9.

If history is anything to go by, we may see one or two additional Java EE 7 implementations in a few months, while after a little more than a year from now most servers should be available in a Java EE 7 flavor. At the moment of writing it looks like Web Profile implementations TomEE 2.0 and Liberty.next (both are actually Web Profile++ or a Full Profile--) indeed aren't that far away.

Arjan Tijms

Sunday, September 14, 2014

Getting the target of value expressions

In the Java EE platform programmers have a way to reference values in beans via textual expressions. These textual expressions are then compiled by the implementation (via the Expression Language, AKA EL spec) to instances of ValueExpression.

E.g. the following EL expression can be used to refer to the named bean "foo" and its property "bar":

#{foo.bar}

Expressions can be chains of arbitrary length, and can include method calls as well. E.g.:

#{foo.bar(1).kaz.zak(test)}

An important aspect of these expressions is that they are highly contextual, specifically where it concerns the top level variables. These consists of the object that starts the chain ("foo" here) and any EL variables used as method arguments ("test" here). Because of this, it's not a totally unknown requirement for wanting to resolve the expression when it's still in context in order to obtain the so-called final base and the final property/method, the last one including the resolved and bound parameters.

Now the EL API does provide a method to get the final base and property of an expression if there is one, but this one unfortunately only supports properties, not methods. When method invocations were introduced in EL 2.2 for usage in ValueExpressions and chains (which is subtly different from a MethodExpression that existed before that) this seems to have been done in the most minimal way. As a result, a lot of JavaDoc and supporting APIs were seemingly not updated.

For instance, the JavaDoc for ValueExpression still says:

For any of the five methods, the ELResolver.getValue[...] method is used to resolve all properties up to but excluding the last one. This provides the base object.
There is no mention here that ELResolver.invoke is used as well if any of the intermediate nodes in the chain is a method invocation (like bar(1) in #{foo.bar(1).kaz.zak(test)}).

The fact that there's a ValueReference only supporting properties and no corresponding MethodReference is extra curious, since method invocations in chains and ValueExpressions and the ValueReference type were both introduced in EL 2.2.

So is there any hope of getting the final base and method if a ValueExpression happens to be pointing to a method? There appears to be a way, but it's a little tricky. The trick in question consists of using a special tracing ELResolver and taking advantage of the fact that some methods on ValueExpression are specified to resolve the expression "up to but excluding the last [node]". Using this we can use the following approach:

  • Instantiate an EL context which contains the special tracing EL resolver
  • Call a method on the ValueExpression that resolves the chain until the next to last node (e.g. getType()) using the special EL context
  • In the tracing EL resolver count each intermediate call, so when getType() returns the length of the chain is known
  • Call a method on the ValueExpression that resolves the entire chain (e.g. getValue()) using the same special EL context instance
  • When the EL resolver reaches the next to last node (determined by counting intermediate calls again), wrap the return value from ElResolver.getValue or ElResolver.invoke
  • If either ElResolver.getValue or ElResolver.invoke is called again later with our special wrapped type, we know this is the final node and can collect all details that we need; the base, property or method name and the resolved method parameters (if any). All of these are simply passed to us by the EL implementation
The return value wrapping of the next to last node (at call count N) may need some extra explanation. After all, why not just wait till we're called the Nth + 1 time? The issue is that this Nth + 1 call may be for resolving variables that are passed as parameters into the final node if this final node is a method invocation. The amount of such parameters is unknown and each parameter can consist of a chain of arbitrary length.

E.g. consider the following expression:

#{foo.bar.kaz(test.a.b.c(x.r), bean.x.y.z(o).p)}
In such a case the first pass of the above given approach will count the calls up until the point of resolving "bar", which is thus at call count N. If "kaz" was a simple property, our EL resolver would be asked to resolve [return value of "bar"]."kaz" at call count N + 1. However, since "kaz" is not a simple property but a complex method invocation with EL variables, the next call after N will be for resolving the base of the first EL variable used in the method invocation ("test" here).

One may also wonder why we do not "simply" get the textual EL representation of an EL expression, chop off the last node using simple string manipulation and resolve that. The reason is two fold. First, it may work for very simple expressions (like #{a.b.c}), but doesn't work in general for complex ones (e.g. #{empty foo? a.b.c : x.y.z}). A second issue is that a given ValueExpression instance all too often contains state (like an embedded VariableMapper instance), which is lost when we just get the EL string from a ValueExpression and evaluate that.

The approach outlined above has been implemented in OmniFaces 2.0. For completeness the most important bit of it, the tracing EL resolver is given below:

class InspectorElResolver extends ELResolverWrapper {

  private int passOneCallCount;
  private int passTwoCallCount;

  private Object lastBase;
  private Object lastProperty; // Method name in case VE referenced a method, otherwise property name
  private Object[] lastParams; // Actual parameters supplied to a method (if any)

  private boolean subchainResolving;

  // Marker holder via which we can track our last base. This should become
  // the last base in a next iteration. This is needed because if the very last property is a
  // method node with a variable, we can't track resolving that variable anymore since it will not have been processed by the
  // getType() call of the first pass.
  // E.g. a.b.c(var.foo())
  private FinalBaseHolder finalBaseHolder;

  private InspectorPass pass = InspectorPass.PASS1_FIND_NEXT_TO_LAST_NODE;

  public InspectorElResolver(ELResolver elResolver) {
    super(elResolver);
  }

  @Override
  public Object getValue(ELContext context, Object base, Object property) {

    if (base instanceof FinalBaseHolder) {
      // If we get called with a FinalBaseHolder, which was set in the next to last node,
      // we know we're done and can set the base and property as the final ones.
      lastBase = ((FinalBaseHolder) base).getBase();
      lastProperty = property;

      context.setPropertyResolved(true);
      return ValueExpressionType.PROPERTY;
    }

      checkSubchainStarted(base);

      if (subchainResolving) {
          return super.getValue(context, base, property);
      }

      recordCall(base, property);

      return wrapOutcomeIfNeeded(super.getValue(context, base, property));
  }

  @Override
  public Object invoke(ELContext context, Object base, Object method, Class<?>[] paramTypes, Object[] params) {

    if (base instanceof FinalBaseHolder) {
      // If we get called with a FinalBaseHolder, which was set in the next to last node,
      // we know we're done and can set the base, method and params as the final ones.
      lastBase = ((FinalBaseHolder) base).getBase();
      lastProperty = method;
      lastParams = params;

      context.setPropertyResolved(true);
      return ValueExpressionType.METHOD;
    }

    checkSubchainStarted(base);

    if (subchainResolving) {
      return super.invoke(context, base, method, paramTypes, params);
    }

    recordCall(base, method);

    return wrapOutcomeIfNeeded(super.invoke(context, base, method, paramTypes, params));
  }

  @Override
  public Class<?> getType(ELContext context, Object base, Object property) {

    // getType is only called on the last element in the chain (if the EL
    // implementation actually calls this, which might not be the case if the
    // value expression references a method)
    //
    // We thus do know the size of the chain now, and the "lastBase" and "lastProperty"
    // that were set *before* this call are the next to last now.
    //
    // Alternatively, this method is NOT called by the EL implementation, but then
    // "lastBase" and "lastProperty" are still the next to last.
    //
    // Independent of what the EL implementation does, "passOneCallCount" should thus represent
    // the total size of the call chain minus 1. We use this in pass two to capture the
    // final base, property/method and optionally parameters.

    context.setPropertyResolved(true);

    // Special value to signal that getType() has actually been called (this value is
    // not used by the algorithm now, but may be useful when debugging)
    return InspectorElContext.class;
  }

  private boolean isAtNextToLastNode() {
    return passTwoCallCount == passOneCallCount;
  }

  private void checkSubchainStarted(Object base) {
    if (pass == InspectorPass.PASS2_FIND_FINAL_NODE && base == null && isAtNextToLastNode()) {
        // If "base" is null it means a new chain is being resolved.
          // The main expression chain likely has ended with a method that has one or more EL variables
        // as parameters that now need to be resolved.
        // E.g. a.b().c.d(var1)
        subchainResolving = true;
        }
  }

  private void recordCall(Object base, Object property) {

    switch (pass) {
      case PASS1_FIND_NEXT_TO_LAST_NODE:

        // In the first "find next to last" pass, we'll be collecting the next to last element
        // in an expression.
        // E.g. given the expression a.b().c.d, we'll end up with the base returned by b() and "c" as
        // the last property.

        passOneCallCount++;
        lastBase = base;
        lastProperty = property;

        break;

      case PASS2_FIND_FINAL_NODE:

        // In the second "find final node" pass, we'll collecting the final node
        // in an expression. We need to take care that we're not actually calling / invoking
        // that last element as it may have a side-effect that the user doesn't want to happen
        // twice (like storing something in a DB etc).

        passTwoCallCount++;

        if (passTwoCallCount == passOneCallCount) {

          // We're at the same call count as the first phase ended with.
          // If the chain has resolved the same, we should be dealing with the same base and property now

          if (base != lastBase || property != lastProperty) {
            throw new IllegalStateException(
              "First and second pass of resolver at call #" + passTwoCallCount +
              " resolved to different base or property.");
          }

        }

        break;
    }
  }

  private Object wrapOutcomeIfNeeded(Object outcome) {
    if (pass == InspectorPass.PASS2_FIND_FINAL_NODE && finalBaseHolder == null && isAtNextToLastNode()) {
      // We're at the second pass and at the next to last node in the expression chain.
      // "outcome" which we have just resolved should thus represent our final base.

      // Wrap our final base in a special class that we can recognize when the EL implementation
      // invokes this resolver later again with it.
      finalBaseHolder = new FinalBaseHolder(outcome);
      return finalBaseHolder;
    }

    return outcome;
  }

  public InspectorPass getPass() {
    return pass;
  }

  public void setPass(InspectorPass pass) {
    this.pass = pass;
  }

  public Object getBase() {
    return lastBase;
  }

  public Object getProperty() {
    return lastProperty;
  }

  public Object[] getParams() {
    return lastParams;
  }

}

As seen, the support for ValueExpressions that point to methods is not optimal in the current EL specification. With some efforts we can workaround this, but arguably such functionality should be present in the specification itself.

Arjan Tijms

Thursday, August 7, 2014

High time to standardize Java EE converters?

A common task when developing Java EE applications is that of converting data. In JSF we convert objects to a string representation for rendering it inside an (HTML) response, and convert it back to an object after a postback. In JPA we convert objects from and to types known by our database, in JAX-RS we convert request parameters strings into objects etc etc.

So given the pervasiveness of this task, is there any common converter type or mechanism in the Java EE platform?

Unfortunately it appears such a common converter type is not there. While rather similar in nature, many specs in Java EE define their very own converter type. Below we take a look at the various converter types that are currently in use by the platform.

JSF

One of the earlier converter types in the Java EE platform is contributed by JSF. This converter type is able to convert from String to Object and the other way around. Because it pre-dates Java SE 5 it doesn't use a generic type parameter. While its name and methods are very general, the signature of both methods takes two JSF specific types. These specific types however are rarely if ever needed for the actual conversion, but are typically used to provide feedback to the user after validation has failed.

The main API class looks as follows:

public interface Converter {
    Object getAsObject(FacesContext context, UIComponent component, String value);
    String getAsString(FacesContext context, UIComponent component, Object value);
}
See: javax.faces.convert.Converter

JAX-RS

JAX-RS too defines its very own converter type; ParamConverter. Just like the JSF Converter it's able to convert from a String to any Java Object, but this time there is a generic type parameter in the interface. There's also a method defined to convert the Object back into a String, but this one is curiously reserved for future use.

The main API class looks as follows:

public interface ParamConverter<T> {
    T fromString(String value);
    String toString(T value);
}
See: javax.w.rs.ext.ParamConverter

JPA

One of the most flexible converters in terms of its interface is the JPA converter AttributeConverter. This one is able to convert between any two types in both directions as denoted by 2 generic type parameters. The naming of the converter methods are very specific though.

The main API class looks as follows:

public interface AttributeConverter<X,Y> {
    Y convertToDatabaseColumn (X attribute);
    X convertToEntityAttribute (Y dbData);
}
See: javax.persistence.AttributeConverter

WebSocket

WebSocket has its own converter as well. Architecturally they are a bit different. In contrast with the above shown converters, WebSocket defines separate interfaces for both directions of the conversion whereas the other specs just put two methods in the same type. WebSocket also defines separate interfaces for one of the several supported target types, whereas the other converters support either String or an Object/generic type parameter.

The two supported target types are String and ByteBuffer, with each having a variant where the converter doesn't provide the converted value via a return value, but writes it to a Writer instance that's passed into the converter method as an extra parameter.

Another thing that sets the WebSocket converters apart from the other Java EE converters is that instances have an init and destroy method and are guaranteed to be used by one thread at a time only.

The String to Object API classes look as follows:

public static interface Decoder.Text<T> extends Decoder {
    T decode(String s) throws DecodeException;
    boolean willDecode(String s);
}
public static interface Encoder.Text<T> extends Encoder {
    String encode(T object) throws EncodeException;
}
See: javax.websocket.Decoder.Text See: javax.websocket.Encoder.Text

PropertyEditor (Java SE)

Java SE actually has a universal converter API as well, namely the PropertyEditor. This API converts Objects from and to String, just as the JSF converter. As demonstrated before this type of converter is often used in Java EE code as well.

A PropertyEditor converter is almost always registered globally and inherently stateful. You first set a source value on an instance and then call another method to get the converted value. Remarkable for this converter type is that it contains lots of unrelated methods, including a method specific for painting in an AWT environment: paintValue(Graphics gfx, Rectangle box). This highly unfocused set of functionality makes the PropertyEditor a less than ideal converter for general usage, but in most cases the nonsense methods can simply be ignored and the ubiquitous availability in Java SE is of course a big plus.

The main API class and main conversion methods look as follows:

public interface PropertyEditor {
    // String to Object
    void setAsText(String text) throws IllegalArgumentException;
    Object getValue();
    // Object to String
    void setValue(Object value);
    String getAsText();
    // Tons of other useless methods omitted
}
See: java.beans.PropertyEditor

Other

There are some specs that use a more implicit notion of conversion and could take advantage of a platform conversion API if there happened to be one. This includes remote EJB and JMS. Both are capable of transferring objects in binary form using what is essentially also a kind of conversion API: Java SE serialization. Finally JAXB has a number of converters as well, but they are build in and only defined for a finite amount of types.

Conclusion

We've seen that there are quite a number of APIs available in Java EE as well as Java SE that deal with conversion. The APIs we looked at differ somewhat in capabilities, and use different terminology for what are essentially similar concepts. The platform as a whole would certainly benefit from having a single unified conversion API; this could eventually somewhat reduce the size of individual specs, makes it easier to have a library of converters available and would surely give the Java EE platform a more consistent feel.

Arjan Tijms

Tuesday, July 22, 2014

JSF 2.3 wish list part I - Components

Over the last days several Java EE specs have published JSR proposals for their next versions. Today JSF published its proposal on the JSF mailing list, titled: Let's get started on JSF 2.3

The JSR groups improvements into 4 categories:

  • Small scale new features
  • Community driven improvements
  • Platform integration
  • Action oriented MVC support

An interesting aspect is the "Community driven improvements", which means it's basically up to the community what will be done exactly. In practice this mostly boils down to issues that have been entered into the JSF issue tracker. It's remarkable how many community filed issues JSF has compared to several other Java EE specs; clearly JSF always has been a spec that's very much community driven. At ZEEF.com we're more than happy to take advantage of this opportunity and contribute whatever we can to JSF 2.3.

Taking a look at this existing issue tracker we see there are quite a lot of ideas indeed. So what should the community driven improvements focus on? Improving JSF's core strengths further, adding more features, incorporating ideas of other frameworks, clarifying/fixing edge cases, performance? All in all there's quite a lot that can be done, but there's as always only a limited amount of resources available so choices have to be made.

One thing that JSF has been working towards is pushing away functionality that became available in the larger Java EE platform, therefore positioning itself more as the default MVC framework in Java EE and less as an out of the box standalone MVC framework for Tomcat et al. Examples are ditching its own managed bean model, its own DI system, and its own expression language. Pushing away these concerns means more of the available resources can be spend on things that are truly unique to JSF.

Important key areas of JSF for which there are currently more than a few issues in the tracker are the following:

  • Components
  • Component iteration
  • State
  • AJAX

In this article I'll look at the issues related to components. The other key areas will be investigated in follow-up articles.

Components

While JSF is about more than just components, and it's certainly not idiomatic JSF to have a page consisting solely out of components, arguably the component model is still one of JSF's most defining features. Historically components were curiously tedious to create in JSF, but in current versions creating a basic component is pretty straightforward.

The simplification efforts should however not stop here as there's still more to be done. As shown in the above reference, there's e.g. still the required family override, which for most simple use cases doesn't make much sense to provide. This is captured by the following existing issues:

A more profound task is to increase the usage of annotations for components in order to make a more "component centric" programming model possible. This means that the programmer works more from the point of view of a component, and threats the component as a more "active thing" instead of something that's passively defined in XML files and assembled by the framework.

For this at least the component's attributes should be declared via annotations, making it no longer "required" to do a tedious registration of those in a taglib.xml file. Note that this registration is currently not technically required, but without it tools like a Facelet editor won't be able to do any autocompletion, so in practice people mostly define them anyway.

Besides simply mimicking the limited expressiveness for declaring attributes that's now available in taglib.xml files, some additional features would be really welcome. E.g. the ability to declare whether an attribute is required, its range of valid values and more advanced things like declaring that an attribute is an "output variable" (like the "var" attribute of a data table).

A nonsensical component to illustrate some of the ideas:

@FacesComponent
public class CustomComponent extends UIComponentBase {
 
    @Attribute
    @NotNull
    private ComponentAttribute<String> value;

    @Attribute
    @Min(3)
    private int dots;

    @Attribute(type=out)
    @RequestScoped
    private String var;
    
    private String theDots;

    @PostConstruct
    public void init() {
        theDots = createDots(dots);
    }
    
 
    @Override
    public void encodeBegin(FacesContext context) throws IOException { 
        ResponseWriter writer = context.getResponseWriter();
        writer.write(value.getValue().toUpperCase());
        writer.write(theDots);

        if (var != null) {
            getRequestParameterMap().put(var, theDots);
        }
    }
}
In the above example there are 4 instance variables, of which 3 are component attributes and marked with @Attribute. These last 3 could be recognized by tooling to perform auto-completion in e.g. tags associated with this component. Constraints on the attributes could be expressed via Bean Validation, which can then partially be processed by tooling as well.

Attribute value in the example above has as type ComponentAttribute, which could be a relatively simple wrapper around a component's existing attributes collection (obtainable via getAttributes()). The reason this should not directly be a String is that it can now be transparently backed by a deferred value expression (a binding that is lazily resolved when its value is obtained). Types like ComponentAttribute shouldn't be required when the component designer only wants to accept literals or immediate expressions. We see this happening for the dots and var attributes.

Finally, the example does away with declaring an explicit name for the component. In a fully annotation centric workflow a component name (which is typically used to refer to it in XML files) doesn't have as much use. A default name (e.g. the fully qualified class name, which is what we always use in OmniFaces for components anyway) would probably be best.

This is captured by the following issues:

Creating components is one thing, but the ease with which existing components can be customized is just as important or perhaps even more important. With all the moving parts that components had in the past this was never really simple. With components themselves being simplified, customizing existing ones could be simplified as well but here too there's more to be done. For instance, often times a user only knows a component by its tag and sees this as the entry point to override something. Internally however there's still the component name, the component class, the renderer name and the renderer class. Either of these can be problematic, but particularly the renderer class can be difficult to obtain.

E.g. suppose the user did get as far as finding out that <h:outputText> is the tag for component javax.faces.component.html.HtmlOutputText. This however uses a renderer named javax.faces.Text as shown by the following code fragment:

public class HtmlOutputText extends javax.faces.component.UIOutput {

    public HtmlOutputText() {
        setRendererType("javax.faces.Text");
    }

    public static final String COMPONENT_TYPE = "javax.faces.HtmlOutputText";
How does the user find out which renderer is associated with javax.faces.Text? And why is the component name javax.faces.HtmlOutputText as opposed to its fully qualified classname javax.faces.component.html.HtmlOutputText? To make matters somewhat worse, when we want to override the renderer of an component but keep its existing tag, we also have the find out the render-kit-id. (it's a question whether the advantages that all these indirection names offer really outweigh the extra complexity users have to deal with)

For creating components we can if we want ignore these things, but if we customize an existing component we often can't. Tooling may help us to discover those names, but in absence of such tools and/or to reduce our dependencies on them JSF could optionally just let us specify more visible things like the tag name instead.

This is captured by the following issues:

Although strictly speaking not part of the component model itself, one other issue is the ease with which a set of components can easily be grouped together. There's the composite component for that, but this has as a side-effect that a new component is created that has the set of components as its children. This doesn't work for those situations where the group of components is to be put inside a parent component that does something directly based on its children (like h:panelGrid). There's the Facelet tag for this, but it still has the somewhat old fashioned requirements of XML registrations. JSF could simplify this by giving a Facelet tag the same conveniences as were introduced for composite components. Another option might be the introduction of some kind of visit hint, via which things like a panel grid could be requested to look at the children of some component instead of that component itself. This could be handy to give composite components some of the power for which a Facelet tag is needed now.

This is partially captured by the following issues:

Finally there's an assortment of other issues on the tracker that aim to simplify working with components or make the model more powerful. For instance there's still come confusion about the encodeBegin(), encodeChildren(), encodeEnd() methods vs the newer encodeAll() in UIComponent. Also, dynamic manipulation of the component tree (fairly typical in various other frameworks that have a component or element tree) is still not entirely clear. As it appears, such modification is safe to do during the preRenderView event, but this fact is not immediately obvious and the current 2.2 spec doesn't mention it. Furthermore even if it's clear for someone that manipulation has to happen during this event, then the code to register for this event and handle it is still a bit tedious (see previous link).

Something annotation based may again be used to simplify matters, e.g.:

@FacesComponent
public class CustomComponent extends UIComponentBase {
 
    @PreRenderView // or @ModifyComponentTree as alias event
    public void init() {
        this.getParent().getChildren().add(...);
    }
}

This and some other things are captured by the following issues:

That's it for now. Stay tuned for the next part that will take a look at component iteration (UIData, UIRepeat, etc).

Arjan Tijms