Add OAuth to your web application with Apache Camel

OAuth is an open protocol to allow secure API authorization from desktop and web applications. Google, for example, already supports OAuth for authorizing 3rd-party applications to access Google services on behalf of a user.

Recently, I added the gauth component to Apache Camel. You can use it to implement OAuth consumer functionality for any web application with only a few lines of code. gauth endpoints take care of exchanging authorization and access tokens between a web application and an OAuth service provider. At the moment, the gauth component can be used to interact with Google's OAuth services, later versions will support other OAuth providers as well.

From a user's perspective, an example OAuth scenario might look as follows:

• The user logs into a web application that uses the Google Calendar API, for example.
• To authorize access, the user is redirected to a Google Accounts authorization page where access for the requesting web application can be granted or denied.
• After granting access the user is redirected back to the web application and the web application can now access the user's calendar data.
  
• The user can revoke access at any time within Google Accounts.
  

To implement that scenario with Apache Camel, two routes are needed. The first route obtains an unauthorized request token from Google and then redirects the user to the Google Accounts authorization page:

String encodedCallback = URLEncoder.encode(
 "https://example.org/handler", "UTF-8");
String encodedScope = URLEncoder.encode(
 "http://www.google.com/calendar/feeds/", "UTF-8");

from("jetty:http://0.0.0.0:8080/authorize")
.to("gauth://authorize"
 + "?callback=" + encodedCallback
 + "&scope=" + encodedScope);

In this example, the authorization request is triggered by the user by sending a GET request to http://example.org/authorize (e.g. by clicking a link in the browser). The gauth://authorize endpoint then obtains an unauthorized request token from Google. The scope parameter in the endpoint URI defines which Google service the web application wants to access. After having obtained the token, the endpoint generates a redirect response (302) which redirects the user to the Google Accounts authorization page. After granting access, the user is redirected back to the web application (callback parameter). The callback now contains an authorized request token that must finally be upgraded to an access token. Handling the callback and upgrading to an access token is done in the second route.

from("jetty:https://example.org/handler")
.to("gauth://upgrade")
.to(new StoreTokenProcessor())

The jetty endpoint receives the callback from Google. The gauth://upgrade endpoint takes the authorized request token from the callback and upgrades it to an access token. The route finally stores the long-lived access token for the current user. The next time the user logs into the web application, the access token is already available and the application can continue to access the user's Google Calendar data without needing further user interaction. The user can invalidate the access token at any time within Google Accounts.

Only these two routes are needed to integrate with Google's OAuth provider services. The routes can perfectly co-exist with any other web application framework. Whereas the web framework provides the basis for web application-specific functionality, the OAuth service provider integration is done with Apache Camel. This approach allows for a clean separation of integration logic from application or domain logic.

For handling OAuth requests, web applications can also use other components than Camel's jetty component, such as the servlet component. For adding OAuth to Google App Engine applications, the jetty component needs to be replaced with Camel's ghttp component. Here's an example:

String encodedCallback = URLEncoder.encode(
 "https://camelcloud.appspot.com/handler", "UTF-8");
String encodedScope = URLEncoder.encode(
 "http://www.google.com/calendar/feeds/", "UTF-8");

from("ghttp:///authorize")
.to("gauth://authorize"
 + "?callback=" + encodedCallback
 + "&scope=" + encodedScope);

from("ghttp:///handler")
.to("gauth://upgrade")
.to(new StoreTokenProcessor())

The following figure gives an overview how the OAuth sequence of interactions relate to the gauth://authorize and gauth://upgrade endpoints.



Accessing a Google service with an access token (step 9) is application-specific and not covered by the gauth component. To get access to a user's Google Calendar data with an access token, one could use the GData client library. The gauth component documentation contains an example.

The gauth component is the first step towards a broader support of security standards such as OAuth and OpenID in Apache Camel. I'm currently thinking of the following extensions

• A Camel OpenID component
• A Camel OpenID/OAuth hybrid component
• Support OAuth providers other than Google

The gauth component is currently part of the Camel 2.3 development snapshot (sources).

Accessing a security-enabled Google App Engine service from a Java client

After a rather long search on Google pages and forums I could only find fragmented information how to programmatically access a Google App Engine service that requires users to authenticate. In this blog post I'm going to summarize my findings for a Java client application.

With programmatic access I mean that the user doesn't need to enter username and password into a login form created by Google but rather into an installed client application and the client coordinates the authentication and authorization process programmatically. The mechanism used here is the ClientLogin for installed applications.

The first step is to obtain an authentication token from the Google Accounts API. The easiest way to do that is with the GData client library for Java.

import java.net.URLEncoder;

import com.google.gdata.client.GoogleAuthTokenFactory;
import com.google.gdata.util.AuthenticationException;

public class AuthExample {

  public static void main(String[] args) throws Exception {

    String username = "myusername@gmail.com";
    String password = "mypassword";
    String serviceName = "ah";

    GoogleAuthTokenFactory factory = new GoogleAuthTokenFactory(serviceName, "", null);
    // Obtain authentication token from Google Accounts
    String token = factory.getAuthToken(username, password, null, null, serviceName, "");

    ...
  }
}

One has to provide username an password and the name of the Google service that should be accessed. For Google App Engine the service name is always ah, regardless of the name of the deployed application. The next step is to do a login at Google App Engine. The login URL is https://example.appspot.com/_ah/login?continue=https%3A%2F%2Fexample.appspot.com%2Fexample&auth=DQAAAJc...qNUA8. The continue query parameter instructs the login service where to rederict after successful login. In this example the redirect goes to https://example.appspot.com/example. The auth query parameter contains the authentication token obtained before.

import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;

public class AuthExample {

  public static void main(String[] args) throws Exception {
    ...

    String token = ...
    String serviceUrl = "https://example.appspot.com/example";
    String loginUrl = "https://example.appspot.com/_ah/login?continue=" +
    URLEncoder.encode(serviceUrl, "UTF-8") + "&auth=" + token;

    HttpClient httpclient = new DefaultHttpClient();
    HttpGet httpget = new HttpGet(loginUrl);
    HttpResponse response = httpclient.execute(httpget);
    // process response
    // ...

    httpclient.getConnectionManager().shutdown();
  }
}

When the login service sends a redirect after successful login, it also returns a cookie that allows the client to finally access the protected App Engine service at https://example.appspot.com/example. The redirect and cookie handling is done by the httpclient automatically. For the duration of the session the protected App Engine service can be accessed with that cookie.

Update: If the service expects POST requests instead of GET requests then an automated redirect is not an option. In this case, redirect must be disabled for the for the httpclient and a POST request to the serviceUrl must be created manually. Also, the authorization cookie must be set explicitly.

import org.apache.http.Header;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.client.params.ClientPNames;
import org.apache.http.impl.client.DefaultHttpClient;

public class AuthExample {

  public static void main(String[] args) throws Exception {
    ...

    String token = ...
    String loginUrl = "https://example.appspot.com/_ah/login?auth=" + token;
    String serviceUrl = "https://example.appspot.com/example";

    HttpClient httpclient = new DefaultHttpClient();
    httpclient.getParams().setBooleanParameter(ClientPNames.HANDLE_REDIRECTS, false);
    HttpGet httpget = new HttpGet(loginUrl);
    HttpResponse response = httpclient.execute(httpget);
    // Get cookie returned from login service
    Header[] headers = response.getHeaders("Set-Cookie");
    httpclient.getConnectionManager().shutdown();   

    httpclient = new DefaultHttpClient();
    HttpPost httppost = new HttpPost(serviceUrl);
    // set cookie returned by login service
    for (Header header : headers) {
      httppost.addHeader("Cookie", header.getValue());
    }
    // set request entity body
    // ...

    response = httpclient.execute(httppost);
    // process response
    // ...

    httpclient.getConnectionManager().shutdown();   
  }
}

Update: Login to a local development server. To get access to a security-enabled application on the local development server there's no need for getting an authentication token. Instead, POST an email address and a redirect URL to http://localhost:<port>/_ah/login and the server returns an authorization cookie. Here's an example:

HttpClient httpClient = new DefaultHttpClient();
httpClient.getParams().setBooleanParameter(
  ClientPNames.HANDLE_REDIRECTS, false);
// POST login data to GAE SDK dev server
HttpPost httpPost = new HttpPost(
  "http://localhost:8888/_ah/login");
httpPost.setHeader("Content-Type",
  "application/x-www-form-urlencoded");
String email = URLEncoder.encode(
  "test@example.com", "UTF-8");
String redirectUrl = URLEncoder.encode(
  "http://localhost:8888", "UTF-8");
httpPost.setEntity(new StringEntity(
  "email=" + email + "&continue=" + redirectUrl));
HttpResponse response = httpClient.execute(httpPost);
// Extract authorization cookie from response
String cookie = response.getFirstHeader("Set-Cookie").getValue();
httpClient.getConnectionManager().shutdown();
// Create a new client and access the secured
// service with the authorization cookie
httpClient = new DefaultHttpClient();
HttpGet httpget = new HttpGet("http://localhost:8888");
httpget.addHeader("Cookie", cookie);
response = httpClient.execute(httpget);
System.out.println(IOUtils.toString(response.getEntity().getContent()));
httpClient.getConnectionManager().shutdown();

New features in grails-jaxrs 0.3

In this blog post I present some new features of the recently released grails-jaxrs 0.3 plugin. A complete list of new features is available in the release notes. A feature overview and links to the complete documentation is on the plugin home page.

grails-jaxrs is a Grails plugin that supports the development of RESTful web services based on the Java API for RESTful Web Services (JSR 311: JAX-RS). It is targeted at developers who want to structure the web service layer of an application in a JSR 311 compatible way but still want to continue to use Grails' powerful features such as GORM, automated XML and JSON marshalling, Grails services, Grails filters and so on. This plugin is an alternative to Grails' built-in mechanism for implementing RESTful web services.

The following example shows how to do content negotiation for Grails domain objects. Grails domain classes like

class Person {
  String firstName
  String lastName
}

can now be used in JAX-RS resource methods directly (e.g. Person parameter in the create method):

import static javax.ws.rs.core.UriBuilder.fromPath

import javax.ws.rs.Consumes
import javax.ws.rs.Path
import javax.ws.rs.Produces
import javax.ws.rs.POST
import javax.ws.rs.core.Response

@Path('/api/person')
@Consumes(['application/xml','application/json'])
@Produces(['application/xml','application/json'])
class PersonCollectionResource {

  @POST
  Response create(Person person) {
      person.save() // use GORM
      URI uri = fromPath(person.id as String).build()
      Response.created(uri).entity(person).build()
  }

  // ...
       
}

Content negotiation and conversion between domain objects and their XML or JSON representations is done by domain object providers. There's no need any more for application code to deal with representation formats directly.

The PersonCollectionResource.create method handles POST requests for creating new Person objects in the database. The method uses GORM to persist the domain object. Clients can send either XML or JSON representations for POSTing person data (see Content-Type header):

POST /hello/api/person HTTP/1.1
Content-Type: application/xml
Accept: application/xml
Host: localhost:8080
Content-Length: 78

<person>
<firstname>Sam</firstname>
<lastname>Hill</lastname>
</person>

or

POST /hello/api/person HTTP/1.1
Content-Type: application/json
Accept: application/json
Host: localhost:8080
Content-Length: 58

{"class":"Person","firstName":"Fabien","lastName":"Barel"}

In either case, the plugin will convert it to a Person object, as required by the person parameter. For creating a response the method uses the JAX-RS API. It first creates a URI for the response Location header and uses the Response builder to set the status code to 201 (created) and the response entity. Note that the method itself doesn't create an XML or JSON representation of the response domain object. This is again done by a domain object provider which uses the Accept request header to determine the response representation format. The responses to the above POST requests are:

HTTP/1.1 201 Created
Content-Type: application/xml
Location: http://localhost:8080/hello/api/person/1
Transfer-Encoding: chunked
Server: Jetty(6.1.14)

<?xml version="1.0" encoding="UTF-8"?>
<person id="1">
<firstname>Sam</firstname>
<lastname>Hill</lastname>
</person>

and

HTTP/1.1 201 Created
Content-Type: application/json
Location: http://localhost:8080/hello/api/person/2
Transfer-Encoding: chunked
Server: Jetty(6.1.14)

{"class":"Person","id":"2","firstName":"Fabien","lastName":"Barel"}

The PersonCollectionResource.create method is even more verbose than necessary. It could equally be written as

import static org.grails.jaxrs.response.Responses.*

@Path('/api/person')
@Consumes(['application/xml','application/json'])
@Produces(['application/xml','application/json'])
class PersonCollectionResource {

  @POST
  Response create(Person person) {
      created person.save()
  }

  // ...
       
}

using helper methods (a mini-DSL) from org.grails.jaxrs.response.Responses. That's exactly the code that is generated when using scaffolding for the Person domain class i.e.

grails generate-resources person

With the grails-jaxrs scaffolding feature, one can generate RESTful service interface for domain objects supporting the HTTP methods POST, GET, PUT and DELETE. A scaffolding example is given in the Scaffolding section of the grails-jaxrs documentation, a walk through the generated code is in the Using GORM section.

By default, grails-jaxrs uses Grail's XML and JSON converters for converting between domain objects and their XML or JSON representations. Applications can easily customize this conversion logic as explained in the Custom entity providers section.

Besides usage of GORM, grails-jaxrs also supports auto-injection of Grails services into JAX-RS resource and provider classes or usage of Grails filters, to mention a few. With version 0.3 the included JAX-RS implementations have been upgraded to their latest versions: Jersey 1.1.4.1 and Restlet 2.0-M6.

Camel components for Google App Engine

The upcoming Apache Camel version 2.1 will include components for connecting to the cloud computing services of Google App Engine. At the moment the following three components are available.

• ghttp: Provides connectivity to the GAE URL fetch service but can also be used to receive messages from servlets
• gtask: Supports asynchronous message processing on GAE by using the task queueing service as message queue.
• gmail: Supports sending of emails via the GAE mail service. Receiving mails is not supported yet but will be added later.

Camel components for the other Google App Engine cloud computing services such as Memcache service, XMPP service, Images service, Datastore Service and the Authentication service are planned.

There's also a tutorial that explains how to develop a non-trivial Camel GAE application using the Camel components for GAE.

From a conceptual point of view, connecting to cloud computing services via Camel components introduces an abstraction-layer that decouples Camel applications from provider-specific cloud service interfaces. Supporting several cloud computing environments in Camel can significantly reduce the burden of migrating Camel applications from one provider to another. The Camel components for Google App Engine are a first step into this direction.

First steps with Apache Camel on Google App Engine

This post describes how to get a simple Camel 2 application running on Google App Engine (GAE). I'll focus on the workarounds and fixes that were necessary to succeed. Please note that the following descriptions are by no means best-practices or recommendations. They only describe my first steps for which better solutions will likely exist in the future. I plan to work on improvements to

• make Camel deployments on GAE easier and to
• allow Camel applications access GAE services via Camel components

For my experiments, I was using a Camel 2.1 development snapshot, the App Engine SDK 1.2.6 and the Google Plugin for Eclipse which makes local testing and remote deployment very easy. The Camel components I used are:

• camel-core
• camel-spring
• camel-servlet
• camel-http

The following snippet shows the route definition of the sample application. It uses the camel-servlet component to receive input via HTTP, converts the HTTP request body to a String, prepends a "Hello " to the body and returns the result.

package example;

import org.apache.camel.builder.RouteBuilder;

public class ExampleRoute extends RouteBuilder {

  @Override
  public void configure() throws Exception {
     from("servlet:/test")
     .convertBodyTo(String.class)
     .transform(constant("Hello ").append(body()));
  }
}

The route doesn't make use of any GAE services (URL fetch, tasks queues, storage, mail ...) Also, message processing is synchronous because GAE doesn't allow applications to create their own threads. For example, using SEDA or JMS queues will not work.

For processing HTTP requests, I created my own servlet class and extended the CamelHttpTransportServlet from the camel-servlet component.

package example;

import org.apache.camel.component.servlet.CamelHttpTransportServlet;
import org.apache.camel.management.JmxSystemPropertyKeys;

public class ExampleServlet extends CamelHttpTransportServlet {

 static {
     System.setProperty(JmxSystemPropertyKeys.DISABLED, "true");
 }

}

The only thing this servlet does is to disable all JMX-related functionality because the GAE JRE doesn't support JMX. All request processing and dispatching is done by the CamelHttpTransportServlet. Configuring the servlet in the web.xml was done as follows.

<servlet>
<servlet-name>CamelServlet</servlet-name>
<servlet-class>example.ExampleServlet</servlet-class>
<init-param>
   <param-name>contextConfigLocation</param-name>
   <param-value>context.xml</param-value>
</init-param>
</servlet>

<servlet-mapping>
<servlet-name>CamelServlet</servlet-name>
<url-pattern>/camel/*</url-pattern>
</servlet-mapping>

The servlet init-param points to the Spring application context that configures the route builder and the Camel context:

<beans xmlns="http://www.springframework.org/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">

<bean id="camelContext"
   class="org.apache.camel.spring.CamelContextFactoryBean">
   <property name="builderRefs">
       <list>
           <ref bean="routeBuilderRef"/>
       </list>
   </property>
</bean>

<bean id="routeBuilderRef"
   class="org.apache.camel.model.RouteBuilderDefinition">
   <constructor-arg value="routeBuilder" />
</bean>

<bean id="routeBuilder"
   class="example.ExampleRoute">
</bean>

</beans>

A severe limitation is that one cannot use the Camel-specific configuration XML schema from the http://camel.apache.org/schema/spring namespace for configuring the Camel context. The problem is that the CamelNamespaceHandler uses JAXB to parse bean definitions which isn't supported by GAE either. One has to fallback to plain old <bean> definitions (POBD?) to configure the Camel context in Spring. Using Spring JavaConfig or something similar would make more sense here but I didn't try it.

Another JAXB-releated problem arises with Camel's Spring DSL. It is also processed with JAXB and therefore cannot be used on GAE.

Going completely without Spring leads to another problem. In this case the CamelContext uses a JndiRegistry by default that depends on javax.naming.InitialContext. This class isn't on the JRE whitelist either. Writing a simple Map-based implementation of org.apache.camel.impl.Registry and configuring the CamelContext with it does the trick.

The last obstacle to get the sample application running was to replace the Camel's UuidGenerator with another one that uses java.util.UUID from the JRE. Camel's original UuidGenerator also uses a class that is not on the JRE whitelist. Since replacement by configuration was not possible, changes to the Camel code base were necessary (patch already submitted).

After deploying the application to GAE and POSTing a request containing "Martin" to http://<appname>.appspot.com/camel/test I was able to send myself greetings. In the URL, <appname> must of course be replaced with the name of an existing application.

IPF 2.0 milestone 2 and IPF Tools milestones released

I'm pleased to announce the following milestone releases from the IPF core project and the IPF Tools project.

IPF 2.0 milestone 2 (release notes)
IPF Tools
- IPF Runtime 2.0 milestone 2 (release notes)
- IPF Manager 2.0 milestone 2 (release notes)
- IPF IDE 1.0 milestone 2 (release notes)

IPF 2.0 milestone 2

This release is feature-equivalent to IPF 1.7.0 but runs on Camel 2.0. Users who plan to upgrade to IPF 2.0 or Camel 2.0 in the near future are highly recommended to use this milestone release. Please note that IPF 2.0 is not backwards-compatible to IPF 1.x. This is mainly due to non-backwards compatible API changes in Camel 2.0. It is therefore important to carefully read the Camel 2.0.0 release notes as well as the IPF 2.0-m2 upgrade notes.

With the release of IPF 2.0-m2 and IPF 1.7.0, IPF 1.x development will go into maintainance mode and new features will be developed on the IPF 2.0 development branch. We leave it open whether to backport selected IPF 2.x features to IPF 1.x. Please add any backport requests to the IPF issue tracker.

Other changes compared to IPF 1.7.0 are:

• The platform manager has been moved to the IPF Tools project.
• The IPF OSGi distributable (IPF runtime) and the IPF OSGi documentation have been moved to the IPF Tools project.
• The HL7-independent parts of the mapping service have been factored out into a new commons-map component.
• The IPF 2.0 documentation has been forked from the IPF 1.7 documentation and revised.

IPF Runtime 2.0 milestone 2

The IPF Runtime is an IPF distribution that is running on the Equinox OSGi platform. It is available as Eclipse plugin or as standalone package. The runtime is used to develop OSGi-based IPF applications.

IPF Manager 2.0 milestone 2

IPF Manager is an Eclipse application for managing IPF services and applications. It is available as Eclipse plugin or as standalone package. In its current state it provides a flow management user interface and a general-purpose JMX client. The IPF Manager is compatible with IPF Runtime 2.0-m2.

IPF IDE 1.0 milestone 2

The IPF IDE supports developers in creating, testing and packaging IPF applications within the Eclipse plugin development environment (PDE) on top of the IPF runtime. The IPF IDE is compatible with IPF Runtime 2.0-m2.

IPF 1.7.0 released

I'm pleased to announce the release of IPF 1.7.0. The main focus of this release was support for clinical standards, in particular the IHE profiles XDS.a, XDS.b, PIX, PDQ and support for the clinical document architecture (CDA) and the Continuity of Care Document (CCD) content profile. The release notes are here.

With IPF's IHE support, IHE actor interfaces can be implemented in IPF routes via URIs. This is as simple as using other Camel or IPF components such as the HTTP or JMS components. The URIs denote individual transactions (ITI) in IHE profiles. For example

from('xds-iti18:myIti18Service')
...

implements the 'XDS Registry Stored Query' service interface of an XDS document registry and can be used from any XDS ITI18-compliant consumer. Such a consumer can also be implemented using the same IPF xds-iti18 component on client side e.g.

...
.to('xds-iti18://somehost:8080/myWebApp/services/myIti18Service')


All the low-level details like communicating with ebXML messages over SOAP etc. is handled by that component. IPF routes deal with easy-to-use object representations of messages exchanged within IHE transactions. The full list of supported transactions is given in the IHE quick reference.

With IPF's CDA and CCD support clinical documents can be created, parsed, rendered and queried/analyzed using a domain-specific language (DSL). This DSL hides away most of the technical details you usually encounter when dealing with the complex XML-representation of clinical documents. In addition to these content-DSL extensions, IPF also provides some route DSL extension for parsing, validating and marshalling CDA documents in IPF routes.

Here's an excerpt of new IPF 1.7.0 features added since 1.6.0

* IHE support
** IHE XDS.a+b transactions (ITI 14-18, 41-43)
** IHE PIX transactions (ITI 8-10)
** IHE PDQ transactions (ITI 21-22)
** IHE ATNA for all the above transactions
* CDA support
** Generic CDA support
** CCD profile support
* Advanced XML processing
** Caching XSLT transmogrifier
** Schematron validator
* Detailed XDS tutorial
* Performance measurement support
* Scheduled flow management database cleanup
* ...

IPF 1.7.0 is based on Camel 1.6. In parallel, a Camel 2.0-based version is developed on the IPF 2.0 branch. The current development snapshot is feature-equivalent with IPF 1.7.0 but runs on Camel 2.0. The next IPF 2.0 milestone 2 release will follow within the next one or two weeks. I recommend you to use the 2.0 milestone releases unless you upgrade from an older IPF 1.x release. After releasing IPF 2.0.0, work on IPF 1.x will go into maintainance mode (but we leave it open whether to backport selected IPF 2.x features).

Exciting new features in IPF 2.0 which didn't make it into IPF 1.7 are Eclipse-based IPF development tools and improvements to IPF's OSGi support. These features are developed and documented in a separate IPF Tools project. The Eclipse-based IPF management client has been moved to this project as well. I'll give a more detailed IPF 2.0 overview in a separate post.

Many thanks to the whole development team and contributors for their excellent and high-quality work!