Building Your Own Presentation Server

Trojan_Room_coffee_pot_xcoffee I gave a presentation recently on the topic of mobile enabling enterprise applications. The context was that if you have a Java EE based application tucked away somewhere in your organization how do you make that application mobile and HTML5 friendly.

As I was creating the presentation, it occurred to me that perhaps my presentation should be like a mini demo of sorts to illustrate some of the principles I was sprouting. I really liked the idea of the Keynote Remote app (which is no longer available) or something equivalent like Slideshow Remote.

So I wrote a simple presentation server that will allow me to control the presentation from my mobile phone. Instead of using Bluetooth or WIFI, my remote control will rely on the web. The following diagram shows how the overall architecture of the presentation server

prezo_2

  1. You start a presentation (the left side of the diagram above) by selecting the presentation from a browser. Yes the presentation is done on a browser like Google Presentation or Prezi.
  2. You then use your mobile phone to open the presenter’s console (right side of the diagram) to control the presentation. The console allows you to advance the slides; it also shows you speaker’s note.
  3. Participants can asked question using the comment tool. Again this is a web application. All comments are displayed on the presentation. The yellow box on the top left corner of the presentation is an example of a comment.
  4. After answering the question, the presenter can dismiss the comment by clicking on the Delete button on the presenter’s console

For the impatient, the source is available here.

Starting A Presentation

Now that we have describe the main features of the presentation server, lets see how this whole shebang works. Every presentation must be started/loaded using the following URL

http://myserver:8080/notppt/presentation/presentation.html

The presentation.html is the name of the presentation file found in the document root of a Java EE web application. The presentation file is a HTML presentation developed using Reveal.js. (more later). Presentations must be loaded from the path /presentation/; this is mapped to the following Servlet

@WebServlet("/presentation/*")
public class PresentationServlet extends HttpServlet {

   @EJB PresentationMap map; 

   protected void doGet(HttpServletRequest req, HttpServletResponse resp) {
      //Get the presentation file name
      String presoFile = req.getPathInfo();
      String uuid = ...//Generate a presentation id
      //Load the presentation file, ctx is ServletContext
      InputStream is = ctx.getResourceAsStream(presoFile);
      String baseURI = req.getScheme() + "://"
            + req.getServerName() + ":"
            + req.getServerPort() + ctx.getContextPath();
      //Read it into memory as DOM
      Document doc = Jsoup.parse(is, "UTF", baseURI);
      //Register it
      map.create(uuid, doc);
      //Load the presenation using a redirect
      resp.sendRedirect(url + "?pid=" + uuid);
   }
}

As you can see the PresentationServlet sets up the initial data structure for the presentation. It generates a unique id for it. After generating the id, it loads it into memory as a DOM-like object. The ‘DOM’ is then associated with the generated id and saved in a map which in here is represented very simplistically by a Singleton EJB. We now get the browser to load the presentation by sending the following URL back as a redirect with the presentation id (pid).

http://myserver:8080/notppt/presentation.html?pid=12345

As I was working on this, I found a really nice HTML parser call Jsoup. Jsoup is a Java HTML parser that lets you access the HTML document using CSS selectors, just like jQuery. Bye bye XPath! We use Jsoup to parses our HTML presentation; it returns returns a Document object which we will  use later to extract the presenter’s note. To learn about Jsoup, see the tutorial.

Presentation Client

The presentation is a HTML presentation developed using the Reveal.js framework. See this link on how to develop presentation with Reveal.js. To add speaker’s note into the slide, embed <div> (one for each point) inside an <aside> tag with a data-notes attribute like so

<section>
   <h1>This is my presentation</h1>
   <aside data-notes=””>
      <div>Speaker notes</div>
      <div>One div for each point</div>
   </aside>
</section>

You can look at presentation.html in the source for a complete example. Every presentation loads a jQuery script call presentation.js. The script performs the following 2 things

  1. Initializes the presentation using Reveal’s API. See the documentation here.
  2. Once Reveal have been initialized, it opens a server sent event connection back to the presentation server. The presentation uses this channel to listen to commands (eg. next slide, previous slide) and control the presentation according to the received commands.

The following is a simplified outline of the above 2 steps

$(function() {
   //Reads the presentation id from the URL
   var pid = getParameterByName("pid");
   //When Reveal has completed its initialization
   $(Reveal).on("ready", function() {
      //Create a SSE, only listen for commands of our id
      var sse = new EventSource("slide-event"); 
      $(sse).on(pid, slideControl);
   });
   ...
});
//Callback to execute when we receive commands
function slideControl(result) {
   //Read the data
   var data = JSON.parse(result.originalEvent.data);
   //If it is a movement command, then move the slide
   if (("next" === data.cmd)|| ("prev" === data.cmd))
      Reveal.slide(data.slide); {
      return;
   }
   //Else process other commands
   ...
}

You can see the entire script in web/app/presentation.js in the source.

Before proceeding any further, lets see how every piece of the system communicates with each other

prezo_3

After the presentation opens, it creates a SSE channel back to the presentation server. The presentation will only listen to commands with its presentation id; it does this by listening for specific SSE packets with its presentation id .

Presentation SSE Channel

When the presentation client displays the presentation in also creates a SSE channel back to the server to listen to commands from the presenter’s console. The SSE’s URL is /slide-event. Since JavaEE does not support SSE, I’ll use this SSE library. You can read more about the SSE library from this blog here and here. Our SSE implementation is a follows

@RequestScoped
@ServerSentEvent("/slide-event")
public class SlideNotificationChannel extends ServerSentEventHandler {
   public void publishEvent(final String id, final JsonObject obj) {
      ServerSentEventData data = new ServerSentEventData();
      data.event(id).data(obj.toString());
      //connection is inherited from ServerSentEventHandler
      connection.sendMessage(data);
   }
}

We create a SSE data, fill in the event id which is the presentation id and the command (more details of this in Presentation Control section). The command tells the presentation client to move to the next slide, return to previous slide, etc.

The SSE data that we are publishing is associated with an event name (presentation id here). This ensures that only client listening to that id will get the data. This is the line

$(sse).on(pid, slideControl);

in the presentation client (presentation.js) that registers the pid with the SSE’s event id.

Side note: this is not strictly true in this implementation as all presentation clients are connected to the same SSE channel. All presentation client will receive commands regardless of whether its for them; but because they are only listening to their own pid, they will only respond if the SSE’s event id matches their pid. Ideally the SSE channel name should be partition with the presentation id viz. clients should connect to /slide-event/<pid> instead of /slide-event. The current SSE library is not able to parameterize the SSE channel name.

Presenter Console

Lets look at the presenter console. The purpose of the presenter console is to allow the presenter to control the presentation and also to look at the speaker’s note for the current slide.

When you open the presenter’s console, you have to select which presentation you wish to control from a combo box.

The presenter’s console is build with jQuery and jQuery Mobile. The following code shows how commands are sent to the server

$(document).on("pageinit", "#main", function() {
   ...
   $("#next").on("click", function() {
      $.getJSON("control"), {
         "cmd": "next",
         "value": pid, //presentation id
      }).done(function(result) {
         //Display the speaker's note
         var notes = "";
         for (var i in result)
            notes = "* " + result.notes[i] + "\n";
         $("#notes").val(notes);
      });
      ...
   });
});

When you click on the next button (code shown above), the console will issue a command to the server using jQuery’s $.getJSON() API. The following request is made to the server to move to the next slide for presentation 123456.

http://.../control?cmd=next&value=123456

If the server manage to successfully advance the slide, the server will return the new slide’s speaker’s notes. The notes are then extracted from the result and displayed on the console. The following is an example of the response from the server

{
   "cmd": "next", "slide": 2, "totalSlides": 10,
   "notes": [ "speaker note", "First point", "Second point" ]
}

Presentation Control

Commands from the presenter’s console to the presentation server are intercepted and processed by ControlServlet; if you notice the $.getJSON() from above sends the command to control URL. This URL is mapped to ControlServlet

@WebServlet("/control")
public class ControlServlet extends HttpServlet {
   //Get the presentation map
   @EJB PresentationMap map;

   //Get the SSE channel
   @Inject @ServerSentEventContext("/slide-event")
   ServerSentEventHandlerContext<SlideNotificationChannel> slideEvents;

   @Override
   protected void doGet(HttpServletRequest req, HttpServletResponse resp) {
      //Get the command and the pid from the request
      String cmd = req.getParameter("cmd");
      ActivePresentation preso = map.get(req.getParameter("value"));
      JsonObjectBuilder builder = Json.createObjectBuilder();
      switch (cmd) {
         //Handle the next command 
         case "next":
            builder.add("cmd", "next")
               .add("slide": preso.currentSlide())
               .add("totalSlides": preso.totalSlides())
               .add("notes": preso.get())
            break;

                     …
      }

      //Send command back to the console
      Json json = builder.build();
      resp.setStatus(HttpServletResponse.SC_OK);
      resp.setContentType("application/json");
      try (PrintWriter pw = resp.getWriter()) {
         pw.println(json.toString());
      }

      //Broadcast the command to all presentations
      for (SlideNotificationChannel chan: slideEvents.getHandlers())
            chan.publishEvent(preso.getPid(), json);
      }
      ...
   }

The code for ControlServlet is quite straightforward. We start by first constructing the command object in JSON using the new JSON API from JavaEE 7. The notes attribute is the speaker notes for the current slide. The speaker notes are extracted from under <aside data-notes=””> element using Jsoup. Look at the source to see how this is done. They are returned as a JSON array.  After creating the command object, we send the object back to the presenter’s console indicating that the command have been accepted.

Now we have to inform the presentation client to execute the command, which is move the to next slide in this case. Using injection, we get the handler to all SSE channels listening on /slide-event. We then loop through all the connections and publish the event.

The publish event now goes back to the presentation client; the slideControl function is invoked (see Presentation Client section above)  and this advances the slide.

Trying Out the Demo

The source for the presentation server is available here. All the dependencies can be found under lib directory. The presentation server is written to JavaEE 7. I’ve tested in on Glassfish 4.0. Once you’ve deploy the application, point your browser to http://your_server:your_port/notppt. You will see the landing page.

The first item is to start a Start Presentation. You must have at least 1 presentation started. Click on this (open it in a new window) to start your first presentation.

Now you can select either the Control or the Comment link. The Control will take you to the presenter’s console. Try opening this on your mobile phone browser. Select a presentation to control from the combobox. Comment allows the audience to comment or ask questions on the slide.

It is very easy to take the Control and/or the Comment portion and make that into an app (Android or otherwise) by wrapping the web portion only with Cordova/Phonegap.

The presentation server currently has only 1 presentation. If you’re thinking of using it here are some suggestions.

  • Add an interface to to manage the presentations.
  • Automatically inject presentation.js into every uploaded presentation.
  • Add security (Java EE container managed).
  • Some jQuery Mobile gesture like swipe in the presenter console to control the slides instead of buttons
  • Support for other HTML presentation toolkit in particular  impress.js and deck.js

Similar Tools

It’s worth mentioning that Reveal.js do have a presentation console with speaker’s notes. There is another excellent blog on the same subject. Both of these approaches uses node.js as their server.

I also just found out that slid.es, the people behind Reveal.js, have rolled out something similar. See the following video. The idea is basically the same except that they have done it with more panache. Go check it out.

Till next time.

Using JSONP with JAX-RS

JSONP is a format used to work around the same origin policy. As the name implies it is a modification on the JSON format. Instead of returning a JSON object like so

{ "customerId": 123, "name": "fred", "email": "fred@bedrock.com" }

the JSON is return as a method call to a function like so

foo({ "customerId": 123, "name": "fred", "email": "fred@bedrock.com" })

JSONP is typically used in cross origin AJAX calls using the GET method. You can learn more about JSONP from this Stack Overflow discussion. The JSON-P website talks about why and when it is used.

Handling JSONP

We will examine JSONP by first looking at how we use it from the ever popular jQuery. To make an cross origin AJAX call with JSONP we use $.ajax() like so

$.ajax({
   type: "GET",
   url: "webresources/customer/1",
   dataType: 'jsonp' })
.done(function(result) {
   //Do something with the result
   console.log("----> " + JSON.stringify(result));
}).fail(function() {
   //Handle failure
   ...
});

Lets assume that our AJAX call is going to get a customer record. The HTTP request goes across to the server

GET /JSONPTest/webresources/customer/1?callback=jQuery210012143543711863458_1393227259817 HTTP/1.1
Accept: text/javascript, application/javascript, application/ecmascript, application/x-ecmascript, */*; q=0.01

and the response

HTTP/1.1 200 OK
Content-Type: application/javascript
jQuery210012143543711863458_1393227259817({"name":"barney","id":1,"email":barney@gmail.com})

Notice that the callback function name is in the query string of the request.

So to write a JAX-RS resource to handle we need to

  • Handle the HTTP GET method
  • Read the callback parameter in the query string. We will use the name of the function to wrap our result
  • Produce the MIME type in the Accept header

The following JAX-RS resource should handle the JSONP request

@RequestScoped
@Path("/customer/{id}")
public class CustomerResource {

   @PersistenceContext EntityManager em;

   @GET
   @Produces("application/javascript")
   public String get(@PathParam("id") Integer id ,
         @QueryParam("callback") String callback) {

      Customer cust = em.find(Customer.class, id); 
      if (null == cust)
         throw WebApplicationException(Response.Status.NOT_FOUND);

      //Convert object to JSON
      JsonObjectBuilder builder = Json.createObjectBuilder();
      ...
      return (callback + "(" + builder.build().toString() + ")");
   }
}

The code is quite straightforward. We inject the callback query string into the handler which we subsequently use to wrap the result.

As you can see, its quite simple to handle JSONP.

JSONP as MessageBodyWriter

JAX-RS has a pair of SPI call MessageBodyReader and MessageBodyWriter that allows anyone to handle marshalling and unmarshalling of objects from HTTP to JAX-RS resources. I wrote a simple MessageBodyWriter , called JSONP4JAXRS, that simplifies the entire process (if you don’t mind adding 12K to to your project).

If you’re using JSONP4JAXRS, then the above CustomerResource can rewritten as follows

@RequestScoped
@Path("/customer/{id}")
public class CustomerResource {

   @PersistenceContext EntityManager em;

   @GET
   @Produces("application/javascript") 
      public String get(@PathParam("id") Integer id ) {

      Customer cust = em.find(Customer.class, id); 
      if (null == cust)
         throw WebApplicationException(Response.Status.NOT_FOUND);

      //Convert object to JSON
      JsonObjectBuilder builder = Json.createObjectBuilder();
      ...
      return (builder.build()); 
   }
}

No more fiddling around with callback parameter. The MessageBodyWriter will automatically pad the JsonObject with the callback.

But wait! There is more!

JSONP4JAXRS will automatically pad any return types according to the following rules:

Return Type Padding (assume fn is the name of the callback function)
JsonObject fn({ … }) where { … } is the JsonObject
JsonArray fn([ … ]) where [ … ] is the JsonArray
Any primitives including their  wrappers fn({“value”: the_value })
Any object fn({“value”: toString_value_of_object})
Iterable  fn([ … ]) The Iterable is converted to a JsonArray and the elements are converted according the the above rule

Don’t forget to wrap your Iterable (eg. List<JsonObject>) in a GenericEntity if you are returning the result via a Response.

The JSONP4JAXRS contains a class call JsonWithPadding that defines the following constants for JSONP MIME types

Constant Name MIME Type
APPLICATION_JAVASCRIPT application/javascript
APPLICATION_JSON application/json
APPLICATION_ECMASCRIPT application/ecmascript
APPLICATION_X_ECMASCRIPT application/x-ecmascript
TEXT_JAVASCRIPT text/javascript

JSONP4JAXRS works with Java EE 7 only. JSONP4JAXRS can be downloaded here. The source and binaries are in the compile NetBeans project file. The binaries is in dist directory.

Till next time.

Integrating TOTP with Java EE’s Container Managed Security

clip_image002Edited on Monday Febuary 1 2014

After reading the following blog on time-based one time password (TOTP), I wanted to see if I could add an additional layer of security into Java EE’s  security. I installed Google Authenticator on my phone and started experimenting it with Java EE security.

I’ll describe what I did and some of the issues I found. You can get a copy of the source when you reach the end of the blog. Hopefully the journey is as pleasant as the destination ;-)

The blog is broken into the following sections

  1. In the first part, I’ll give a brief intro to Google Authenticator/time based onetime password and how it works
  2. Setup JDBC realm for Glassfish application server in preparation for using it in our form based authentication
  3. Create the login and user registration with JSF Faces Flow
  4. Configure container managed security to use TOTP for login

Time-base One Time Password

So lets start with TOTP. We are all familiar with the concept of password. Password are typically static; they do not change over time; so if someone knows your password then your account is compromised.

In most operating system you can configured a password expiry policy where users are forced to change their password at regular intervals eg. 6 months. This password expiry policy gets around the above mention problem every 6 months. But the risk still exist.

TOPT takes the password expiry policy to the extreme by expiring your password at an even faster rate like say every 30 seconds! The problem here is that you cannot change your password every 30 seconds; so to make this work, TOTP uses an algorithm to change the password for us.

Very briefly this is how the TOTP works: you pass to the TOTP password generator algorithm a secret key and the current time; out pops your password; So if you call the TOTP password generator now and then again 30 seconds later (assuming that your window is 30 seconds) using the same secret key, you will get 2 completely different passwords.

To verify a password generated by TOTP algorithm, the TOTP password verification must know your secret key and the time that the TOTP password is generated. Since its a 30 second expiry period in our example, the password verifier will be able to validate the password as long as it is run inside the same 30 second window when the password is generated. All the wonderful math can be found here. TOTP is an IETF standard RFC6238.

Examples of TOTP can be found in online banking like DBS’s OTP where the account holders are issued dongles that implement the TOTP algorithm along with their unique secret keys embedded in the dongles. 

Another way of using TOTP is to augment it with your regular password as in Google’s 2-Step Verification (the focus of this blog). Google’s 2-Step Verification (G2SV) relies Google Authenticator which replaces the dongle in the above example. You have to first create an account. The Authenticator captures your TOTP details of this account by scanning a QR code. The QR code is created from the following 3 pieces of information

  1. A base 32 encoding of a 10 digit hexadecimal number. Eg the bytes from the string 8-polytope (“8-polytope”.getBytes()) will produce the following base 13 encoding: HAWXA33MPF2G64DF. This is the secret key and is unique to each user
  2. An issuer, typically the name of a company or the issuing authority eg. Yoyodyne Propulsion Systems
  3. A label, which I like to call the subject, is the user’s name or some identifier eg. johnwhorfin

All the information have to be encoded in the otpauth format. If we were to encode the values from above, we will get the following

otpauth://totp/Yoyodyne+Propulsion+Systems%3Ajohnwhorfin%3Fsecret%3DHAWXA33MPF2G64DF%26issuer%3DYoyodyne+Propulsion+Systems

Note that the otpauth string have to be URL safe. We can how use this otpauth to generate a QR using Google’s QR generator. You can find the query string parameters here. So to get the QR code of 350×350 pixel in size, enter the following

https://www.google.com/chart?chs=350x350&cht=qr&chl=otpauth://totp/Yoyodyne+Propulsion+Systems%3Ajohnwhorfin%3Fsecret%3DHAWXA33MPF2G64DF%26issuer%3DYoyodyne+Propulsion+Systems

See the QR in its full glory here.  Of course the entire process of generating the OTP can be automated.

Clearly the secret key in which the entire TOTP scheme hinges is the weakest link. If we minimize the exposure of the secret key (like keeping it from the user, using HTTPS to generate the QR) then we can potentially make using TOPT reasonably secure. Well get to that a little later.

 

Glassfish JDBC Realm

Java EE security realms is a place where username, password and groups are stored. When you configure security for your application you need to specify which security realm you application will be associated with. When a user login to your application, it is from this realm that the application will get the user’s credentials, password and group id. See here for more details.

A JDBC realm is a realm where usernames, passwords, etc are stored in a RDBMS. You first create the required tables in a RDBMS; next you create a connection pool on top of it followed by a JDBC resource.

Since I’m using Glassfish and Java DB/Apache Derby, I’ll be creating and configuring JDBC realm based on these. There are lots of blogs and tutorial on creating JDBC realm for Glassfish so I won’t go into it in great detail. I’ll just list out my configuration.

The following are the table schemas, based on Apache Derby, which I’ve copied from this blog. Adapt it to whatever database you’re using.

create table logintable (
   username varchar(128) NOT NULL CONSTRAINT usernamePK primary key,
   password varchar(128) NOT NULL,
   otp_secret varchar(128) NOT NULL
);

create table grouptable (
   username varchar(128) NOT NULL,
   groupid varchar(128) NOT NULL,
   CONSTRAINT GROUP_PK PRIMARY KEY(username, groupid),
   CONSTRAINT USER_FK FOREIGN KEY(username) REFERENCES logintable(username)
);

I’ve added an extra column in logintable called otp_secret. This column will be used to store our OTP secret key.

Create a JDBC resource to the database that has the above 2 tables; see here if you do not know how. Assume that the JDBC resource name is jdbc/security. Next define a JDBC realm for Glassfish with the following configuration according to the table schemas

Property Name Property Value
JAAS Context jdbc-realm (note: this is the realm’s name)
JNDI jdbc/security
User Table logintable
User Name Column username
Password Column password
Group Table grouptable
Group Table User Name Column username (note: this is the username column in grouptable)
Group Name Column groupid
Password Encryption Algorithm none (note: enter this value)
Digest Algorithm SHA-256 (note: this is important as you’ll need to know what message digest to use)

Login Flow for TOTP

The next thing that we’re going to do is to define a flow for a user to login or to register themselves. Registration is important here because this is where we generate the TOTP secret key. The following diagram shows the authentication and registration process

totp_flow

  • totp-login – this is the login page and the main entry point into this flow. The page need to have the same name as the flow name to indicate that this is the starting page
  • register – if we are a new user, then we click on a button on totp-login to get us here
  • qrcode – after we have successfully registered, the qrcode page will display the QR code for our TOTP which we have to use Google Authenticator to scan. After scanning, we return to totp-login to login
  • error – the error page

totp_flowI’ll be implementing the process using JSF using the new Faces Flows; so the files are created in a directory called totp-login; this incidentally will also be the name of the flow. 

Let start with the registration (register.xhtm) page. Note all JSF pages are at its bare minimum.

<h:form id="form">
   <h:inputText id="username" value="#{newUser.username}"/>
   <h:inputSecret id="password0" value="#{newUser.password}"/>
   <h:inputSecret id="password1" value="#{newUser.passwordAgain}"/>
   <h:commandButton value="Register Me" action="#{newUser.register()}"/>
</h:form>

And this is the backing bean for register.xhtml

@FlowScoped("totp-login")
@Named
public class NewUser {
   @EJB private UserManager mgr; 
   //JavaBean properties - not shown
   ...
   public String register() {
      //Checks
      ...
      Map<String, String> result = OneTimePassword.generateQR(username, Authentication.ISSUER);
      mgr.register(username, password, result.get(OneTimePassword.SECRET32));
      qrUrl = result.get(OneTimePassword.OTPAUTH);
      return ("qrcode");
   }
}

The code is quite simple. When the user clicks on Register Me button, we will call register(). After verify the fields, us use an EJB call UserManager to insert the username, password and the base32 encoded secret key into the JDBC realm. This corresponds to logintable above. It also assigns the user to a group called registered.

Recall that in our JDBC realm configuration, we say that we are using SHA-256 for the password digest; so in effect we don’t save the password but its digest. This hashing operation is performed by the register() method.

Before we register the new user into our database, we have to generate a secret key for it. We use generateQR(); generateQR() takes in 2 parameters, the subject and the issuer. In the example above, for the subject we will be using the user’s login name and the issuer is hardcoded to “Yoyodyne Propulsion Systems”. generateQR() returns a Map which hold a few entries. The name of the keys holds the following values

Key Name Value Description
secret32 Base32 encoded String of a randomly generated 10 byte secret
otpauth otpauth URI
url Full URL, with the otpauth, to the Google site that generates the QR code
secret Stringify 10 byte secret. Note in the demo, we only use the first 2 keys viz. secret32 and otpauth

We assign the otpauth URI to a property and move to the qrcode.xhtml page to display the QR code shown below

<h:outputText value="
   Please scan the QR code on the right using 
   your Google Authenticator App. After that you can 
   proceed to login."/>           
<h:commandButton value="Login" action="totp-login"/>
<h:graphicImage url="#{newUser.qrUrl}"/>     

A digression: I found that graphicImage cannot load images from an external source viz. if I pass it http://www.google.com/chart?… I’ll get a broken image. Not sure if this is a bug with JSF (Majorra) or its part of the specs. Anyways, this blog pointed out a possible workaround by proxying the image via a Servlet. The otpauth is appended to /qrcode. So the url attribute in graphicImage references /qrcode/the_otpauth instead of http://www.google.com/chart?… A Servlet is mapped to /qrcode/* to handle the image loading. See QRCodeServlet for more details.

All that remains now is the login page which is shown below

<h:form>
   <h:inputText id="username" value="#{authentication.username}"/>
   <h:inputSecret id="password" value="#{authentication.password}"/>
    <h:inputText id="otp" value="#{authentication.otpCode}"/>
   <h:commandButton value="New User" action="register"/>
   <h:commandButton value="Login" action="#{authentication.login()}"/>
</h:form>

and its corresponding backing bean

@FlowScoped("totp-login")
@Named
public class Authentication implements Serializable {
   //Getters and Setters not shown
   ...
   @EJB private UserManager mgr;
   public String login() {
      FacesContext ctx = FacesContext.getCurrentInstance();
      HttpServletRequest req = (HttpServletRequest)ctx.getExternalContext().getRequest();
      if (null != req.getRemoteUser())
         return ("protected/menu");
      if (null == req.getAuthType()) {
         ctx.addMessage(null, new FacesMessage("Not configured for container managed login"));
         return ("error");
      }
      try {
         req.login(username, password);
      } catch (ServletException ex) {
         ctx.addMessage(null, new FacesMessage("Incorrect login"));
         return ("error");
      }
      User user = mgr.getUser(username);
      if (null == user) {
         ctx.addMessage(null, new FacesMessage("Cannot find user."));
         return ("error");
      }
      try {
         if (!OneTimePassword.checkCode(user.getOtpSecret(), otpCode, System.currentTimeMillis())) {
            req.logout();
            ctx.addMessage(null, new FacesMessage("Incorrect login"));
            return ("error");
         }
      } catch (InvalidKeyException | NoSuchAlgorithmException | NumberFormatException ex) {
         ctx.addMessage(null, new FacesMessage("Exception: " + ex.getMessage()));
         return ("error");
      }
      return ("success");
   }
}

The login() method takes a user name and password and will throw a ServletException if the that combination is incorrect. It will also throw ServletException under 2 other circumstances

  • If the application have not been configured for security. We check that using getAuth() method or
  • If we attempt to login a use that have been authenticated. So we check if we have the user name by calling getRemoteUser(). If this is not null, meaning that this request has been authenticated, then we redirect the request into the secure area. More on secure area later.

After successfully login, we perform the 2nd validation by checking the OTP. We pass in the secret key, which is returned when we get the User from getUser(), the OTP password that the user entered and the current time to OneTimePassword.checkCode(). checkCode() method is  this algorithm that I found here. checkCode() verifies the password using a 120 second window viz. if you give it the current time, it’ll take the current, past and future 30 second slot. If we don’t do that then the time on the OTP generator (the mobile phone that runs Google Authenticator) and the time on the server that host the web application cannot be more than 30 seconds apart.

Note that if the TOTP password is incorrect we perform a logout (logout()) before returning; the other method that you can use is HttpSession.invalidate() which performs a logout as well. The difference is that logout() will not destroy the HttpSession. One reason you might not want to invalidate the session is to keep a count of failed attempts and disable the username for 5 minutes for example.

If the OTP is correct, we exit the TOTP authentication flow by returning the success token. I’m using explicit flow definition, so the definition is as follows

<flow-definition id="totp-login">
    <flow-return id="success">
        <from-outcome>/protected/menu</from-outcome>           
    </flow-return>
</flow-definition>

where everything under the /protected  directory requires authentication and authorization before access.

 

Form Based Authentication with TOTP

We’ll not turn our attention to configuring the web application with the totp-login flow described above using the JDBC realm that we have created. So if you want to secure an application in Java EE, you have to specify 3 things

  1. Authentication scheme – how are we going to collect username and password and to perform the authenticatoin
  2. Authorized role(s) – what are the application defined security roles
  3. Security constraint – what are the resources that needs to be authenticated and who is authorized to access it. The constraint ties authentication and authorization into a neat little package

Authentication Scheme

Let start the discussion by looking at authentication scheme. Java EE supports container managed security. If we’re dealing with securing web application, then common ways to authenticate a user are either basic or form-based (see here for other ways).

When you use basic authentication, you get a dialog box to request for your user name and password. If a correct combination of username and password is presented, then you will be redirected to the protected resource. Basic authentication does not give you control over the login page and the authentication process. Say you want to enter a third field, eg our TOTP field, as part of the the login, there is just no way of doing this.

Enter form based authentication. With form based authentication, you create a login and an error page in the web.xml file. The login page needs to contain j_username and j_password field; these values are POSTed to j_security_check. Form based authentication gives you control over the L&F of your login page but you’re still out of  the loop when it comes to the authentication process. Again it does not allow us to arbitrarily include any additional data into the authentication process. You can do it after you’ve passed security check but not during. Another common gripe with form based authentication is that its not very JSF friendly; you have to fallback to <form> and <input>.

So in Java EE 6, the Servlet API introduced programmatic security. At its heart, it is still form based authentication. However instead of relying of j_security_check to collect username and password, your application is now be responsible for collecting these information and performing the authentication. This opens up a lot of possibilities; you are no longer restricted to just username and password. You can include additional fields; see Authentication.login() and totp-login page above.

The following XML fragment shows form based authentication for our TOTP authentication

<login-config>
   <auth-method>FORM</auth-method> 
   <realm-name>jdbc-realm</realm-name> 
   <form-login-config>
      <form-login-page>/faces/index.xhtml</form-login-page> 
      <form-error-page>/faces/index.xhtml</form-error-page> 
   </form-login-config>
</login-config>

Note that in a typical form based login, you have 2 different views for login and for error. The container will redirect the request to the error page if you’ve login incorrectly. In our “semi” container managed security, our application controls this (see the flow above). So you don’t actually need an error page; I therefore set it to index.xhtml.

Another issue is cannot enter a JSF flow with a HTTP GET. In a typical form based login configuration, you provide the actual login page. In this scenario, we cannot have /faces/totp-login; this will not start the flow. What you need to do here is provide a view which have a POST back to start the totp-login flow; in this case index.xhtml looks like this

<h:form> 
   <h:commandLink action="totp-login" value="Login"/>
</h:form>

Authorized Roles

Once you’ve setup the authentication mechanism, you now need to define a security role. Our demo application have a single role which is authenticated.

<security-role>
    <description/>
    <role-name>authenticated</role-name>
</security-role>

The authenticated role needs to be mapped to users and groups from the jdbc-realm. This mapping is application server specific. Glassfish uses a file called glassfish-web.xml to perform this mapping. The following XML fragment shows how we map authenticated to registered group.

<security-role-mapping>
  <role-name>authenticated</role-name>
  <group-name>registered</group-name>
</security-role-mapping>

Security Constraints

A security constraint defines what resources are to be protected and what roles are allowed to authorized to access it. Lets say we wish to secure everything under the /protected path. So when we try to access /faces/protected/menu, the application server will require you to be authenticated first; then check if you are authorized with the role to access that resource. In our case our security constraints looks like this

<security-constraint>
    <display-name>Constraint1</display-name>
    <web-resource-collection>
        <web-resource-name>protected</web-resource-name>
        <description/>
        <url-pattern>/faces/protected/*</url-pattern>
    </web-resource-collection>
    <auth-constraint>
        <description/>
        <role-name>authenticated</role-name>
    </auth-constraint>
</security-constraint>

The above constraint says that what we are protecting is everything under the /faces/protected path; only those uses who are authenticated to the authenticated role will be authorized to access it.

 

Screenshots

The following are some screenshots with a running commentary of what’s happening

totp0

This is the login page. Assume that we are a new user, so we click on New User button. This brings us to the following screen

totp1

We enter the new user details. Then click on Register Me button. This will call NewUser.register(). A secret key will be generated and the appropriate QR code link is created based on the username, issuer and the secret key. The following page is displaying asking the user johnbigboote to scan that into his Google Authenticator.

totp2

After scanning in you should see the following in the Google Authenticator

jb

The number 732803 is the OTP and the ‘pie chart’ to the right is the 30 seconds count down timer. After login you’ll be redirected to the secure area.

totp3

You can use this approach to integrate other authentication mechanism like adding CAPTCHA.

I’ve listed a few caveats in the blog. They are mostly related to JSF and not security. So let me just state one here. One of the big advantage of container managed security is that if you are going to access /faces/protected/menu resource and your are not authentication, the application server will redirect you to the login page (if we’re using form based authentication). After you have authenticated and you’re authorized to access that resource, the container will now redirect you to that resources which is /faces/protected/menu in our case.

But using programmatic authentication, you lose this ability. I have to go to a specific view after a successful login. I cannot find a way of redirecting the request back to the original resource. Let me know if you have a standard and portable way of doing this.

The source code can be found here. It’s been an extremely long blog. Didn’t start off that way. Thanks for reading.

Till next time.

jsocketio – A Simple Java EE 7 Backend for socket.io

A while back, I blogged about how to use Websocket and Servlet support in Java EE 7 to implement a backend for socket.io library. After my experimentation, I concluded that the entire process (of writing the backend) can be wrapped in a framework.

After a few false start, I’ve finally got the first version out. The framework is called jsocketio (I’m using the time honoured way of prefixing a “j” to name the framework).

 

@SocketIO

Use the @SocketIO annotation to annotate your message handler. The annotation only takes one parameter which is the event name. The default event name is message if you omit the name.

The following handler

@SocketIO //Defaults to message event
public class CustomerHandler {
   ...

will be used to handle the following

var sock = io.connect("http://localhost:8080&quot;); sock.send("hello jsocketio");

If you require a custom event name, then the following handler

@SocketIO("customer")
public class CustomerHandler {

will be used on the following code

sock.send("customer", "hello jsocketio");

Like JAX-RS, jsocketio also supports simple path tempates. However, jsocketio’s path templates is based on regular expression (see what Jamie Zawinski say about RE) and uses groups, instead of named parameters in JAX-RS, for isolating values in event names. For example the following

@SocketIO("customer/(([0-9]+))")
public class CustomerHandler {

will match this

sock.send("customer/1", "fred");
sock.emit("customer/3", "barney");
sock.json.emit("customer/5", {name: "wilma"});

To retrieve the matched value, eg. getting 1 from customer/1, you’ll have to get group 1. More on this later.

 

Message Handler Annotations

There are 2 types of annotation for regular message (@WhenMessage) and events (@WhenEvent).

@WhenMessage is used with the send() method. The message payload can be converted into any primitive type, JsonObject, JsonArray or to any Java object so long as the class have

  • a constructor with a single String parameter, or
  • a public static method call valueOf with a single String parameter eg Integer.valueOf(String)

@SocketIO
public class ToUppercase {
   @WhenMessage
   public void handle(String data, Session session) {
      session.getBasicRemote()
         .sendObject(Frame.send(data.toUpperCase()));
   }
}

First off, you will notice that message handler, the method with @WhenMessage, have 2 parameters: one of these must be the Websocket’s Session object unless your handler is not planning on sending response back to the client. The other parameter is the injected value from the packet. jsocketio will try to convert the payload to the target type according to the parameter type.

When you are sending reply back to the client, the replies must be in a socket.io frame. As you can see in the example above, jsocketio provide a utility method called Frame to construct frames. Once the frame has been created, use sendObject() method to send the frame.

Here are some more methods in Frame

public static SocketIOFrame send(JsonStructure json)
public SocketIOFrame void send(String message)
/* values are send as JSON array */
public SocketIOFrame void emit(String event, String... values)
/* Key/value pairs are converted to JSON object */
public SocketIOFrame void emit(String event, Map<String , String> map)
public static SocketIOFrame emit(String event, List<Map<String , String> items)
public static SocketIOFrame emit(String event, JsonArray array)
public static SocketIOFrame emit(String event, JsonObject object)

/* The following methods sends a control frames, meaning should be obvious */
public static SocketIOFrame connect()
public static SocketIOFrame disconnect()
public statsic SocketIOFrame heartbeat()

I’ve tried to make Frame intuitive by mirroring socket.io’s method; send() and emit()have the same semantics as its JavaScript counterpart.

Here is a slightly more complex example involving JPA and resource injection. I’m using the customer database from Derby. The entity classes, not shown, are generated using NetBeans.

@SocketIO("customer/(([0-9]+))
public class CustomerResource {
   @PersistenceContext EntityManager em;
   public void handle(@Group(1) int id, Session session){
      Customer customer = em.find(Customer.class, id);
      if (null == customer) {
         session.getBasicRemote().sendObject(Frame.send( 
            Json.createObjectBuilder().build());
         return;
      }
      JsonObjectBuilder builder = Json.createObjectBuilder(); 
      builder.add("name", customer.getName())
         .add("custId", customer.getCustomerId())
         ...;
      session.getBasicRemote().sendObject(Frame.send(
         builder.build()));
   }
}

 

Livecycle Events

To listen to jsocketio’s lifecycle event, use @WhenConnect and @WhenDisconnect. Since I’ve only implemented websocket support, so the parameters for the lifecycle events are the same as that of Websocket. @OnOpen and @OnClose.

/* Parameter are optional */
@WhenConnect
public void connect(Session session, EndpointConfig config) {
   ...
}

@WhenDisconnect
public void disconnect(Session session, CloseReason reason) {
   ...
}

 

Miscellaneous Items

One of the issue that I’ve raised in my previous blog on socket.io is the that socket.io browser clients need to set the resource parameter when connecting to a JavaEE backend. The reason is that all JavaEE web applications are deployed with an application context. So if you are connecting to a JavaEE backend with mysockapp as the context, then you must do the following to connect to it

var socket = io.connect("http://server:8080&quot;
   , { resource: "mysockapp/socket.io" });

jsocketio automatically adjust the resource parameter to match its context. So all you have to do from the client is

var socket = io.connect(“http://server:8080”);

One last item,  you have to set bean-discovery-mode=”all”  attribute in your beans.xml file. CDI 1.1 changes the default behaviour of how beans are looked up. See this blog. However Websocket handles does not support any of the CDI scopes apart from when the connection is made (@OnOpen). So to work around this issue, you have to change the discovery mode from annotated to explicit archive.

 

What Next?

I would like to add support for a fallback protocol. The obvious choice would be xhr-polling. Also I would like to abstract websocket specific objects from leaking in jsocketio’s handlers.

The source for jsocketio can be found here. The compiled JAR file can be found here.

Let me know what you think.

Till next time.

Persisting Backbone.js Models and Collections to JAX-RS

There are many competing Javascript framework available; one of the more common and popular framework is Backbone.js. One of its claim to fame over other Javascript frameworks its persistence. (No flames please)

Like other framework, Backbone.js allows you to define models and collections to represent your data; you can then use Backbone.js build in REST features to persist your models and collections to a backend system.

I was recently given the opportunity to use Backbone.js with Java EE, specifically connecting from Backbone.js to a some RESTful resource. I’ll document what I’ve learnt in this blog; the database used here is not the actual database.

The blog will be broken into 3 parts

  • The first part we will look at how to map a model to a JAX-RS root resource and a JPA entity
  • The second part we will extend from mapping a model to a collection
  • The third part we will look at PATCH support

We will be using the following table throughout this blog

customer_table

The customer table (from JavaDB in NetBeans) is mapped to the following JPA entity. Note: for the sake of simplicity, I’ve not mapped all the fields and also I’ve not included any relationship

@Entity @XmlRootElement
public class Customer implements Serializable {
   @Id @Column(name=”customer_id”) private Integer customerId;
   private String name;
   private String addressLine1, addressLine2;
   private String city, state;
   private String email, fax, phone
   private Integer creditLimit;
   //Properties for the above members – not shown
   . . .

The @XmlRootElement annotation allows JAX-RS to automatically convert the a customer entity to JSON (as well as XML).

 

Exposing Customer Entity as a RESTful Resource

We will now use JAX-RS to provide a REST interface for the Customer entity. If you’re not familiar with JAX-RS, see this tutorial.

But before we proceed to code the class, we need to decide the resource identifier; the obvious choice is customer/id where id is the primary key entity.

@Path(“/customer/{id:\\d+}”)
@RequestScoped
public class CustomerResource {
   @PersistenceContext EntityManager em;
   @GET
   @Produces(MediaType.APPLICATION_JSON)
   public Response get(@PathParam(“id”) int custId) {
      Customer customer = em.find(Customer.class, custId);
      if (null == customer)
         return (Response.status(Response.Status.NOT_FOUND)
               .build());
      return (Response.ok(customer).build());
   }
   . . .

Using a REST client for testing Customer resource

rest_get

Click to enlarge

So if you look closely at the resource URI you will notice the following URI

http://localhost:8080/customer/webresources/customer/1

Reading the URI from left to right, the first customer is the the application context. This is followed by webresources. The webresource is the base URI and is used as a prefix for all other resource URI. This value specified in the @ApplicationPath annotation on the Application subclass. If you are using NetBean, this subclass is generated for you and the base URI defaults to “webresources”.

Finally customer/1 is the actual URI of CustomerResource; this is defined in defined in @Path.

 

Mapping to Backbone Model

We will now define a Backbone model to map to customer resource. There are 3 things that we need to know:

  • The first is associating the Model’s id to the JPA’s entity primary key;
  • The second is mapping the resource identifier to JAX-RS root resource’s path.
  • Finally the HTTP method. This will largely depend on what  the model is performing. eg a Model.fetch() will generate a GET method

Lets look at JPA first. By default all Backbone model assumes that the name of the primary key is called id. In our case above, the primary key is called customerId. We have to tell Backbone this otherwise a Backbone will not know the identity of the model if we perform any of the CRUD operations on the model or collections. To map the primary key use the idAttribute.

To retrieve and update backend data, all Backbone models need the URI fo the REST resource. Since we are using JAX-RS, we need to know the URI of the resource. If we map the model to the above defined REST resource, then we need to assign the URI  webresources/customer to the urlRoot property (assuming that our JavaScript is loaded from http://localhost:8080/customer/index.html).

var Customer = Backbone.Model.extend({
   initialize: function() {
      console.log(“created: “ + this.get(“name”));
   },
   idAttribute: “customerId”, //PK mapping
   urlRoot: “webresources/customer” //REST URI mapping
});

To retrieve customerId 1, from we do the following

var customer = new Customer();
customer.set({customerId: 1})
customer.fetch()

The result of the fetch() shown below

result Click to enlarge

If you are unfamiliar with the workings of Backbone.js persistence, read the excellent free book by Addy Osmani here.

I will leave CUD as an exercise for the reader.

 

Mapping to Backbone Collections

One of the concept of RESTful resource is the idea of a container; a container is just a collection of RESTful resource. So we can have a container for Customers. Unlike Java collections, RESTful containers you should not hold the Customer data but a URI to that data. See “Link things together” in this article. So the following is an example of Customer container in JSON

[
   { “id”: 1, “name”: “Jumbo Eagle Corp",
"url":
      “http://localhost:8080/customer/webresources/customer/1”},
   { “id”: 2, “name”: . . .
]

Notice that although there are information like customer id and name, the key is the url which the URI to the actual resource. Lets look at how we can create just such a collection

@Path(“/customers”)
@RequestScoped
public class CustomersResource {
   @PersistenceContext EntityManager em;
   @Context UriInfo ui;
   @GET
   @Produces(MediaType.APPLICATION_JSON)
   public Response get(
         @DefaultValue(“0”) @QueryParam(“start”) int start,
         @DefaultValue(“-1”) @QueryParam(“count”) int count) {
      TypeQuery<Customer> query = em.createNamedQuery(
            “Customer.findAll”, Customer.class);
      query.setFirstResult(start);
      if (count > 0)
         query.setMaxResult(count);
      List<CustomerReference> refs = new LinkedList<>();
      UriBuilder builder = ui.getBaseUriBuilder()
            .path(CustomerResource.class);
      for (Customer c: query.getResultList()) {
         URI uri = builder.clone().build(c.getCustomerId());
         ref.setId(c.getCustomerId());
         ref.setName(c.getName());
         ref.setUrl(uri.toString());
         refs.add(ref);
      }
      GenericEntity<List<CustomerReference>> response =
            new GenericEntity<List<CustomerReference>>(refs){};
      return (Response.ok(response).build());
   }
   . . .

The URI of the Customer container is /customers. The resource allows you to specify a cursor (start) and also the number of records to retrieve (count). You can of course add other query string to the GET like a pattern to match etc.  The rest of the code is quite straight forward.

The CustomerReference is a class to hold the result. It is defined as follows

@XmlRootElement
public class CustomerReference {
   private String id;
   private String name;
   private String url;
   //Getter and setters – not shown

Mapping a collection to /customers URI is very similar to mapping a model. The only difference is you use url instead of urlRoot.

var Customers = Backbone.Collection.extend({
   model: Customer,
   url: “webresources/customers”
});

Once you have setup the model, you can say get 5 Customers starting from position 3

var custSet = new Customers();
custSet.fetch({data: {start: 3, count: 5}});

Partial Model Update with PATCH

One common problem with model is that if you just update one field, Backbone will send all the attributes of that model to the server to be persisted. The HTTP PATCH method allows you to specify only the subset of attributes to send back . Say you want to update the customer’s name and get Backbone to only send that attribute back, set the patch option to true when you are performing a save.

var cust = custSet.get(2);
cust.save({name: "Fred"}, {patch: true});

Backbone will send a PATCH method with /customer/id URL back to the root resource. You can see this in the following

patch Click to enlarge

Now to write JAX-RS support. As I’m using Java EE 7 for this, I get to use new Json classes to map the values in. If you’re using EE 6, use an InputStream as a parameter to read to the method. Then use one of the Java libraries from json.org to parse it to JSON.

First we define a HTTP PATCH method

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
@HttpMethod("PATCH")
public @interface PATCH { }

Next we add the method to handle PATH

@Path(“/customer/{id:\\d+}”)
@RequestScoped
public class CustomerResource {
   . . .
   @PATCH
   @Consumes({"application/json"})
   @Produces("application/json")
   public Response updateValues(JsonObject json) {
      //Update record – not shown
      . . .
      return (Response.ok().build());
   }
. . .

The full source including all the missing methods is available here. I have to disguise the source as a Word file because Word Press will not allow me to upload zips. Just download it and rename the suffix to zip. Its a NetBeans project file.

Let me know what you think. Thanks for reading.

Till next time…

Marrying Socket.IO Client with Java EE 7’s WebSocket

just_marriedSocket.IO is a JavaScript library that uses a variety of transport (WebSocket, XHR polling, etc) to allow you to develop real-time web application. Socket.IO hides the underlying transport mechanism by figuring out what is the best transport to use between the browser and the server. It performs a handshake to determine this. Most Socket.IO servers are written in JavaScript running in node.js.

Java EE 7 was released recently. One of the key themes of EE 7 is better support for HTML 5 application. The support for modern web application started in Java EE 6 with the inclusion of RESTful web services. In EE 7, the powers that be added WebSocket and JSON.

I was recently learning and mucking around with Socket.IO. A thought struck me; can I connect a Socket.IO client to a Java EE server? I did a little investigation; I found 2 server side Java implementation of Socket.IO. They are netty-socketio and Socket.IO-Java. While both are excellent implementations and have been used in production, they are tied to specific servers. netty-socketio is based on Red Hat’s async IO framework called Netty and Socket.IO-Java runs on top of Jetty leveraging its WebSocket and continuation support.

I wanted something cross platform using standard API and  more importantly, for me at least, to use Java EE’s programming model (what I’m familiar with) and utilize its resources (JPA, EJB, etc).

The following is a sort of step by step guide on how you integrate a Socket.IO client to Java EE 7 using the new WebSocket API (JSR-356). What I’m doing here is based on looking at Socket.IO-Java sources.

Something Old

When a Socket.IO client connects, it does 2 things

  • Sends a HTTP GET to /socketio to download the socket.io.js JavaScript file. This is the result of <script>
  • Sends a second HTTP GET to /socketio/1/ for the handshake. The server should respond with a colon delimited string comprising of the session id, heart beat timeout, default timeout and a list of supported transport separated by comma. Eg 123456:15000:10000:websocket,xhr-polling

The easiest way to handle the handshake part is to use Servlet and that has to be mapped to /socket.io/*. Override doGet() and doPost() to handle the client request.

//Bare minimum, no error checking

@WebServlet(urlPatterns={“/socket.io/*”})

public class SocketIOServlet extends HttpServlet {

   private static final String CONFIG = “:15000:10000:websocket”;

   @Override

   protected void doGet(HttpServletRequest req, HttpServletResponse resp)

         throws IOException, ServletException {

      serve(req, resp);

   }

   @Override

   protected void doPost(HttpServletRequest req, HttpServletResponse resp)

         throws IOException, ServletException {

      serve(req, resp);

   }

   private void serve(HttpServletRequest req, HttpServletResponse resp)

         throws IOException, ServletException {

      String path = req.getPathInfo().trim();

      path = path.startsWith(“/”)? path.substring(1): path;

      String[] paths = path.split(“/”);

      //GET /socket.io/socket.io.js – returns the socket.io.js library

      if (“GET”.equals(req.getMethod()) && “socket.io.js”.equals(path[0])) {

         resp.setContentType(“text/javascript”);

         //Copy socket.io.js back to client, then return, not shown

        ….

      } else if (parts.length <= 2) {

         //Return handshake

         try (PrintWriter pw = resp.getWriter()) {

            pw.print(req.getSession().getId() + CONFIG);

         }

      }

   }

}

The important thing to note is that in the handshake back to the client, we indicated that we only support WebSocket. See CONFIG constant.

Something New

Once the handshake is successful, the client will now attempt to communicate with us using WebSocket. WebSocket is the newest API in the EE 7 stack. Its very similar, in terms of programming style, to JAX-RS. See here for a comprehensive WebSocket tutorial.

To communicate with the client (Socket.IO), the WebSocket end point must be mapped to /socket.io/protocol_version/websocket/sessionId eg /socket.io/1/websocket/123456. The session id is the variable part of the path so we need to create an endpoint with path parameter to match that but we will not be using it unless you are interested in specific clients.

@ServerEndpoint(value=”/socket.io/1/websocket/{sessionId}”)

public class SocketIOWebSocketHandler {

   @OnOpen

   public void open(Session session) {

      session.getBasicRemote().sendObject(connect());

   }

   @OnClose

   public void close(Session session) {

      session.getBasicRemote().sendObject(close());

      session.close();

   }

   @OnMessage

   public void message(Session session, String msg) {

      System.out.println(msg);

   }

}

So if you look at handling WebSocket transport with the client, its actually quite straightforward. The part that is might be a little confusing is why sendObject()? And what is in connect() and close(). Another question might be the format of msg.

 

Something Borrowed

Socket.IO defines a frame/packet based wire protocol that it uses for all communications between the 2 ends. A typical frame looks something like this

   3:::hello world

So if we were to receive some data from the client and print that out, we will see something similar to above.

As I’ve mentioned at the start of this blog, the code here is largely based on studying Socket.IO-Java sources. Instead of writing my own parser for the Socket.IO frames, I’m going to just use Socket.IO-Java’s. The class in question is SocketIOFrame. Its quite easy to use.

To parse a message from the client

List<SocketIOFrame> frames = SocketIOFrame.parse(msg);

SocketIOFrame has the following methods to identify the frame

  • getFrameType() – whether its a data frame or control frames
  • getMessageType() – text or JSON message
  • getData() – the data

So message() method above can be written as follows (we’ll only be covering the message)

@OnMessage

public void message(Session session, String msg) {

   for (SocketIOFrame f: SocketIOFrame.parse(msg))

      switch (f.getFrameType()) {

            …

         case EVENT:

         case JSON_MESSAGE:

         case MESSAGE:

      }

}

Lets look at how a Socket.IO client sends each of these message type and how we handle it on the server. Assume that socket is a Javascript Socket.IO socket.

Client

socket.send(“hello world”);

Server

case MESSAGE:

   System.out.println(f.getData());


Client

socket.json.send({message: “hello world”})

Server

case JSON_MESSAGE:

   StringReader data = new StringReader(f.getData());

   JsonReader reader = Json.createReader(data);

   JsonObject jsonData = reader.readObject();

   System.out.println(jsonData.getString(“message”));

Client

socket.emit(“greetings”, {message: “hello world”});

Server

case EVENT:

   StringReader data = new StringReader(f.getData());

   JsonReader reader = Json.createReader(data);

   JsonObject jsonData = reader.readObject();

   String eventName = jsonData.getString(“name”); // greetings

   JsonArray payload = jsonData.getJsonArray(“args”); // { … }

When you are sending data back to the client, the message have to be framed as well. Assume we have the following message handlers on the client

socket.on(“message”, function(data) {

   console.log(“MESSAGE: “ + data);

});

socket.on(“greetings”, function(data) {

   console.log(“greetings: “ + data);

});

The following shows how to send message to each of the above handlers

Sending as text

SocketIOFrame f = new SocketIOFrame(

      SocketIOFrame.FrameType.MESSAGE

      , SocketIOFrame.TEXT_MESSAGE_TYPE, message);

session.getBasicRemote().sendObject(f);

Sending as JSON

StringWriter w = new StringWriter();

try (JsonWriter writer = Json.createWriter()) {

   writer.write(json);

}

SocketIOFrame f = new SocketIOFrame(

      SocketIOFrame.FrameType.JSON_MESSAGE

      , SocketIOFrame.JSON_MESSAGE_TYPE, w.toString());

session.getBasicRemote().sendObject(f);

Sending as custom event

JsonArray array = Json.createArrayBuilder()

      .add(/* add data as JsonObject to array */).build();

JsonObject json = Json.createObjectBuilder()

      .add(“name”, “greetings”)

      .add(“args”, array).build();

StringWriter w = new StringWriter();

try (JsonWriter writer = Json.createWriter()) {

   writer.write(json);

}

SocketIOFrame f = new SocketIOFrame(

      SocketIOFrame.FrameType.EVENT

      , SocketIOFrame.JSON_MESSAGE_TYPE, w.toString());

session.getBasicRemote().sendObject(f);

A connect SocketIO frame (connect() method) is created like so

new SocketIOFrame(FrameType.CONNECT

      , SocketIOFrame.TEXT_MESSAGE_TYPE, “”)

and close SocketIO frame (close() method) is

new SocketIOFrame(FrameType.CLOSE

      , SocketIOFrame.TEXT_MESSAGE_TYPE, “server”)

Something Blue

The Socket.IO client makes the following assumption when communicating with the server

  • All communication endpoint are at socket.io viz. when you do io.connect(“http://myserver:8080”), it automatically look for the server side end point to be at http://myserver:8080/socket.io
  • All data send between the client and the server are enclosed in a frame defined by Socket.IO library
  • Following from the first point, the client always talks to the same end point. In the above example, all data are directed to http://myserver:8080/socket.io. So the handler must look at the data and dispatch to appropriate handlers. It would be nice if Socket.IO can use end points to discriminate the message handlers eg.  http://myserver:8080/socket.io/news

Lets address each of the above issues. When you deploy a Java EE web application, you always deploy it under an application context. Lets say the web application uses jsocketio as the application context, then you can use the resource option to remap the endpoint.

By default if you do the following

var socket = io.connect(“http://localhost:8080”);

socket.io will look for the end point at http://localhost:8080/socket.io as discussed above. To include the web application context, use the resource option

var socket = io.connect(“http://localhost:8080”

      { resource: “jsocketio/socket.io”});

See here for a detailed description of other options.

The workaround for SocketIO frames can handle using WebSocket’s encoders and decoders.

An incoming message first be passed to a decoder which converts the messages into a List<SocketIOFrame>. We then inject this list into the @OnMessage method. Note that in the above example we perform the decoding ourself. We can actually delegate this to decoders.

For outgoing messages, we have to create SocketIOFrame. Then use sendObject() to send it back to the client. The SocketIOFrame encoder will intercept this object and convert it to a frame.

public class SocketIOFrameDecoder

      implements Decoder.Text<List<SocketIOFrame>> {

   @Override

   public List<SocketIOFrame> decode(String value)

         throws DecodeException {

      return (SocketIOFrame.parse(value));

   }

   @Override

   public boolean willDecode(String value) { return (true); }

   …

}

public class SocketIOFrameEncoder

      implements Encoder.Text<SocketIOFrame> {

   @Override

   public String encode(SocketIOFrame frame)

         throws EncodeException {

      return (frame.encode());

   }

   …

}

We now need to associate the encoder/decoder pair with SocketIOWebSocketHandler

@ServerEndpoint(value=”/socket.io/1/websocket/{sessionId}”

      , encoders = { SocketIOFrameEncoder.class}

      , decoders = {SocketIOFrameDecoder.class}

)

public class SocketIOWebSocketHandler {

   @OnMessage

   public void message(Session session, List<SocketIOFrame> frames) {

      //Process inbound messages

      for (SocketIOFrame f: frames)

         //Do something

      …

      //Send reply back

      SocketIOFrame sioFrame = new SocketIOFrame(…);

      session.getBasicRemote().sendObject(sioFrame);

   }

You can find more details on encoders and decoders here.

The third and final caveat when working with a Socket.IO client is that all messages are directed at the same endpoint URL. If you look at how Socket.IO is used with node.js, you’ll notice that you can attach handlers to events. I’ve talked about this above so I won’t be repeating myself here. It would be nice to be able to map event names, in particular those send by using emit(), to WebSocket handlers.

 

I’m currently working on a simple framework so I’ll post the complete code when I complete the framework.

Everything that I’ve described I’ve tested with Chrome, Glassfish 4 and NetBeans 7.4 beta. For some strange reason, Socket.IO client on refuses to use WebSocket on Firefox.

Till next time.

Implementing a Multi-Player XMPP Game – Part 2

In my previous blog, I looked at implementing a multi-player set game. Barring any bugs, the engine is working fine. We now turn our attention to writing the client and I’m faced with the following 2 questions:

Firstly what are we going to use to develop the client? What technology, language and platform are we going to implement this in?

Second question: once we’ve written the client, how are we going to get people to install and use it.

The answer I think is quite obvious: the web. By the web, we mean HTML(5), JavaScript and the browser.  So if this is the current thinking, is there a way that we can apply that here?

As you know, Vorpal applications are naturally web in the sense that they are masquerading as web application in a WAR file deployed into a web container. So far we have not really exploited this schizophrenic nature of Vorpal. I have flirted with it in a previous blog on Server Sent Events. It occurred to me that I could I could write the client for the Set game as a web application and deploy it along with server/engine component just like a regular web application.

The next question is how do you write do you write a HTML application that does XMPP? There are lots of JavaScript libraries that does this really well eg. Strophe.js. While these are excellent techniques for developing XMPP application on the web, they strike me as something of an add-on viz. running XMPP inside a browser whose primary conversation of choice is HTTP.

What I was thinking of was is there a XMPP client that has a browser component where the primary protocol is XMPP.  I could not find what I wanted so I wrote a prototype using JavaFX’s WebView. Here’s how it would work:

web_interface_1

A user would login to a XMPP server using this XMPP client (shown above on the left). The XMPP client performs the usual service discovery by sending  disco#info packet over to the server. If any Vorpal component (external component) has web interface enabled, then it’ll reply with the disco#info packet with the following as one of its <feature>

<feature var="urn:web:interface"/>

Web interface is enabled on the component by adding vorpal.component.web.interface property to the xep-0114.xml file. This property points to the URL of the landing page for the component (like the welcome page in web.xml). For example in the Set game we have the following setting

http://someserver:8080/{__context__}/set.html

In the tradition of Vorpal, the {__context__} is a macro for the context root (or application context)  of the WAR. Remember that even though Vorpal is not a web application, it  is packaged in a WAR file for deployment. The application server will treat it as a web application with a valid application context. So {__context__} will be replaced with the application context. The  unfortunate thing is that there is no way for me to use standard Servlet API to find out about the server portion (http://someserver:8080) at startup, so you’ll have to key this part in.

Once the XMPP client discovers that an external component supports web interface, the client will now send a <query xmlns=”urn:web:interface”/> to get the the URL of the interface. The component will now reply with the the URL. Once the client gets the URL, it then sends a HTTP GET to get the web page. See diagram above.

The Web Interface

The client exports a JavaScript object into the web interface call xmppConnection. As the name implies, this is the XMPP connection to the server. The xmppConnection object provides the following methods

xmppConnection.registerHandler(type, JavaScript_method_name) – the first parameter specifies the type of XMPP message that the JavaScript method (second parameter) should handle. The valid values for type are “message”, “presence”, “iq” and “selector”. For the second parameter, you can just enter the name of the JavaScript method name. The name has to be a String (sorry, limitation of the WebView). Say you have the following

xmppConnection.registerHandler("message", "handleMessage")

then whenever the client receives a XMPP message type, handleMessage method will be called. All JavaScript message handler has the following signature

function handlerName(msg)

where the msg is the XMPP message in String. I have written a few utilities to convert the String to XML. One of these is text2xml(). See Set Game example.

Every message send between the client and the external component is marked with the web interface’s name. This allows the XMPP client to know to which view a packet should be routed. In some cases packets arriving from some other sources that are not marked, then XMPP client will let the web view client decide if it should handle the packet. This is the “selector”. The selector behaves exactly like a message handler, has the same signature except that a message selector method returns a boolean telling the client if it wants to handle the message. If it is a true, then message will be routed to the appropriate handler.

xmppConnection.send(pkt) – sends a packet. The send() behaves like Vorpal client. It’ll try to use the default if certain information is missing from the packet eg. if you send a packet without the from, then it’ll use the current user’s name. There are also some macro from Vorpal for that you can use: {__domain__}, {__subdomain__}, {__self__}, {__username__} and {__resource__}.

xmppConnection.send(pkt, JavaScript_method_name) – If you’re sending an IQ packet and you want a specific method to handle the reply, then use this method to send. Otherwise the IQ reply will be routed to a general handler described above.

Finally, if your web page has a JavaScript function initVorpal(), then this function will be called after the HTML document have been loaded viz. if you’re using jQuery, then after $(document).ready(). initVorpal() is used to perform the web interface initialization, most notably registering message handlers. You cannot use xmppConnection until after the document has been loaded.

Set Game Web Interface

So lets look at how we go about writing a web interface by looking at an example. You can get the source here (Multiplayer Set Game). If you expand the zip file, the important files are

  • set.html – this the interface/web view for the set game
  • resources/set.js – most of the behaviour
  • resources/set.css – the stylesheet

I’ve also used

  • jQuery – for handling interaction. As it turns out jQuery is also extremely useful in manipulating and extracting information from XMPP packets
  • jStrophe – for creating XMPP message
  • vorpalutils – a set of homegrown utilities with functions culled from the Internet
    Lets walk through set.js

$(document).ready() – get jQuery to bind the action handlers to the clear, submit and refresh button. The clear button clears the Set selection; submit button sends the 3 cards that you’ve selected to the set game server in the external component and the refresh button, refreshes the 12 cards in your view. We also setup 12 blank cards

initVorpal() – does 2 things here. The first is that we setup the various message handlers.

The second is to constructs a disco#items to the JID of the subcomponent. If you recall, the first part of this describes the set game room’s name is discoverable under the external component’s JID. We set a reply handler for this IQ packet. When the reply returns (handleRoomName(msg)), the client enters the room by sending a presence packet. We get the room’s name using jQuery and then construct a presence packet with jStrophe.

Once we have joined the game room (handlePresence(msg)), we setup the set.html by displaying the number of players and also by sending a get_board message (see first part) to get the 12 cards to fill the view.

I’ve recorded a screencast of this

 

The latest Vorpal along with the experimental XMPP client can be downloaded from here. The Set Multiplayer can be downloaded from here. To run the XMPP client, type bin/lglass . Yeah its a bash script.

Let me know what you think.

Follow

Get every new post delivered to your Inbox.