jsocketio – A Simple Java EE 7 Backend for socket.io

A while back, I blogged about how to use Websocket and Servlet support in Java EE 7 to implement a backend for socket.io library. After my experimentation, I concluded that the entire process (of writing the backend) can be wrapped in a framework.

After a few false start, I’ve finally got the first version out. The framework is called jsocketio (I’m using the time honoured way of prefixing a “j” to name the framework).

 

@SocketIO

Use the @SocketIO annotation to annotate your message handler. The annotation only takes one parameter which is the event name. The default event name is message if you omit the name.

The following handler

@SocketIO //Defaults to message event
public class CustomerHandler {
   ...

will be used to handle the following

var sock = io.connect("http://localhost:8080"); sock.send("hello jsocketio");

If you require a custom event name, then the following handler

@SocketIO("customer")
public class CustomerHandler {

will be used on the following code

sock.send("customer", "hello jsocketio");

Like JAX-RS, jsocketio also supports simple path tempates. However, jsocketio’s path templates is based on regular expression (see what Jamie Zawinski say about RE) and uses groups, instead of named parameters in JAX-RS, for isolating values in event names. For example the following

@SocketIO("customer/(([0-9]+))")
public class CustomerHandler {

will match this

sock.send("customer/1", "fred");
sock.emit("customer/3", "barney");
sock.json.emit("customer/5", {name: "wilma"});

To retrieve the matched value, eg. getting 1 from customer/1, you’ll have to get group 1. More on this later.

 

Message Handler Annotations

There are 2 types of annotation for regular message (@WhenMessage) and events (@WhenEvent).

@WhenMessage is used with the send() method. The message payload can be converted into any primitive type, JsonObject, JsonArray or to any Java object so long as the class have

  • a constructor with a single String parameter, or
  • a public static method call valueOf with a single String parameter eg Integer.valueOf(String)

@SocketIO
public class ToUppercase {
   @WhenMessage
   public void handle(String data, Session session) {
      session.getBasicRemote()
         .sendObject(Frame.send(data.toUpperCase()));
   }
}

First off, you will notice that message handler, the method with @WhenMessage, have 2 parameters: one of these must be the Websocket’s Session object unless your handler is not planning on sending response back to the client. The other parameter is the injected value from the packet. jsocketio will try to convert the payload to the target type according to the parameter type.

When you are sending reply back to the client, the replies must be in a socket.io frame. As you can see in the example above, jsocketio provide a utility method called Frame to construct frames. Once the frame has been created, use sendObject() method to send the frame.

Here are some more methods in Frame

public static SocketIOFrame send(JsonStructure json)
public SocketIOFrame void send(String message)
/* values are send as JSON array */
public SocketIOFrame void emit(String event, String... values)
/* Key/value pairs are converted to JSON object */
public SocketIOFrame void emit(String event, Map<String , String> map)
public static SocketIOFrame emit(String event, List<Map<String , String> items)
public static SocketIOFrame emit(String event, JsonArray array)
public static SocketIOFrame emit(String event, JsonObject object)

/* The following methods sends a control frames, meaning should be obvious */
public static SocketIOFrame connect()
public static SocketIOFrame disconnect()
public statsic SocketIOFrame heartbeat()

I’ve tried to make Frame intuitive by mirroring socket.io’s method; send() and emit()have the same semantics as its JavaScript counterpart.

Here is a slightly more complex example involving JPA and resource injection. I’m using the customer database from Derby. The entity classes, not shown, are generated using NetBeans.

@SocketIO("customer/(([0-9]+))
public class CustomerResource {
   @PersistenceContext EntityManager em;
   public void handle(@Group(1) int id, Session session){
      Customer customer = em.find(Customer.class, id);
      if (null == customer) {
         session.getBasicRemote().sendObject(Frame.send( 
            Json.createObjectBuilder().build());
         return;
      }
      JsonObjectBuilder builder = Json.createObjectBuilder(); 
      builder.add("name", customer.getName())
         .add("custId", customer.getCustomerId())
         ...;
      session.getBasicRemote().sendObject(Frame.send(
         builder.build()));
   }
}

 

Livecycle Events

To listen to jsocketio’s lifecycle event, use @WhenConnect and @WhenDisconnect. Since I’ve only implemented websocket support, so the parameters for the lifecycle events are the same as that of Websocket. @OnOpen and @OnClose.

/* Parameter are optional */
@WhenConnect
public void connect(Session session, EndpointConfig config) {
   ...
}

@WhenDisconnect
public void disconnect(Session session, CloseReason reason) {
   ...
}

 

Miscellaneous Items

One of the issue that I’ve raised in my previous blog on socket.io is the that socket.io browser clients need to set the resource parameter when connecting to a JavaEE backend. The reason is that all JavaEE web applications are deployed with an application context. So if you are connecting to a JavaEE backend with mysockapp as the context, then you must do the following to connect to it

var socket = io.connect("http://server:8080&quot;
   , { resource: "mysockapp/socket.io" });

jsocketio automatically adjust the resource parameter to match its context. So all you have to do from the client is

var socket = io.connect(“http://server:8080”);

One last item,  you have to set bean-discovery-mode=”all”  attribute in your beans.xml file. CDI 1.1 changes the default behaviour of how beans are looked up. See this blog. However Websocket handles does not support any of the CDI scopes apart from when the connection is made (@OnOpen). So to work around this issue, you have to change the discovery mode from annotated to explicit archive.

 

What Next?

I would like to add support for a fallback protocol. The obvious choice would be xhr-polling. Also I would like to abstract websocket specific objects from leaking in jsocketio’s handlers.

The source for jsocketio can be found here. The compiled JAR file can be found here.

Let me know what you think.

Till next time.

Implementing a Multi-Player XMPP Game – Part 2

In my previous blog, I looked at implementing a multi-player set game. Barring any bugs, the engine is working fine. We now turn our attention to writing the client and I’m faced with the following 2 questions:

Firstly what are we going to use to develop the client? What technology, language and platform are we going to implement this in?

Second question: once we’ve written the client, how are we going to get people to install and use it.

The answer I think is quite obvious: the web. By the web, we mean HTML(5), JavaScript and the browser.  So if this is the current thinking, is there a way that we can apply that here?

As you know, Vorpal applications are naturally web in the sense that they are masquerading as web application in a WAR file deployed into a web container. So far we have not really exploited this schizophrenic nature of Vorpal. I have flirted with it in a previous blog on Server Sent Events. It occurred to me that I could I could write the client for the Set game as a web application and deploy it along with server/engine component just like a regular web application.

The next question is how do you write do you write a HTML application that does XMPP? There are lots of JavaScript libraries that does this really well eg. Strophe.js. While these are excellent techniques for developing XMPP application on the web, they strike me as something of an add-on viz. running XMPP inside a browser whose primary conversation of choice is HTTP.

What I was thinking of was is there a XMPP client that has a browser component where the primary protocol is XMPP.  I could not find what I wanted so I wrote a prototype using JavaFX’s WebView. Here’s how it would work:

web_interface_1

A user would login to a XMPP server using this XMPP client (shown above on the left). The XMPP client performs the usual service discovery by sending  disco#info packet over to the server. If any Vorpal component (external component) has web interface enabled, then it’ll reply with the disco#info packet with the following as one of its <feature>

<feature var="urn:web:interface"/>

Web interface is enabled on the component by adding vorpal.component.web.interface property to the xep-0114.xml file. This property points to the URL of the landing page for the component (like the welcome page in web.xml). For example in the Set game we have the following setting

http://someserver:8080/{__context__}/set.html

In the tradition of Vorpal, the {__context__} is a macro for the context root (or application context)  of the WAR. Remember that even though Vorpal is not a web application, it  is packaged in a WAR file for deployment. The application server will treat it as a web application with a valid application context. So {__context__} will be replaced with the application context. The  unfortunate thing is that there is no way for me to use standard Servlet API to find out about the server portion (http://someserver:8080) at startup, so you’ll have to key this part in.

Once the XMPP client discovers that an external component supports web interface, the client will now send a <query xmlns=”urn:web:interface”/> to get the the URL of the interface. The component will now reply with the the URL. Once the client gets the URL, it then sends a HTTP GET to get the web page. See diagram above.

The Web Interface

The client exports a JavaScript object into the web interface call xmppConnection. As the name implies, this is the XMPP connection to the server. The xmppConnection object provides the following methods

xmppConnection.registerHandler(type, JavaScript_method_name) – the first parameter specifies the type of XMPP message that the JavaScript method (second parameter) should handle. The valid values for type are “message”, “presence”, “iq” and “selector”. For the second parameter, you can just enter the name of the JavaScript method name. The name has to be a String (sorry, limitation of the WebView). Say you have the following

xmppConnection.registerHandler("message", "handleMessage")

then whenever the client receives a XMPP message type, handleMessage method will be called. All JavaScript message handler has the following signature

function handlerName(msg)

where the msg is the XMPP message in String. I have written a few utilities to convert the String to XML. One of these is text2xml(). See Set Game example.

Every message send between the client and the external component is marked with the web interface’s name. This allows the XMPP client to know to which view a packet should be routed. In some cases packets arriving from some other sources that are not marked, then XMPP client will let the web view client decide if it should handle the packet. This is the “selector”. The selector behaves exactly like a message handler, has the same signature except that a message selector method returns a boolean telling the client if it wants to handle the message. If it is a true, then message will be routed to the appropriate handler.

xmppConnection.send(pkt) – sends a packet. The send() behaves like Vorpal client. It’ll try to use the default if certain information is missing from the packet eg. if you send a packet without the from, then it’ll use the current user’s name. There are also some macro from Vorpal for that you can use: {__domain__}, {__subdomain__}, {__self__}, {__username__} and {__resource__}.

xmppConnection.send(pkt, JavaScript_method_name) – If you’re sending an IQ packet and you want a specific method to handle the reply, then use this method to send. Otherwise the IQ reply will be routed to a general handler described above.

Finally, if your web page has a JavaScript function initVorpal(), then this function will be called after the HTML document have been loaded viz. if you’re using jQuery, then after $(document).ready(). initVorpal() is used to perform the web interface initialization, most notably registering message handlers. You cannot use xmppConnection until after the document has been loaded.

Set Game Web Interface

So lets look at how we go about writing a web interface by looking at an example. You can get the source here (Multiplayer Set Game). If you expand the zip file, the important files are

  • set.html – this the interface/web view for the set game
  • resources/set.js – most of the behaviour
  • resources/set.css – the stylesheet

I’ve also used

  • jQuery – for handling interaction. As it turns out jQuery is also extremely useful in manipulating and extracting information from XMPP packets
  • jStrophe – for creating XMPP message
  • vorpalutils – a set of homegrown utilities with functions culled from the Internet
    Lets walk through set.js

$(document).ready() – get jQuery to bind the action handlers to the clear, submit and refresh button. The clear button clears the Set selection; submit button sends the 3 cards that you’ve selected to the set game server in the external component and the refresh button, refreshes the 12 cards in your view. We also setup 12 blank cards

initVorpal() – does 2 things here. The first is that we setup the various message handlers.

The second is to constructs a disco#items to the JID of the subcomponent. If you recall, the first part of this describes the set game room’s name is discoverable under the external component’s JID. We set a reply handler for this IQ packet. When the reply returns (handleRoomName(msg)), the client enters the room by sending a presence packet. We get the room’s name using jQuery and then construct a presence packet with jStrophe.

Once we have joined the game room (handlePresence(msg)), we setup the set.html by displaying the number of players and also by sending a get_board message (see first part) to get the 12 cards to fill the view.

I’ve recorded a screencast of this

 

The latest Vorpal along with the experimental XMPP client can be downloaded from here. The Set Multiplayer can be downloaded from here. To run the XMPP client, type bin/lglass . Yeah its a bash script.

Let me know what you think.

Implementing a Multi-Player XMPP Game – Part 1

In most of my previous blogs, I have focused on what are the new features in the Vorpal framework. Vorpal has now reach a stage of development where I feel that I can stop adding new features and let the existing one stabilizes. One aspect that really concerns me is how useful are those features that I’ve added?

So for the next few blogs I’ll focus on using Vorpal to develop a moderately complex (okay hello_world++) XMPP application; the aim is to show how to use Vorpal to develop something useful (as oppose to code snippets) and also the larger issue of using XMPP in your application.

I’m no XMPP expert and mostly what I know comes from reading the standards, books (one book in particular) and this excellent series of blogs on the design of Chesspark (part 1, 2 and 3).

Set Game

The Set game is a real-time, party style card game. 12 cards are laid out in 4 rows of 3 on the table. Each card has the following four attributes: colour, number, symbol and shading; each attribute has 3 different values. The example card shown on the right has the following attributes: colour: red, number: 2, symbol: diamond, shading: open.

The game is very simple; you are to pick 3 cards that have the same 3 attributes; this constitute a set. Alternatively you can pick 3 cards, all with different attributes. You can see some examples here. The cards are replenished from the deck after you have removed your set. Game ends when the deck runs out of card and there are no more set on the table. The winner is the player with the most set.

Detailed rules and examples can be found here.

Design

We will implement the Set game using MUC (multi-user chat); the reason is that since this is a real-time, multi-player party style game, we think that MUC would be a good fit. The other option is to use pubsub. Every MUC set game room has a ‘dealer’. The dealer is the entity that is responsible for keeping score, checking if you have a set, replenishing cards, etc. The dealer is our Set game component.

Players join the game by entering the game room. Once they’re in the game room, they

  • will be notified of who else is in the game room. Also when player leaves, you will also be notified. All these are supported through MUC.
  • find the game room by sending a disco#items to the set game service
  • get the state of the board from the from the dealer
  • try to take a set of 3 cards by informing the dealer. This is between the dealer and the player only (‘chat’ message)
  • dealer will inform everyone if a player is successful in taking a set (‘groupchat’ message)

Game Protocol

The game protocol can be roughly broken up into 3 parts

  1. A player entering and leaving the game room – these are standard MUC messages
  2. Game specific messages – these are game messages. We will use <message> for all Set game moves. Game messages from the dealer are also <message> type.
  3. Game room membership – messages sent out by MUC about players joining and leaving the game room

Implementation

Setting up the Game Room

When the game starts up, the first thing that it’ll do is to setup the game room. In this implementation, we only support one game room; you are welcome to take the code and modify to support more. We do this by listening to the ComponentPostStart event which is after the Set game component has successfully connect to the XMPP server; see here for more details on lifecycle events.

private void postStart(@Observes ComponentPostStart postStartEvt) {
   String confRoom = randomString(16) + "@conference."
+ postStartEvt.getDomain();
   gameState.setRoomName(confRoom);
   JoinRoom join = new JoinRoom(Constants.DEALER + "@"
+ postStartEvt.getSubdomain(), confRoom + "/" + Constants.DEALER);
    try {
        postStartEvt.getComponentContext().send(join);
    } catch (ComponentException ex) { }
}

The game room’s name, called confRoom above, is randomly generated. The Set game component then joins the room with the nick name dealer. The following message will be sent out.

<presence from="dealer@set.batcomputer"
      to=somerandomstringname@conference.batcomputer/dealer>
   <x xmlns="
http://jabber.org/protocol/muc"/>
</presence>

We perform the reverse when the component shuts down by destroying the room.

gameState is an @ApplicationScoped CDI managed object. It stores information like the game room, card deck, scores, etc. As you can see above, once we have generated the game room, we save that in the gameState object.

Once we have send the join room message, we now have to wait for a confirmation from the XMPP server. The message will come in the form of a <presence> message.

@Presence
@From("{room}@conference." + PredefinedBindings.PARAMETER_DOMAIN + "/{ignore}")
public class ConfirmChatRoom {

   @Inject GameState gameState;
   @Inject ComponentContext compCtx;

   @XmlPath(namespace={"n", Constants.MUC_USER}, path="/presence/n:x/n:item[@role=’moderator’]")
   private ResponseContext roomCreated() {

      ResponseContext respCtx = new ResponseContext(ResponseContext.Type.IQ);
      respCtx.add(IQ.Type.set)
            .add(new IQChildElement("query", Constants.MUC_OWNER))
            .add(DataFormBuilder.create(DataForm.Type.submit));

      compCtx.add(new ItemSpecification(gameState.getRoomName(), "Set game room name"));
      compCtx.add(new FeatureSpecification(PredefinedBindings.DISCO_ITEMS));

      return (respCtx);
    }
}

We look for the confirmation message be examining the <presence> message with an <item> tag in the MUC namespace; furthermore the <item> must have the value moderator in the role attribute. This is our cue; the role moderator tell us that we are the room owner. So when we received this message, we can now enter the room.

To enter the room, we create an empty data form of Submit type and send that back to the room in an IQ-set message. This is shown in green.

The other thing that we need to do is to make our game room discoverable by disco#info and disco#item. Since the game room’s name is not going to change throughout the life time of the Set game component, we use static registration (in blue). We add disco#item feature to our Set game component JID; then we add the game room’s JID as in <item>.

Note on using @XmlPath annotation for matching; I’ve often @XmlElement. @XmlElement is the simpler of the 2. If you want to either match immediate child under the packet’s root (<message>, <presence> or <iq>) use @XmlElement. Use @XmlPath, based on XPath, gives you more control and matching capability at the expense of simplicity.

Handling Player Membership

As a result of using MUC, our component don’t really have to do that much when players join or leave the game room. Since the Set component is in the game room, it will receive <presence> message for every player that joins or leave the room. Furthermore, the MUC will also send notification of the new player to every other participant in the game room. Nice!

The following is an example message of a player joining the game room

<presence from=”fred@batcomputer/pidgin”
      to=”somerandomstringname@conference.batcomputer/fred”>
   <x xmlns=”http://jabber.org/protocol/muc#user”>
      <item affiliation=”member” role=”participant”/>
   </x>
</presence>

The following presence message handler handles both players joining and leaving the game room.

@Presence
@From("{roomName}@conference." + PredefinedBindings.PARAMETER_DOMAIN + "/{alias}")

public class PlayerMembershipHandler {

   @Inject GameState gameState;

   @XmlPath(namespace={"n", Constants.MUC_USER}, path="/presence/n:x/n:item[@role=’participant’]")
   private void playerJoining(@Named("alias") String player) {
      gameState.createPlayer(player);
   }

   @XmlPath(namespace={"n", Constants.MUC_USER}, path="/presence/n:x/n:item[@role=’none’]")
   private void playerLeaving(@Named("alias") String player) {
      gameState.removePlayer(player);
   }

}

The way that these membership messages are matched is exactly the same as confirming the chat room. We can actually combine the PlayerMembershipHandler and ConfirmChatRoom by assigning the @XmlPath element to a capture parameter of the type Element. I’ll leave this as an exercise to the reader.

Playing Set

The Set implementation have 2 types of game message both using <message> as envelop. The first of these 2 message is the get_board message. This message is for a new participant to get the current state of the board. The message is send directly to the dealer.

<message from="fred@batcomputer/pidgin"
      to=”somerandomstringname@conference.batcomputer/dealer”  type="chat">
   <set xmlns="uri:game:set" command="get_board"/>
</message>

The handler for get_board message is as follows

@Message @XmlElement(tag=Constants.SET_TAG)
@To(Constants.DEALER + "@" + PredefinedBindings.PARAMETER_SUBDOMAIN)
public class BoardLayoutCommandHandler {
   @Inject GameState gameState;
   @XmlPath(namespace={"n", Constants.SET_NAMESPACE}, path="/message/n:set[@command=’get_board’]")
   @From("{from}")
   public List<Object> handle(@Named(“from”) String from) {
      //Construct and return the board – see source
   }

which generates the following message

<message type="chat" to=somerandomstringname@conference.batcomputer/fred 
     from=”somerandomstringname@conference.batcomputer/dealer“>
   <ns2:set xmlns:ns2="uri:game:set" command="get_board">
      <ns2:data>
         <board_layout>
            <board id="0">22</board>
            <board id="1">33</board>
            <board id="2">78</board>
            <board id="3">43</board>
              …
            <board id="9">16</board>
            <board id="10">59</board>
            <board id="11">69</board>
         </board_layout>
      </ns2:data>
   </ns2:set>
</message>

The second message is a try_take message

<message to="somerandomstringname@setgame.batcomputer/dealer" type="chat"
      from=”
fred@batcomputer/pidgin”>
   <set xmlns="uri:game:set" command="try_take">
      <data>
         <cards_taken>
            <board id="9"/>
            <board id="7"/>
            <board id="10"/>
         </cards_taken>
      </data>
   </set>
</message>

As the name implies, a try_take is where a player tries to take 3 card to form a set. The cards are identified by their board position. Like the get_board message, the try_take command is sent directly to the dealer (message type is chat) and not broadcast to the game room.

The message handler for try_take is very similar to get_board, so I’m not going to list it here. Look at the source if you’re interested.

Finally if the try_take is successful, dealer will send a take_card message into the game room (message type is groupchat) notifying everyone of the new cards and the score

<message type="groupchat" to="somerandomstringname@conference.batcomputer"
      from=”somerandomstringname@conference.batcomputer/dealer”>
   <ns2:set xmlns:ns2="uri:game:set" command="take_card">
      <ns2:data>
         <cardsInDeck>66</cardsInDeck>
         <cards_taken>
            <board id="9">42</board>
            <board id="7">16</board>
            <board id="10">20</board>
         </cards_taken>
         <cards_new>
           <board id="9">08</board>
           <board id="7">60</board>
           <board id="10">44</board>
         </cards_new>

         <name>fred</name>
         <score>1</score>
         <taken>true</taken>
      </ns2:data>
   </ns2:set>
</message>

If try_take is not successful, the take_card message will not have the <cards_new> section and the value in <taken> element will be false.

I’ve implemented most of the game except for the end game; at the moment the component does not known when the game ends and declares a winner. I’ll try fixing this in the future.

You can find the source code of this in playground. Let me know what you think.

My next blog talk about writing a client for this Set game component.

Vorpal2 Client

Since rewriting Vorpal2 core framework, I’ve made a number of changes to the client side. I first introduce Vorpal client here; in Vorpal2 client, the programming model remains the same but bootstrapping and login in XMPP server have been change. Lets look at these changes.

Component Class

Every Vorpal application, whether its server or client side, requires a configuration file for startup. This is an XML file. The only change here is the component-class entry. This has changed from com.kenai.jabberwocky.framework.client.JabberwockyClientComponent to c.k.j.f.c.XMPPClientComponent. If you have written a client application, the following in red is what you need to change in your configuration file

<connection port=”5222″ domain=”batcomputer”>
   <component-class name=”com.kenai.jabberwocky.framework.client.XMPPClientComponent”/>
   <classes-root>classes</classes-root>
   <packages>
      <package name=”com.acme.xmpp.client”/>
   </packages>
   <properties>
      <property name=”vorpal.component.description” value=”A component to query the customer’s database”/>
      <property name=”vorpal.component.name” value=”Customer Query Client”/>
   </properties>
</connection>

Bootstrapping Vorpal2 Client

A typical Vorapl2 client starts in 2 different phases; the first is to start CDI (using Weld). You can read about starting CDI in Java SE here. The second phase is to start Vorpal2 framework which will in turn start your Vorpal2 Client application. This is typically done by capturing the ContainerInitialized event.

The following code snippet shows how this is done.

@Inject ClientConnectionManager mgr;

private void containerInitialized(@Observes ContainerInitialized ciEvt) {
   String username = …
   String password = …
   ClientConnectionManager.Connection conn = mgr.createConnection();
   conn.load(new File(“client.xml”));
   if (!conn.login(username, password)) {
         //Authentication failed
   }

   //Vorpal2 client is now started
}

private void containerShutdown(@Observes BeforeShutdown bsEvt) {
   mgr.getConnection().logout();
}

Once we got the ContainerInitialized event, you can create a connection from ClientConnectionManager. The ClientConnectinManager is an ApplicationScoped object which is injected into your startup class. The next step is to load the configuration file. The load() method accepts the configuration either as an InputStream, a File or a ClientComponentConfiguration object. The ClientComponentConfiguration is the object representation of the configuration file; it is JAXB enabled so you can use JAXB to unmarshall a XML configuration file. Alternatively you can also use the helper method ClientComponentConfiguration.read() to return an instance of ClientComponentConfiguration. read() takes either a File or an InputStream as parameter.

After successfully loading the configuration, you can now attempt to login. The login() method will return a false if it fails to login. You can optionally pass a resource into the login() method.

To close a connection, retrieve the default connection from ClientConnectionManager.getConnection() and call the logout() method.

Running Vorpal2 Client

Vorpal2 client applications are dependent on the following 4 runtime JAR files

  • vorpal2.jar
  • weld-se-core.jar
  • weld-se.jar
  • jabberwocky_support2.jar

Due to classloading issues with log4j, jabberwocky_support2.jar MUST the the last JAR file in your CLASSPATH; otherwise Weld will fail initialization.

Client support currently is still very basic. Only clear connections are supported; SSL and other features are planned.

To try Vorpal2 client support, download the latest bundle from here.

NetBeans Plugin for Vorpal2

tools-beltsI’ve finally manage to get some time to update NetBeans plugin for Vorpal2. One major change the plugin is that as Vorpal2 no longer supports Component and AbstractComponent, the plugin is now reflects this.

Lets step through the new tool. You can also treat this as a tutorial for creating a Vorpal2 application.


screen0Step 1. Create a web application


screen1Step 2. Give a name to your web application. The XMPP component will be packaged in a WAR


screen2Step 3. Make sure that you’re using Java EE 6. You can either use Web Profile or the full EE 6 platform. You must also enable CDI. Vorpal2 relies on this heavily.

If you’ve forgotten to select CDI here, you can manually create a beans.xml under WEB-INF directory.


screen3Step 4. Select Jabberwocky from the Frameworks list. Then enter all the parameters of your component.

Note: I’ll talk about Web Interface in a separate blog


screen4Step 5. Click on Properties tab. You can provide a name and a description for your component. You can also customize the category and type of your component. See this document for some pre defined categories and types.

You can also add your own properties to be used by your component.

Click Finish when you are done.


screen5

The plugin will generate a xep-0114.xml file for you and add in all the relevant libraries into your project.

You can find the latest Vorpal2 bundle with the NetBeans plugin here.

I’ve also create a new project call playground to hold the 2 reference application; customer query and eliza. They have been ported to Vorpal2.

New Beginnings Again!

exploring-calvin-and-hobbes

 

They say third time is a or should be charm! In this blog, I’ll talk about Vorpal2, a reworked version of Vorpal.

 

Excuses, excuses, excuses

The original Vorpal grew out of an experiment and the framework was implemented on top of AbstractComponent class from Tinder. It was a really easy way for someone like me with very little knowledge of XMPP to learn about XMPP and get the framework up and running. However due to the way AbstractComponent class works, I had to repeat a same piece of code in different places. There is no single choke point for the XMPP packets coming in and leaving the framework (for that I had to modify AbstractComponent which was not very appealing); this make implementing certain features extremely difficult and in some cases impossible.

The second issue was that I made the framework dependent on a custom Glassfish container call Jabberwocky. Whilst Jabberwocky is great in that it provided a very robust base to run the framework, integrated extremely well with Glassfish asadmin command and provided all sort of statistics on a Vorpal application, the down side was that it effectively ties Vorpal to Glassfish. Also Jabberwocky is build with Maven. So whenever there is a new release of Glassfish I have to figure out which dependencies have change, etc.

Finally testing the framework engine’s becomes difficult because of the 2 dependencies above.

So I’ve decided to reimplement Vorpal; the good news for me is that most of the framework is fine. I just have to rewrite how the Vorpal is loaded. I’ve removed Vorpal’s dependence on AbstractComponent and Jabberwocky; the startup is now done in ServletContainerInitializer. Its lightweight and in theory should run on any JavaEE 6 Web Profile compliant container. Glassfish will still be the best supported platform simply because this is where I develop and test Vorpal.

For the rest of this blog, I’ll talk about migrating from Vorpal to Vorpal2, packet filters which is a new feature available only in Vorpal2.

 

The Great Migration

No code change (except for ComponentManager)

I’m happy to say that there are no code changes going from Vorpal to Vorpal2 but if your application uses ComponentManager then you’ll have to rewrite that portion of it.  ComponentManager is an artefact of AbstractComponent. In the past I’ve allowed you to inject ComponentManager into your message handlers. This is no longer true; all the functions in ComponentManager can now be found in ComponentContext.

New JARs

The new Vorpal2 bundle can be found here (vorpal2). As before, the framework is split into 2 JARs

  • vorpal2.jar – the framework
  • jabberwocky_support2.jar – framework supporting files

If you have been using Vorpal, you’ll need to uninstall them. Delete jabberwocky.jar and jabberwocky_support.jar from $GLASSFISH_HOME/glassfish/modules directory.

As before, vorpal2.jar must be bundled with your application in WEB-INF/lib.

There are 2 ways you can install jabberwocky_support2.jar. You can either bundle that with your application in WEB-INF/lib or install it at your application’s CLASSPATH. For Glassfish this will be the $GLASSFISH_HOME/glassfish/modules directory.

At the writing of this blog, I’ve not fully migrated the NetBeans plugin to support Vorpal2 (hence the missing update directory from the download bundle). Hopefully I’ll have the plugins done by the next release. In the mean time, you’ll have to manage the build and package yourself. In NetBeans, add the the JARs from the bundle to your project’s library and remove the previous version of the libraries. Also remember that these JARs, or at the minimum vorpal2.jar, must be in WEB-INF/lib of the final WAR file.

Deployment descriptor

Vorpal uses a deployment file to connect to an XMPP server. The file is called xep-0114.xml. This file is found in WEB-INF directory. If you’re developing Vorpal applications with the NetBeans plugin, the template of this deployment file is in your project directory and its called xep-0114.txmlt.

The deployment file contains the following pieces of information

  • The XMPP domain, port number to connect to
  • The subdomain name we wish to use
  • The class name of the component to start – this is very specific to the frame
  • An option to specify the shared secret when connecting to the XMPP domain
  • An optional list of properties to be passed to the application

The file looks like the following

<subdomain domain=”batcomputer” port=”5275” name=”customer”>
   <component-class name=”com.kenai.jabberwock.framework.core.JabberwockyComponent”/>
   <create-subdomain shared-secret=”AlwaysRight” create=”true” cleanup=”true”/>
   <properties>
      <property name=”vorpal.component.name” value=”Customer service”/>
      <property name=”vorpal.component.description” value=”Customer service”/>
   </properties>
</subdomain>

What you need to do is the following

  • Change the component-class name from com.kenai.jabberwock.framework.core.Jabberwocky to com.kenai.jabberwocky.framework.core.XMPPComponentServer
  • Always specify the shared secret with create-domain. The create and cleanup attributes must be present but their values are ignored.

So the new deployment file xep-0114.xml looks like below

<subdomain domain=”batcomputer” port=”5275” name=”customer”>
   <component-class  name=”com.kenai.jabberwock.framework.core.XMPPComponentServer”/>
   <create-subdomain shared-secret=”AlwaysRight” create=”true” cleanup=”true”/>
   <properties>
      <property name=”vorpal.component.name” value=”Customer service”/>
      <property name=”vorpal.component.description” value=”Customer service”/>
   </properties>
</subdomain>

Recompile your application after you’ve made these changes.

New Features

Packet filtering

Packet filtering is one of the feature that I wanted to implement into Vorpal. But I could not do this easily under the old code base until now.

Packet filter are filters that are applied to incoming and outgoing XMPP packets. To implement a packet filter, first annotate your class with @PacketFilter and either implement RequestFilter or ResponseFilter (or both!). The following is an example of a request filter that filters out packets that are not send from the local domain

@PacketFilter
public class LocalEntityOnlyFilter implements RequestFilter {
    @Inject @Named(PredefinedBindings.DOMAIN) String domain;
    //Inspired by AbstractComponent.sentByLocalEntity(Packet packet)
    @Override
    public Packet incoming(final Packet in
          , ComponentContext context)
{
        final JID jid = in.getFrom();
        if (null != jid)
            return (in);
        String pktDomain = jid.getDomain();
        if (pktDomain.equals(domain)
               || pktDomain.endsWith("." + domain))
            return (in);
        return (null);
    }
}

Firstly we see that our filter is annotated with @PacketFilter and implements RequestFilter. So what this mean is that this filter will only monitor incoming packets viz. packets sent to the Vorpal application.

Filters are managed by CDI. so you can injected values into it. In this case we are injecting the domain name that the subcomponent is connected to. Filters are statesless so you should not use any CDI scopes on it.

In the filter, you can do anything to the packet; you can add stanzas, remove stanzas and change certain values in the packet. When you return a packet, this can be the original packet from the parameter or a totally new packet, Vorpal will take the packet and proceed with finding an appropriate handler. If you return a null, then Vorpal will discard the packet.

A ResponseFilter behaves the same way as a RequestFilter. When you implement ResponseFilter, you have to override the outgoing(Packet, ComponentContext) method which returns a Packet as well. If you return a null, then Vorpal will not send the packet.

If you have more that one request or response filter, Vorpal does not impose any ordering on them. Also if one of the filter rejects a packet, then that packet will not be processed by other filters. For example if you have 3 request filters; the first of these 3 filters rejects the packet, then that packet will not be passed to the other 2 to be processed.

Ordering sent packets

One side effect of this new version is you can now impose some order on how the packets are send out by returning a List in your message handler. The following code snippet illustrates this

@Body(“{body}”)
private List<ResponseContext> handle(@Named(“body”) String body) {
   …
}

handle() returns a List; so Vorpal will send out the packet according to the order of the packets in the list.

Let me know what you think of Vorpal2. If you have any issues, please post them to the forum. The new bundle can be found here.

Using Server Sent Events with Vorpal

bridge

I came across the following 2 blogs (here and here) on a CDI framework for server sent event (SSE) several months ago. I downloaded the source then tried it with Vorpal running on Glassfish 3.1.x. The result was a really long exception stack.

4 months later.

After publishing part 4 of on conversations, I decided to see if I have any luck with the SSE framework. I wasn’t really expecting it to work. Well it did. What is even more surprising is that it worked with Vorpal. 

What I’m going to do in this blog is explain how the SSE framework works and more importantly how you can use it with Vorpal. Its really easy.

Setup the push event stream

Firstly create a notification handler. The notification handler will maintain the connection between the browser and the web application. The following is a simple notification handler

@ServerSentEvents(“/notifications”)

public class NotificationHandler extends ServerSentEventHandler {

   public void sendMessage(String msg) {

      try {

         connection.sendMessage(msg);

      } catch (IOException ex) {

      }

   }

A few points to note: the notification handler must extend ServerSentEventHandler; this class is from the framework. You also have to annotate the class with @ServerSentEvents and specify the URL of the event stream. In this case it is notifications. More on this later.

Now you need to write a HTML/JSP/JSF/whatever page to invoke this event stream. Here is a snippet of the HTML page along with a snippet of Javascript that loads loads the event stream

<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:h="http://java.sun.com/jsf/html">
    <h:head>
        <title>XMPP with SSE</title>
        <script src="resources/app.js"></script>
    </h:head>
    <h:body>
        <h1>Using Vorpal with SSE</h1>       
    </h:body>
</html>

The Javascript that creates the event stream is in app.js. Here is the code

var url = ‘http://’ + document.location.host

      + “/xmppsse/notifications”;

eventSource = new EventSource(url);

eventSource.onmessage = {

   document.body.innerHTML += msg + ’<br>’;

};

The important thing here is to get the event stream URL right. If our example, this is made up of the application context of our web application (in blue) and the value that you’ve specified in @ServerSentEvent annotation (in green).

 

Pushing data to the browser

We now hook the event handler to a Vorpal application. I did a screencast on developing a customer query service which allows you to use a Jabber client to query a database, which you can find here and the source here.

Lets extend this so that whenever someone performs a query, the result is also pushed to the browser using SSE.

@Message

public class MessageHandler {

   @EJB CustomerManager mgr;

   @Inject @ServerSentEventContext(“/notifications”)

        ServerSentEventHandlerContext<NotificationHandler> ctx;

   @Body(“{id}”)

   private String handle(@Named(“id”) int id) {

      //Assume we find every record and no NPE

      String record = mgr.find(id).toString();

      //Loop through all handlers

      for (NotificationHandler nh: ctx.getHandlers())

         try {

            nh.sendMessage(record);

         } catch (Exception ex) {

            nh.close();

         }

      return (record);

   }

After we got the record from CustomerManager, which uses JPA, we loop through all the registered handlers for an event stream channel and pushes the record to them. You have to get the correct handler to push to.

In our example, we are pushing to all browser registered on notifications. So we inject an instance of ServerSentEventHandlerContext with qualifying it with @ServerSentEventContext(“notifications”).

Example code

You can download the example source from here. I’ve bundled the SSE library into the download which is a NetBeans project.

Use the following steps to setup, deploy and run the example

  1. Configure an external component on your Jabber server. The example uses customer for subdomain and AlwaysRight for shared secret (Caps!). You can always change these settings. I use Openfire for the Jabber server
  2. Make sure that you’ve installed your Glassfish with Jabberwocky container.
  3. Deploy the project to Glassfish either from NetBeans or using asadmin command. It’s probably easier if you deploy if from NetBeans
  4. Open your browser and point to http://yourserver:port/xmppsse. I’ve tested with Firefox and Chrome, latest versions.
  5. Open your Jabber client, open a chat with query@customer.your_jabber_server. Send 1
  6. You should now see your reply display in your Jabber client and in your browser.

The source code for the SSE can be found here if you wish to look at it.

Part of the code (the app.js) is taken from Bhakti Mehta’s blog. Big thanks.

Till next time.

Art of Conversation: Miscellaneous Topics – Part 4

Programmatically looking up Beans

As you may know, you can use Vorpal client to create client side application. I talked about using client side Vorpal here. For the most part, Vorpal client has the same programming model as its server counterpart.

Vorpal relies heavily on CDI for resources; this is not a problem if you’re on the server side, but on the client side you have to lookup the CDI instances yourself. The trick is this is to get a reference to either BeanManager or WeldContainer.

Which of those 2 objects you get depends largely on how you bootstrap CDI in the Java SE environment. See this for the various ways. Since BeanManager is standard Java EE API, we’ll use BeanManager.

One way that I’ve found that works is to get CDI to inject an instance of BeanManager by listening to the container starting up event. Then you can cache a reference of BeanManager in a singleton. The following shows how we go about doing this

@ApplicationScoped

public class MySingletons {

   @Inject private BeanManager bm;

   @Inject ComponentContext ctx;

  

   private static MySingletons instance = null;

  

   @PostConstruct private void init() {

      instance = this;

   }

   private void containerInitialized(

         @ObservesContainerInitialized ciEvt) { }

   public BeanManager getBeanManager() {

      return (bm);

   }

   public ComponentContext getComponentContext() {

      return (ctx);

   }

   public static MySingletons getInstance() {

      return (instance);

   }

}

During container startup we use MySingletons to listen to the container initializing and use this opportunity to hold a reference to BeanManager. Same with ComponentContext. Now wherever you need to access either BeanManager or ComponentContext, you can use the static method MySingletons.getInstance() .

Note: Vorpal client is suppose to be CDI implementation agnostic, but I’ve only ever tested it with Weld.

Client side conversation setup

When you’re working with conversation on the client side, you have make sure that you are not working with conversation artifacts from a previous processing cycle.

On the server, before Vorpal calls your message handler, it initializes the component context and populate it with the appropriate bindings and meta data that your message handler will need to process a message correctly. Server side is mostly reactive processing to incoming messages and Vorpal uses this behaviour to perform a lot of mundane task under the hood.

However on the client side, message processing is mostly proactive viz. you click on a button and your application uses Vorpal client to send out a message. Since there is no initializing the component context; you may actually be picking up artifacts from the previous processing cycle. What you have to do is to clear the component context before you perform your processing by calling ComponentContext.startProcessingCycle().

The following code shows you how you clear the component context on every button click

public void actionPerformed(ActionEvent aEvt) {

   String cmd = aEvt.getActionCommand();

   switch (cmd) {

      case “New Conversation”:

         ComponentContext compCtx = MySingletons

               .getInstance().getComponentContext();

         compCtx.startProcessingCycle();

         Conversation conv = lookup(Conversation.class

               , new NamedQualifier(“__conversation__”));

         ConversationContext convCtx = lookup(

               ConversationContext.class);

         conv.begin();

         ResponseContext respCtx =

               new ResponseContext(…);

                    

         compCtx.send(respCtx);

         break;

      …

   }

}

private <T> T lookup(Class<T> type

      , Annotation… qualifiers) {

   T instance = null;

   BeanManager bm = MySingletons.getInstance();

   Bean<T> bean = (Bean<T>)bm.resolve(

         bm.getBeans(type, qualifiers));

   if (null != bean) {

      CreationalContext<T> ctx =

            bm.createCreationalContext(bean);

      instance = bean.create(ctx);

   }

   return (instance);

}

The above shows how you can lookup an instance of Conversation and ConversationContext programmatically. I’ll leave the NamedQualifier class as an exercise for the reader. Hint: use AnnotationLiteral

If you’re just interested in just creating a new conversation but not about associating any states with it then you can manually add a ThreadID to ResponseContext like so:

case “ New Conversation”:

   ComponentContext compCtx = MySingletons

         .getInstance().getComponentContext();

   compCtx.startProcessingCycle();

   ResponseContext respCtx = new ResponseContext(…);

   respCtx.add(ThreadID.generate());

       …

   compCtx.send(respCtx);

   break;

which is a lot simpler. I’ve blog about this in my previously.

Warning: you should not use ComponentContext.startProcessingCycle() when you are developing server side components or bad things will happen.

Turning off conversation inheritance

Vorpal allows conversation states to be inherited viz. if you create a new conversation in the context of an existing conversation, then that new conversation will inherit all the states from the existing ConversationContext.

If you want to suppress the new conversation from inheriting the states, add a property call com.kenai.jabberwocky.framework.property.inheritConversationContext to the existing conversation’s ComponentContext and set the value to false. This will prevent new conversation from inheriting the existing conversation’s state.

//ConversationContext from an existing conversation

convCtx.setAttribute(

      FrameworkProperty.INHERIT_CONVERSATION_CONTEXT

      , false);

//Outgoing packet will not inherit conversation states

ResponseContext respCtx = new ResponseContext(…);

respCtx.add(ThreadID.generate());

   …

compCtx.send(respCtx);

The com.kenai.jabberwocky.framework.property.inheritConversationContext is not a permanent attribute as it is cleared after every processing cycle. So you will have to set it again on the next processing cycle if you wish to further suppress conversation inheritance.

The September 2012 bundles supports the latest conversation features. You can download them here. I’ve just uploaded a new bundle with bug fixes.

Till next time.

Art of Conversation: Explicit Conversation – Part 3

In my previous blog entry, I talked about implicit conversations. A quick recap: implicit conversations are conversations that are created for you by the Vorpal framework whenever you send or receive an IQ get or set. These conversation are terminated when you send or receive an IQ result or error.

Conversation gives you the ability to store states either in a ConversationContext object or in the message handler class annotated with @ConversationScoped. For example, you can associate some state with an outgoing IQ get packet; so when the reply returns, you can look at the state know what to do with it. When Vorpal creates another implicit conversation in the context of an ongoing conversation, the newly created conversation will inherit the states from the ongoing conversation. This is known as conversation inheritance.

In this blog, we will introduce explicit conversation, viz conversation that are directly under the control of the application; explicit conversation share lots of similar properties as implicit conversations but build on these concepts.

What are explicit conversations?

In XMPP, conversation between 2 Jabber entities are marked with <thread>; see this document. You will typically find <thread> element in message. This allows both parties to track the packets between the 2 conversing entities. Below is an example of a pair of message exchange between a patient and his psychiatrist.

<message from=”normanb@arkham” to=”eliza@arkham” type=”chat”>
<body>I want my mother</body>
<thread>1234567890</thread>
</message>

<message from=”eliza@arkham” to=”normanb@arkham” type=”chat”>
<body>Why do you need your mother?</body>
<thread>1234567890</thread>
</message>

To see how to use explicit conversation, lets write s service, call Eliza (which is a bot), that provides psychiatric help.

@ConversationScoped
@Message
@To(“eliza@{__subdomain__}”)
public class ElizaService {
   @Inject @Named(“__conversation__”) Conversation conv;
   private Eliza eliza = null;

   @PostConstruct private void init() {
      conv.begin();
      //Initialize Eliza
  
   eliza = new Eliza();
        
   }

   @Body(“{body}”)
   private String handleMessage(@Named(“body”) String body) {
      if (“bye”.equals(body))
         conv.end();
      return (eliza.process(body));
   }
}

Assume we receive a new message with a thread id of 1234567890; also assume that this is the first time that Eliza service is seeing this new thread id.

So what we want to do is associate an instance of Eliza object with this thread. We will continue to use the same Eliza instance for messages with the same thread id until the sender decides to terminate the conversation.

To do that we annotate the message hander with @com.kenai.jabberwocky.framework.ConversationScoped. Do note that the @ConversationScoped is from com.kenai.jabberwocky.framework package and not from javax.enterprise.context. Any class that is annotated with this will be associated with the same conversation for the life time of that conversation.

We also inject an instance of Conversation object into the handler. The Conversation object is use to start and terminate a conversation. For explicit conversation, if you do not explicitly start them, the conversation will not begin. It is also important that we qualify the conversation with @Named(“__conversation__”) or else the appserver will try to inject a Conversation instance from JSF instead of from Vorpal.

This is how the code works: after the handler has been instantiated; the @PostConstruct method will start the conversation by calling conv.begin(). Since ElizaService class is annotated with @ConversationScoped, this particular instance will be associated with this thread id for the life time of the conversation.

After invoking @PostConstruct, normal method processing occurs viz. handleMessage() will be called. When we receive an ‘bye‘ in the message body, we terminate the conversation by calling conv.end(). After that the ElizaService instance will be discarded. A new instance will be created even when the next message contains the same thread id.

Once you have started a conversation, any handlers annotated with @ConversationScoped that process messages from this open conversation will also be automatically associated with the conversation. This is known as a conversation group.

@ConversationScoped
@Message
@To(“eliza@{__subdomain__}”)
@DataForm
   public class Configure {

For example if the following is the one of the handler that is invoked after the patient has started normanb@arkham has started the conversation, then Configure object will be associated with the conversation. When the conversation terminates, all objects in the conversation group will be released.

Using ConversationContext

The above can be rewritten using ConversationContext. A CovnersationContext is a map and behaves like HttpSession. You can also use this to store the conversation state. The following shows ElizaService rewritten using ConversationContext

@Message
@To(“eliza@{__subdomain__}”)
public class ElizaService {
   @Inject @Named(“__conversation__”) Conversation conv;
   @Inject ConversationContext convCtx;

   @PostConstruct private void init() {
      conv.begin();
      Eliza eliza = new Eliza();
      convCtx.setAttribute(“elizaInstance”, eliza);
   }

@Body(“{body}”)
private String handleMessage(@Named(“body”) String body) {
   Eliza eliza = (Eliza)convCtx.getAttribute(“elizaInstance”);
   if (“bye”.equals(body))
      conv.end();
   return (eliza.process(body));
   }
}

You can use a combination of @ConversationScoped objects and ConversationContext to hold your conversation state.

How do you decide which to use? If you have a really complex object then @ConversationScoped objects are the way to go; but if you decide to share conversation states with all handlers (including those that are not annotated with @ConversationScoped) then you should consider ConversationContext.

Creating new conversation threads

Vorpal does not support nested conversations; furthermore a message handler can only deal with 1 conversation at a time. Let say we have the following use case; in the course of normanb@arkham‘s conversation with Eliza, Eliza would like to start a new conversation thread on a new topic with normanb. In other words, normanb is have 2 open conversation with Eliza (2 unique thread id). To do this, you need to manually insert a new thread id into an outgoing message packet. The following code shows how this is done

@Message
@To(“eliza@{__subdomain__}”)
public class ElizaService {
   @Inject @Named(“__conversation__”) Conversation conv;
   @Inject ConversationContext convCtx;

   @PostConstruct private void init() {
      conv.begin();
      Eliza eliza = new Eliza();
      convCtx.setAttribute(“elizaInstance”, eliza);
   }

   @Body(“{body}”)
   private ResponseContext handleMessage(@Named(“body”) String body) {
      Eliza eliza = (Eliza)convCtx.getAttribute(“elizaInstance”);
      ResponseContext reply = new ResponseContext(
      ResponseContext.Type.Message);

      if (“bye”.equals(body))
         conv.end();
      else if (body.toLowerCase().startsWith(“new topic”)) {
         body = body.substring(9).trim();
         //Insert a new ThreadID into the response
        
responseContext.add(ThreadID.generate());
      }           
      responseContext.add(eliza.process(body));
      return (responseContext);
   }
}

Assume that when the response from normanb starts with the phrase ‘new topic‘ then we want to start a new conversation thread. Under normal circumstances, if there are no open conversation, then what we do is inject an instance of Conversation object in, call begin(). The out going message will then be marked with a randomly generated thread id.

However in the situation above, we already are in the context of an open conversation; invoking begin() again will result in an IllegalStateException. In this situation, to let Vorpal know that the out going message is under a new conversation, you have to manually add the <thread> element into the message packet. You can either manipulate the Message packet or use ResponseContext as show above to add a thread id.

When Vorpal sends out the message, it’ll create a new a Conversation and a ConversationContext objects for this new conversation; it then
automatically starts the conversation for you viz. calls Conversation.begin(). This is the only time an explicit conversation behaves like an implicit conversation. You’ll still need to manually terminate it. The ConversationContext will inherit all the values from the existing open conversation.

This new way of creating conversation works in all handlers including IQ and Presence; for example, you can create a new conversation with another Jabber entity when you are handling a disco#info. The new conversation’s ConversationContext will inherit all the values from the ConversationContext of the currently opened implicit conversation.

The only restriction is that this works with messages (<message>) only.

Flash Conversations

Inheriting conversation only works when a new conversation is created and started in the context of an existing conversation; for example in the case of implicit conversation (eg. sending out disco#info get in a disco#item result handler), the out going IQ packet’s conversation is automatically started by Vorpal so it’ll inherit the conversation states of the existing conversation. Similar situation with inserting a new thread id into messages described above.

There are cases where you would like to inherit the conversation states from an existing conversation but you only wish to start the conversation in the next packet that you receive. Here is an example

  1. fred@myjabber sends a message to chess@playground.myjabber to create a chess game
  2. chess@playground.myjabber randomly generates a game room name abcde. Saves abcde@conference.myjabber and fred@myjabber in ConversationContext.
  3. sends a presence to abcde@conference.myjabber to create a chat room to host the game
  4. conference.myjabber returns after creating the room
  5. chess@playground.myjabber now sends a direct invitation to fred@myjabber. Also updates ConversationContext with any other data need for the game

Since the invitation to fred@myjabber is a message (<message>), no conversation will be started. What Vorpal does at this point is this; if there is an existing conversation, it’ll take all the states in the ConversationContext and save that in the flash conversation map. Vorpal constructs a pseudo thread id made up of the ‘to’ and ‘from’ attributes from the message.

Sometime in the future, when fred@myjabber confirms the chess game, Vorpal will now pickup the existing ConversationContext (and Conversation) object from flash and starts the conversation.

So flash conversations are conversations that are not started but have the potential to start sometime in the near future; its model after Ruby on Rails’ flash. If a flash conversation is not started within a certain time period (default is 3 mins), its discarded.

See Arkham in playground especially CreateChatroom.java and ConfirmChatroom.java.

You can find the latest bundle here.

Art of Conversation: Implicit Conversations – Part 2

You say it best, when you say nothing at all” – When You Say Nothing At All, Ronan Keating

In my last blog, I reintroduce conversation support in Vorpal. Conversation allows Vorpal application to keep states about ongoing covnersation. One of the side effect of conversation support is a notion call implicit conversation. 
What is an implicit conversation? It is when a Vorpal application
  • Either sends/receives an IQ get/set packet, and
  • Receives/sends an IQ result/error packet in response
So an implicit conversation is an exchange between a pair of corresponding IQ packets, having the same packet id; the conversation is started automatically when you sends or receives an IQ get or set and is terminated automatically when you receives or sends an IQ result or error packet.

Starting and Terminating Implicit Conversations

Note: the following examples makes heavy use of service discovery. If you’re not familiar with service discovry have a look at XEP-0030 and also this blog.
Lets look at how implicit conversation works with the following use case: assume that we are writing a service  that upon receiving a ‘chat’ message will send and disco#info to the server. When the reply comes back, the services will send a message back to the message sender with the number of features that the server supports.
Lets first look at the code to send the disco#info and receive the result

   @Body("{ignore}")

   private void handleMessage(@From JID from) { 

      ResponseContext iqCtx = new ResponseContext(ResponseContext.Type.IQ);   
      iqCtx.to(postStartEvt.getDomain()).add(IQ.Type.get)
            .add(new IQChildElement(“query”, “http://jabber.org/protoco/disco#info&#8221;);
      //Implicit conversation starts the moment we send the IQ packet  
      componentContext.send(iqCtx); 
   } 

   @Query(namespace=”http://jabber.org/protoco/disco#info&#8221;) 
   @IQType(type=IQ.Type.result) 
   private void discoInfoReply(@Feature List<FeatureSpecification> features) {
      //Do something 
   } //Implicit conversation ends the moment we exit the handle method 

In the example above, we have 2 different methods  handleMessage() and discoInfoReply()in 2 difference classes (one class annotated with @Message and the other with @IQ). On receiving a chat message, handleMessage() constructs a disco#info and sends it to the server. If every thing goes well, a reply will return to be handled by discoInfoReply() handler. 

An implicit conversation begins the moment we send an IQ get/set packet (the green lines); the implicit conversation resumes when we receive an IQ result/error reply and do not end until we exit the handler (the red lines). In the case where this is no handler to the IQ response, the implict conversation will resume and then terminates immediately. The only time that we have an incomplete implicit conversation is if we did not receive any reply.  More on this later. 
The reverse is also true viz. when our service receives a IQ get, the implicit conversation starts the moment the handler is invoked and ends when we exit the handler method.
Vorpal tracks IQ request/response packets by their packet id. In XMPP, IQ packets are required to have packet ids; furthermore an IQ request/response packet pair must have the same packet id. Vorpal uses this to track the implicit conversation.

Saving States in ConversationContext

Lets continue with our example. We have fulfilled the first part of our use case and that is by sending a disco#info to our server. However to send the result back to message sender we need to know the sender’s JID. But the sender’s JID is only available in handleMessage(). For this we will need to use ConversationContext

   @Message

   public class MessageHandler {
      @Inject ComponentContext componentContext;
      @Inject ConversationContext convCtx;
      @Body(“{ignore}”)
      private void handleMessage(@From JID from) { 
         //Save the send in the conversation context
         convCtx.setAttribute(“sender” from);

         ResponseContext iqCtx = new ResponseContext(ResponseContext.Type.IQ);   
         iqCtx.to(postStartEvt.getDomain()).add(IQ.Type.get)
               .add(new IQChildElement(“query”, “http://jabber.org/protoco/disco#info&#8221;);

         //Implicit conversation starts the moment we send the IQ packet

         componentContext.send(iqCtx); 

      }

   @IQ
   public class IQHandler {
      @Inject ComponentContext componentContext;

      @Inject ConversationContext convCtx;

      @Query(namespace=”http://jabber.org/protoco/disco#info&#8221;) 
      @IQType(type=IQ.Type.result) 
      private void discoInfoReply(@Feature List<FeatureSpecification> features) {
         JID from = (JID)convCtx.getAttribute(“sender”);
         ResponseContext msgCtx = new ResponseContext(ResponseContext.Type.Message);
         msgCtx.to(from)
               .add(“There are ” + features.size() + ” features”);
         componentContext.send(msgCtx);
      //Implicit conversation ends the moment we exit the handle method
When we receive the chat message, we save the send’s JID in a ConversationContext. A ConversationContext is like a HttpSession object. So when we send the disco#info, the ConversationContext will be associated with the implicit message. Upon receiving a reply, Vorpal will reinject the same ConversationContext object based on the id of the IQ packet.
Note: Observant reader may recall that in handleMessage() the implicit conversation doesn’t start until we send the IQ packet (the green line). So the ConversationContext object in this case is said to be an unbounded context as oppose to a bounded one in discoInfoReply().

Inheriting Conversations

ComponentContext.send() allows you to send more than 1 packet. What if in handleMessage() method, we send more than one disco#info packet to different JID entities eg. conference.mydomain and pubsub.mydomain like the following code snippet

   componentContext.send(toConference, toPubsub);

where toConference and toPubsub are disco#info packets. Firstly, Vorpal will start two separate implicit conversation since the two IQ packets will have different ids. Secondly both the implicit conversations will have their own separate ConversationContext but these two ConversationContexts will inherit all the attributes from the existing convCtx. Conversation inheritance applies to all implicit conversations irregardless of whether a ConversationContext is bound or unbound.
Lets consider another scenario. Assume that instead of sending back the number of features that a XMPP server supports, we want to return a list of all Jabber entities discoverable through disco#items. The algorithm to do that is roughly like this:
  1. Send a disco#info to the Jabber entity. If this is the start, then it’ll be the JID of the XMPP server
  2. Check if the replies contains disco#items
  3. Send a get disco#item to the Jabber entity. Again the first JID will be the XMPP server
  4. Save a list of all the item when the disco#item reply returns. 
  5. For each JID in <item>, repeat from step 1 until there are no more disco#items to be found
The following code kicks off this process. when we receives a message

   @Message

   public class MessageHandler {

      @Inject ConversationContext convCtx;

      @Body(“{ignore}”)
      private void handleMessage(@From JID from) { 

         ResponseContext iqCtx = new ResponseContext(ResponseContext.Type.IQ);
         iqCtx.add(IQ.Type.get).to(postStartEvt.getDomain())
              .add(new IQChildElement(“query”, “http://jabber.org/protocol/disco#info&#8221;);
         //Setup the conversation context
         convCtx.setAttribute(“requestor”, from);
         convCtx.setAttribute(“jids”, new HashSet<JID>());
         convCtx.setAttribute(“disco#info”, new AtomicInteger(1));
         convCtx.setAttribute(“disco#items”, new AtomicInteger(0));
         postStartEvt.getComponentContext().send(iqCtx);
   }

The only important thing here is that we setup a collection to hold our discovered JIDs. Also we create 2 AtomicInteger to count the number of disco#info and disco#items that we have sent.  This is how the counters work: every time we send a get disco#info we increment the count. When we receive a reply for disco#info, we decrement the counter. When the counter reaches zero, we know that we have completed our search.  The same goes for the disco#items counter. We also keep the client’s JID for the reply.
The following class shows the real work of collating the Jabber entities.

   @IQ

   public class CollateJID {


      @Inject ConversationContext convCtx;

      @Query(namespace=”http://jabber.org/protocol/disco#info&#8221;)
      public ResponseContext discoInfoResult(@From JID from) {
         //Decrement the disco#info counter cause we have just got a reply
         discoInfo(-1);
         ResponseContext iqCtx = new ResponseContext(ResponseContext.Type.IQ);
         iqCtx.add(IQ.Type.get).to(from)
               .add(new IQChildElement(“query”, “http://jabber.org/protocol/disco#items&#8221;);

         //Decrement the disco#items counter cause we have just got a reply
         discoItems(1);

         return (iqCtx);
      }

      @Query(namespace=”http://jabber.org/protocol/disco#info&#8221;)
      public ResponseContext discoInfoResult() {
         //Decrement the disco#info counter cause we have just got a reply
         discoInfo(-1);
         //Check to see if 
         if (shouldEnd())
            return (createReply());
         return (null);        
      }
      @Query(namespace=”http://jabber.org/protocol/disco#items&#8221;)
      public ResponseContext discoItemsResult(@Item List<ItemSpecification> jids) {
         //Decrement the disco#items counter cause we have just got a reply
         discoItems(-1);
         if (jids.isEmpty() && shouldEnd())
            return (createReply());
         
         Set<JID> jidSet = (Set<JID>)convCtx.getAttribute(“jids”);
         ResponseContext iqCtx = ResponseContext(ResponseContext.Type.IQ);
         iqCtx.add(IQ.Type.get)
               .add(new IQChildElement(“query”, “http://jabber.protocol.disco#info&#8221;));
         for (ItemSpecification j: jids) {
            iqCtx.to(j.getJid());
            jidSet.add(j.getJid);
         }
         //Increment the disco#info counter. We are about to send a bunch of these
         discoInfo(jids.size());
         return (iqCtx);
      }
      //Helper methods
      private int discoInfo(int i) {
         return (((AtomicInteger)convCtx.getAttribute(“disco#info”)).addAndGet(i));
      }
      private int discoItems(int i) {
         return (((AtomicInteger)convCtx.getAttribute(“disco#items”)).addAndGet(i));
      }
      private boolean shouldEnd() {
         return ((discoInfo(0) <= 0) && (discoItems(0) <= 0));
      }
      private ResponseContext createReply() {
         Set<JID> jidSet = (Set<JID>)convCtx.getAttribute(“jids”);
         ResponseContext reply = new ResponseContext(ResponseContext.Type.Message);
         reply.to((JID)convCtx.getAttribute(“requestor”))
               .add(jidSet.toString());
         return (reply);
      }
   }

The code is long but its acutally quite simple. After sending the initial disco#info to the server (in handleMessage()), the returns the result. This is handle by our IQ handler class CollateJID above. The disco#info reply can produce 2 different response; if the reply does not contain disco#items feature (meaning the server does not have any Jabber entities), then this will be handled by discoInfoResult() method (in blue). We decrement the disco#info counter in the conversation context and checks if we should end. 
If the disco#info reply do contain disco#items feature, then we send a disco#items to the sender of the reply packet. At the same time we decrement disco#info and increment disco#items counter. This is handled by discoInfoResult() (in red).
When we receive the disco#items reply, for every JIDs in the reply we send need to further send a disco#info packet. This is handle by discoItemsResult() (in green). After that, the entire service discovery process repeats itself.
What is interesting to note is the conversation inheretance happening here. Once we have initialized the initial conversation context object with the sender’s JID and the counters, these values are inherited in all subsequent implicit conversations. For example, when we are in discoInfoResult() (the red one) in response to a disco#info reply,  Vorpal will inject the previously created conversation context into the handler. Now when we now send out disco#items, Vorpal will create a new conversation context for this implicit conversation. But since there is an existing conversation context, the new conversation context for the outgoing disco#items packet will inherit all the values.
What happens if there is no response to a previously sent disco#items or disco#info packet? Wouldn’t Vorpal be holding all these objects in conversation context? Memory leak? All conversations have timeouts unless you turn them off. When timeout occurs for a particular conversation, the conversation context object for that conversation will be destroyed. We will look at timeout and handling them in a future blog.
If you’re testing implicit conversation and find a bug or have issues or questions, please post them to the forum.
%d bloggers like this: