Developing XMPP Components – The Setup

There are few ways to develop ‘server’ type XMPP applications; by server I mean a bot or an application connecting to a XMPP server and providing some sort of service like weather, fortunes, die roll, magic 8 ball or even act as mediator for games like Diplomacy and chess. For the sake of this blog, lets call this a xmpp service. The following are 3 possible ways you can implement xmpp service

1. As a ‘regular’ XMPP user – what I mean by this is that the xmpp service is like a regular user. The service would create an account with say jabber.org, logs in and waits messages. The XMPP service can also be part of a real user’s (human) login. When a person logs in, the service starts. XMPP clients like Spark provides a plugin architecture (sparkplug) to develop such services. This method is by far the easiest method to develop the services; the disadvantage is that your service has a limited reach because in order to use the xmpp service other users must discover the JID and must also explicitly add that JID into their roster before you can do anything with it.
2. XMPP Server plugins – another way of developing these services is to make them part of the server. Most servers have some sort of a plugin architecture; see Openfire’s as an example. The advantage of this approach is that the service does not need to in a users roster for the user to use it. And discovering the service boils down to sending a disco#item packet to the server. For example sending a disco#item to jabber.org returns the following <item jid=’conference.jabber.org’ name=’Multi-User Chat’/> in part of the reply. We can now start a conference with that JID. The disadvantage of this approach is that the service that you have developed may not be portable across to other servers.
3. XMPP components  – the XMPP Foundation has defined a standard protocol for services to connect to XMPP servers. This is defined in Jabber Component Protocol (XEP-0114). The way this works is that the service will communicate with a standard XMPP server using a predefine mechanism for identification and authentication. Once the handshake is successful, the XMPP component effectively becomes part of the server and behaves like a server plugin in that it can be discoverable using the standard disco#item as well as disco#info mechanism. The component runs outside (in a separate process) of the XMPP server. So you can shutdown the service without affecting the server. An added advantage is that because XEP-0114 is a standard, in theory , you can connect the service to different XMPP servers.

There are a few XEP-0114 Java libraries like XMPP Gateway and Whack. We will be using Whack. The first step is to setup the CLASSPATH the the appropriate libraries

• Start by downloading the Whack source from the repository as shown here. The reason why you download the source instead of just the binaries is because when you eventually run your XMPP service, the Whack runtime will need a few more libraries that are not present in the binary download. So its better to bite the bullet and get the source which contains all the dependencies. There is also 2 sample services which you can reference. They are actually the same service  (weather) but implemented differently
• Build it either with Ant or Maven. For Ant the whack.jar will be target directory and for Maven it’ll be in sample/weatherabstract/build/lib/merge/whack.jar.
• Now set your CLASSPATH to include whack.jar and all the JARs in build/lib/dist and build/lib/merge and other libraries that you may need for your application. You can exclude the Jetty JARs if you want to run your component as a standalone Java application. Tip: if you are using JavaSE 6 or later, you can use the wildcard character like so
  CLASSPATH=’/whack_src/build/lib/dist/*:/whack_src/build/lib/merge/*:<other_jars>’.

The second step is to give a subdomain name and a secret key that is shared between the XMPP server and the service. Both the subdomain and secret key are setup on the XMPP server. A subdomain is a name that will be used to identify your service; eg in the example above, jabber.org MUC’s JID is conference.jabber.org; the subdomain here is conference and jabber.org is the domain. So if we call our subdomain rot13 because we are going to provide a ROT13 service, then the JID will be rot13.jabber.org (assuming its on jabber.org).

The following diagram shows how to set these 2 values if you are using Openfire.

• Login to the admin console http://localhost:9090
• Click on Server Settings then External Components
• Type in the subdomain name and the secret key and click on Add Component to save the setting

In my next blog we will look at developing the XMPP service.

Difference between Pubsub and PEP

I’ve had lots of opportunity over the past few weeks to work with both PEP and Pubsub. While both delivers data/item/payload, the way the 2 services are structured are quite different. The following table list their respective differences

	Pubsub	PEP
Service discovery	Typically discovered by sending a disco#item to the server	Discovered via entity caps in the presence
Node	Can create/delete multiple nodes if authorized. Node can be configured during creating	Only one node, your JID; and cannot be deleted. Not configurable
Publish/subscribe	Depending on configuration, anyone can publish/subscribe to nodes (list-single_access-model, list-single_publish_model)	Declare in your disco#info your intention to publish certain type of info or receive certain type of info Only those in your roster and have declared their intention to receive the type of info will receive it
Payload	Configurable number of items that you can persist in the node (persist_items, max_items) Can either deliver a notification of the payload or the payload itself (deliver_payloads) Payload may be marked delayed and timestamped if they are delivered after the fact	Persist only the last payload published No timestamped/delay delivery

Hopefully this will help you decide which model is suitable for your need. Let me know if you spot any mistakes or give me feedback.

Entity Caps and PEP

I’ve been testing my User Gaming via PEP on a locally installed copy of openfire and things are looking pretty good until I tried on jabber.org. Suddenly none of my packets are getting through. Upon further investigation, I found that you must advertise what information you are publishing and what info you are interested in before jabber.org will forward your PEP packets or route others PEP packets to you. This is in keeping with the PEP specs. Surprise, surprise. And the way this information is published is using a method call Entity Capabilties (XEP-0115).

Here is roughly how it works. When you come online, you Jabber client will send out a presence packet like so

<presence id=”N2ttf-4″/>

However XMPP clients with entity caps sends presences packet looks something like this

<presence id=”N2ttf-6″>
   <c xmlns=”http://jabber.org/protocol/caps&#8221; hash=”sha-1″
         node=”http://www.igniterealtime.org/projects/smack&#8221;
         ver=”FqngNglmgiLwmdc3F7/O7kMua0E=”/>
</presence>

Note the additional <c> fragment (formatted for readability) and inparticular the ver attribute. As you might have guess this is entity caps and the ver is a hash of your entity’s, in this case your JID, capabilities. This is typically what features your client support. I use JID and clients here interchangeably; a client is a XMPP client like Pidgin and JID is what you use to get on say jabber.org.

Now remember the presence packet is going out from the entity who has just come online; as an example and in keeping with most XMPP literature, when romeo@jabber.org comes online it will send out a presences packet to every one in his roster. Juliet (juliet@jabber.org) happens to be in his roster, so Juliet Jabber client receives this packet. Juliet’s client notices that Romeo’s presence packet has a entity cap hash string but does not know what that is. So the client sends a IQ to query Romeo. The following packet goes out to romeo@jabber.org

<iq id=”purple2686f3a5″ to=”romeo@jabber.org/games” from=”juliet@jabber.org/pidgin” type=”get”>
   <query xmlns=”http://jabber.org/protocol/disco#info&#8221;
         node=”http://www.igniterealtime.org/projects/smack#FqngNglmgiLwmdc3F7/O7kMua0E=“/>
</iq>

Romeo’s client then respond to this query with a list of the features that the hash represents them

<iq id=”purple2686f3a5″ to=”juliet@jabber.org/pidgin” type=”result”>
   <query xmlns=”http://jabber.org/protocol/disco#info&#8221;
         node=”http://www.igniterealtime.org/projects/smack#FqngNglmgiLwmdc3F7/O7kMua0E=“>
      <identity category=”client” name=”Smack” type=”pc”/>
      <feature var=”http://jabber.org/protocol/caps”/&gt;
      <feature var=”http://jabber.org/protocol/xhtml-im”/&gt;
      <feature var=”http://jabber.org/protocol/muc”/&gt;
      <feature var=”http://jabber.org/protocol/bytestreams”/&gt;
      <feature var=”http://jabber.org/protocol/commands”/&gt;
      <feature var=”urn:xmpp:gaming:0″/>
      <feature var=”urn:xmpp:gaming:0+notify”/>
   </query>
</iq>

Juliet’s client will now cache the hash string against these features. The next time she sees the same hash string, her client will know exactly what those features are without ever sending an IQ.

So what has this to do with PEP? Simple PEP leverages entity caps to tell other what it is interested in receiving and what it is publishing.

Smack do not support entity cap. However there is a project on github that adds this support. You can find it here. I’ve made a fork of this since mine works with the stock Smack rather than a fork version of Smack as the original did. You can find my implementation here and I’ll be using this for the rest of the article.

The following code snippet shows how you publish your entity cap in your presence

//Create a connection, connection and login
XMPPConnection connection = …
connection.connect();
connection.login(“juliet”, “act2scene2”, “balcony”);

//Get an instance of EntityCapsManager
EntityCapsManager capsManager = EntityCapsManager.getInstanceFor(connection);

//Eg. want to support user gaming (XEP-0196) through PEP (XEP-0163)
capsManager.addFeature(“urn:xmpp:gaming:0”);
capsManager.addFeature(“urn:xmpp:gaming:0+notify”);

//Republish our presence so that it will include our entity caps
connection.sendPacket(new Presence(Presence.Type.available));

//Now publish or receive data
…

Once you have created a connection, use the connection to retrieve an instance of EntityCapsManager.  Then what you do is now add the namespace XEP-0196 (urn:xmpp:gaming:0) as our feature. Note that there are two lines. The first indicates that we are publishing user gaming data and the second with a +notify appended indicates that we are interested in receiving user gaming data published by others. You can continue adding features here. Once you are done, you now resend your presences. The presence will now include an entity cap fragment with the latest hash code of all your features, including those supported by the client by default.

One important thing to remember is that if you want a feature or extended info to be included in the hash value of your entity caps, use addFeature(), removeFeature(), setExtendedInfo() and removeExtendedInfo() from EntityCapsManager. Do not use those the 4 corresponding methods from ServiceDiscoveryManager. Wherever you add or remove a feature or extended info, don’t forget to republish your presence.

jabber.org is one public servers I’ve tried that supports PEP. To find out what other public servers supports PEP, issue a disco#info to the JID (your login) or and look for the following

<identity category=”pubsub” name=”null” type=”pep“/>

In Smack that would be

XMPPConnection conn = … //Create a connection and login
ServiceDiscoveryManager mgr = ServiceDiscoveryManager.getInstanceFor(conn);
DiscoverInfo discoInfo = mgr.discoverInfo(conn.getUser());
Iterator<DiscoverInfo.Identity> iter = discoInfo.getIdentities();
while (iter.hasNext()) {
   DiscoveryInfo.Identity id = iter.next();
   if (“pubsub“.equals(id.getCategory() && “pep“.equals(id.getType()))
      //Eureka! our server supports PEP!
}