Photostream App – Part 3

In the final part, we will create a simple hybrid mobile application that will snap a photo and upload it to the file upload service.

I’m using the following framework and tools

  1. Ionic – this is the responsive web/mobile framework based on AngularJS
  2. Cordova camera plugin – https://github.com/apache/cordova-plugin-camera
  3. Cordova file transfer plugin – https://github.com/apache/cordova-plugin-file-transfer

 

The Ionic source is available here.

Till next time.

Advertisements

Photostream App – Part 2

In the the second part of the “Photostream” trilogy, we will look at how the server notifies all the clients who are observing its stream. There are 2 ways we can do this; either by using server sent events or with websocket.

Since JavaEE 7 supports websocket I’ll be using websocket. Websocket is a 2 way communication channel. In photostream app, the notification/data only flows on way viz. from the server to the client (those observing our photostream) so server sent event (SSE) is actually more appropriate. But to use SSE in JavaEE (in Glassfish/Payara in particular), you have to use add additional libraries.

In you’re using Glassfish/Payara and you would like to try SSE, see this document.

The source code for part 2 is available here.

Till next time.

Photostream App Trilogy

I’ve started a new screencast. In the screencast, I’ll show you how to develop a photostream application. The photostream application has 2 parts

1. Allow users to upload images

2. Push the newly uploaded images (a photostream) to connected clients.

The following is a list of open source projects and technologies that I’m using to write this application

    Rather than creating one long screencast, I’ve split it into 3 parts.

The first part – upload service and and Angular client.

The second part – server notification using websocket

The third part – hybrid mobile app to take pictures and uploading them.

Here is the first part

 

Some points to note about

  • I’m using asynchronous Servlet to handle the file upload. This is to improve performance and to better utilize server resources by releasing the request thread when the image file is being persisted. We can resume the request once the file have been saved
  • Since this is a simple application, I’ve saved the images directly in the document root. You should use a CMS or something like a GridFS to save your images
    Till next time.

Screencast: Creating a Hybrid Mobile Application with Ionic and Cordova

p { font-size: 1.3em; }

I’ve just published a new screencast on how to create hybrid mobile application using the Ionic framework. This screencast is a remake of an older version.

This remakes includes infinite scrolling, integration with 2 Cordova plugins as well as customizing the generated Android ‘shell’.

The source code and the APK is available here.

Till next time.

A Cordova Plugin for Getting the Phone Number on Your Android Phone

Something that I put together while working on a project. There is one available. There is even a nice blog by the author.

My plugin is almost exactly the same as Simon’s. The only difference is that mine is about 3 years late. I did this because I wanted to learn about Cordova plugin development and also to use try out Android Studio.

You can get the details of the plugin here.

BTW the plugin will not work if the phone number is not available on your phone. Go to Settings | About device | Status | My phone number to see if the number is available before you use.

Till next time

Screencast: Creating a Hybrid Mobile Application

Instead of writing a blog for this month, I’ve done a short (30 mins) screencast on how to use Ionic, Angular and Cordova to create a hybrid mobile app.

 

The source (just the www directory and the APK) can be found here.

The app is incomplete; one of the things that you’ll notice is that that when you try to follow a link, the app will open that in a browser. But when you click on the back button, it does not go back to the app. You need to handle the link and display that in the inappbrowser plugin. I’ll leave that as an exercise for the reader.

I initially wanted to display the blogs using drawer navigation; this would make the video longer that I intended. So if your’re curious have a look at Ionic’s side menus.

Till next time.

Building Your Own Presentation Server

Trojan_Room_coffee_pot_xcoffee I gave a presentation recently on the topic of mobile enabling enterprise applications. The context was that if you have a Java EE based application tucked away somewhere in your organization how do you make that application mobile and HTML5 friendly.

As I was creating the presentation, it occurred to me that perhaps my presentation should be like a mini demo of sorts to illustrate some of the principles I was sprouting. I really liked the idea of the Keynote Remote app (which is no longer available) or something equivalent like Slideshow Remote.

So I wrote a simple presentation server that will allow me to control the presentation from my mobile phone. Instead of using Bluetooth or WIFI, my remote control will rely on the web. The following diagram shows how the overall architecture of the presentation server

prezo_2

  1. You start a presentation (the left side of the diagram above) by selecting the presentation from a browser. Yes the presentation is done on a browser like Google Presentation or Prezi.
  2. You then use your mobile phone to open the presenter’s console (right side of the diagram) to control the presentation. The console allows you to advance the slides; it also shows you speaker’s note.
  3. Participants can asked question using the comment tool. Again this is a web application. All comments are displayed on the presentation. The yellow box on the top left corner of the presentation is an example of a comment.
  4. After answering the question, the presenter can dismiss the comment by clicking on the Delete button on the presenter’s console

For the impatient, the source is available here.

Starting A Presentation

Now that we have describe the main features of the presentation server, lets see how this whole shebang works. Every presentation must be started/loaded using the following URL

http://myserver:8080/notppt/presentation/presentation.html

The presentation.html is the name of the presentation file found in the document root of a Java EE web application. The presentation file is a HTML presentation developed using Reveal.js. (more later). Presentations must be loaded from the path /presentation/; this is mapped to the following Servlet

@WebServlet("/presentation/*")
public class PresentationServlet extends HttpServlet {

   @EJB PresentationMap map; 

   protected void doGet(HttpServletRequest req, HttpServletResponse resp) {
      //Get the presentation file name
      String presoFile = req.getPathInfo();
      String uuid = ...//Generate a presentation id
      //Load the presentation file, ctx is ServletContext
      InputStream is = ctx.getResourceAsStream(presoFile);
      String baseURI = req.getScheme() + "://"
            + req.getServerName() + ":"
            + req.getServerPort() + ctx.getContextPath();
      //Read it into memory as DOM
      Document doc = Jsoup.parse(is, "UTF", baseURI);
      //Register it
      map.create(uuid, doc);
      //Load the presenation using a redirect
      resp.sendRedirect(url + "?pid=" + uuid);
   }
}

As you can see the PresentationServlet sets up the initial data structure for the presentation. It generates a unique id for it. After generating the id, it loads it into memory as a DOM-like object. The ‘DOM’ is then associated with the generated id and saved in a map which in here is represented very simplistically by a Singleton EJB. We now get the browser to load the presentation by sending the following URL back as a redirect with the presentation id (pid).

http://myserver:8080/notppt/presentation.html?pid=12345

As I was working on this, I found a really nice HTML parser call Jsoup. Jsoup is a Java HTML parser that lets you access the HTML document using CSS selectors, just like jQuery. Bye bye XPath! We use Jsoup to parses our HTML presentation; it returns returns a Document object which we will  use later to extract the presenter’s note. To learn about Jsoup, see the tutorial.

Presentation Client

The presentation is a HTML presentation developed using the Reveal.js framework. See this link on how to develop presentation with Reveal.js. To add speaker’s note into the slide, embed <div> (one for each point) inside an <aside> tag with a data-notes attribute like so

<section>
   <h1>This is my presentation</h1>
   <aside data-notes=””>
      <div>Speaker notes</div>
      <div>One div for each point</div>
   </aside>
</section>

You can look at presentation.html in the source for a complete example. Every presentation loads a jQuery script call presentation.js. The script performs the following 2 things

  1. Initializes the presentation using Reveal’s API. See the documentation here.
  2. Once Reveal have been initialized, it opens a server sent event connection back to the presentation server. The presentation uses this channel to listen to commands (eg. next slide, previous slide) and control the presentation according to the received commands.

The following is a simplified outline of the above 2 steps

$(function() {
   //Reads the presentation id from the URL
   var pid = getParameterByName("pid");
   //When Reveal has completed its initialization
   $(Reveal).on("ready", function() {
      //Create a SSE, only listen for commands of our id
      var sse = new EventSource("slide-event"); 
      $(sse).on(pid, slideControl);
   });
   ...
});
//Callback to execute when we receive commands
function slideControl(result) {
   //Read the data
   var data = JSON.parse(result.originalEvent.data);
   //If it is a movement command, then move the slide
   if (("next" === data.cmd)|| ("prev" === data.cmd))
      Reveal.slide(data.slide); {
      return;
   }
   //Else process other commands
   ...
}

You can see the entire script in web/app/presentation.js in the source.

Before proceeding any further, lets see how every piece of the system communicates with each other

prezo_3

After the presentation opens, it creates a SSE channel back to the presentation server. The presentation will only listen to commands with its presentation id; it does this by listening for specific SSE packets with its presentation id .

Presentation SSE Channel

When the presentation client displays the presentation in also creates a SSE channel back to the server to listen to commands from the presenter’s console. The SSE’s URL is /slide-event. Since JavaEE does not support SSE, I’ll use this SSE library. You can read more about the SSE library from this blog here and here. Our SSE implementation is a follows

@RequestScoped
@ServerSentEvent("/slide-event")
public class SlideNotificationChannel extends ServerSentEventHandler {
   public void publishEvent(final String id, final JsonObject obj) {
      ServerSentEventData data = new ServerSentEventData();
      data.event(id).data(obj.toString());
      //connection is inherited from ServerSentEventHandler
      connection.sendMessage(data);
   }
}

We create a SSE data, fill in the event id which is the presentation id and the command (more details of this in Presentation Control section). The command tells the presentation client to move to the next slide, return to previous slide, etc.

The SSE data that we are publishing is associated with an event name (presentation id here). This ensures that only client listening to that id will get the data. This is the line

$(sse).on(pid, slideControl);

in the presentation client (presentation.js) that registers the pid with the SSE’s event id.

Side note: this is not strictly true in this implementation as all presentation clients are connected to the same SSE channel. All presentation client will receive commands regardless of whether its for them; but because they are only listening to their own pid, they will only respond if the SSE’s event id matches their pid. Ideally the SSE channel name should be partition with the presentation id viz. clients should connect to /slide-event/<pid> instead of /slide-event. The current SSE library is not able to parameterize the SSE channel name.

Presenter Console

Lets look at the presenter console. The purpose of the presenter console is to allow the presenter to control the presentation and also to look at the speaker’s note for the current slide.

When you open the presenter’s console, you have to select which presentation you wish to control from a combo box.

The presenter’s console is build with jQuery and jQuery Mobile. The following code shows how commands are sent to the server

$(document).on("pageinit", "#main", function() {
   ...
   $("#next").on("click", function() {
      $.getJSON("control"), {
         "cmd": "next",
         "value": pid, //presentation id
      }).done(function(result) {
         //Display the speaker's note
         var notes = "";
         for (var i in result)
            notes = "* " + result.notes[i] + "\n";
         $("#notes").val(notes);
      });
      ...
   });
});

When you click on the next button (code shown above), the console will issue a command to the server using jQuery’s $.getJSON() API. The following request is made to the server to move to the next slide for presentation 123456.

http://.../control?cmd=next&value=123456

If the server manage to successfully advance the slide, the server will return the new slide’s speaker’s notes. The notes are then extracted from the result and displayed on the console. The following is an example of the response from the server

{
   "cmd": "next", "slide": 2, "totalSlides": 10,
   "notes": [ "speaker note", "First point", "Second point" ]
}

Presentation Control

Commands from the presenter’s console to the presentation server are intercepted and processed by ControlServlet; if you notice the $.getJSON() from above sends the command to control URL. This URL is mapped to ControlServlet

@WebServlet("/control")
public class ControlServlet extends HttpServlet {
   //Get the presentation map
   @EJB PresentationMap map;

   //Get the SSE channel
   @Inject @ServerSentEventContext("/slide-event")
   ServerSentEventHandlerContext<SlideNotificationChannel> slideEvents;

   @Override
   protected void doGet(HttpServletRequest req, HttpServletResponse resp) {
      //Get the command and the pid from the request
      String cmd = req.getParameter("cmd");
      ActivePresentation preso = map.get(req.getParameter("value"));
      JsonObjectBuilder builder = Json.createObjectBuilder();
      switch (cmd) {
         //Handle the next command 
         case "next":
            builder.add("cmd", "next")
               .add("slide": preso.currentSlide())
               .add("totalSlides": preso.totalSlides())
               .add("notes": preso.get())
            break;

                     …
      }

      //Send command back to the console
      Json json = builder.build();
      resp.setStatus(HttpServletResponse.SC_OK);
      resp.setContentType("application/json");
      try (PrintWriter pw = resp.getWriter()) {
         pw.println(json.toString());
      }

      //Broadcast the command to all presentations
      for (SlideNotificationChannel chan: slideEvents.getHandlers())
            chan.publishEvent(preso.getPid(), json);
      }
      ...
   }

The code for ControlServlet is quite straightforward. We start by first constructing the command object in JSON using the new JSON API from JavaEE 7. The notes attribute is the speaker notes for the current slide. The speaker notes are extracted from under <aside data-notes=””> element using Jsoup. Look at the source to see how this is done. They are returned as a JSON array.  After creating the command object, we send the object back to the presenter’s console indicating that the command have been accepted.

Now we have to inform the presentation client to execute the command, which is move the to next slide in this case. Using injection, we get the handler to all SSE channels listening on /slide-event. We then loop through all the connections and publish the event.

The publish event now goes back to the presentation client; the slideControl function is invoked (see Presentation Client section above)  and this advances the slide.

Trying Out the Demo

The source for the presentation server is available here. All the dependencies can be found under lib directory. The presentation server is written to JavaEE 7. I’ve tested in on Glassfish 4.0. Once you’ve deploy the application, point your browser to http://your_server:your_port/notppt. You will see the landing page.

The first item is to start a Start Presentation. You must have at least 1 presentation started. Click on this (open it in a new window) to start your first presentation.

Now you can select either the Control or the Comment link. The Control will take you to the presenter’s console. Try opening this on your mobile phone browser. Select a presentation to control from the combobox. Comment allows the audience to comment or ask questions on the slide.

It is very easy to take the Control and/or the Comment portion and make that into an app (Android or otherwise) by wrapping the web portion only with Cordova/Phonegap.

The presentation server currently has only 1 presentation. If you’re thinking of using it here are some suggestions.

  • Add an interface to to manage the presentations.
  • Automatically inject presentation.js into every uploaded presentation.
  • Add security (Java EE container managed).
  • Some jQuery Mobile gesture like swipe in the presenter console to control the slides instead of buttons
  • Support for other HTML presentation toolkit in particular  impress.js and deck.js

Similar Tools

It’s worth mentioning that Reveal.js do have a presentation console with speaker’s notes. There is another excellent blog on the same subject. Both of these approaches uses node.js as their server.

I also just found out that slid.es, the people behind Reveal.js, have rolled out something similar. See the following video. The idea is basically the same except that they have done it with more panache. Go check it out.

Till next time.

%d bloggers like this: