I’ve been experimenting with ways to tie the activity of some of the systems we use together. In this case I would like activity information from Confluence to be streamed into Socialcast. There are several ways to go about this but none are perfect or “out of the box”.

Before I go on, here’s what this looks like in Socialcast with the verb and actor mentioned and object name and link:

A Little Background

You could configure a stream import in Socialcast with an RSS feed from Confluence though there are two problems with this. First, Confluence requires authentication and doesn’t support any token type convention so you have to hardcode the username and password of an account that has access to the activity you’re interested in in Confluence. Second, Socialcast appears to truncate long URLs for RSS feeds (consider this a reported bug ). So the only way I got this to work was to use a URL shortener like bit.ly but a Confluence username and password was hardcoded at bit.ly and the bit.ly URL was used in the RSS feed import. And finally what was imported wasn’t linkable back to Confluence. So there were many stumbling blocks in this, though the simplest approach.

Second I created a confluence-bot user account in both Confluence and Socialcast (with the account in Socialcast having the email address that Confluence would send from) and set this account as a watcher to all of the spaces I wanted activity information to be published. Then I created a group in Socialcast where activity would be published. The email address of the confluence-bot account was set to the email address of the Socialcast group. So when Confluence sent the watch emails out for confluence-bot, they would be sent to the Socialcast group. For some reason this just didn’t work; the emails were never arriving. I suspect Socialcast was filtering the emails as spam, as they contained a ‘Precedence: bulk’ header. It was worth a try but much too kludgy without any control so not worth pursuing in my opinion.

Ideally I wish Confluence had more dynamic notification options. More specifically I wish I could register a webhook in confluence that would let me process watch events or notifications. Even cooler would be to let each user configure a webhook URL. This would decouple the notification from the event and eliminate the need to write a Confluence plugin to handle something like this. For example, if I wanted to get Growl notifications on my desktop for Confluence activity, I could configure a webhook to post to Jeff Lindsay’s Noftify.io service (awesome by the way!).

Anyway, in lieu of that, I decided to hack together watcher emails from Confluence to Socialcast using Gaelyk on Google App Engine and the Socialcast API.

Confluence -> GAE -> Socialcast

Using Gaelyk and Google App Engine as the Glue

Confluence can send out email notifications based on watched content, and Google App Engine applications can act as an SMTP endpoint so since I’m a Groovy fan and been looking for an excuse to kick the tires on Gaelyk, I created a very simple Gaelyk application with an email handler that parsed incoming messages from Confluence and called the RESTful Socialcast API to create new messages. Unfortunately the HTML email Confluence produces isn’t valid xhtml, with unclosed tags and such, so I can’t just parse the XML and pull out the bits I need. I used some quick and dirty regular expressions to grab what I needed, making assumptions that the format of the emails wouldn’t change. I suppose I could fix the Confluence email templates, but I’m trying not to touch Confluence programmatically in this case. He’s a stripped down version of the email handler (<my_project>/war/WEB-INF/groovy/email.groovy):

// have access to message object of javax.mail.internet.MimeMessage
import groovyx.gaelyk.logging.GroovyLogger
import java.io.BufferedReader;
import java.io.OutputStreamWriter;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.Authenticator;
import java.net.PasswordAuthentication;
import java.net.URL;
import java.net.HttpURLConnection;

def log = new GroovyLogger("emailLogger")

// the email notification message from confluence is a bit messy, try to pull out the important bits from it
def theLink = ""
def m1 = message.content =~ /<\/?(?i:a)(.|\n)*?>/
if(m1?.size() > 0) {
    def firstAnchor = m1[0][0]
    def m2 = firstAnchor =~  /href="(.)*"/
    if(m2?.size() > 0) {
        theLink = m2[0][0] - "href=\"" - "\""
    }
}

String content = message.content;
int start = content.indexOf("<h4>") + 4;
int end = content.indexOf("</h4>");
def details = content.substring(start, end).replaceAll(/<\/?(?i:a|b)(.|\n)*?>/, '')

def user = "confluence-bot@mydomain.com";
def pass = "mypassword";
def addr = "https://mydomain.socialcast.com/api/messages.xml"

def authString = "${user}:${pass}".getBytes().encodeBase64().toString()
def conn = addr.toURL().openConnection()
conn.setRequestProperty("Authorization", "Basic ${authString}")
conn.setRequestMethod("POST")
conn.doOutput = true

def queryString = "message[title]=${URLEncoder.encode(message.subject)}&" + "message[body]=${URLEncoder.encode(details)}" + "&message[url]=${URLEncoder.encode(theLink)}" + "&message[group_id]=6845";

def writer = new OutputStreamWriter(conn.outputStream)
writer.write(queryString)
writer.flush()
writer.close()
conn.connect()

def res = conn.content.text
log.info(res)

There’s some configuration required to turn on the inbound email service and map the incoming email to the proper Groovlet.

Add this to your appengine-web.xml:

<inbound-services>
    <service>mail</service>
</inbound-services>

In your web.xml file, add the new servlet mapping and security constraint:

...
<servlet>
    <servlet-name>EmailServlet</servlet-name>
    <servlet-class>groovyx.gaelyk.GaelykIncomingEmailServlet</servlet-class>
</servlet>
...
<servlet-mapping>
    <servlet-name>EmailServlet</servlet-name>
    <url-pattern>/_ah/mail/*</url-pattern>
</servlet-mapping>
...
<!-- Only allow the SDK and administrators to have access to the incoming email endpoint -->
<security-constraint>
    <web-resource-collection>
        <url-pattern>/_ah/mail/*</url-pattern>
    </web-resource-collection>
    <auth-constraint>
        <role-name>admin</role-name>
    </auth-constraint>
</security-constraint>
...

There are obvious improvements that can be made to this approach and much better ways to go about this, along with a lot of considerations that I ignored (e.g. security/permissions) but for a few hours of hacking this is hard to beat!