m i k e b r e v o o r t » Technology

Grails Solr Plugin Progress Screencast

Mike — Wed, 03 Mar 2010 07:34:51 +0000

I started work on this plugin back in December based on the work I did for Patheos.com and by the graces of my employer Avalon Consulting LLC and Patheos, they allowed me to open source the plugin and continue working on it. This week I had some time to get back to it, and today I started creating a basic reference implementation application that will accompany the documentation. Below is a screencast demo of this application which indexes an export of songs from my iTunes library metadata and makes them searchable.

The code for the plugin is at Github and is still a work in progress. I’m pushing towards a 0.1 release next week with the bulk of the work I still need to do in the form of documentation and clean-up.

So please watch the screencast, and I would love feedback. I will certainly take offers to help continue the development of the plugin but would like to get 0.1 out first to round out my train of thought and not further delay that basic milestone.

Grails Solr Plugin pre-0.1 Demo from mbrevoort on Vimeo.

Issue negotiating SSL connections from Weblogic

Mike — Wed, 17 Feb 2010 03:41:25 +0000

This was one of those tough to track down issues that yielded very little in terms of actionable solutions vs. confirmation of similar problems while I Googled the problem symptoms. I recently upgraded a Grails app from Weblogic 8 to 10.3. The app allowed users to upload videos to Youtube using the Youtube APIs, using ClientLogin for authentication. The problem was after the upgrade the connection to https://www.google.com/accounts/ClientLogin was failing during the SSL negotiation phase. The errors in the log were

java.lang.IllegalStateException: Cipher not initialized
at javax.crypto.Cipher.c(DashoA13*..)
at javax.crypto.Cipher.update(DashoA13*..)
at com.certicom.tls.provider.Cipher.update(Unknown Source)
…

and

java.security.InvalidKeyException: Illegal key size

Thankfully I found this thread on a Korean Oracle forum. The solution is to add this JVM parameter:

-Dweblogic.security.SSL.nojce=true

This enables Weblogic to use a FIPS 140-2 compliant crypto module in the server’s SSL implementation. FIPS 140-2 is a standard that describes U.S. Federal government requirements for sensitive, but unclassified use.

If you have this problem, I hope you stumble upon this post and it helps you.

DOSUG Solr in 5 Minutes Ingnite Presentation

Mike — Thu, 04 Feb 2010 14:57:41 +0000

On Tuesday (2/2/2010) I participated in the Denver Open Source User Group’s first Ignite night. My presentation was an quick overview of Solr, the java based open-source search engine. This was my first Ignite style presentation, and the format is challenging! The presentations were each 5 minutes with a 20 slide deck auto advancing every 15 seconds. I stumbled a bit out of the gate with the cold start but was able to get it back on the rails though I felt as if I was trying to dig myself out of quicksand through the rest of it. Overall it was a lot of fun.

The room was packed, somewhere in the neighborhood of 100 people, and overall the quality of the presentations were very good. You can check out the slide decks on Slideshare here. Here are my slides:

And here is a dry-run of my presentation I recorded while practicing the day of:

This is Not Comcastic

Mike — Sat, 10 Oct 2009 06:40:34 +0000

I wrote a script that pings the IP of the first node outside of my house into Comcast’s network. It pings every two seconds and logs when the connection is up or down based on the ping success. Last week Comcast came and checked my line and said everything looked great and that it may be the onslaught of the cold weather. I’ve been working through Comcast support via Twitter which has been sporadic but convenient when they respond. Today I couldn’t get a response from them (@comcastmelissa and @comcaststeve) and today sucked:

Fri Oct 09 08:20:12 MDT 2009 DOWN!
Fri Oct 09 08:23:41 MDT 2009 UP
Fri Oct 09 09:31:39 MDT 2009 DOWN!
Fri Oct 09 09:31:39 MDT 2009 UP
Fri Oct 09 09:32:15 MDT 2009 DOWN!
Fri Oct 09 09:32:26 MDT 2009 UP
Fri Oct 09 09:33:27 MDT 2009 DOWN!
Fri Oct 09 09:33:27 MDT 2009 UP
Fri Oct 09 09:33:48 MDT 2009 DOWN!
Fri Oct 09 09:33:48 MDT 2009 UP
Fri Oct 09 09:34:14 MDT 2009 DOWN!
Fri Oct 09 09:34:14 MDT 2009 UP
Fri Oct 09 09:34:30 MDT 2009 DOWN!
Fri Oct 09 09:34:30 MDT 2009 UP
Fri Oct 09 09:34:56 MDT 2009 DOWN!
Fri Oct 09 09:34:56 MDT 2009 UP
Fri Oct 09 09:36:13 MDT 2009 DOWN!
Fri Oct 09 09:36:13 MDT 2009 UP
Fri Oct 09 09:37:24 MDT 2009 DOWN!
Fri Oct 09 09:37:46 MDT 2009 UP
Fri Oct 09 09:38:42 MDT 2009 DOWN!
Fri Oct 09 09:38:42 MDT 2009 UP
Fri Oct 09 09:39:28 MDT 2009 DOWN!
Fri Oct 09 09:39:51 MDT 2009 UP
Fri Oct 09 09:41:37 MDT 2009 DOWN!
Fri Oct 09 09:41:37 MDT 2009 UP
Fri Oct 09 09:42:43 MDT 2009 DOWN!
Fri Oct 09 09:42:43 MDT 2009 UP
Fri Oct 09 09:44:50 MDT 2009 DOWN!
Fri Oct 09 09:44:50 MDT 2009 UP
Fri Oct 09 09:45:21 MDT 2009 DOWN!
Fri Oct 09 09:45:21 MDT 2009 UP
Fri Oct 09 09:46:02 MDT 2009 DOWN!
Fri Oct 09 09:46:13 MDT 2009 UP
Fri Oct 09 09:47:25 MDT 2009 DOWN!
Fri Oct 09 09:47:25 MDT 2009 UP
Fri Oct 09 09:47:56 MDT 2009 DOWN!
Fri Oct 09 09:47:56 MDT 2009 UP
Fri Oct 09 09:48:22 MDT 2009 DOWN!
Fri Oct 09 09:48:22 MDT 2009 UP
Fri Oct 09 09:50:08 MDT 2009 DOWN!
Fri Oct 09 09:50:08 MDT 2009 UP
Fri Oct 09 09:51:40 MDT 2009 DOWN!
Fri Oct 09 09:51:40 MDT 2009 UP
Fri Oct 09 09:52:06 MDT 2009 DOWN!
Fri Oct 09 09:52:06 MDT 2009 UP
Fri Oct 09 09:52:37 MDT 2009 DOWN!
Fri Oct 09 09:52:37 MDT 2009 UP
Fri Oct 09 09:53:08 MDT 2009 DOWN!
Fri Oct 09 09:53:08 MDT 2009 UP
Fri Oct 09 09:53:44 MDT 2009 DOWN!
Fri Oct 09 09:53:44 MDT 2009 UP
Fri Oct 09 09:54:25 MDT 2009 DOWN!
Fri Oct 09 09:54:25 MDT 2009 UP
Fri Oct 09 09:54:51 MDT 2009 DOWN!
Fri Oct 09 09:54:51 MDT 2009 UP
Fri Oct 09 09:55:38 MDT 2009 DOWN!
Fri Oct 09 09:55:49 MDT 2009 UP
Fri Oct 09 09:57:00 MDT 2009 DOWN!
Fri Oct 09 09:57:00 MDT 2009 UP
Fri Oct 09 09:57:16 MDT 2009 DOWN!
Fri Oct 09 09:57:16 MDT 2009 UP
Fri Oct 09 09:57:42 MDT 2009 DOWN!
Fri Oct 09 09:57:42 MDT 2009 UP
Fri Oct 09 09:57:58 MDT 2009 DOWN!
Fri Oct 09 09:57:58 MDT 2009 UP
Fri Oct 09 09:58:39 MDT 2009 DOWN!
Fri Oct 09 09:59:12 MDT 2009 UP
Fri Oct 09 09:59:28 MDT 2009 DOWN!
Fri Oct 09 09:59:28 MDT 2009 UP
Fri Oct 09 09:59:55 MDT 2009 DOWN!
Fri Oct 09 10:00:17 MDT 2009 UP
Fri Oct 09 10:00:43 MDT 2009 DOWN!
Fri Oct 09 10:00:43 MDT 2009 UP
Fri Oct 09 10:02:24 MDT 2009 DOWN!
Fri Oct 09 10:02:24 MDT 2009 UP
Fri Oct 09 10:02:45 MDT 2009 DOWN!
Fri Oct 09 10:02:45 MDT 2009 UP
Fri Oct 09 10:03:11 MDT 2009 DOWN!
Fri Oct 09 10:03:11 MDT 2009 UP
Fri Oct 09 10:03:37 MDT 2009 DOWN!
Fri Oct 09 10:03:37 MDT 2009 UP
Fri Oct 09 10:04:14 MDT 2009 DOWN!
Fri Oct 09 10:04:25 MDT 2009 UP
Fri Oct 09 10:04:56 MDT 2009 DOWN!
Fri Oct 09 10:04:56 MDT 2009 UP
Fri Oct 09 10:05:12 MDT 2009 DOWN!
Fri Oct 09 10:05:12 MDT 2009 UP
Fri Oct 09 10:05:33 MDT 2009 DOWN!
Fri Oct 09 10:05:55 MDT 2009 UP
Fri Oct 09 10:06:11 MDT 2009 DOWN!
Fri Oct 09 10:06:11 MDT 2009 UP
Fri Oct 09 10:06:27 MDT 2009 DOWN!
Fri Oct 09 10:06:49 MDT 2009 UP
Fri Oct 09 10:07:05 MDT 2009 DOWN!
Fri Oct 09 10:07:05 MDT 2009 UP
Fri Oct 09 10:07:21 MDT 2009 DOWN!
Fri Oct 09 10:07:32 MDT 2009 UP
Fri Oct 09 10:07:48 MDT 2009 DOWN!
Fri Oct 09 10:07:59 MDT 2009 UP
Fri Oct 09 10:08:15 MDT 2009 DOWN!
Fri Oct 09 10:08:37 MDT 2009 UP
Fri Oct 09 10:08:53 MDT 2009 DOWN!
Fri Oct 09 10:08:53 MDT 2009 UP
Fri Oct 09 10:09:14 MDT 2009 DOWN!
Fri Oct 09 10:09:14 MDT 2009 UP
Fri Oct 09 10:10:10 MDT 2009 DOWN!
Fri Oct 09 10:10:11 MDT 2009 UP
Fri Oct 09 10:10:37 MDT 2009 DOWN!
Fri Oct 09 10:10:37 MDT 2009 UP
Fri Oct 09 10:11:38 MDT 2009 DOWN!
Fri Oct 09 10:11:38 MDT 2009 UP
Fri Oct 09 10:13:04 MDT 2009 DOWN!
Fri Oct 09 10:13:04 MDT 2009 UP
Fri Oct 09 10:14:21 MDT 2009 DOWN!
Fri Oct 09 10:14:21 MDT 2009 UP
Fri Oct 09 10:15:07 MDT 2009 DOWN!
Fri Oct 09 10:15:07 MDT 2009 UP
Fri Oct 09 10:15:43 MDT 2009 DOWN!
Fri Oct 09 10:15:43 MDT 2009 UP
Fri Oct 09 10:15:59 MDT 2009 DOWN!
Fri Oct 09 10:15:59 MDT 2009 UP
Fri Oct 09 10:30:30 MDT 2009 DOWN!
Fri Oct 09 10:30:30 MDT 2009 UP
Fri Oct 09 10:30:56 MDT 2009 DOWN!
Fri Oct 09 10:30:56 MDT 2009 UP
Fri Oct 09 10:32:07 MDT 2009 DOWN!
Fri Oct 09 10:32:07 MDT 2009 UP
Fri Oct 09 10:33:18 MDT 2009 DOWN!
Fri Oct 09 10:33:18 MDT 2009 UP
Fri Oct 09 10:33:34 MDT 2009 DOWN!
Fri Oct 09 10:33:34 MDT 2009 UP
Fri Oct 09 10:36:26 MDT 2009 DOWN!
Fri Oct 09 10:36:26 MDT 2009 UP
Fri Oct 09 13:12:12 MDT 2009 DOWN!
Fri Oct 09 13:15:30 MDT 2009 UP
Fri Oct 09 13:52:12 MDT 2009 DOWN!
Fri Oct 09 13:53:08 MDT 2009 UP
Fri Oct 09 14:34:14 MDT 2009 DOWN!
Fri Oct 09 14:35:09 MDT 2009 UP
Fri Oct 09 15:08:11 MDT 2009 DOWN!
Fri Oct 09 15:11:29 MDT 2009 UP
Fri Oct 09 16:26:15 MDT 2009 DOWN!
Fri Oct 09 16:29:22 MDT 2009 UP
Fri Oct 09 17:26:26 MDT 2009 DOWN!
Fri Oct 09 17:26:26 MDT 2009 UP
Fri Oct 09 17:27:17 MDT 2009 DOWN!
Fri Oct 09 17:27:17 MDT 2009 UP
Fri Oct 09 17:29:14 MDT 2009 DOWN!
Fri Oct 09 17:29:14 MDT 2009 UP
Fri Oct 09 17:30:35 MDT 2009 DOWN!
Fri Oct 09 17:30:35 MDT 2009 UP
Fri Oct 09 17:31:21 MDT 2009 DOWN!
Fri Oct 09 17:31:22 MDT 2009 UP
Fri Oct 09 17:31:58 MDT 2009 DOWN!
Fri Oct 09 17:31:58 MDT 2009 UP
Fri Oct 09 17:33:14 MDT 2009 DOWN!
Fri Oct 09 17:33:14 MDT 2009 UP
Fri Oct 09 18:25:03 MDT 2009 DOWN!
Fri Oct 09 18:25:03 MDT 2009 UP

Yammer, Socialcast, Present.ly… and SocialWok

Mike — Sat, 10 Oct 2009 06:16:25 +0000

I’ve spent the last several weeks diving into many of the “enterprise”/private SaaS microblogging services available (a.k.a Twitter for the enterprise). Wow it’s been quite a while since I blogged! I’ve spend most of my “sharing” time on Twitter the last year.

This all began when I was trying out the new “social” features of Confluence. I really like confluence, and I like the new features like status updates and following other users, but it still is a bit limited. So I started poking around and tried out SocialText and my first impression was that it felt disheveled; it’s much more than a micro-blogging platform and more comparable to Confluence in fact. I decided to turn my focus on the more pure play microblogging apps. So I began a series of twitter searches to see what people were mentioning most with respect to “enterprise twitter”, “micro-blogging”, etc. So I decided to take a look at: Yammer, Socialcast and Present.ly.

I created accounts in each of these and invited a couple coworkers. My general first impression of each was as follows. Yammer was very clean and unintimidating; it felt like you could just jump in and start using it. Socialcast’s UI turned me off right away as it “felt” a bit dated (a little petty I know, but a first impression none the less). There seemed to be a little more to it than yammer with the categorizations and the ability to import from a list of outside services (Flickr, YouTube, Digg, Facebook, Slideshare, etc.). Present.ly was the only one of the three to impose the 140 character status limit but would allow you to attach text or a file, and it was “busy”. There were giant question mark icons and @ symbols next to some updates. Given these first impressions, I picked Yammer for my organizational experiment. I wanted to see how micro-blogging fit culturally and within the dynamics of my organization.

So we’re trying out Yammer, but also spent some additional time with Present.ly and Socialcast. Overall the micro-blogging “experiment” is going really well and here’s my quick take on each service.

Yammer

Overall Yammer has a very low barrier to entry, easy to register and easy to begin contributing. The feed can display as threaded or mixed chronological (like Twitter). The threaded view is only a single level threading though. Yammer comes in three levels: Basic (free), Silver ($3/user/mo) and Gold ($5/user/mo) and only has a SaaS hosting model so there is no longer a behind the firewall option. Yammer segments your network strictly by email domain and you cannot invite members outside of your domain unless you are at a paid level. At the Basic level each user “owns” their data where with the paid version the user owns the data and you can’t get an export of the data unless you are at the Gold level.

Yammer’s desktop client is mediocre and the search capability is basically dysfunctional. There is a 3rd party Mac OS native application called Gabble that is a big improvement over the official yammer desktop client. You have a threaded view capability but no search capability. There’s a Firefox extension that works much like the Twitter extension that works well to keep you in the loop announcing “YAM” every time a new message was posted and twitter integration allowing twitter updates to flow into Yammer if you add the tag #yam to your tweet. There’s several other extension that I didn’t try. Yammer’s new iPhone app is really, really good. There’s a push capability and the ability to post photos. One of my coworkers said the 3rd party Android app worked well, but as I tried to find a link for it at nullwire all of the information is gone and the site is pretty much stripped.

Yammer’s group capability allows for public or private groups and what is really nice is the ability for each user to specify notification preferences for each group (email, IM or SMS). The profile has a good mix of data but doesn’t allow for custom fields. There’s also a very nice self directed org chart feature that allows each user to specify who they report to, manage and work with. I did not try the API.

Socialcast

Though I didn’t pick Socialcast for the experiment, there’s something about them that makes me want to like them. Maybe it’s how responsive and friendly they are (thanks @carrieyoung @socialcast); I don’t know. This week Socialcast released a new version that included a a UI overhaul which is MUCH better. They also launched a Social Business Intelligence capability that looks really compelling. With Socialcast you own your own data and can request a full data export even in the free version. They offer two versions: Basic (Free, SaaS) and Enterprise (Behind the firewall virtual appliance, price? offered as SaaS as well?). Even in the basic version you can “claim” your administration rights and then set your theme, upload a custom logo, manage various metadata and view reports. You can also send a broadcast messages to the entire community.

In Socialcast you can create custom streams based on people or tags/keywords. There is similar group functionality as in Yammer, but Socialcast doesn’t have the custom notification settings I liked in Yammer. The profile is similar to Yammer though you can add custom profile questions.

The desktop app is functional, bland from a UI perspective. You can’t seem to filter by custom streams or groups but you can filter by categories or types of messages. The search works much better than Yammer’s desktop search. They have an iPhone optimized mobile site that works well, but no native mobile apps. They do have a nice Google gadget though that’s perfect for those organizations using Google Apps. The REST API looks very capable.

Present.ly

I spent the least amount of time using Present.ly though it is a very capable service as well. There is a free SaaS version and an enterprise behind the firewall version with perpetual licenses by number of users. Even the free version has a superior array of configurability allowing you to set custom mail servers, LDAP servers, etc. There’s a nice array of apps including an iPhone, Android, Blackberry and Windows mobile versions. I did discover that if you upload a video file (in this case an avi) that Present.ly would process it and render in an integrated player – that was sweet.

SocialWok

At the tail end of my research I stumbled upon SocialWok – basically a social app for Google Apps that runs on Google App Engine. Since we use Google Apps internally I was excited by the prospect of this app. With many thousands of companies using Google Apps, already having taken the SaaS cloud plunge, I believe SocialWok is in an incredible sweet spot. IMO there’s still some key areas of development to be done, specifically there are not private groups and though the UI is a blatant (and intentional) Google copy it’s still rough around the edges in places. However, the core functionality is there and there’s a solid foundation to build on. I’m certainly going to keep my eye on SocialWok.

Overall I learned a lot from my survey of these services. Yammer was the simplest to use and had a stable of applications and even some 3rd party apps. They seem to have the most momentum and the largest user base. Thus you have to pay for data “ownership”, the ability to export, etc. Socialcast in my mind is the most exciting given their new Social Business Intelligence capabilities and the strength of their platform as a whole. Like I said before, for some reason I just like them. Present.ly seems very capable and would certainly warrant more time if I had it. SocialWok has reinvented itself exclusively for Google Apps, and they are going to develop the hell out of the niche they are in.

So there are some key differences between each of these services, some others I didn’t mention and many features and capabilities that I just didn’t have time to mention. In truth I just scratched the surface. The best way to learn about these services is to sign up for account and try them out first hand. It’s so easy to get going that it’s scary. I can only imagine that there are 1000’s of rogue yammer communities out there operating under the noses of organizations. These things can be very difficult to control once they proliferate. Now I wonder when Twitter will join the party? With a $1B valuation and very little revenue you would think that enterprise micro-blogging would be a strategic opportunity for growth…

Making Relative URLs Absolute with Groovy

Mike — Sun, 21 Dec 2008 22:47:11 +0000

I’m working on a project where I am using Grails as a content delivery layer for an XML based content management system. The content management system has the ability to publish XHTML content with inline links and images; however the inline img and a tags reference the content as relative paths. I need the links to internal pages to be absolute from the server root (ie /sub/content.html rather than sub/content.html) and I am hosting all static content like images at Amazon S3 so I need the image links to be absolute to a different DNS name (ie http://media.mywebsite.com/images/image.jpg rather than images/image.jpg).

Instead of implementing some hacks in the CMS itself, I opted to do the transformation in Grails as the content was rendered allowing greater flexibility. I created a taglib for rendering the XHTML block that does a Groovy replaceAll with regular expressions. The tag gets called like this: . My GSP has reference to a content variable that is an XMLSlurped xml document with a mainBody tag. Here is the code for the tag:

/**
 * Given XHTML content will return the same content with any relative a tag hrefs prefixed with a /
 * and any relative images prefixed with the mediaURL
 */

def processIngeniuxXHTML = {attrs ->
  def content = attrs.content
  def rootURL = grailsApplication.config.mediaURL

  if (content) {
    // first look for links within the content that are relative URLS and prefix with a slash so they are
    // resolved from the root rather than the current context
    def regex = /(< \s*a\s+[^>]*href\s*=\s*[\"'])(?!http)([^\"'>]+)[\"'>]/
    def replace = { fullMatch, a, b ->
      if (b[0] != "/"){
        // for some reason it's not outputing the closing quote so I added it at then end explicitly
        "${a}/${b}\""
      } else
        "${fullMatch}"
    }

    content = content.replaceAll(regex, replace)

    // second if there are any images referenced with relative URLs, prefix with the DNS for the S3 bucket
    regex = /(< \s*img\s+[^>]*src\s*=\s*[\"'])(?!http)([^\"'>]+)[\"'>]/
    replace = { fullMatch, a, b ->
      if (b[0] != "/"){
        // for some reason it's not outputing the closing quote so I added it at then end explicitly
        "${a}${rootURL}/${b}\""
      } else
        "${fullMatch}"
    }

    out < < content.replaceAll(regex, replace)
  }
}

And here are the test conditions to demonstrate what’s expected:

void testProcessXHTML() {
  ContentUtilTagLib tl = new ContentUtilTagLib()

  def i = """"""
  def o = """
"""

  tl.processXHTML(content: i)
  assertEquals o, out.toString()
  out.getBuffer().setLength(0)

  i = """
"""
  o = """
"""

  tl.processXHTML(content: i)
  assertEquals o, out.toString()
  out.getBuffer().setLength(0)

  i = """
"""
  o = """
"""

  tl.processXHTML(content: i)
  assertEquals o, out.toString()
  out.getBuffer().setLength(0)

  i = """
"""
  o = """
"""

  tl.processXHTML(content: i)
  assertEquals o, out.toString()
  out.getBuffer().setLength(0)

  i = """
"""
  o = """
"""

  tl.processXHTML(content: i)
  assertEquals o, out.toString()
  out.getBuffer().setLength(0)

}

More related to this project to come…

NFJS Rocky Mountain Software Symposium Debrief

Mike — Mon, 17 Nov 2008 06:45:25 +0000

This weekend I attended the NFJS Rocky Mountain Software Symposium in Lone Tree, CO. It was hard to pass up a NFJS conference that’s 15 minutes from home. Overall the sessions were excellent. There wasn’t a single session I attended that sucked or was a waste of time, a first for any conference I’ve attended. I seemed to gravitate toward sessions by Ken Sipe and Stu Halloway followed by Groovy and Grails presentations by Jeff Brown and Scott Davis.

I gleaned several consistent overarching themes at the conference:

Java is essentially Dead (Stu Halloway leading this sentiment)
BUT the JVM is alive and well. In fact the JVM has transitioned from a write once run Java anywhere runtime to a dynamic platform for many languages (Groovy, JRudy, Scala, Clojure, Javascript, etc…. oh yeah and Java)
Not doing Test Driven Development is basically irresponsible, especially with the explosion of dynamic languages
Not doing Continuous Integration is basically irresponsible

I learned the Jeff Brown has developed and is about to announce a new Hudson plugin for building Grails projects in Hudson, a big and welcome improvement on specifying new ant targets, using the shell script option or whatever. Jeff also echoed much of what has already been said about the recent Spring Source acquisition of G2:

Deeper integration between Grails and Sprint MVC
The Groovy Spring bean builder will be moved into Spring
And of course there will be closer collaboration between the Groovy, Grails, and Spring development teams, though the team will remain distinct.

I’m super excited about Git though Matt McCullough urged me to take a look at Mercurial…

Anyway, fantastic event and kudos to No Fluff Just Stuff.

Rocky Mountain Software Symposium

Mike — Sat, 15 Nov 2008 05:51:35 +0000

This weekend I’m attending the NFJS Rocky Mountain Software Symposium. Today I attended a few really good sessions specifically Java Memory, Performance and the Garbage Collector presented by Ken Sipe and a very enthusiastic Groovy Metaprogramming tour by Scott Davis. Though I’ve been using Groovy for quite a while, primarily inconjuction with Grails, Scott’s presentation really helped fill some gaps for me. Ken’s presentation on JVM memory managemant was a great primer. I’m looking forward to profiling a few of the Grails apps I’m working on vith VisualVM to see how I can optimize New vs Old space as well as preset the PermGen space to speed up Grails start-up.

I’m looking forward to the coming two days of sessions though I REALLY could have used a weekend. At least the conference is 15 minutes from home. I’ll post my thoughts/reactions to the other session this weekend.

After a nudging from Ken during his presentation to ‘increase your digital footprint’ and Cannon-Brookes return to blogging (congrats on your recent position on the Gartner Magic Quadrant for Social Software – well deserved…) I’m renewing my commitment to start blogging again and this time more focused on my “work” life, choosing to use Facebook for sharing personal updates.

Congrats to G2 for the Spring Source aquisition and the release today of Grails 1.0.4. And I just saw that the Groovy/Grails Experience has been scheduled for February 2009… in Denver!

Active Directory Authentication Part II

Mike — Sat, 27 Sep 2008 06:19:01 +0000

This is a long overdue follow-up to my Active Directory Authentication post. I had all the best intentions of quickly following this post up with a second post so this is long overdue. It’s been just over a year though so there are lots of cobwebs so this is going to be a stream of consciousness dump…

First, there are two books that I found indespensible and the best resources I could find for detailed information about integrating with Active Directory and Windows security. The LogonUser and SSPI approach I mentioned in my first post a year ago originated from these books.

In an effort to just get some follow-up out there since I tend to get regular inquiries about my original post, Here’s a zip file of some sanitized code that includes the code for LogonUser, SSPI and a few other classes I used.
In the end I settled on LogonUser which seemed to give the best overall performance in a very decentralized, many sub-domain environment. There were also less anomalies as well. When using SSPI I found that if typical authentication took 140ms, some calls were taking 60.140s and 120.140s in one or two domains. This was rare and most likely a result of some sort of timeout and retry or fallback to another domain controller. I was not able to identify the root cause.

One side note regarding the LogonUse API. When the authentication type is set to “LOGON32 LOGON NETWORK” the LastLogon and LastLogonTimeStamp attributes on the domain controllers are not updated when authentication occurs. If you have any logic that relies on these attributes (For example if you to run a script against AD to check for user account inactivity), change the authentication type to “LOGON32 LOGON INTERACTIVE” to have these attributes updated as expected.

Lastly to test the differences between the three authentication types, I created a method to log detailed information about each authentication attempt including: elapsed time, samAccountName, domain, authentication result, user ip, domain server authenticated against (if I could figure it out), authentication web server. Without going into too much detail, I logged this info into a row in a database table on each attempt (this also inherently created a great resource for security auditing). This way I could query the data very easily to roll-up statistics based on which domain, what server, the IP subnet of the user, particular users, etc. Key to this though was to avoid any additional latency on the authentication request or a database dependency on the success of the authentication request, I created a BlockingQueue that waited for a log message to be placed on the queue and inserted it into the database. This happened out of band in another thread and if there was some failure, the queue would just continue to grow until the problem was resolved. This was very effective in measuring the performance of these different authentication methods.

Feel free to comment with any questions, I’ll do my best to recollect.

Expanding on Jira’s LDAP Integration

Mike — Tue, 08 Apr 2008 22:23:32 +0000

I recently had the opportunity to do a fresh installation of Atlassian Jira and was faced again with how to integrate Jira with LDAP (in this case Active Directory). Jira does support simple authentication integration with LDAP via OSUser but requires that each LDAP user exist in Jira first. So if an LDAP user needs access to Jira, I have to go to Jira and create a new corresponding user record with the same user id as LDAP (sAMAccountName in this case). Ideally I would like Jira to synchronize LDAP users based on some query filter.

Luckily Jira has a perfect way to address this “opportunity”: Services. Jira Services are asynchronous scheduled units of work configured to run within Jira. I created a new service using JNDI called LDAPUserSyncService that can periodically query LDAP user objects based on configurable filter and create new Jira users for new users it finds in LDAP. I also added the ability to put a subset of users in a group called “internal-users” for users whose email matched a particular domain.

Its pretty simple. When the service runs, it queries LDAP, iterates through each user checking if the user already exists in LDAP, if not it created the user and puts it in the jira-users group and then checks if the email address contains myco.com (fictitious domain for this example) and if so put the user in the internal group as well. I just realized as I’m writing this though that I didn’t do anything to handle paging or results if the number of users is greater than the max returned by LDAP. Oh well, maybe another time…

public class LDAPUserSyncService extends AbstractService {

    private static org.apache.log4j.Category log =
            org.apache.log4j.Category.getInstance(LDAPUserSyncService.class);

    public static String INITCTX = "com.sun.jndi.ldap.LdapCtxFactory";
    public static String GROUP_INTERNAL =  "Internal-Users";
    public static String LDAP_USER_ID_ATTRIBUTE = "sAMAccountName";

    public void run() {
        try {
            log.debug("Running com.myco.jira.service.LDAPUserSyncService");
            String LDAP_HOST = getProperty("LDAP_HOST");
            String LDAP_USER = getProperty("LDAP_USER");
            String LDAP_PASSWORD = getProperty("LDAP_PASSWORD");
            String LDAP_SEARCHBASE = getProperty("LDAP_SEARCHBASE");
            String LDAP_FILTER = getProperty("LDAP_FILTER");   

            Hashtable env = new Hashtable();
            env.put(Context.INITIAL_CONTEXT_FACTORY, INITCTX);
            env.put(Context.PROVIDER_URL, LDAP_HOST);
            env.put(Context.SECURITY_AUTHENTICATION, "simple");
            env.put(Context.SECURITY_PRINCIPAL, LDAP_USER);
            env.put(Context.SECURITY_CREDENTIALS, LDAP_PASSWORD);

            DirContext ctx = new InitialDirContext(env);

            SearchControls constraints = new SearchControls();
            constraints.setSearchScope(SearchControls.SUBTREE_SCOPE);

            NamingEnumeration results = ctx.search(LDAP_SEARCHBASE, LDAP_FILTER, constraints);

            while(results != null && results.hasMore()) {
                SearchResult sr = (SearchResult) results.next();
                Attributes attrs = sr.getAttributes();
                log.debug("Processing " + attrs.get("dn"));

                // need sn, givenName, mail
                String first = (attrs.get("givenName") != null) ? (String) attrs.get("givenName").get() : null;
                String last = (attrs.get("sn") != null) ? (String) attrs.get("sn").get() : null;
                String mail = (attrs.get("mail") != null) ? (String) attrs.get("mail").get() : null;
                String userId = (attrs.get(LDAP_USER_ID_ATTRIBUTE) != null) ? (String) attrs.get(LDAP_USER_ID_ATTRIBUTE).get() : null;

                if(first != null && last != null && mail != null && userId != null) {

                    // check if user is in Jira
                    if(!userExists(userId.toLowerCase())) {

                        // create user, if a myco company user then add to internal group
                        if(createJiraUser(userId, first, last, mail) != null) {
                            if(mail.contains("myco.com"))
                                addUserGroup(userId, GROUP_INTERNAL);
                        } else {
                            log.error("LDAP User " + userId + " could not be created");
                        }
                    }
                }
            }
        } catch (NamingException nex) {
            log.error("Caught exception trying to Synch LDAP Users: " + nex.toString());
        } catch (ObjectConfigurationException oce) {
            log.error("Caught exception trying to Synch LDAP Users.  Configuration setup failed: " + oce.toString());
        }
    }

    private User createJiraUser(String userId, String fname, String lname, String email) {

        UserManager userMgr = UserManager.getInstance();
        User osUser = null;
        try {
            osUser = userMgr.createUser(userId.toLowerCase());
            osUser.setFullName(fname + " " + lname);
            osUser.setEmail(email);

            Group jiraUserGroup = userMgr.getGroup("jira-users");
            osUser.addToGroup(jiraUserGroup);
            osUser.store();
            log.debug("Successfully added LDAP user "+ userId);
        } catch (ImmutableException e) {
            log.error("Error creating User in Jira: " + userId, e);
        } catch (DuplicateEntityException e) {
            log.error("Error creating User in Jira: " + userId, e);
        } catch (EntityNotFoundException e) {
            log.error("Could not find group jira-users.  Error creating User in Jira: " + userId, e);
        }

        // to be sure the user was created, return the reloaded user from Jira
        return osUser;

    }

    private boolean userExists(String userId) {
        UserManager userMgr = UserManager.getInstance();
        boolean exists = false;
        try {
            if(userMgr.getUser(userId) != null)
                exists = true;
        } catch (EntityNotFoundException e) {
            exists = false;
        }

        return exists;
    }

    private void addUserGroup(String userId, String groupName){
        UserManager userMgr = UserManager.getInstance();

        try {
            User osUser = userMgr.getUser(userId);

            Group group = userMgr.getGroup(groupName);
            osUser.addToGroup(group);
            osUser.store();
            log.debug("Successfully added LDAP user " + userId + " to group: " + groupName);
        } catch (ImmutableException e) {
            log.error("Error adding User to acis-users group: " + userId, e);
        } catch (EntityNotFoundException e) {
            log.error("Could not find group acis-users.  Error adding User to acis-users group:  " + userId, e);
        }
    }

    public ObjectConfiguration getObjectConfiguration() throws ObjectConfigurationException {
        return getObjectConfiguration("LDAPUSERSYNCSERVICE", "com/myco/jira/service/LDAPUserSyncService.xml", null);
    }
}

To configure a service within Jira navigate to Services in the System section of the Administration interface:

Here I’ve added the service you can see the parameters that can be customized for the service:

The parameters are configured in an XML file that is referenced by the getObjectConfiguration() method in the service code above:


    
    
    

        
            LDAP_HOST
            LDAP Host (format: ldap://myco.com:389)
            string
        
        
            LDAP_USER
            LDAP User DN
            string
        
                
            LDAP_PASSWORD
            LDAP Password)
            string
        
        
            LDAP_SEARCHBASE
            LDAP Search Base DN
            string
        
        
            LDAP_FILTER
            LDAP Search Filter
            string

And that’s it. New users are automatically created in Jira within 30 minutes of being created in LDAP. Ooo Rah.