m i k e b r e v o o r t

Round Two of my quest to cluster the Grails Searchable Plugin index with Terracotta! In my first post I outlined my intent to use Terracotta to cluster the Grails Searchable Plugin (Lucene + Compass) using the Lucene RAMDirectory approach. Incidentally, my first attempt failed miserably.

As a quick confidence booster before setting out again, I successfully clustered the Grails Searchable Plugin using a JDBCDirectory store for the index. Since my last attempt I received some guidance from the guys at Terracotta and learned that Shay Banon is working with Terrocotta to develop a Compass module for Terracotta. Good stuff… check out the details of that here.

The first hurdle I needed to overcome was the classloader error:

Classloader name not set, instances defined from this
loader not supported in Terracotta
(loader: org.codehaus.groovy.grails.cli.support.GrailsRootLoader)

Before an object can be shared by Terracotta, the ClassLoader that loaded the object’s class has to be known to Terracotta. For core java and supported frameworks/container Terracotta takes care of registering relevant classloaders. For the GrailsRootLoader I can do this one of two ways:

1. hack the grails source where the class loader is instantiated, registering it with Terracotta
2. this can be converted into a config module that will inject the proper byte code at runtime so as to avoid any code tweaking.

The guys at Terracotta offered to help with #2 if I could get #1 working.

So I checked out the Grail trunk from SVN, and added these lines after the GrailsRootLoader is created:

 // create loader and execute main class
GrailsRootLoader loader = new GrailsRootLoader(lc);
((com.tc.object.loaders.NamedClassLoader) loader).__tc_setClassLoaderName("Grail Classloader");
com.tc.object.bytecode.hook.impl.ClassProcessorHelper.registerGlobalLoader((com.tc.object.loaders.NamedClassLoader) loader);

I built the Grails jars and replaced them in my grails-1.0-final-SNAPSHOT/dist directory. That resolved my classloader error! Next I went through about 100 iterations of root and lock configuration for my tc-config.xml as I dug and spun my way around the Compass source. I finally ended up with this for my tc-config.xml (keep in mind this can surely be optimized and is pretty brute force):

<?xml version="1.0" encoding="UTF-8" ?>

<tc:tc-config xmlns:tc="http://www.terracotta.org/config">
  <system>
    <configuration-model>development</configuration-model>
  </system>

  <servers>
    <server name="localhost" />
  </servers>

  <clients>
    <logs>%d/client-logs-%h</logs>
    <dso>

       <debugging>
        <runtime-logging>
          <lock-debug>true</lock-debug>
        </runtime-logging>
        <runtime-output-options>
          <full-stack>true</full-stack>
        </runtime-output-options>
      </debugging>

    </dso>
    <modules>
      <module name="clustered-lucene-2.0.0" version="2.5.0"/>
    </modules>
  </clients>

  <application>
    <dso>
      <instrumented-classes>
        <include>
          <class-expression>org.compass.core.lucene.engine..*</class-expression>
        </include>
        <include>
          <class-expression>org.apache.lucene..*</class-expression>
        </include>
      </instrumented-classes>
      <roots>
        <root>
          <field-name>org.compass.core.lucene.engine.store.RAMLuceneSearchEngineStore.ramIndexes</field-name>
        </root>
      </roots>
      <locks>
	<named-lock>
          <method-expression>* org.compass.core.lucene.engine.manager.DefaultLuceneSearchEngineIndexManager.*(..)</method-expression>
          <lock-level>write</lock-level>
          <lock-name>theLockName</lock-name>
        </named-lock>
        </locks>
     </dso>
  </application>
</tc:tc-config>

It worked! I was able to test with the Terracotta server running, and two Grails apps running in separate JVMs that all of the Domain class data that the first JVM indexed, could be searched by the second JVM and they both returned identical results. I also verified the existence of the objects under the root in the Terracotta Admin GUI. And finally, I shut both Grails app JVMs down and brought them back up and my index was still available.

What’s next? Work to get a config module for Terracotta and optimize my root and locking configuration. I’m very interested in what will come of the promised Compass/Terracotta integration. Also Clustering Grails is a much taller mountain. I was told that past attempts to cluster Groovy with Terracotta had not been too successful. For now I’m happy just to have climbed my little hill…