I’m on a quest to figure out how to cluster the Grails Searchable plugin that’s based on Lucene/Compass across multiple nodes with as little intrusion and in a way that’s as turnkey as possible. I’m in immediate need of a solution, and I think it would be a good contribution to the Searchable plugin.

To start here’s what I’m considering:

1. Manage a local index per node. Probably disable mirrorChanges via Compass::GPS and rebuild the index on someinterval. This is not very desirable since it would eliminate the Compass:GPS capability, and there just must be a more elegant way…
2. Use the JDBCDirectory implementation of Lucene and store the index in a sql database. This option is viable but has obvious performance implications that somewhat defeat the purpose. The configuration would be fairly clean using the native Compass XML config. This maybe a good fall back option to do some A/B testing against.
3. Use Terracotta and RAMDirectory to handle synchronization of the index across nodes. This has a lot of promise and will be the path I head down first. Though I have no experience whatsoever with Terracotta

Another possibility is to look at Compass’ new support for Gigaspaces. Thanks Marcos for passing that on. This sounds interesting but “feels” fairly heavy compared to the others, though I know nothing about GigaSpaces or Coherence so I’m just talking out of my arse at this point!

I made my first attempt with Terracotta and Compass today. I dug deep enough into Compass to find that it manages the RAMDirectory objects in this class: org.compass.core.lucene.engine.store.RAMLuceneSearchEngineStore and manages each subindex RAMDirectory in a java.util.HashMap called ramIndexes. The first issue I had to overcome was that HashMap isn’t a synchronized data structure, and I was having problems configuring the locking in Terracotta for ramIdexes. I kept getting UnlockedSharedObjectException exceptions not matter what I tried.

After a little help from the Terracotta users list I ended up with a named lock. My tc-config.xml looked like this:

<?xml version="1.0" encoding="UTF-8" ?>
<tc:tc-config xmlns:tc="http://www.terracotta.org/config">
  <system>
    <configuration-model>development</configuration-model>
  </system>

  <servers>
    <server name="localhost" />
  </servers>

  <clients>
    <logs>%d/client-logs-%h</logs>
    <dso>
      <debugging>
        <runtime-logging>
          <lock-debug>true</lock-debug>
        </runtime-logging>
        <runtime-output-options>
          <full-stack>true</full-stack>
        </runtime-output-options>
      </debugging>
    </dso>
    <modules>
      <module name="clustered-lucene-2.0.0" version="2.5.0"/>
    </modules>
  </clients>

  <application>
    <dso>
      <instrumented-classes>
        <include>
          <class-expression>org.compass.core.lucene.engine.store.RAMLuceneSearchEngineStore</class-expression>
        </include>
      </instrumented-classes>
      <roots>
        <root>
          <field-name>org.compass.core.lucene.engine.store.RAMLuceneSearchEngineStore.ramIndexes</field-name>
        </root>
      </roots>
      <locks>
	<named-lock>
          <method-expression>* org.compass.core.lucene.engine.store.RAMLuceneSearchEngineStore.*(..)</method-expression>
          <lock-level>write</lock-level>
          <lock-name>theLockName</lock-name>
        </named-lock>
      </locks>
     </dso>
  </application>
</tc:tc-config>

That at least got me past the UnlockSharedObjectException, but tonight I hit a bigger roadblock. Terracotta is choking on Grails custom ClassLoader. I did a preliminary search on the error message below but came up short. I followed up on the Terracotta users’ list but I thought I’d share my progress and my intentions here in case anyone out there has any ideas…

Error creating bean with name 'compassGps': Cannot resolve reference to
bean'compass' while setting bean property 'compass'; nested exception is
org.springframework.beans.factory.BeanCreationException: Error creating
bean with name 'compass': FactoryBean threw exception on object creation;
nested exception is java.lang.IllegalStateException: Classloader name not set,
instances defined from this loader not supported in Terracotta (loader:
org.codehaus.groovy.grails.cli.support.GrailsRootLoader):
java.lang.IllegalStateException: Classloader name not set, instances
defined from this loader not supported in Terracotta (loader:
org.codehaus.groovy.grails.cli.support.GrailsRootLoader)

Stay tuned…

Update (Jan 29, 2008): See part 2 here