Solr on NFS issues
Our application uses a built-in Solr instance for searching. The data directory is on NFS and I cannot change it. Using Solr is very simple, there is one thread that updates the index periodically, and there are several read threads - all of them inside the same Java process. No other Solr interaction occurs.
By default, "solrconfig.xml" I sometimes run "java.nio.channels.OverlappingFileLockException". As I understand it, the "SimpleFSLockFactory" does not actually work properly with NFS.
Questions:
-
Given the above application scenario (no concurrent index modifications), shouldn't NoLockFactory be enough? Are there any downsides to using NoLockFactory? If I install NoLockFactory I get several error log entries saying "CONFIGURATION WARNING: Locks are disabled". Why is this message entering the error log? Is this really considered a bug and why?
-
Maybe there is a better solution than using "NoLockFactory"?
-
Not sure if this is NFS related, but sometimes (quite quietly) my index gets corrupted and I get a lot of "java.io.FileNotFoundException: _i.fdx" when trying to update the index. There's no way out of this other than manually deleting the entire index directory and starting from scratch. Why might this happen, and is there any nifty way to automatically detect a broken index and repair it?
source to share
Storing your indexes over NFS is prone to problems, but if it were to be run over NFS, I predict this problem occurs, possibly because you are not using NFSv4 or are not using it correctly. NFSv4 - first version of byte range locking support , NFSv2 and v3 (weakly) support whole files, and without running portmap, rpc.lockd and rpc.statd - locks are probably advisory only (as opposed to mandatory), but definitely will not be closed to lock the byte range.
java.nio.channels.OverlappingFileLockException says
Unchecked exception thrown when an attempt is made to acquire a lock on a region of a file
that overlaps a region already locked by the same Java virtual machine, or when another
thread is already waiting to lock an overlapping region of the same file.
A quick search of the Lucene mailing list returns many results that seem to indicate that using Lucene (and by extension, Solr) over NFS is a bad idea .
Blocking issues aside, performance is likely to be pretty poor.
I know this is not the answer you were hoping for, but this is the answer you need.
source to share