Lucene NIOFSDirectory and SimpleFSDirectory with multiple threads

My main question is what is the correct way to create / use instances of NIOFSDirectory and SimpleFSDirectory when there are multiple threads that need to process requests (reads) on the same index. Moreover: should an instance of XXXFSDirectory be instantiated for each thread that should execute the request and get some results (and then on the same thread close it immediately), or should I make a "global" (singleton?) That is passed to all threads, and then they all use it at the same time (and it no longer depends on each thread to close it when it was running with a request)?

Here's more details:

I read the docs in both NIOFSDirectory and SimpleFSDirectory and I got:

  • they support multithreading:

NIOFSDirectory: "An FSDirectory implementation that uses a java.nio FileChannel positional read that allows multiple threads to read from a single file without synchronization."

SimpleFSDirectory: "Direct implementation of FSDirectory using java.io.RandomAccessFile. However, this class has poor concurrency performance (multiple threads will be a bottleneck) because it synchronizes when reading multiple threads from the same file. It is usually better to use NIOFSDirectory or MMapDirectory."

  • NIOFSDirectory is better suited (mostly faster) than SimpleFSDirectory in a multithreaded context (see above)

  • NIOFSDIrectory does not work on Windows. On Windows, SimpleFSDirectory is recommended. However, on * nix OS, NIOFSDIrectory works great and due to better multithreading performance it is recommended over SimpleFSDirectory.

"NOTE: NIOFSDirectory is deprecated on Windows due to a bug in the way FileChannel.read is implemented in the Sun JRE. Internally, the position appears to be in sync."

The reason I am asking is that I have seen some actual projects where the target OS is Linux, the NIOFSDirectory is used to read from the index, but an instance of it is created for every request (from every thread) and as soon as the request is done and results will be returned, the thread closes this instance (only to create a new one on next request, etc.). So I was wondering if this is really a better approach than just creating one instance of NIOFSDirectory that will be used by all threads, and just opening it when the application starts up and closing it later when a certain (multi-threaded) job is finished ...

Moreover, for a web application, is it better not to have something like a context listener that instantiates NIOFSDirectory, puts it in the application context, all servlets share and use it, and then that same context listener closes it when the application shuts down?

+3


source to share


1 answer


Lucene's official answers suggest the following:

Share one IndexSearcher between requests and threads in your application.



IndexSearcher

requires a single one IndexReader

, and the latter can be created with a help DirectoryReader.open(Directory)

that only requires one instance Directory

.

+2


source







All Articles