Issue Type: Bug Created: 2008-01-23T15:12:19.000+0000 Last Updated: 2008-03-21T16:25:41.000+0000 Status: Resolved Fix version(s): - 1.5.0 (17/Mar/08)
Reporter: Alexander Veremyev (alexander) Assignee: Alexander Veremyev (alexander) Tags: - Zend_Search_Lucene
Related issues: Attachments: - test.php
I have found a lurking bug which is critical (applies to the most revision 7584 in trunk SVN): If two long-running threads are doing multiple updates on the index at the same time, the index can be left in a corrupted/unusable.
Specific situations I have found to cause index corruption: - two different long-running threads adding and/or deleting many different documents at the same time - calling the $index->optimize() command while another thread is adding and/or deleting documents.
An exception thrown: 'Zend_Search_Lucene_Exception' with message 'File '[some_index_file_in_the_data_directory]' is not readable.'... The index remains unusable/corrupted for all future requests.
I believe there to be a bug in how flock is being used somewhere in the framework code (possibly due to race conditions). You might suggest that my filesystem doesn't support the flock feature. I checked this out and it shouldn't be an issue. I am using this on a PHP's flock() function works as expected (testing on updated CentOS with most recent official Redhat backported security patched PHP 5.1.6).
It should be noted that certain aspects of the locking do work: - if a single thread is writing (updating) the index, and other threads are just reading (searching) the index, the writing/reading threads seem to block each other properly, without any problems.
I hope this problem isn't too difficult to fix. To you help reproduce it, using a fresh copy of the same test script I sent before (see attachment) do the following:
Posted by Alexander Veremyev (alexander) on 2008-01-25T11:24:59.000+0000
Have you found an issue?
See the Overview section for more details.