What I do:

            if(file_exists($this->config->searchIndexFileName))
                    $this->index = Zend_Search_Lucene::open($this->config->searchIndexFileName);
            else
                    $this->index = Zend_Search_Lucene::create($this->config->searchIndexFileName);

            Zend_Search_Lucene_Analysis_Analyzer::setDefault(new Zend_Search_Lucene_Analysis_Analyzer_Common_TextNum_CaseInsensitive());
            $document = new Zend_Search_Lucene_Document();

            $document->addField(Zend_Search_Lucene_Field::Keyword('name', $item->name));
            $document->addField(Zend_Search_Lucene_Field::Text('description', $item->description));
            $document->addField(Zend_Search_Lucene_Field::Keyword('extension', $item->extension));
            $document->addField(Zend_Search_Lucene_Field::UnIndexed('creationdate', $item->creationDate));
            $document->addField(Zend_Search_Lucene_Field::Keyword('type', $item->objectType));
            $document->addField(Zend_Search_Lucene_Field::Text('itemgroupnames', $itemGroupNames));
            $document->addField(Zend_Search_Lucene_Field::Text('itemgroupdescriptions', $itemGroupDescriptions));

            $this->index->addDocument($document);
            $this->index->commit();

When multiple instances of this page are loaded concurrently, the following error appears often:

Fatal error: Uncaught exception 'Zend_Search_Lucene_Exception' with message 'File '/tmp/lucene.index/_2s_6jebf52bb.del' is not readable.' in /usr/src/zend/library/Zend/Search/Lucene/Storage/File/Filesystem.php:59 Stack trace: #0 /usr/src/zend/library/Zend/Search/Lucene/Storage/Directory/Filesystem.php(358): Zend_Search_Lucene_Storage_File_Filesystem->__construct('/tmp/lucene.ind...') #1 /usr/src/zend/library/Zend/Search/Lucene/Index/SegmentInfo.php(357): Zend_Search_Lucene_Storage_Directory_Filesystem->getFileObject('_2s_6jebf52bb.d...') #2 /usr/src/zend/library/Zend/Search/Lucene/Index/Writer.php(548): Zend_Search_Lucene_Index_SegmentInfo->__construct(Object(Zend_Search_Lucene_Storage_Directory_Filesystem), '_2s', 1, 1.84467440737E+19, NULL, 1, true) #3 /usr/src/zend/library/Zend/Search/Lucene/Index/Writer.php(790): Zend_Search_Lucene_Index_Writer->_updateSegments() #4 /usr/src/zend/library/Zend/Search/Lucene/Index/Writer.php(399): Zend_Search_Lucene_Index_Writer->commit() #5 /usr/src/zend/library/Zend/Search/Lucene/Inde in /usr/src/zend/library/Zend/Search/Lucene/Storage/File/Filesystem.php on line 59

Looking into the index dir, /tmp/lucene.index/_2s_6jebf52bb.del does indeed not exist:

ls -l /tmp/lucene.index/

total 84 -rw-rw-rw- 1 apache apache 347 2008-09-13 18:30 segments_3b -rw-rw-rw- 1 apache apache 369 2008-09-13 18:30 segments_3c -rw-rw-rw- 1 apache apache 347 2008-09-13 18:30 segments_3d -rw-rw-rw- 1 apache apache 2318 2008-09-13 17:02 _18.cfs -rw-rw-rw- 1 apache apache 2244 2008-09-13 17:02 _1j.cfs -rw-rw-rw- 1 apache apache 2239 2008-09-13 17:02 _1u.cfs -rw-rw-rw- 1 apache apache 2264 2008-09-13 17:02 _25.cfs -rw-rw-rw- 1 apache apache 2293 2008-09-13 17:02 _2g.cfs -rw-rw-rw- 1 apache apache 614 2008-09-13 18:29 _2x.cfs -rw-rw-rw- 1 apache apache 2619 2008-09-13 18:29 _2y.cfs -rw-rw-rw- 1 apache apache 615 2008-09-13 18:29 _2z.cfs -rw-rw-rw- 1 apache apache 615 2008-09-13 18:29 _30.cfs -rw-rw-rw- 1 apache apache 607 2008-09-13 18:30 _31.cfs -rw-rw-rw- 1 apache apache 607 2008-09-13 18:30 _32.cfs -rw-rw-rw- 1 apache apache 614 2008-09-13 18:30 _33.cfs -rw-rw-rw- 1 apache apache 614 2008-09-13 18:30 _34.cfs -rw-rw-rw- 1 apache apache 614 2008-09-13 18:30 _35.cfs -rw-rw-rw- 1 apache apache 2282 2008-09-13 17:02 _b.cfs -rw-rw-rw- 1 apache apache 2251 2008-09-13 17:02 _m.cfs -rw-rw-rw- 1 apache apache 2263 2008-09-13 17:02 _x.cfs -rw-rw-rw- 1 apache apache 0 2008-09-13 18:30 optimization.lock.file -rw-rw-rw- 1 apache apache 0 2008-09-13 18:30 read.lock.file -rw-rw-rw- 1 apache apache 0 2008-09-13 18:30 read-lock-processing.lock.file -rw-rw-rw- 1 apache apache 0 2008-09-13 18:30 write.lock.file -rw-rw-rw- 1 apache apache 20 2008-09-13 18:30 segments.gen

This is not a permission problem, the index (directory) was created by another php page shortly before.

This sounds a little like bug 3514 or 2448, except its using today's SVN. The filesystem containing the index is "ext3 (rw,noatime)", I'm using 2.6.24-gentoo, apache 2.2.8 and php 5.2.6.