Issues

ZF-4071: Lucene doesn't work on 64-bit

Issue Type: Bug Created: 2008-08-26T05:54:34.000+0000 Last Updated: 2010-05-03T03:19:13.000+0000 Status: Resolved Fix version(s): - 1.8.4 (23/Jun/09)

Reporter: Igor D'Astolfo (dusty) Assignee: Alexander Veremyev (alexander) Tags: - Zend_Search_Lucene

Related issues: - ZF-5141

Attachments: - lucene-64bit.diff

Description

I tried Zend_Search_Lucene on a 32 bit and all was fine. Trying on a 64 bit simply doesn't work. The same index correctly opened on the 32 bit machine raises an exception on 64 bit. Looking in the code I discovered that there's a problem with integer precision: on line 422 of Zend/Search/Lucene.php the code:

$numField = $segmentsFile->readInt();

gives -1 on 32 bit, 4294967295 on 64 bit, reading the same index.

moreover, on line 425 the code

if ($numField != (int)0xFFFFFFFF)

fails, since 0xFFFFFFFF has a different value on 32 bit and 64 bit. As workaround I changed numField to -1 and skipped the check and it works. So I think it's sufficent to change the test and the readInt function.

Comments

Posted by Piotr Gabryjeluk (gabrys) on 2009-01-20T10:01:26.000+0000

Hello guys,

I've just experienced this issue, quick-looked into the code and just made a patch for this. It's changing -1 to (int)0xFFFFFFFF in more places, like in other places it is done already.

The patch (looks like this is reverse patch, but you'll deal with it):

Index: library/Zend/Search/Lucene.php

--- library/Zend/Search/Lucene.php (revision 13711) +++ library/Zend/Search/Lucene.php (working copy) @@ -413,7 +413,7 @@ if ($this->_formatVersion == self::FORMAT_2_3) { $docStoreOffset = $segmentsFile->readInt();

  • if ($docStoreOffset != -1) {
  • if ($docStoreOffset != (int)0xFFFFFFFF) { $docStoreSegment = $segmentsFile->readString(); $docStoreIsCompoundFile = $segmentsFile->readByte();

Index: library/Zend/Search/Lucene/Index/Writer.php

--- library/Zend/Search/Lucene/Index/Writer.php (revision 13711) +++ library/Zend/Search/Lucene/Index/Writer.php (working copy) @@ -496,7 +496,7 @@ if ($srcFormat == Zend_Search_Lucene::FORMAT_2_3) { $docStoreOffset = $segmentsFile->readInt();

  • if ($docStoreOffset != -1) {
  • if ($docStoreOffset != (int)0xFFFFFFFF) { $docStoreSegment = $segmentsFile->readString(); $docStoreIsCompoundFile = $segmentsFile->readByte();

Posted by Piotr Gabryjeluk (gabrys) on 2009-01-20T10:06:21.000+0000

This is patch to hopefully fix the Lucene addDocument on 64bit systems. It seems it is a reverse patch.

Posted by Piotr Gabryjeluk (gabrys) on 2009-02-15T05:52:01.000+0000

Any information on this, ZF team?

Posted by Alexander Veremyev (alexander) on 2009-06-19T04:44:27.000+0000

Fixed

Posted by Piotr Gabryjeluk (gabrys) on 2009-06-19T15:25:12.000+0000

This: http://framework.zend.com/code/viewrep/… seams to include what I suggested.

I'll test it. Thanks!

Posted by John (bj) on 2010-05-03T03:19:11.000+0000

Hello guys, I need your help.

I've installed Drupal (http://drupal.org/), module Search Lucene API (http://drupal.org/project/luceneapi) with Zend Framework components (http://drupal.org/node/521604) and module Search Lucene DidYouMean (http://drupal.org/project/luceneapi_dym). First partial indexings were successful, but now "Index statistics" for Search Lucene DidYouMean is: "Number of documents in index: 13500; Number of terms: 90598", and I'm getting "Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 71 bytes) in /var/www//data/www//modules/luceneapi_dym/luceneapi_dym.index.inc on line 64" when I'm trying to perform indexing. Replacing -1 with (int)0xFFFFFFFF, as you suggested, doesn't solve a problem. Applying fixes from http://framework.zend.com/code/viewrep/… makes search inoperative. Replacing library folder with one, extracted from http://framework.zend.com/download/latest, too.

Please upload to somewhere an archive, which I should to use instead of Zend Framework components (http://sourceforge.net/projects/luceneapi) to get things working. Or give me to know what I need to change in code. I'll be very grateful!

Have you found an issue?

See the Overview section for more details.

Copyright

© 2006-2016 by Zend, a Rogue Wave Company. Made with by awesome contributors.

This website is built using zend-expressive and it runs on PHP 7.

Contacts