ZF-88: PHP allowed memory exhausted fatal error

Description

In playing around with the Lucene_Search tool, I ran into a PHP Fatal error when trying to load large amounts of data (actually, not all that much data, but around 200,000 varchar(100) text fields): PHP FATAL ERROR: Allowed memory size of 104857600 bytes exhausted (tried to allocated 119 bytes) in Zend/Search/Lucene/Analysis/Analyzer/Common/Text.php on line 65. and PHP FATAL ERROR: Allowed memory size of 104857600 bytes exhausted (tried to allocated 64 bytes) in Zend/Search/Lucene/Index/SegmentWriter.php on line 480.

I am curios if there are any known restrictions on the amount of data you can use when building the Lucene search documents?

Comments

It depends from segment merging functionality.

addDocument() accumulates documents in memory and writes down new segment only at commit() time.

Frequent commits make index segments small and non-optimal. Take a look on a http://framework.zend.com/manual/en/…

Segment merging functionality will allow make index optimization automatically.

Done. MergeFactor, MaxMergeDocs and MaxBufferedDocs index options define auto-optimization behaviour.

This is patch that hopefully fixes Zend_Search_Lucene for 64bit systems. It seems that it is a reverse patch.

I'm sorry about attaching a patch file in wrong place. The right place is here http://framework.zend.com/issues/browse/ZF-4071