Details
-
Type:
Bug
-
Status:
Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: None
-
Fix Version/s: 0.2.0
-
Component/s: Zend_Search_Lucene
-
Labels:None
Description
In playing around with the Lucene_Search tool, I ran into a PHP Fatal error when trying to load large amounts of data (actually, not all that much data, but around 200,000 varchar(100) text fields): PHP FATAL ERROR: Allowed memory size of 104857600 bytes exhausted (tried to allocated 119 bytes) in Zend/Search/Lucene/Analysis/Analyzer/Common/Text.php on line 65. and PHP FATAL ERROR: Allowed memory size of 104857600 bytes exhausted (tried to allocated 64 bytes) in Zend/Search/Lucene/Index/SegmentWriter.php on line 480.
I am curios if there are any known restrictions on the amount of data you can use when building the Lucene search documents?
It depends from segment merging functionality.
addDocument() accumulates documents in memory and writes down new segment only at commit() time.
Frequent commits make index segments small and non-optimal.
Take a look on a http://framework.zend.com/manual/en/zend.search.index-creation.html#zend.search.index-creation.updating
Segment merging functionality will allow make index optimization automatically.