Zend Framework

memory segmen storage provider

Details

  • Type: Improvement Improvement
  • Status: Resolved Resolved
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: 0.8.0
  • Fix Version/s: 0.8.0
  • Component/s: Zend_Search_Lucene
  • Labels:
    None

Description

Zend_Search_Lucene_Storage_Directory_RAM and Zend_Search_Lucene_Storage_File_RAM classes should be added to store segment in memory.
It should improve indexing performance.

Issue Links

This issue depends on:
ZF-87 Index optimization (segment merging) Major Resolved

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Alexander Veremyev added a comment -

In-memory segment creation is not significant without segment mergin functionality

Show
Alexander Veremyev added a comment - In-memory segment creation is not significant without segment mergin functionality
Hide
Permalink
Alexander Veremyev added a comment -

No, it's Zend_Search issue, which doesn't depends from Zend_Memory.

Zend_Memory goal is to use filesystem (or other backend) as a temporary storage for data, when we don't have enough memory.

memory segmen storage provider is a 'virtual filesystem' in RAM. Goal is to speed up document indexing, which performs a lot of small read/write operations.

Show
Alexander Veremyev added a comment - No, it's Zend_Search issue, which doesn't depends from Zend_Memory. Zend_Memory goal is to use filesystem (or other backend) as a temporary storage for data, when we don't have enough memory. memory segmen storage provider is a 'virtual filesystem' in RAM. Goal is to speed up document indexing, which performs a lot of small read/write operations.
Hide
Permalink
Bill Karwin added a comment -

Changing fix version to 0.6.0.

Show
Bill Karwin added a comment - Changing fix version to 0.6.0.
Hide
Permalink
Alexander Veremyev added a comment -

obsolete.

Zend_Search_Lucene uses MaxBufferedDocs option to accumulate documents in memory.

Show
Alexander Veremyev added a comment - obsolete. Zend_Search_Lucene uses MaxBufferedDocs option to accumulate documents in memory.
Hide
Permalink
Chris Mattmann added a comment -

Hi,

In the case where there is a separate Lucene index, built from e.g., Java Lucene, then wouldn't a Zend_Search_Lucene RAMDirectory that loads that separately created index into memory and then supports querying of it as persisted in RAM significantly speed up the performance of ZSL?

I can see how MaxBufferedDocs affects indexing performance, but how does it affect search performance?

Thanks,
Chris

Show
Chris Mattmann added a comment - Hi, In the case where there is a separate Lucene index, built from e.g., Java Lucene, then wouldn't a Zend_Search_Lucene RAMDirectory that loads that separately created index into memory and then supports querying of it as persisted in RAM significantly speed up the performance of ZSL? I can see how MaxBufferedDocs affects indexing performance, but how does it affect search performance? Thanks, Chris
Hide
Permalink
Alexander Veremyev added a comment -

Chris, I've just got your comment. MaxBufferedDocs doesn't affect search performance. But Java Lucene RAMDirectory implementation also used only during indexing. OS supported RAM disk or OS level file system caching may be used to increase search performance.

Show
Alexander Veremyev added a comment - Chris, I've just got your comment. MaxBufferedDocs doesn't affect search performance. But Java Lucene RAMDirectory implementation also used only during indexing. OS supported RAM disk or OS level file system caching may be used to increase search performance.
Hide
Permalink
Chris Mattmann added a comment -

Thanks for your comment, Alexander. Is an OS supported RAM disk or OS level file system caching something that we could implement in Zend_Search_Lucene? I'm willing to help, just point me in the right direction...

Show
Chris Mattmann added a comment - Thanks for your comment, Alexander. Is an OS supported RAM disk or OS level file system caching something that we could implement in Zend_Search_Lucene? I'm willing to help, just point me in the right direction...

People

Vote (0)
Watch (1)

Dates

  • Created:
    Updated:
    Resolved:
Atlassian JIRA (v4.4.3#663-r165197) | Report a problem
Powered by a free Atlassian JIRA open source license for Zend Framework. Try JIRA - bug tracking software for your team.