Issue Type: Performance Improvement Created: 2009-10-13T06:49:44.000+0000 Last Updated: 2009-10-16T08:22:18.000+0000 Status: Resolved Fix version(s): Reporter: Alexander Mazalov (alex347) Assignee: Alexander Veremyev (alexander) Tags: - Zend_Search_Lucene
Related issues: Attachments:
Please add a limit for n top results to Zend_Search_Lucene. I have a big database and setResultSetLimit is not satisfactory.
Posted by Alexander Veremyev (alexander) on 2009-10-16T06:58:57.000+0000
Generating "N top results" needs to compose whole result set and calculate all scores. That takes exactly the same time as returning all results.
One suggestion which may help, you should be careful with using results. First access to the returned hit (access to any field instead id and score) provokes whole document retrieving (stored fields retrieving). It takes additional time. So, only use top N results (first N results from whole result set) adn don't touch others. That will give you exactly the same functionality as "returning top N results".
If you still see such functionality useful, even it doesn't give any performance improvement, please reopen issue.
Posted by Alexander Mazalov (alex347) on 2009-10-16T08:22:18.000+0000
I have 10M records in database. One query could match 500k records. I will never need more than 1k results.
Yes, it still needs to calculate all scores but don't need to sort and keep in memory all other 499k records. CPU usage will decrease noticeably.
Have you found an issue?
See the Overview section for more details.