ZF-7738: Adding multi-token awareness to rewrite() in Query_Preprozessing_... and Query_Phrase (patch)

Issue Type: New Feature Created: 2009-08-31T10:06:07.000+0000 Last Updated: 2012-12-22T21:46:45.000+0000 Status: Reopened Fix version(s): Reporter: Corvus Corax (corvuscorax) Assignee: None Tags: - Zend_Search_Lucene

Related issues: - ZF-7688

Attachments: - MultiTokenAbility.patch


class Zend_Search_Lucene_Search_Query_Preprocessing_Term had a couple of @todo lines (and thrown exceptions) regarding handling of term text when it was parsed as multiple tokens, either due to a stemming or synonym analyzer, or due to space interpreted characters in the term (like underscore)

The same with Zend_Search_Lucene_Search_Query_Preprocessing_Fuzzy

This patch adds awareness to both types of multi-tokens, and rewrites such queries (even when including wildcards) as Zend_Search_Lucene_Search_Query_Phrase

(instead of Query_MultiToken as before)

thereby handling $token->getPositionIncrement() to find out wether a token is to be handled as an alternative (0) or part of a "phrase" (>0) and setting the correct position within the phrase.

This patch works standalone for searching, however in order to have highlighting working correctly, it needs the Temp-Index highlighting patch ( ZF-7736 ).


Corvus Corax

PS: this extends the patches attached to ZF-7688, supposedly fixing that bug


Posted by Corvus Corax (corvuscorax) on 2009-09-02T08:31:00.000+0000


the test in Zend_Search_Lucene_Search_Query_Preprocessing_Term wether wildcard term alternatives have been found for all positions in the to be created Phrase query doesn't take on-purpose gaps into account and assumes there need to be tokens present for ALL positions.

Basically that means, the case "$token->getPositionIncrement() >= 2" is not taken care of and may need additional work to give usefull results.

It may not be very relevant in practice, but, as they say, never say "never" ;)

this issue should be much more easy to fix in Zend_Search_Lucene_Search_Query_Preprocessing_Fuzzy.

but I have no time doing that now, I am off for a holiday trip for the next two weeks.

Posted by Rob Allen (rob) on 2012-11-20T20:53:12.000+0000

Bulk change of all issues last updated before 1st January 2010 as "Won't Fix".

Feel free to re-open and provide a patch if you want to fix this issue.

Posted by Corvus Corax (corvuscorax) on 2012-11-21T08:13:48.000+0000

Bug has been closed by default in "Bulk change". However patch is already attached to bug, what has been missing is a maintainer

Posted by Rob Allen (rob) on 2012-12-22T21:46:45.000+0000

I tried to apply the patch, but it failed, so I applied it manually. However it causes the unit tests to fail.

Are you able to fix the patch?

Have you found an issue?

See the Overview section for more details.


© 2006-2018 by Zend, a Rogue Wave Company. Made with by awesome contributors.

This website is built using zend-expressive and it runs on PHP 7.