ZF-2929: Problems with German Umlaute since ZF 1.5

Issue Type:
Bug
Created:
2008-03-19T12:48:56.000+0000
Last Updated:
2012-03-21T11:32:08.000+0000
Status:
Resolved
Fix version(s):
  • 1.5.2 (15/May/08)
Reporter:
Dominik Bors (dom)
Assignee:
Satoru Yoshida (satoruyoshida)
Tags:
  • Zend_Filter
Related issues:
Attachments:

Description

We have some strange behaviour with Zend_Filter_Alnum & Zend_Filter_Alpha since ZF 1.5. Everything works correct by using ZF 1.0.4.

By folowing code line echo Zend_Filter_Alnum::filter("Lüge"); // in utf-8 encoding delivers Lge instead of Lüge until ZF 1.0.x the result was correct.

Corresponding code block in Zend/Filter/Alnum.php: if (!self::$_unicodeEnabled) { // POSIX named classes are not supported, use alternative a-zA-Z0-9 match $pattern = '/[^a-zA-Z0-9' . $whiteSpace . ']/'; } else if (extension_loaded('mbstring')) { // Unicode safe filter for the value with mbstring $pattern = '/[^[:alnum:]' . $whiteSpace . ']/u'; } else { // Unicode safe filter for the value without mbstring $pattern = '/[^\p{L}\p{N}' . $whiteSpace . ']/u'; }

The problem is we have the mbstring enabled and this line don't work
$pattern = '/[^[:alnum:]' . $whiteSpace . ']/u'; but this line $pattern = '/[^\p{L}\p{N}' . $whiteSpace . ']/u'; The internal-encoding is correctly set to UTF-8.

The question is why the [:alnum:] line don't work, and why it is useful to handle on another way if the extension is enabled ?

Comments

Posted by old of Satoru Yoshida (yoshida@zend.co.jp) on 2008-03-19T19:19:51.000+0000

Hello, Dominik. Is the mbstring extension used in German ?

In the last version, before changing in ZF-2107, only "^\p{L}\p{N}" pattern is used.

But I found it causes error in Japanese. The problem is all character of Japanese is passed. So, I changed because I thought mbstring extension used only in the language that has many multibyte characters.

But if the mbstring extension used in German , (or Czeck, Polish...etc) ZF-2107 happens your problems.

Do you have any idea instead of using "if (extension_loaded('mbstring'))" ? It seems to be better if we use language location in if statement.

Posted by Wil Sinclair (wil) on 2008-03-25T21:09:03.000+0000

Please categorize/fix as needed.

Posted by Dominik Bors (dom) on 2008-04-07T04:25:45.000+0000

Hello Satoru,

sorry for my late answer.

I don't think mbstring extension is used in German, it shouldn't. But the mbextension often is installed by default.

I think we should add a second condition to the if statement, maybe something that asks if the default zend_locale is japanese.

Best regards Dominik

Posted by old of Satoru Yoshida (yoshida@zend.co.jp) on 2008-04-10T08:47:32.000+0000

Hello, Dominik. Thank you for your reply.

Ok, I try to add some condition by using Zend_Locale. :-)

Posted by Kirill (kupuyc) on 2008-04-16T18:05:46.000+0000

I have same problem with russian characters:

$filter = new Zend_Filter_Alnum(true);  
Zend_Debug::dump($filter->filter('это странненько - mbstring enabled'));  
//string(19) "   mbstring enabled"
$filter = new Zend_Filter_Alnum(true);  
Zend_Debug::dump($filter->filter('это странненько - mbstring disabled'));  
//string(48) "это странненько  mbstring disabled"

In my case $pattern = '/[^[:alnum:]' . $whiteSpace . ']/u'; is used by filter.

Posted by old of Satoru Yoshida (yoshida@zend.co.jp) on 2008-04-16T20:10:41.000+0000

Thank You for Your information, Kirill.

Posted by old of Satoru Yoshida (yoshida@zend.co.jp) on 2008-04-21T01:14:24.000+0000

Resolved in SVN r9266