ZF-7987: Zend_Validate_Float locale

Description

My locale is et_EE

(Zend_Locale::setDefault(LOCALE); is set)

Float validator fails if I use construct method without locale - it won't chek default locale but uses localeconv() instead

So I tried to set php locale to et_EE (decimal_point is ",") but there is a bug when comapearing values in Float.php

        $locale        = localeconv();
        $valueFiltered = str_replace($locale['thousands_sep'], '', (string) $value);
        $valueFiltered = str_replace($locale['decimal_point'], '.', $valueFiltered);

        if (strval(floatval($valueFiltered)) != $valueFiltered) {
            $this->_error(self::NOT_FLOAT);
            return false;
        }

$valueFiltered is something like 1.1 but floatval($valueFiltered) return 1,1 if you resest PHP's locale with setlocale()

Comments

Why should Zend_Validate_Float work with the fallback locale from Zend_Locale ?

You set no locale to use for Zend_Validate_Float. This means that Zend_Validate_Float works locale-independent.

It would be better when you say what's your problem.

Closing as non issue.

I tested with "de_AT" which itself also uses "," as delimiter and found no problem.

problem comes when you use:

setlocale(LC_ALL, 'de_AT'); (locale must be installed on OS too) or setlocale(LC_ALL, 'de_AT.UTF-8');

no try

$valitator = Zend_Validate_Float() // without locale

then validator takes system locale and bug comes out. It would be nice validator would chek Zend_Locale::getDefault()

Using your code, after a little correction because it does not conform PHP, I get a true or false in return regardless of what the input is with the actual release.

There are no warnings, notices, errors, or exceptions, so there is no issue. Check your installation.

The issue still remains when using the float validator without locale. This is the default.

Code to reproduce:


$v = new Zend_Validate_Float();
setlocale(LC_ALL, 'C');
var_dump($v->isValid(1.3));   //true
var_dump($v->isValid("1.3")); //true
var_dump($v->isValid("1,3")); //false as expected

$v = new Zend_Validate_Float(Zend_Locale::findLocale('de_AT'));
var_dump($v->isValid(1.3));   //true
var_dump($v->isValid("1.3")); //true
var_dump($v->isValid("1,3")); //true

$v = new Zend_Validate_Float();
setlocale(LC_ALL, 'de_AT');
var_dump($v->isValid(1.3));   //false!!
var_dump($v->isValid("1.3")); //true
var_dump($v->isValid("1,3")); //false!!

Insufficient informations: C seems to rely on "en" in your environment.

Regarding using setlocale 'de_AT' I would like to know why you think that 1.3 or 1,4 are wrong.

In german the "," is used as precision separator. This means 1.3 is wrong as it should be 1,3. In english this is reversed.

Using setlocale the settings are completly dependend on your system as localeconv is used to get the correct settings in this case.

Therefor this issue will not be reopened.

{quote} Regarding using setlocale 'de_AT' I would like to know why you think that 1.3 or 1,4 are wrong. {quote}


$v = new Zend_Validate_Float();
setlocale(LC_ALL, 'de_AT');
var_dump($v->isValid("1,3")); //false!!

"1,3" should obviously be valid, as you wrote "," is used as decimal separator.

About the second case I'm not that sure:


$v = new Zend_Validate_Float();
setlocale(LC_ALL, 'de_AT');
var_dump($v->isValid(1.3)); //false!!

The thing is that 1.3 is not a string but a "real" float (as php type), so imho it should be valid. If you execute "echo 1.3;" you get "1,3" - it's just a "." as it's hardcoded - that won't happen in a real world application. Other than that "1.3" as string is valid too.

Once again:


1.) $v = new Zend_Validate_Float();    
2.) setlocale(LC_ALL, 'de_AT');
3.) var_dump($v->isValid("1,3")); //false!!

1.) locale = "en"; decimal = ","; thousand = "."; Set locale = environment as no locale has been set

Be aware that it COULD be that you registered a locale previously within the registry or your application. Those set objects would be used in such a case.

2.) locale = "de_AT"; decimal = "."; thousand = ",";

3.) Value "1,3" string As locale is de_AT it is converted to the value "1x3" where the x stands for a wrong character. It should be "1.3".

You can't depend on localized input and then expect that localization is not taken into account.

Additionally integer values (when you say "1x3" is stripped to "13") are also no valid float values. See into the PHP manual for is_float().

I can not verify your second example. It returns true for me which is correct in my eyes.

Ok, I'm not sure if we understand each other :D

I created a test that should show the problem.

  • setup and tearDown I copied from Zend_Measure_TemperatureTest (resets setlocale and Zend_Registry::get('Zend_Locale'))
  • testNoZendLocaleButPhpLocale shows my problem (fails for me)
  • testLocaleDeFloatType passes (I added that to make sure if "real" floats are considered valid when using Zend_Locale de)
  • testPhpLocaleDeFloatType shows that is is not the case when setlocale is used (fails for me)

(tested with latest trunk)

I think that I understood very well.

Regarding your example you gave:


1.) $valueFiltered = str_replace($locale['thousands_sep'], '', (string) $value);
2.) $valueFiltered = str_replace($locale['decimal_point'], '.', $valueFiltered);

3.) if (strval(floatval($valueFiltered)) != $valueFiltered) {

Think of what you said an you will see that this is not the problem at all.

Line 1.) The value is converted into a string!! So any float value like 1.1 would be converted into the string "1,1" when your locale enforces this.

Line 2.) The string has now be normalized to computional notation... in our case also "1.1"

Line 3.) Now there is what you really don't think of... floatval($valueFiltered) converts the string to a float value... so we have now a "float 1.1".... BUT and this is what you've missed, strval converts the float back to a string using the systems locale.

Ok, you explained why it's broken by showing me the implementation. You still think this is not a bug?


setlocale(LC_ALL, 'de_AT');
$v = new Zend_Validate_Float();    
var_dump($v->isValid("1,3"));

I added your tests to the testbed and the repo and it are is no failures. So eighter your testcases are broken when they show different behaviour, or you are not using latest trunk as you stated.

PS: I used latest trunk, actually r20521 and your testcases have been added with r20515.

Oh - so it's a php or system issue. I'm using PHP 5.2.6-1+lenny4 on debian.

Thanks for adding the test.

one issue however:


Index: FloatTest.php
===================================================================
--- FloatTest.php       (Revision 20521)
+++ FloatTest.php       (Arbeitskopie)
@@ -144,7 +144,7 @@
     {
         setlocale(LC_ALL, 'de');
         $valid = new Zend_Validate_Float();
-        $this->assertTrue($valid->isValid(123,456));
+        $this->assertTrue($valid->isValid('123,456'));
     }

     /**

And why should I add an invalid test?

Did you try your example yourself? According to your test the following lines should always return the same output:


setlocale(LC_ALL, 'de');
$value = '123,456';
var_dump($value);
var_dump((string) $value);
var_dump(floatval($value));

But as you can see PHP itself does NOT detect float values localized. And when you don't give a locale, then PHP is used to detect the value. On the other hand, when you give a locale then localized values will be detected.

Simply said... when PHP would be locale aware then your test would return true instead of false.

Zend Framework will not change PHP internal behaviour. Therefor this issue will still not be reopened.

Use the locale parameter when you want localized detection or use it not when you want to use PHP native detection.

Output from your code: string(7) "123.456" string(7) "123.456" float(123,456) So yes, if the correct locale is set and it is supported, float values are converted from strings localized.

The code you added to the test: $valid->isValid(123,456) is just plain wrong - it gives two parameters, the first one being an int.

How floatval behaves depends on the OS, not on PHP. PHP differs therefor depending on the used OS.

In my environment the third var_dump of floatval returns 123 and not 123.456.

And you should know as you have stated yourself before, that within german the string '123,456' is equal to (float) 123.456.

Then you don't have a 'de' locale avaliable on your system. setlocale returns the actually used locale, please verify that it returns 'de' for you.


var_dump(setlocale(LC_ALL, array('de_DE', 'de_AT', 'de')));

I am a austrian person... my native language is german... my OS is a german version... my OS returns de... my browser is set to de.

And as you can see in the code I wrote I even force the locale DE in the first line.

Additionally I am the I18n team lead and have build this components... and you want to say me that I don't know how to check if I am really within a specified locale where I said that the test is OK? ;-)

I think there is nothing more to say :-)

Sorry, I didn't mean to upset you, and I'm sure you know more about I18n than me. Please ignore my last two comments.

But this is wrong:


$this->assertTrue($valid->isValid(123,456));

PS: I'm Austrian too :D

Once again: A string is not always converted into a float. This depends on the OS.

I used this example:


var_dump(setlocale(LC_ALL, 0));
$v = new Zend_Validate_Float();
var_dump($v->isValid("123,456"));

$value = "123,456";
var_dump((float) $value);

and the returned value was:


string 'LC_COLLATE=C;LC_CTYPE=German_Austria.1252;LC_MONETARY=C;LC_NUMERIC=C;LC_TIME=C' (length=78)
boolean false
float 123

This means for me that the string "123,456" is converted into a float 123 in my OS and a float 123,456 in your OS... so the behaviour is OS dependend.

Therefor when the input is a string within Zend_Validate_Float, the result can not always be correct.

So, to be independend from OS and PHP implementation you must give a locale.

But the issue is that this is not a string - the quotes are missing!

I can not add a test for a not supported feature which seems to work in one environment but not in anothers.

For localized inputs you need to use the localized feature.

This function call always passes two arguments, the first 123, the second 456: $valid->isValid(123,456) This does never depend on any locale setting.

And testNoZendLocaleButPhpLocale with quotes added fails for me. That's what is my original issue.

If Float validator doesn't listen OS locale - so what, I will survive. But at least it would be nice if it would listen Zend Framework default locale:

Zend_Locale::setDefault('et_EE'); // also floats are presented like "1.3" $validator = new Zend_Validate_Float(); var_export($validator->isValid('1,3')); //return false;

It is not normal to set a locale every time I haveto use Float_Validate.

Sorry: Zend_Locale::setDefault('et_EE'); // also floats are presented like * "1,3" *

And I can confirm:

var_dump(setlocale(LC_ALL, 'et_EE')); // returns 'et_EE' $v = new Zend_Validate_Float(); var_dump($v->isValid("123,456")); // returns false !!!!!!!!!!!!

I am sure that this is not what a developer wants

Take a look into the manual how to see how to set a locale which is used within the complete application. setDefault is not the application wide locale. Look into Zend_Locale or Zend_Application for details about this feature.

Related to OS dependend detection: As I already said, when you give a string without a locale then the detection is based on OS behaviour. The problem is, that PHP knows only english locale settings for floats.

Therefor as I already stated before, the detection can not be guaranteed in such a case. And this is also the reason why such a unittest will not be added, because we can't add a test for something which is not supported.

PHP itself is not locale aware for floats. And when you depend on PHP internals (by explicitly NOT giving a locale) you declare to use PHPs internals. Run the above code to see results.

It has been verified under Windows, CentOS and Ubuntu that conversion to float does not work setlocale aware.

For details to this PHP internal bug please take a look into ZF-8919 and vote for the solution you prefer (there are 2 ways we could go).

This issue will stay closed as this is actually a PHP bug.