Issue Type: Bug Created: 2007-08-14T15:49:50.000+0000 Last Updated: 2010-11-24T03:53:28.000+0000 Status: Resolved Fix version(s): Reporter: Alex Adriaanse (alexadriaanse) Assignee: Shahar Evron (shahar) Tags: - Zend_Http_Client
Related issues: - ZF-5407
Attachments: - Http.diff
I am using Zend_Http_Client against a web application that runs on IIS 4.0. I noticed that I wasn't able to log into this application using Zend_Http_Client, but it worked fine from Firefox. I was able to narrow the cause of this problem down to the cookie getting urlencode()d where the web server/application didn't like this.
After logging in, this is the response from the server (sensitive parts are X'ed out):
<pre class="literal">
HTTP/1.1 302 Object moved
Server: Microsoft-IIS/4.0
Date: Tue, 14 Aug 2007 20:27:13 GMT
MicrosoftOfficeWebServer: 5.0_Pub
Location: XXXXXX.asp
Content-Length: 131
Content-Type: text/html
Set-Cookie: XXXXXX=HID=XXXXXX&UN=XXXXXXX&UID=XXXXX; path=/
Cache-control: private
In the subsequent request Zend_Http_Client sends the following:
<pre class="literal">
GET /XXXXXXX/XXXXXX.asp HTTP/1.1
Host: <a href="www.XXXXXXX">www.XXXXXXX</a>
Accept-encoding: gzip, deflate
User-agent: Zend_Http_Client
Cookie: ASPSESSIONIDXXXXXXXX=XXXXXXXXXXXXXXXXXXXXX;XXXXXX=HID%3DXXXXXX%26UN%3DXXXXXXX%26UID%3DXXXXX;
Note how the cookie that was sent to us has been urlencode()d in our request. This web application apparently does not recognize this cookie and thus prevents us from logging in.
If I remove the urlencode()ing from Zend_Http_Cookie::__toString(), logging into this application works fine:
<pre class="literal">
--- Zend/Http/Cookie.php (revision 1597)
+++ Zend/Http/Cookie.php (working copy)
@@ -240,7 +240,7 @@
*/
public function __toString()
{
- return $this->name . '=' . urlencode($this->value) . ';';
+ return $this->name . '=' . $this->value . ';';
}
/**
The new request will look like this:
<pre class="literal">
GET /XXXXXXX/XXXXXX.asp HTTP/1.1
Host: <a href="www.XXXXXXX">www.XXXXXXX</a>
Accept-encoding: gzip, deflate
User-agent: Zend_Http_Client
Cookie: ASPSESSIONIDXXXXXXXX=XXXXXXXXXXXXXXXXXXXXX;XXXXXX=HID=XXXXXX&UN=XXXXXXX&UID=XXXXX;
By the way, this (un-urlencode()d) version is also how Firefox sends the cookie to the server.
I'm not sure if urlencode()ing is required by any RFCs dealing with cookies or whether this is simply a case where IIS or the web application in question is broken. If we need to keep urlencode() in Zend_Http_Cookie::__toString() then it'd be nice if we could specify an option to turn off urlencode()d cookies for broken web applications.
Thanks.
Posted by Thomas Weidner (thomas) on 2007-08-16T13:32:06.000+0000
Assigned to Shahar
Posted by Alexander Cheshchevik (than) on 2008-05-03T00:38:10.000+0000
The same problem.
For example: Receive: Set-Cookie: JSESSIONID=0000j8CydACPu_-J9bE8uTX91YU:12a83ks4k; Path=/
And we have urlencoded session incorrect value: Cookie: JSESSIONID=0000j8CydACPu_-J9bE8uTX91YU%3A12a83ks4k;
Cookie value can contain ":" and other escaped symbols.
Posted by Lukas Smith (lsmith) on 2008-06-13T15:08:47.000+0000
I also got bitten by this really nasty bug. Very hard to figure out what's going on. The following bug in PEAR along with the fix might also be worth a look (if clean IP does not scare you off from looking at it): http://pear.php.net/bugs/bug.php?id=1080
Posted by Lukas Smith (lsmith) on 2008-06-16T07:33:20.000+0000
The relevant RFC's fail to discuss encoding of specialchars like semicolon, equal or newline all together: http://www.faqs.org/rfcs/rfc2109.html http://www.faqs.org/rfcs/rfc2965
General consensus on the web seems to be that v0 cookies do not define any encoding.
Using urlencoding seems to be a popular approach though, supposedly for v1 cookies. That being said in this case the encoding needs to be applied to both the name and the value.
There are however obviously servers that do not support v1 cookies and which will run into issues with urlencoding. For these there should be an option to skip urlencoding entirely.
Posted by Lukas Smith (lsmith) on 2008-06-16T23:48:41.000+0000
Ok after some discussion with Dave Kristol (http://kristol.org/cookie/errata.html) it seems obvious that there needs to be another way to set the cookie. Indeed the standard is quite lax about encoding. The idea was that the browser should send back whatever was set and it is the job of the server to be able to interpret whatever it gets back again.
So when sending a cookie, it should only do whatever encoding that was decoded previously. In many cases urlencoding might be a reasonable default, but in many cases it isnt. So I suggest to do the following:
Posted by Lukas Smith (lsmith) on 2008-06-18T09:02:54.000+0000
Zend_Http_Client is also affected by this issue, as there is also code in there that forced urlencode() on cookie values.
Posted by Lukas Smith (lsmith) on 2008-06-18T09:21:27.000+0000
Here is a solution for just passing through the cookie that was send to PHP. This way when PHP is sitting between the user browser and the original server that set the cookie as a proxy, the cookie is passed through without change to the server that set the cookie.
class My_Zend_Http_Client extends Zend_Http_Client { /** * Add a cookie to the request. If the client has no Cookie Jar, the cookies * will be added directly to the headers array as "Cookie" headers. * * @param Zend_Http_Cookie|string|bool $cookie * @param string|null $value If "cookie" is a string, this is the cookie value. * @return Zend_Http_Client */ public function setCookie($cookie, $value = null) { if ($cookie !== true) { return parent::setCookie($cookie, $value); }
$this->headers['cookie'] = array('Cookie', $_SERVER['HTTP_COOKIE']);
return $this;
}
}
Posted by Shahar Evron (shahar) on 2008-06-18T23:24:26.000+0000
Because this issue touches both Zend_Http_Cookie, Zend_Http_Client and in a way Zend_Http_Cookiejar - I think an ideal solution would be:
As a general rule, how does this sound?
Posted by Lukas Smith (lsmith) on 2008-06-19T00:13:31.000+0000
So to be able to pass through the cookies of the current request without any parsing I could do the following?
$client = new Zend_Http_Client($url, $opts); $cookie = Zend_Http_Cookie::fromString($_SERVER['HTTP_COOKIE']); $client->setCookie($cookie); $response = $client->request();
That seems like a workable approach to me.
Posted by Shahar Evron (shahar) on 2008-06-19T01:34:18.000+0000
Yeah - that would work.
It might also make good sense to add an API to quickly do that - but honestly I think this is quite advanced usage so just to save a line of code I think it's not required.
Posted by Lukas Smith (lsmith) on 2008-06-19T01:46:28.000+0000
Also, while I am focusing on the encode part. The same situation also needs to be addressed for the decoding stage. The raw cookie needs to be available in the response.
I do not agree that its so advanced. This situation will come up whenever PHP sits as a proxy between a browser and a server, where the server originally set the cookie. In my case its relevant because of a mashup we are doing. The reason why few users get bitten by this issue is that indeed most servers support url encoding. So the scenario is not so obscure, only that it creates problems is. Now if we make this something the user has to do on his own, then you will have all sorts of users fall into this trap over and over again (even if you document this). But if there is a method "passBrowserCookieToServer()", then it will be clear for people that do write a proxy what method they should use to send the cookie to the server.
Thanks for looking into this!
Posted by Michael Rehbein (tech13) on 2010-01-13T07:13:33.000+0000
Attempts to fix both: http://framework.zend.com/issues/browse/ZF-1850 http://framework.zend.com/issues/browse/ZF-5407
Adds an option to Zend_Http_Client to override the old behavior
Adds an optional parameter to Zend_Http_Cookie to override the old behavior and effects how __toString behaves.
Posted by Michael Rehbein (tech13) on 2010-01-13T07:45:25.000+0000
Matching English documentation changes for previous patch
Posted by Shahar Evron (shahar) on 2010-02-11T06:27:25.000+0000
Thanks for the patch Michael - I've applied it in r. 21018, and it will be merged to the 1.10 release branch soon.
To not encode cookies, please set the 'encodecookies' config option to false.
Posted by Danila V. (dvershinin) on 2010-02-17T18:47:57.000+0000
Hi, not really fixed. I tested the patch: using http client, still getting encoded cookies even though the "encodecookies" option is set. When going through several requests and a cookie is added via CookieJar->addCookie:
public function addCookie($cookie, $ref_uri = null) { if (is_string($cookie)) { $cookie = Zend_Http_Cookie::fromString($cookie, $ref_uri); .... }
The fromString does not pass the preferred option onto config, thus fromString still gets $encodeValue = true
Posted by Matt Pinkston (mpinkston) on 2010-11-15T12:46:26.000+0000
The encodecookies preference is still not being passed along to cookies that are added to the cookiejar from the response (as pointed out by Danila V.)
Posted by Matt Pinkston (mpinkston) on 2010-11-15T12:49:24.000+0000
Here's a patch that passes along the encodecookies value to cookies added to the cookiejar through addCookiesFromResponse. My first patch, so I apologize if I missed a step.
http://framework.zend.com/issues/secure/…
Posted by Shahar Evron (shahar) on 2010-11-16T12:18:12.000+0000
The patch looks good, I have two questions:
Posted by Matt Pinkston (mpinkston) on 2010-11-17T11:46:07.000+0000
I did sign a CLA (Feb of 2009).
I haven't created any unit tests for this issue. (though I will vouch that it works in my real-world scenario)
Posted by Shahar Evron (shahar) on 2010-11-24T03:53:26.000+0000
Patch applied, unit test added to verify resolution. Committed to trunk in r23442, merged into 1.11 branch in 23443.