Issues

ZF-1850: urlencode()d cookies break IIS application

Issue Type: Bug Created: 2007-08-14T15:49:50.000+0000 Last Updated: 2010-11-24T03:53:28.000+0000 Status: Resolved Fix version(s): Reporter: Alex Adriaanse (alexadriaanse) Assignee: Shahar Evron (shahar) Tags: - Zend_Http_Client

Related issues: - ZF-5407

Attachments: - Http.diff

Description

I am using Zend_Http_Client against a web application that runs on IIS 4.0. I noticed that I wasn't able to log into this application using Zend_Http_Client, but it worked fine from Firefox. I was able to narrow the cause of this problem down to the cookie getting urlencode()d where the web server/application didn't like this.

After logging in, this is the response from the server (sensitive parts are X'ed out):

<pre class="literal">
HTTP/1.1 302 Object moved
Server: Microsoft-IIS/4.0
Date: Tue, 14 Aug 2007 20:27:13 GMT
MicrosoftOfficeWebServer: 5.0_Pub
Location: XXXXXX.asp
Content-Length: 131
Content-Type: text/html
Set-Cookie: XXXXXX=HID=XXXXXX&UN=XXXXXXX&UID=XXXXX; path=/
Cache-control: private

In the subsequent request Zend_Http_Client sends the following:

<pre class="literal">
GET /XXXXXXX/XXXXXX.asp HTTP/1.1
Host: <a href="www.XXXXXXX">www.XXXXXXX</a>
Accept-encoding: gzip, deflate
User-agent: Zend_Http_Client
Cookie: ASPSESSIONIDXXXXXXXX=XXXXXXXXXXXXXXXXXXXXX;XXXXXX=HID%3DXXXXXX%26UN%3DXXXXXXX%26UID%3DXXXXX;

Note how the cookie that was sent to us has been urlencode()d in our request. This web application apparently does not recognize this cookie and thus prevents us from logging in.

If I remove the urlencode()ing from Zend_Http_Cookie::__toString(), logging into this application works fine:

<pre class="literal">
--- Zend/Http/Cookie.php        (revision 1597)
+++ Zend/Http/Cookie.php        (working copy)
@@ -240,7 +240,7 @@
      */
     public function __toString()
     {
-        return $this->name . '=' . urlencode($this->value) . ';';
+        return $this->name . '=' . $this->value . ';';
     }

     /**

The new request will look like this:

<pre class="literal">
GET /XXXXXXX/XXXXXX.asp HTTP/1.1
Host: <a href="www.XXXXXXX">www.XXXXXXX</a>
Accept-encoding: gzip, deflate
User-agent: Zend_Http_Client
Cookie: ASPSESSIONIDXXXXXXXX=XXXXXXXXXXXXXXXXXXXXX;XXXXXX=HID=XXXXXX&UN=XXXXXXX&UID=XXXXX;

By the way, this (un-urlencode()d) version is also how Firefox sends the cookie to the server.

I'm not sure if urlencode()ing is required by any RFCs dealing with cookies or whether this is simply a case where IIS or the web application in question is broken. If we need to keep urlencode() in Zend_Http_Cookie::__toString() then it'd be nice if we could specify an option to turn off urlencode()d cookies for broken web applications.

Thanks.

Comments

Posted by Thomas Weidner (thomas) on 2007-08-16T13:32:06.000+0000

Assigned to Shahar

Posted by Alexander Cheshchevik (than) on 2008-05-03T00:38:10.000+0000

The same problem.

For example: Receive: Set-Cookie: JSESSIONID=0000j8CydACPu_-J9bE8uTX91YU:12a83ks4k; Path=/

And we have urlencoded session incorrect value: Cookie: JSESSIONID=0000j8CydACPu_-J9bE8uTX91YU%3A12a83ks4k;

Cookie value can contain ":" and other escaped symbols.

Posted by Lukas Smith (lsmith) on 2008-06-13T15:08:47.000+0000

I also got bitten by this really nasty bug. Very hard to figure out what's going on. The following bug in PEAR along with the fix might also be worth a look (if clean IP does not scare you off from looking at it): http://pear.php.net/bugs/bug.php?id=1080

Posted by Lukas Smith (lsmith) on 2008-06-16T07:33:20.000+0000

The relevant RFC's fail to discuss encoding of specialchars like semicolon, equal or newline all together: http://www.faqs.org/rfcs/rfc2109.html http://www.faqs.org/rfcs/rfc2965

General consensus on the web seems to be that v0 cookies do not define any encoding.

Using urlencoding seems to be a popular approach though, supposedly for v1 cookies. That being said in this case the encoding needs to be applied to both the name and the value.

There are however obviously servers that do not support v1 cookies and which will run into issues with urlencoding. For these there should be an option to skip urlencoding entirely.

Posted by Lukas Smith (lsmith) on 2008-06-16T23:48:41.000+0000

Ok after some discussion with Dave Kristol (http://kristol.org/cookie/errata.html) it seems obvious that there needs to be another way to set the cookie. Indeed the standard is quite lax about encoding. The idea was that the browser should send back whatever was set and it is the job of the server to be able to interpret whatever it gets back again.

So when sending a cookie, it should only do whatever encoding that was decoded previously. In many cases urlencoding might be a reasonable default, but in many cases it isnt. So I suggest to do the following:

  1. also url encode/decode the name by default 2) provide a setRawCookie() method where its up to the user to ensure proper encoding 3) maybe also provide a method to "pass along" a raw cookie in the current request, this method would parse the $_SERVER raw cookie value in a simple a way as possible. Maybe even just pass along the entire cookie in the raw form to prevent any chance of misinterpretation

Posted by Lukas Smith (lsmith) on 2008-06-18T09:02:54.000+0000

Zend_Http_Client is also affected by this issue, as there is also code in there that forced urlencode() on cookie values.

Posted by Lukas Smith (lsmith) on 2008-06-18T09:21:27.000+0000

Here is a solution for just passing through the cookie that was send to PHP. This way when PHP is sitting between the user browser and the original server that set the cookie as a proxy, the cookie is passed through without change to the server that set the cookie.

class My_Zend_Http_Client extends Zend_Http_Client { /** * Add a cookie to the request. If the client has no Cookie Jar, the cookies * will be added directly to the headers array as "Cookie" headers. * * @param Zend_Http_Cookie|string|bool $cookie * @param string|null $value If "cookie" is a string, this is the cookie value. * @return Zend_Http_Client */ public function setCookie($cookie, $value = null) { if ($cookie !== true) { return parent::setCookie($cookie, $value); }

    $this->headers['cookie'] = array('Cookie', $_SERVER['HTTP_COOKIE']);
    return $this;
}

}

Posted by Shahar Evron (shahar) on 2008-06-18T23:24:26.000+0000

Because this issue touches both Zend_Http_Cookie, Zend_Http_Client and in a way Zend_Http_Cookiejar - I think an ideal solution would be:

  1. Whatever is received from the server (through Zend_Http_Cookiejar, Zend_Http_Cookie::fromString() etc. is stored and sent back just the way it is
  2. Any cookies added manually by the user can be either encoded or not - depending on an additional parameter to Zend_Http_Client->setCookie(). By default it will be true (do encoding) in order to maintain BC.

As a general rule, how does this sound?

Posted by Lukas Smith (lsmith) on 2008-06-19T00:13:31.000+0000

So to be able to pass through the cookies of the current request without any parsing I could do the following?

$client = new Zend_Http_Client($url, $opts); $cookie = Zend_Http_Cookie::fromString($_SERVER['HTTP_COOKIE']); $client->setCookie($cookie); $response = $client->request();

That seems like a workable approach to me.

Posted by Shahar Evron (shahar) on 2008-06-19T01:34:18.000+0000

Yeah - that would work.

It might also make good sense to add an API to quickly do that - but honestly I think this is quite advanced usage so just to save a line of code I think it's not required.

Posted by Lukas Smith (lsmith) on 2008-06-19T01:46:28.000+0000

Also, while I am focusing on the encode part. The same situation also needs to be addressed for the decoding stage. The raw cookie needs to be available in the response.

I do not agree that its so advanced. This situation will come up whenever PHP sits as a proxy between a browser and a server, where the server originally set the cookie. In my case its relevant because of a mashup we are doing. The reason why few users get bitten by this issue is that indeed most servers support url encoding. So the scenario is not so obscure, only that it creates problems is. Now if we make this something the user has to do on his own, then you will have all sorts of users fall into this trap over and over again (even if you document this). But if there is a method "passBrowserCookieToServer()", then it will be clear for people that do write a proxy what method they should use to send the cookie to the server.

Thanks for looking into this!

Posted by Michael Rehbein (tech13) on 2010-01-13T07:13:33.000+0000

Attempts to fix both: http://framework.zend.com/issues/browse/ZF-1850 http://framework.zend.com/issues/browse/ZF-5407

Adds an option to Zend_Http_Client to override the old behavior

Adds an optional parameter to Zend_Http_Cookie to override the old behavior and effects how __toString behaves.

Posted by Michael Rehbein (tech13) on 2010-01-13T07:45:25.000+0000

Matching English documentation changes for previous patch

Posted by Shahar Evron (shahar) on 2010-02-11T06:27:25.000+0000

Thanks for the patch Michael - I've applied it in r. 21018, and it will be merged to the 1.10 release branch soon.

To not encode cookies, please set the 'encodecookies' config option to false.

Posted by Danila V. (dvershinin) on 2010-02-17T18:47:57.000+0000

Hi, not really fixed. I tested the patch: using http client, still getting encoded cookies even though the "encodecookies" option is set. When going through several requests and a cookie is added via CookieJar->addCookie:

public function addCookie($cookie, $ref_uri = null) { if (is_string($cookie)) { $cookie = Zend_Http_Cookie::fromString($cookie, $ref_uri); .... }

The fromString does not pass the preferred option onto config, thus fromString still gets $encodeValue = true

Posted by Matt Pinkston (mpinkston) on 2010-11-15T12:46:26.000+0000

The encodecookies preference is still not being passed along to cookies that are added to the cookiejar from the response (as pointed out by Danila V.)

Posted by Matt Pinkston (mpinkston) on 2010-11-15T12:49:24.000+0000

Here's a patch that passes along the encodecookies value to cookies added to the cookiejar through addCookiesFromResponse. My first patch, so I apologize if I missed a step.

http://framework.zend.com/issues/secure/…

Posted by Shahar Evron (shahar) on 2010-11-16T12:18:12.000+0000

The patch looks good, I have two questions:

  1. Have you signed a CLA?
  2. Do you have any unit tests that reproduce the issue you think should be committed along with this patch?

Posted by Matt Pinkston (mpinkston) on 2010-11-17T11:46:07.000+0000

I did sign a CLA (Feb of 2009).

I haven't created any unit tests for this issue. (though I will vouch that it works in my real-world scenario)

Posted by Shahar Evron (shahar) on 2010-11-24T03:53:26.000+0000

Patch applied, unit test added to verify resolution. Committed to trunk in r23442, merged into 1.11 branch in 23443.

Have you found an issue?

See the Overview section for more details.

Copyright

© 2006-2016 by Zend, a Rogue Wave Company. Made with by awesome contributors.

This website is built using zend-expressive and it runs on PHP 7.

Contacts