ZF-6477: Zend_Service_Amazon_S3 : Does not work for buckets located in EU

Description

When you try to put an object into a bucket located in EU, you get an error from amazon, that this bucket must be accessed using the specific endpoint (bucketname.s3.amazonaws.com). I have changed the function _makeRequest as follows. It seems to work so far with these changes. Haven't tested intensively.


    /**
     * Make a request to Amazon S3
     *
     * TODO: support bucket.s3.amazon.com style
     *
     * @param  string $method
     * @param  string $path
     * @param  array  $params
     * @param  array  $headers
     * @param  string $data
     * @return Zend_Http_Response
     */
    public function _makeRequest($method, $path='', $params=null, $headers=array(), $data=null)
    {
        $retry_count = 0;

        if (!is_array($headers)) {
            $headers = array($headers);
        }

        $headers['Date'] = gmdate(DATE_RFC1123, time());

        $parts = explode('/', $path);
        self::addSignature($method, $path, $headers);

        $client = self::getHttpClient();

        $client->resetParameters();
        // Work around buglet in HTTP client - it doesn't clean headers
        // Remove when ZHC is fixed
        $client->setHeaders(array('Content-MD5' => null, 'Expect' => null, 'Range' => null));
        
        $endpoint = self::S3_ENDPOINT;
        if (count($parts) > 1){
            $endpoint = str_replace('http://', 'http://' . $parts[0] . '.', $endpoint);
            array_shift($parts);
            $client->setUri($endpoint.'/'.implode('/', $parts));
        } else {
            $client->setUri(self::S3_ENDPOINT.'/'.$path);
        }
        $client->setHeaders($headers);

        if (is_array($params)) {
            foreach ($params as $name=>$value) {
                $client->setParameterGet($name, $value);
            }
         }

         if (($method == 'PUT') && ($data !== null)) {
             if (!isset($headers['Content-type'])) {
                 $headers['Content-type'] = self::getMimeType($path);
             }
             $client->setRawData($data, $headers['Content-type']);
         }

         do {
            $retry = false;

            $response = $client->request($method);
            $response_code = $response->getStatus();

            // Some 5xx errors are expected, so retry automatically
            if ($response_code >= 500 && $response_code < 600 && $retry_count <= 5) {
                $retry = true;
                $retry_count++;
                sleep($retry_count / 4 * $retry_count);
            }
            else if ($response_code == 307) {
                // Need to redirect, new S3 endpoint given
                // This should never happen as Zend_Http_Client will redirect automatically
            }
            else if ($response_code == 100) {
                //echo 'OK to Continue';
            }
        } while ($retry);
        return $response;
    }


Comments

Assigning to Jon

Could your attach a patch file so i apply this and test it?

Thanks,

Here is a patchfile with the changes.

IMHO it would be a good idea to handle Amazon S3 API permanent redirects, which seem to use the HTTP status 301 and do not set a location header (and are therefore not handled by Zend_HTTP_Client).

Kristof,

I didn't write this component to i'm not too entierly sure what you are talking about. Can you please provide the docs page that explains that or submit a patch?

Hi Jon

You can find more information about it at http://docs.amazonwebservices.com/AmazonS3/latest/…, under "Permanent Request Redirection". After thinking a bit more about this, it might be better to use domain-style requests after all, as Ivo suggests. That works in all cases (EU and US-located buckets).

This has been resolved and checked-in to svn.

Hello,

are you shure that is suitable for all operations that can be done? When I try to DELETE an object from a bucket (in the EU) then I get an positive response from S3. But my object is still there. I guess the problem is, that Amazon's API also returns this positive response when you try to delete an object that does not even exist. I did not test for other operations yet.