ZF-11383: S3 getObjectStream() gets stuck in an eternal loop in Zend_Http_Client_Adapter_Socket::read() when retrieving an S3 object within a double-level subdir

Description

Put an image file of size ~1.8 MB on S3 in a double-level subdir (i.e. ///file.png

Test if available via Zend's S3 isObjectAvailable() method (it should return true)

Now getObjectStream() for the S3 object

After drilling down a bit, Zend_Http_Client_Adapter_Socket::read() will sit spinning forever, when the original S3 put took only a few moments

Specifically, it gets stuck here:


for ($read_to = $current_pos + $contentLength;
                 $read_to > $current_pos;
                 $current_pos = ftell($this->socket)) {
                 if($this->out_stream) {
                     if(@stream_copy_to_stream($this->socket, $this->out_stream, $read_to - $current_pos) == 0) {
                         
                         $this->_checkSocketReadTimeout();
                          
                          break;
                     }
                                          AG_Log::logThis("5");
                     
                 } else {
                    $chunk = @fread($this->socket, $read_to - $current_pos);
                    if ($chunk === false || strlen($chunk) === 0) {
                        $this->_checkSocketReadTimeout();
                        break;
                    }

                    $response .= $chunk;
                }

                // Break if the connection ended prematurely
                if (feof($this->socket)) break;
            }

You'll see that $current_pos never increases, and thus the for loop becomes an eternal loop.

I've tested using the S3 streamWrapper functionality:


$sourceHandle = fopen("s3:://$s3object","r");
        $targetHandle = fopen($localImageToFormat,"w");
        $length = 102400;
        $i=1;
        while(!feof($sourceHandle))
        {   
            $result = fwrite($targetHandle, fread($sourceHandle, $length));
            $i++;
        }
        
        fclose($sourceHandle);
        fclose($targetHandle);

The feof is never received. I'm guessing the problem is that S3 doesn't send a feof.

Please fix this ASAP.

Comments

Hi,

Are you sure that the 2nd problem you are describing (the S3 stream wrapper never reaches EOF) is really related to a bug in Zend_Http_Client?

It's not true that $current_pos never increases - as you can see there's a call ``` in the iterator of the for loop. I've tested this with a file on S3 using the Socket adapter and write-to-stream interface, with a 5mb file on a 2nd level directory, this works very well.

Can you try to isolate the problem from the s3 stream wrapper? E.g. provide a piece of code (preferrably with a URL to the exact public file on S3 you are testing with) that gets stuck in an infinite loop, but not using the s3 wrapper - only Zend_Http_Client.

If the problem is not in Zend_Http_Client, this bug needs to be reassigned.

Philip, is there any follow up on this?

I also see this issue in one of my environments, but not others. I'm suspecting it may be a system configuration setting that is causing it to happen.

We just gave up, as the only way to offer up an example of this happening was to post a file to our S3 and then provide our personal login details...