ZF-8869: Memory leak in xml-rpc client.

Description

When using "$XmlRpcClient->call($method, $args);" in a loop structure, after a few rounds, I got Php's memory exhaustion message.

I traced the leak to "Zend\XmlRpc\Client.php" line 351 where the code is;


$params[$key] = Zend_XmlRpc_Value::getXmlRpcValue($param, $type);

I did the trace by placing a "memory_get_usage();" before and after the line and see the results (there is probably a better way).

My app is running on a Windows XP / SP3, using php 5.2.12 and ZF 1.9.7.

This is my first time reporting a bug, if I omitted some info or not reporting the right way please let me know and will make the necessary corrections.

Regards, Gonzalo.

Comments

Thanks for the bug report. Could you provide me an example of the parameters you are passing to Zend_XmlRpc_Client::call()?

Hi Lars,

Will attach all the info I think relevant.

I've just did run the loop for about 30 secs, that did perform four xml-rpc client calls, each with $args of around 400K each.

I am attaching all the info I could get from it.

i) graph of process memory consumption.

ii) zip file containing 12 files, 3 for each call, eg; # 1st_args.txt is print_r() of the $args passed in the call. # 1st_request.txt is the captured http request. # 2nd_response.txt is the captured http response.


On the client's side the line making the call goes like;


$result = $XmlRpcClient->call($method, $args);

where $method is a string and $args is "array($queries)" being $queries another array.

On the server's side the head of the method handling the xml-rpc call is;


/*
* @param struct $queries
* @return struct
*/
public function actondbms($queries) {
...
}

Regards, Gonzalo.

See comment for files content description...

I believe this issue is related to the addition of an XML generator based on XMLWriter. If the XMLWriter extension is enabled, Zend_XmlRpc_Value defaults to using it.

Following an update of my local version of Zend Framework, one of my scripts went from comfortably fitting in 512M to needing 2G. I added

Zend_XmlRpc_Value::setGenerator(new Zend_XmlRpc_Generator_DomDocument());

before my call to {{new Zend_XmlRpc_Server()}} and the memory usage is back down to normal levels.

OK... I'll revise my statements. It is now running out of memory only some of the time.

Hi guys,

we can confirm this problem. We have a cron-script which runs in an infinity-loop and uses also the Zend_XmlRpc_Client. The memory usage grows minute for minute.

It really would be nice if we had an solution or workaround for this issue.

Succesfully reproduced it on Zend Server 5.1.0 (php-5.3.5-nts-Win32-VC9-x86). The memory usage increases slightly in my Simulation, but with the fix it is constant and much faster.

Manual for XMLWriter::flush()

I am not sure what the issue is, both the DomDocument and XmlWriter generators operate the same way. They are capturing all values passed through Zend_XmlRpc_Value::getXmlRpcValue() and keeping them in-memory (as I think I expect them to as you are using a static generator). If you remove the generator from the equation, then memory usage is constant.


<?php
set_include_path('path/to/ZFStandardTrunk/library');

//require_once('Zend/XmlRpc/Generator/XmlWriter.php');
require_once('Zend/XmlRpc/Generator/DomDocument.php');
require_once('Zend/XmlRpc/Value.php');

// be sure to use the leaky generator
//$generator = new Zend_XmlRpc_Generator_XmlWriter();
$generator = new Zend_XmlRpc_Generator_DomDocument();
Zend_XmlRpc_Value::setGenerator($generator);

// test a thousand times
for( $i = 0; $i < 1000; $i++ ){
  $targetValue = str_repeat('*', 1000); // 1000 stars
  $xmlRpcValue = Zend_XmlRpc_Value::getXmlRpcValue($targetValue, Zend_XmlRpc_Value::XMLRPC_TYPE_STRING);
  $output = $xmlRpcValue->saveXml();
  
  // report sometimes
  if( $i%100 === 0 ){
    $mem = floor(memory_get_usage()/1024);
    echo "KB $mem\n";
  }
  
  // writing output to disk
  file_put_contents(__DIR__ . '/tmp/' . str_pad($i, 4, '0', STR_PAD_LEFT) . '.output', $output);
  
  // clean
  unset($output, $targetValue, $xmlRpcValue);
}

Try that with a variety of different runs and examine the output files.

I am inclined to close as not an issue.

You're right. I was mislead by the description and only tested Zend_XmlRpc_Value.

I've read through the module now and found out that only $generator->saveXml() gets called. Also $xmlRpcValue->saveXml() is never called. If it were called the user would have to call $generator->flush() beforehand, else it would return xml of other values that we're generated before.

However, that is not the problem, Request/Response and Client flush the generator correctly. I reproduced Zend_XmlRpc_Client::call() and now i'm as sure as you that there is no leak.

I'm voting for a close as not an issue, i also have no issue with this in production, i just wanted to bug hunt :)

It's possible that this is the same issue that was fixed in ZF-9504. I think you should look over that ticket and also ZF-8457 before calling this one closed (duplicate).

Patched in r24402, ready to deploy

Had some additional test runs.

Can't see why it was set to false in the first place (r19561)

I found r24402 fix is reverted at r24480 http://framework.zend.com/code/revision.php/…

So, is this Memory leak issue is resolved?