History | Log In     View a printable version of the current page.  
Issue Details (XML | Word | Printable)

Key: ZF-1498
Type: Bug Bug
Status: Open Open
Priority: Major Major
Assignee: Ralph Schindler
Reporter: Andrew L. Ayers
Votes: 4
Watchers: 4
Operations

If you were logged in you would be able to see more operations.
Google issue summary
Zend Framework

Zend_Db and LONGBLOBs

Created: 05/Jun/07 04:42 PM   Updated: 22/Jun/09 06:47 AM  Due: 23/Jan/09
Component/s: Zend_Db, Zend_Db_Adapter_Mysqli
Affects Version/s: 0.9.2
Fix Version/s: Next Mini Release

Time Tracking:
Original Estimate: 2 days
Original Estimate - 2 days
Remaining Estimate: 2 days
Remaining Estimate - 2 days
Time Spent: Not Specified
Remaining Estimate - 2 days

 Public Fields   Internal Project Management Fields   
Tags:
Participants: Andrew L. Ayers, Bill Karwin, Ralph Schindler, Sander Datema, Toni Wenzel, Wil Sinclair and Zachary Schneider
Fix Version Priority: Should Have


 Description  « Hide
As far as I can tell, this issue is related mainly to Zend_Db, and not Mysqli - however, since I am using Mysqli as my connection, I am including it as a point of reference.

The bug that I am encountering is as follows:

I have a MySQL table for BLOB storage. One of the fields is defined as a LONGBLOB. I can use Zend_Db to write data to this field, but when I go to read it, the data that is returned isn't the same as the data that is stored. I can use both Mysql and Mysqli to read the data, and it appears fine (I can also use the connection object via Zend_Db to access Mysqli directly, and that works OK as well). Reading this data using one of the external methods, and then performing a character-by-character comparison of the data seems to indicate that some form of translation is occurring. Whether this is a bug or by design (ie, there is a flag or parameter I need to set?) is unknown. What I do know is that if I choose to set the field data type to a smaller BLOB data type (ie, MEDIUMBLOB), the issue does not occur.

So, the "quickfix" is only use MEDIUMBLOB (or smaller) data type fields in a table, or use one of the "direct methods" of access if there is no issue with tying your application to a particular DB.

I should also note that I do not know if this issue affects LONGTEXT fields or not. I suspect it may, but it has not been tested for...

Sample testing code:

$id = 11;
// Note - I set the DB up elsewhere, but this code is pretty simple for the test
$table = new blobs();
$blob_data1 = $table->getAdapter()->fetchone("SELECT blob_data FROM blobs WHERE blob_id = $id");

print "ZENDDB BLOB_DATA LEN=".strlen($blob_data1);
print "<br>";
  
///////////////////////////////////////////////////////////////////////////////////////////////////////////////
  
$link = mysql_connect(DB_HOST, DB_USERNAME, DB_PASSWORD);

mysql_select_db(DB_DBNAME);
                
$result = mysql_query("SELECT blob_data FROM blobs WHERE blob_id = $id");
        
$blob_data2 = mysql_result($result, 0);
        
mysql_close($link);
  
print "MYSQL BLOB_DATA LEN=".strlen($blob_data2);
print "<br>";

///////////////////////////////////////////////////////////////////////////////////////////////////////////////
  
$link = mysqli_connect(DB_HOST, DB_USERNAME, DB_PASSWORD, DB_DBNAME);
  
$result = mysqli_query($link, "SELECT blob_data FROM blobs WHERE blob_id = $id");

$data = mysqli_fetch_assoc($result);
  
$blob_data3 = $data["blob_data"];
  
mysqli_close($link);
    
print "MYSQLI BLOB_DATA LEN=".strlen($blob_data3);
print "<br>";

///////////////////////////////////////////////////////////////////////////////////////////////////////////////
  
$result = $table->getAdapter()->getConnection()->query("SELECT blob_data FROM blobs WHERE blob_id = $id");
$data = mysqli_fetch_assoc($result);
$blob_data4 = $data["blob_data"];

print "ZENDDB BLOB_DATA4 LEN=".strlen($blob_data4);
print "<br>";
  
for ($i=0; $i < 5; $i++) {
  print "<hr>";
  print "POS=".$i;
  print "<br>";
  print "SUBSTR_ZF=".substr($blob_data1, $i, 1).", ".ord(substr($blob_data1, $i, 1));
  print "<br>";
  print "SUBSTR_MYSQL=".substr($blob_data2, $i, 1).", ".ord(substr($blob_data2, $i, 1));
  print "<br>";
  print "SUBSTR_MYSQLI=".substr($blob_data3, $i, 1).", ".ord(substr($blob_data3, $i, 1));
  print "<br>";
  print "SUBSTR_MYSQLI2=".substr($blob_data4, $i, 1).", ".ord(substr($blob_data4, $i, 1));
}

Sample output (LONGBLOB):

ZENDDB BLOB_DATA LEN=177669
MYSQL BLOB_DATA LEN=177669
MYSQLI BLOB_DATA LEN=177669
ZENDDB BLOB_DATA4 LEN=177669
--------------------------------------------------------------------------------
POS=0
SUBSTR_ZF=), 41
SUBSTR_MYSQL=ÿ, 255
SUBSTR_MYSQLI=ÿ, 255
SUBSTR_MYSQLI2=ÿ, 255
--------------------------------------------------------------------------------
POS=1
SUBSTR_ZF=, 0
SUBSTR_MYSQL=Ø, 216
SUBSTR_MYSQLI=Ø, 216
SUBSTR_MYSQLI2=Ø, 216
--------------------------------------------------------------------------------
POS=2
SUBSTR_ZF=, 0
SUBSTR_MYSQL=ÿ, 255
SUBSTR_MYSQLI=ÿ, 255
SUBSTR_MYSQLI2=ÿ, 255
--------------------------------------------------------------------------------
POS=3
SUBSTR_ZF=, 0
SUBSTR_MYSQL=à, 224
SUBSTR_MYSQLI=à, 224
SUBSTR_MYSQLI2=à, 224
--------------------------------------------------------------------------------
POS=4
SUBSTR_ZF=¸, 184
SUBSTR_MYSQL=, 0
SUBSTR_MYSQLI=, 0
SUBSTR_MYSQLI2=, 0

Sample output (MEDIUMBLOB):

ZENDDB BLOB_DATA LEN=177669
MYSQL BLOB_DATA LEN=177669
MYSQLI BLOB_DATA LEN=177669
ZENDDB BLOB_DATA4 LEN=177669
--------------------------------------------------------------------------------
POS=0
SUBSTR_ZF=ÿ, 255
SUBSTR_MYSQL=ÿ, 255
SUBSTR_MYSQLI=ÿ, 255
SUBSTR_MYSQLI2=ÿ, 255
--------------------------------------------------------------------------------
POS=1
SUBSTR_ZF=Ø, 216
SUBSTR_MYSQL=Ø, 216
SUBSTR_MYSQLI=Ø, 216
SUBSTR_MYSQLI2=Ø, 216
--------------------------------------------------------------------------------
POS=2
SUBSTR_ZF=ÿ, 255
SUBSTR_MYSQL=ÿ, 255
SUBSTR_MYSQLI=ÿ, 255
SUBSTR_MYSQLI2=ÿ, 255
--------------------------------------------------------------------------------
POS=3
SUBSTR_ZF=à, 224
SUBSTR_MYSQL=à, 224
SUBSTR_MYSQLI=à, 224
SUBSTR_MYSQLI2=à, 224
--------------------------------------------------------------------------------
POS=4
SUBSTR_ZF=, 0
SUBSTR_MYSQL=, 0
SUBSTR_MYSQLI=, 0
SUBSTR_MYSQLI2=, 0


 All   Comments   Work Log   Change History   FishEye   Crucible      Sort Order: Ascending order - Click to sort in descending order
Bill Karwin - 20/Jul/07 02:56 PM
I just talked to a user who confirms that this issue does affect LONGTEXT as well as LONGBLOB.

The workaround for now is to use MEDIUMTEXT or MEDIUMBLOB, but regardless I'll try to fix this as soon as I can.

Keep in mind that in MySQL, the MEDIUMTEXT/MEDIUMBLOB types hold up to 16MB of data. LONGTEXT/LONGBLOB hold up to 4GB of data. In most apps, the capacity of the MEDIUM types should be adequate!

Reference on MySQL text/blob datatypes:
http://dev.mysql.com/doc/refman/5.0/en/string-type-overview.html


Wil Sinclair - 21/Mar/08 05:05 PM
This issue should have been fixed for the 1.5 release.

Wil Sinclair - 25/Mar/08 08:43 PM
Please categorize/fix as needed.

Toni Wenzel - 13/Apr/08 07:48 AM
I can reproduce this problem (ZF 1.5.0). I've a longtext column (stackTrace). When I read the table I get a "out of memory" error.

<?php
error_reporting(E_ALL);

require_once 'Zend/Db.php';
require_once 'Zend/Config.php';

$dbParams = array(
'adapter' => 'mysqli',
'params' => array(
'host' => '10.1.1.110',
'username' => 'rpv2',
'password' => 'rpv2',
'dbname' => 'txdb'
) ,
);

$dbOptions = array(
Zend_Db::CASE_FOLDING => Zend_Db::CASE_UPPER
);

$dbParams['params']['options'] = $dbOptions;

try
{
$db = Zend_Db::factory(new Zend_Config($dbParams));
}
catch (Zend_Db_Adapter_Exception $e)
{
// Möglicherweise ein fehlgeschlagener login, oder die RDBMS läuft möglicherweise nicht
print_r(errorOutput('Config DB: Login failed or server is not available. Details:'.$e->getMessage()));
die();
}

$mandator = '1';
$level = '8';
$count = 100;
$top = 0;

//$sql = 'SELECT idErrorLog FROM ErrorLog WHERE mandatorId= ? AND errorLevel=?';

$sql = 'SELECT stackTrace FROM ErrorLog WHERE mandatorId= ? AND errorLevel=?';

try
{
$result = $db->fetchAll( $db->limit($sql, $count,$top), array( $mandator,$level) );
} catch(Exception $e)
{
echo $e->getMessage();
}
echo "<html><body><pre>";
var_dump($result);
echo "</pre></body></html>";

?>

SQL:
– MySQL Administrator dump 1.4

-- ------------------------------------------------------
– Server version 5.0.45-community

/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */;
/*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */;
/*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */;
/*!40101 SET NAMES utf8 */;

/*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */;
/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */;
/*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */;


-- Create schema txdb

CREATE DATABASE IF NOT EXISTS txdb;
USE txdb;


-- Definition of table `ErrorLog`

DROP TABLE IF EXISTS `ErrorLog`;
CREATE TABLE `ErrorLog` (
`idErrorLog` int(10) unsigned NOT NULL auto_increment,
`mandatorId` tinyint(3) unsigned NOT NULL,
`eventId` bigint(20) unsigned default NULL,
`userId` bigint(20) unsigned default NULL,
`logDate` datetime NOT NULL,
`serverName` varchar(128) collate latin1_german1_ci NOT NULL,
`errorLevel` tinyint(3) unsigned NOT NULL,
`errorCode` int(10) unsigned default NULL,
`extendedErrorCode` int(10) unsigned default NULL,
`className` varchar(255) collate latin1_german1_ci NOT NULL,
`classVersionInfo` varchar(128) collate latin1_german1_ci default NULL,
`methodName` varchar(255) collate latin1_german1_ci default NULL,
`messageText` varchar(255) collate latin1_german1_ci default NULL,
`additionalInfo` varchar(1024) collate latin1_german1_ci default NULL,
`stackTrace` longtext collate latin1_german1_ci,
PRIMARY KEY (`idErrorLog`,`mandatorId`),
KEY `ErrorLog_FKIndex1` (`mandatorId`),
KEY `ErrorLog_FKIndex2` (`eventId`),
KEY `ErrorLog_FKIndex3` (`userId`)
) ENGINE=InnoDB AUTO_INCREMENT=1163 DEFAULT CHARSET=latin1 COLLATE=latin1_german1_ci;


-- Dumping data for table `ErrorLog`

/*!40000 ALTER TABLE `ErrorLog` DISABLE KEYS */;
INSERT INTO `ErrorLog` (`idErrorLog`,`mandatorId`,`eventId`,`userId`,`logDate`,`serverName`,`errorLevel`,`errorCode`,`extendedErrorCode`,`className`,`classVersionInfo`,`methodName`,`messageText`,`additionalInfo`,`stackTrace`) VALUES
(1159,1,NULL,1,'2008-04-13 11:23:28','10.1.1.105',8,NULL,NULL,'/var/www/core/library/Exsportance/Db/TableAdapter.php','1.0.0.0','logLastStatement','DELETE FROM `ErrorLog` WHERE (mandatorId = \'1\' AND errorLevel=\'8\')\r\nQueryParams: ','QueryTime: 0.0303328037262',' at 4 /var/www/core/library/Exsportance/Db/TableAdapter.php (line 296) -> logLastStatement()\r\n at 5 /var/www/core/dataaccess/System/ErrorLog.php (line 44) -> delete(\'ErrorLog\',\'mandatorId = \'1\' AND errorLevel=\'8\'\')\r\n at 6 /var/www/core/services/Logging.php (line 176) -> deleteByLogLevel(\'1\',\'8\')\r\n at 7 /var/www/admin/services/WebService.php (line 1142) -> truncateErrorLogDb(\'1\',\'8\')\r\n at 8 (line ) -> truncateErrorLogDb(\'1\',\'8\')\r\n at 9 /var/www/admin/services/WebService.php (line 34) -> call_user_func_array(\'truncateErrorLogDb\',Array(\'1\',\'8\'))\r\n'),
(1160,1,NULL,1,'2008-04-13 11:23:28','10.1.1.105',8,NULL,NULL,'/var/www/core/library/Exsportance/Db/TableAdapter.php','1.0.0.0','logLastStatement','SELECT Count(idErrorLog) FROM ErrorLog WHERE mandatorId= ? AND errorLevel=?\r\nQueryParams: \'1\',\'8\'','QueryTime: 0.00922608375549',' at 4 /var/www/core/library/Exsportance/Db/TableAdapter.php (line 152) -> logLastStatement()\r\n at 5 /var/www/core/dataaccess/System/ErrorLog.php (line 21) -> fetchOne(\'SELECT Count(idErrorLog) FROM ErrorLog WHERE mandatorId= ? AND errorLevel=?\',Array(\'1\',\'8\'))\r\n at 6 /var/www/core/services/Logging.php (line 169) -> getCountByLogLevel(\'1\',\'8\')\r\n at 7 /var/www/admin/services/WebService.php (line 1126) -> getErrorLogDbCount(\'1\',\'8\')\r\n at 8 (line ) -> getErrorLogDbCount(\'1\',\'8\')\r\n at 9 /var/www/admin/services/WebService.php (line 34) -> call_user_func_array(\'getErrorLogDbCount\',Array(\'1\',\'8\'))\r\n'),
(1161,1,NULL,1,'2008-04-13 11:23:29','10.1.1.105',8,NULL,NULL,'/var/www/core/library/Exsportance/Db/TableAdapter.php','1.0.0.0','logLastStatement','SELECT idErrorLog,mandatorId,eventId,userId,logDate,serverName,errorLevel,errorCode,extendedErrorCode,className,classVersionInfo,messageText,additionalInfo FROM ErrorLog WHERE mandatorId= ? AND errorLevel=? LIMIT 100\r\nQueryParams: \'1\',\'8\'','QueryTime: 0.00233006477356',' at 4 /var/www/core/library/Exsportance/Db/TableAdapter.php (line 48) -> logLastStatement()\r\n at 5 /var/www/core/dataaccess/System/ErrorLog.php (line 16) -> fetchAll(\'SELECT idErrorLog,mandatorId,eventId,userId,logDate,serverName,errorLevel,errorCode,extendedErrorCode,className,classVersionInfo,messageText,additionalInfo FROM ErrorLog WHERE mandatorId= ? AND errorLevel=? LIMIT 100\',Array(\'1\',\'8\'))\r\n at 6 /var/www/core/services/Logging.php (line 162) -> getByLogLevel(\'1\',\'8\',\'0\',\'100\')\r\n at 7 /var/www/admin/services/WebService.php (line 1109) -> getErrorLogDb(\'1\',\'8\',\'0\',\'100\')\r\n at 8 (line ) -> getErrorLogDb(\'1\',\'8\',\'0\',\'100\')\r\n at 9 /var/www/admin/services/WebService.php (line 34) -> call_user_func_array(\'getErrorLogDb\',Array(\'1\',\'8\',\'0\',\'100\'))\r\n'),
(1162,1,NULL,1,'2008-04-13 11:24:07','10.1.1.105',8,NULL,NULL,'/var/www/core/library/Exsportance/Db/TableAdapter.php','1.0.0.0','logLastStatement','SELECT Count(idErrorLog) FROM ErrorLog WHERE mandatorId= ? AND errorLevel=?\r\nQueryParams: \'1\',\'8\'','QueryTime: 0.00160193443298',' at 4 /var/www/core/library/Exsportance/Db/TableAdapter.php (line 152) -> logLastStatement()\r\n at 5 /var/www/core/dataaccess/System/ErrorLog.php (line 21) -> fetchOne(\'SELECT Count(idErrorLog) FROM ErrorLog WHERE mandatorId= ? AND errorLevel=?\',Array(\'1\',\'8\'))\r\n at 6 /var/www/core/services/Logging.php (line 169) -> getCountByLogLevel(\'1\',\'8\')\r\n at 7 /var/www/admin/services/WebService.php (line 1126) -> getErrorLogDbCount(\'1\',\'8\')\r\n at 8 (line ) -> getErrorLogDbCount(\'1\',\'8\')\r\n at 9 /var/www/admin/services/WebService.php (line 34) -> call_user_func_array(\'getErrorLogDbCount\',Array(\'1\',\'8\'))\r\n');
/*!40000 ALTER TABLE `ErrorLog` ENABLE KEYS */;

/*!40101 SET SQL_MODE=@OLD_SQL_MODE */;
/*!40014 SET FOREIGN_KEY_CHECKS=@OLD_FOREIGN_KEY_CHECKS */;
/*!40014 SET UNIQUE_CHECKS=@OLD_UNIQUE_CHECKS */;
/*!40101 SET CHARACTER_SET_CLIENT=@OLD_CHARACTER_SET_CLIENT */;
/*!40101 SET CHARACTER_SET_RESULTS=@OLD_CHARACTER_SET_RESULTS */;
/*!40101 SET COLLATION_CONNECTION=@OLD_COLLATION_CONNECTION */;
/*!40101 SET CHARACTER_SET_CLIENT=@OLD_CHARACTER_SET_CLIENT */;


Wil Sinclair - 18/Apr/08 01:11 PM
This doesn't appear to have been fixed in 1.5.0. Please update if this is not correct.

Sander Datema - 08/May/08 08:48 PM
And 1.5.1 also seems to have this issue. Although I don't get to see an error, I get a blank page. But when reducing the amount of data in the long fields, the query is run fine.

A quick fix is to not select the big fields when doing a quick lookup. But that's not a fix when you need that specific field or when you use e.g. the findDependentRowSet method, as that one takes all fields.


Wil Sinclair - 06/Jan/09 02:26 PM
This issue has gone unaddressed for too long. I'm re-assigning to Ralph for re-evaluation and categorization.

Ralph Schindler - 09/Jan/09 01:51 PM
I will evaluate this within 2 weeks

Zachary Schneider - 14/May/09 08:14 PM
This is pretty old, I know the problem exists in the 1.5 series has it been fixed in 1.8?