<ac:macro ac:name="unmigrated-inline-wiki-markup"><ac:plain-text-body><![CDATA[
<ac:macro ac:name="unmigrated-inline-wiki-markup"><ac:plain-text-body><![CDATA[
This proposal aims to provide a more flexible, consistent and lean strategy for modifying queries derived around the Zend_Db_Table component. This includes the fetch/find methods from a Zend_Db_Table object as well as the relationship-based queries performed from a Zend_Db_Table_Row. Currently the signature for fetchAll follows a ($where, $order, $limit, $offset) approach (similarly for fetchRow) but this does not provide enough flexibility for some more complex requirements (e.g. grouping). Some current methods completely ignore any passed parameters (the magic method relationship queries). It is proposed instead that all fetch/find queries from a Zend_Db_Table/Row provide the capability of supplying a Zend_Db_Select object to modify the query.
Zend Framework: Zend_Db_Table Query Enhancements Component Proposal
Proposed Component Name
Zend_Db_Table Query Enhancements
Developer Notes
http://framework.zend.com/wiki/display/ZFDEV/Zend_Db_Table Query Enhancements
Proposers
Simon Mundy
Darby Felton, Zend liaison
Revision
0.1 - 1 August 2007: Initial proposal. (wiki revision: 8)
Table of Contents
1. Overview
2. References
3. Component Requirements, Constraints, and Acceptance Criteria
4. Dependencies on Other Framework Components
- Zend_Db_Select
- Zend_Db_Table_Abstract
- Zend_Db_Table_Row_Abstract
- Zend_Db_Table_Rowset_Abstract
5. Theory of Operation
The components will continue to operate when called using the method signature as per release 1.0.1 of the Zend Framework. However the new select object usage will become the preferred method and by release 1.2+ of Zend Framework the deprecated methods will emit an E_NOTICE.
A user-supplied Zend_Db_Select object will not allow modification to the 'from' or any 'join' methods - this is to prevent any errors or conflicts with the Zend_Db_Table schema.
6. Milestones / Tasks
- Milestone 1: design notes will be published here
- Milestone 2: [DONE] Working prototype @ http://framework.zend.com/svn/laboratory/library/Zend/Db/Table
- Milestone 3: Unit tests created.
- Milestone 4: Update existing documentation.
7. Class Index
- Zend_Db_Select
- Zend_Db_Table_Abstract
- Zend_Db_Table_Row_Abstract
- Zend_Db_Table_Rowset_Abstract
8. Use Cases
| UC-01 |
|---|
Simple table query
| UC-02 |
|---|
Retrieving related/dependent rows
9. Class Skeletons
The major changes to the class(es) will be the addition of a method to allow retrieval of the Zend_Db_Select object. From a Zend_Db_Table_Row object, for example, it is possible to retrieve via $row->getTable()>getAdapter()>select() but for the sake of readability and leaner code these convenience methods will provide a helpful shortcut.
The addition of the readOnly flag is to prevent partial Zend_Db_Table_Row objects being saved if they contain fields outside of the table schema (including functions, aliases, etc.)
The other subtle change is to modify the signature of the fetch/find methods to detect a select object. If the deprecated method is used, then these parameters will be added to a new select object and passed to the query. In a later release of the framework, using the deprecated signature will emit an E_NOTICE.
]]></ac:plain-text-body></ac:macro>]]></ac:plain-text-body></ac:macro>
19 Comments
comments.show.hideAug 03, 2007
Bill Karwin
<p>Is there any way we can support specifying columns, so that the table and row methods don't always return '<code>*</code>' all columns? This has been requested in e.g. ZF-694.</p>
<p>The Table and Row methods should remove any FROM or JOIN clauses, in case the user tries to add one to the select object. It should enforce that the method queries only a single table.</p>
<p>But the Zend_Db_Select object has a <code>getParts()</code> method that could be used to retrieve any columns specified. So one could specify columns in the following way:</p>
<ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
$rows = $table->fetchAll($table->select()->from('foo', array('col1', 'col2')));
]]></ac:plain-text-body></ac:macro>
<p>Inside the <code>fetchAll()</code> code, it can retrieve parts of the query:</p>
<ul>
<li><code>$select->getParts('from')</code> returns an array keyed by correlation names, each element of which is an array with keys 'joinType', 'schema', 'tableName', 'joinCondition'.</li>
<li><code>$select->getParts('columns')</code> returns an array, each element of which is an array of correlation name, column name, and alias.</li>
</ul>
<p>So what I would suggest is that <code>fetchAll()</code> examine each of these parts. For each column, if its correlation name maps to a table matching the Table class's <code>$this->_name</code> member, then include that column. If no legitimate columns have been found after looping through them, assume '*'. That should eliminate references to the wrong tables.</p>
Aug 14, 2007
Nathan Bell
<p>Can I suggest that we also modify Zend_Db_Select::from() to allow a Zend_Db_Table object as the first parameter?</p>
<p>So that (among other things) Bill's example:</p>
<ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
$rows = $table->fetchAll($table->select()->from('foo', array('col1', 'col2')));
]]></ac:plain-text-body></ac:macro>
<p>... could be instead written as:</p>
<ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
$rows = $table->fetchAll($table->select()->from($table, array('col1', 'col2')));
]]></ac:plain-text-body></ac:macro>
Oct 24, 2007
Adrian Hope-Bailie
<p>Perhaps the 'from' part could also allow for Zend_Db_Statement objects to allow for things like:</p>
<ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
FROM table
SELECT COUNT
]]></ac:plain-text-body></ac:macro>
<p>I would also suggest that rather than forcing older method signatures to raise a warning that this functionality be written to a new method.</p>
<p>By definition "fetchAll" should fetch all columns.</p>
<p>Why not have a Zend_Db_Table_Abstract::fetch(Zend_Db_Select $stmt) method.</p>
<p>'fetchAll' would then become a convenience method which removes all 'from' parts from the Zend_Db_Select object and calls 'fetch'. However it would also take the existing parameters (when, order, count, offset).</p>
Oct 24, 2007
Simon Mundy
<p>Hi Adrian</p>
<p>With allowing a Zend_Db_Select object to modify the query we also get a lot of flexibility with not only the order/limits but also to the query itself.</p>
<p>If you don't supply a 'from' part to the query it automatically assumes a '*' for the table. But you can specify you only wish a subset of columns to be returned:-</p>
<ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
$partial = $MyTable->fetchAll($MyTable->select()->from('mytable', array('id', 'name', 'created')));
]]></ac:plain-text-body></ac:macro>
<p>Taking this a step further we also recognised the usefulness of allowing expressions to be included in the query. You can specify expressions in the $select object - however the rowset that is returned is 'read only' and attempts to write back any of the data throws an exception.</p>
<ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
$partial = $MyTable->fetchAll($MyTable->select()
>from('mytable', array('SUM(logins)', 'name', 'created'))>group('id'));]]></ac:plain-text-body></ac:macro>
<p>You can also specify criteria on JOINed tables as long as the returned fields are all derived from the primary table<br />
E.g:-</p>
<ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
// This is valid as no columns are specified in JOIN
$partial = $MyTable->fetchAll($MyTable->select()->from('mytable', array('name', 'created'))
->join('myjoin', 'myjoin.key = mytable.lookup')
->where('myjoin.status = ?', 'foo'));
// This will throw an exception
$partial = $MyTable->fetchAll($MyTable->select()->from('mytable', array('name', 'created'))
->join('myjoin', 'myjoin.key = mytable.lookup', 'otherfield')
->where('myjoin.status = ?', 'foo'));
]]></ac:plain-text-body></ac:macro>
<p>Hope this clarifies the usage - I'm not sure of the benefits of an ambiguous 'fetch' method for the Zend_Db_Table class.</p>
Oct 25, 2007
Adrian Hope-Bailie
<p>Hi Simon</p>
<p>Don't get me wrong, I think this is the best proposal I have seen yet. Really opens up the possibilities for easy AND flexible data queries.</p>
<p>My concern, as stated in the second part of my last comment, is that fetchAll was written primarily as a convenience function and by definition should "fetch all".</p>
<p>Rather than replacing fetchAll why not do something like:</p>
<ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
public function fetchAll($where = null, $order = null, $count = null, $offset = null)
{
return $this->fetch($this->select()->where($where)
->order($order)
->limit($count, $offset));
}
public function fetch(Zend_Db_Select $stmt){
...
]]></ac:plain-text-body></ac:macro>
<p>Personally I see no reason why we should not have a number of other convenience functions on the Zend_Db_Table_Abstract class.</p>
<p>I have extended the class with the following functions, all of which could be reproduced much easier with your adapted classes.</p>
<p>At the end of the day the Framework is intended to not only be flexible but to facilitate easy and FAST coding, to me that means convenience functions are a good thing... have I been mislead?</p>
<ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
/**
*/
public function findRow($data)
{
if ($data instanceof $this->_rowClass)
return $this->find($data)->current();
}
/**
*
*/
public function fetchColumn($column, $where = null)
{
$rows = $this->fetchAll($where);
$returnArray = array();
foreach ($rows as $row)
return $returnArray;
}
/**
*
*/
public function fetchCount($where = null)
{
// selection tool
$select = $this->_db->select();
// the FROM clause
'), $this->_schema);
$select->from($this->_name, new Zend_Db_Expr('COUNT
// the WHERE clause
$where = (array) $where;
foreach ($where as $key => $val) {
// is $key an int?
if (is_int($key))
else
}
// return the results
$stmt = $this->_db->query($select);
$data = $stmt->fetchColumn();
return $data;
}
/**
*
*
*/
public function fetchAllAndCount($where = null, $order = null, &$count = null, $offset = null)
/**
*
*
*/
public function fetchAllGroupBy($where = null, $order = null, $count = null, $offset = null, $group = null)
{
// selection tool
$select = $this->_db->select();
// the FROM clause
$select->from($this->_name, $this->_cols, $this->_schema);
// the WHERE clause
$where = (array) $where;
foreach ($where as $key => $val) {
// is $key an int?
if (is_int($key))
else
}
if(!empty($group))
// the ORDER clause
if (!is_array($order))
foreach ($order as $val)
// the LIMIT clause
$select->limit($count, $offset);
// return the results
$stmt = $this->_db->query($select);
$data = $stmt->fetchAll(Zend_Db::FETCH_ASSOC);
$rowset = array(
'table' => $this,
'data' => $data,
'rowClass' => $this->_rowClass,
'stored' => true
);
Zend_Loader::loadClass($this->_rowsetClass);
return new $this->_rowsetClass($rowset);
}
/**
*
*
*/
public function fetchCountGroupBy($alias, $where = null, $group = null)
{
// selection tool
$select = $this->_db->select();
// the FROM clause
AS ' . $alias));
$cols = (array) $group;
array_unshift($cols, new Zend_Db_Expr('COUNT
$select->from($this->_name, $cols, $this->_schema);
// the WHERE clause
$where = (array) $where;
foreach ($where as $key => $val) {
// is $key an int?
if (is_int($key))
else
}
if(!empty($group))
// return the results
$stmt = $this->_db->query($select);
$data = $stmt->fetchAll(Zend_Db::FETCH_ASSOC);
$rowset = array(
'table' => $this,
'data' => $data,
'rowClass' => $this->_rowClass,
'stored' => true
);
Zend_Loader::loadClass($this->_rowsetClass);
return new $this->_rowsetClass($rowset);
}
]]></ac:plain-text-body></ac:macro>
Oct 25, 2007
Simon Mundy
<p>fetchAll represents all rows, not all columns - it mirrors the call in your database adapter. It's named to distinguish it readily from fetchRow.</p>
<p>My own thoughts on convenience methods are they're only valuable if they're <em>really</em> convenient. And for the methods above I'm sure they are to you and the context in which you develop. But maybe not for me or for others - it's a personal choice and isn't right or wrong, per se.</p>
<p>Feel free to use these in your own base class that extends Zend_Db_Table - this component is really only meant as a bare-bones structure for you to build onto.</p>
Oct 19, 2007
Bryce Lohr
<p>This seems like a really elegant way to add a lot of flexibility. I hope it gets approved! I also would welcome more of this kind of elegance throughout the framework. <ac:emoticon ac:name="smile" /></p>
Oct 23, 2007
Adrian Hope-Bailie
<p>Excellent proposal. I would like to suggest that the Many to Many relationship query be looked at too.<br />
Currently only intersection tables with 2 columns are supported.<br />
It would be preferable if a where clause could be included which is executed against the intersection table so that these can contain other columns.</p>
<p>Currently only the following type of query is possible:</p>
<ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
SELECT `m`.* FROM `user_gadgets` AS `i`
INNER JOIN `gadgets` AS `m` ON `i`.`gadget_id` = `m`.`id`
WHERE (`i`.`user_id` = '1')
]]></ac:plain-text-body></ac:macro>
<p>by executing something like</p>
<ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
$user->findGadgetByUserGadgets();
]]></ac:plain-text-body></ac:macro>
<p>It would be far more useful if one could execute a query like:</p>
<ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
SELECT `m`.* FROM `user_gadgets` AS `i`
INNER JOIN `gadgets` AS `m` ON `i`.`gadget_id` = `m`.`id`
WHERE (`i`.`user_id` = '1') AND (`i`.`status_id` = 'confirmed')
ORDER BY `m`.`created` DESC LIMIT 10 OFFSET 3
]]></ac:plain-text-body></ac:macro>
<p>by executing something more like</p>
<ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
$user->findGadgetByUserGadgets($user->select()->where('i.status_id = ?', 'confirmed')
->order('m.created DESC')
->limit(10, 3));
]]></ac:plain-text-body></ac:macro>
Oct 23, 2007
Simon Mundy
<p>That works already with the new components - download the laboratory files and give them a whirl! <ac:emoticon ac:name="smile" /></p>
Oct 23, 2007
Adrian Hope-Bailie
<p>Good things, will do!</p>
<p>Any idea how close this is to being incorporated into the next release?</p>
Oct 25, 2007
Adrian Hope-Bailie
<p>Hi Simon,</p>
<p>One more issue. Sorry.</p>
<p>I have looked through your code and have a few suggestions.</p>
<p>You have changed the output of Zend_Db_Table_Abstract::_fetch.<br />
This, to me, is a MAJOR issue. Any extensions of Zend_Db_Table_Abstract that use this function will break.</p>
<p>My suggestion would be to remove your "readonly" checking from this function and do it before calling _fetch.</p>
<p>This takes me back to my proposal of a generic "fetch" function which only takes a Zend_Db_Table_Select as a parameter.</p>
<p>This seperates relative functionality more logically between functions:</p>
<p>Top level<br />
fetchAll, fetchRow etc take simple parameters ($where, $order etc).<br />
The logic of building a Zend_Db_Table_Select object from the parameters is done within these functions. They would then call "fetch" and return relevant results.</p>
<p>Second Level<br />
fetch. Takes only a Zend_Db_Table_Select object.<br />
returns Zend_Db_Table_Rowset_Abstract</p>
<p>Finally<br />
_fetch. Takes either simple params or select object to maintain BC.<br />
returns rows as array</p>
<p>Perhaps a completely different way of approaching this problem would be to do necessary checks when calling Zend_Db_Table_Select::from. That way an internal flag on the select object could be set for readonly results and an error could be raised if the columns are not from the correct table.</p>
<p>Since the Zend_Db_Table_Select object is instantiated via a row, rowset or table it can record, when it is created, what it's parent table is. Following that any calls to "from" are checked against the parent table and the internal "readonly" flag is set or an error thrown if necessary.</p>
<p>When calling a function such as fetchAll and passing in a select object the function sets the readOnly flag on the Rowset or Row it creates based on the flag from the supplied Select object.</p>
<p>There is no way for the user to publicly set this value.</p>
<p>I have made some changes to your code. I will send them over for you to have a look at.</p>
<p>I would love this to go into the core soon as I think it is really taking the framework in the right direction.</p>
Oct 25, 2007
Simon Mundy
<p>To me there's no benefit for a 'fetch' method as it's more ambiguous than fetchAll and really essentially performs the same function anyway. And putting the $where, $order, $limit, $offset back in place means you would have 2 methods with limited functionality for the sake of BC. The way I'd proposed this was to keep BC AND allow a degree of flexibility.</p>
<p>I can definitely see merit in having the 'Zend_Db_Table_Select' providing the read-only status of a query. It would need to provide this post-query as the entire SQL isn't available to interrogate until then. But as you say, it does have the benefit of leaving the existing _fetch method return the same type of result data and would therefore be more BC-friendly... Will do some investigation. I can't use your code as-is unless you've signed a CLA but I'm sure a similar solution can be found.</p>
Oct 23, 2007
Adrian Hope-Bailie
<p>To put my statement above more correctly, the current functions only support matching intersection table columns against match table columns and not actual values.</p>
<p>Perhaps the Zend_DB_Table_Abstract::_referenceMap could also be upgraded such that it can contain, 'filter', 'count', 'offset' fields.</p>
<p>In this way these values can be set as part of the join rule.</p>
<p>Using the examples above:</p>
<ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
class UserGadgets extends Zend_DB_Table_Abstract{
protected $_name = 'user_gadgets';
protected $_referenceMap = array(
'User' => array(
'columns' => 'user_id',
'refTableClass' => 'User',
'refColumns' => 'id'
),
'ConfirmedGadget' => array(
'columns' => 'gadget_id',
'refTableClass' => 'Gadget',
'refColumns' => 'id',
'filter' => array('i.status_id = ?' => 'confirmed'),
'count' => 10,
'offset' => 3
),
'Gadget' => array(
'columns' => 'gadget_id',
'refTableClass' => 'Gadget',
'refColumns' => 'id',
));
]]></ac:plain-text-body></ac:macro>
<p>In which case the following:</p>
<ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
$user->findGadgetViaUserGadgetsByUserAndConfirmedGadget();
]]></ac:plain-text-body></ac:macro>
<p>would execute the second query above.</p>
<p>(NOTE: function calls above should read 'findGadgetVia' not 'findGadgetBy')</p>
Oct 25, 2007
Adrian Hope-Bailie
<p>Hi Simon,</p>
<p>I have made some changes to your code to reflect the ideas I mentioned.<br />
Have left simply a fetchAll, no fetch I promise <ac:emoticon ac:name="smile" /></p>
<p>Most of the changes are in the Zend_Db_Table_Abstract and Zend_Db_Table_Select.</p>
<p>I have signed a CLA so feel free to use and include as you wish. I'm not sure how to submit the code other than to email it to you?</p>
Oct 28, 2007
Simon Mundy
<p>FYI I've updated the laboratory with amendments to the Zend_Db_Table_Abstract, Zend_Db_Table_Select and Zend_Db_Table_Row_Abstract classes. The _fetch method now reverts to returning an array of results only and the 'readOnly' status of a row/rowset can be determined by querying the select object after the query has been made (using $select->isReadOnly()).</p>
<p>The select object also performs validation on the query to ensure that JOINed tables can be made but not referenced in the returned columns.</p>
<p>We're definitely ready for review so will await the Zend team's feedback.</p>
Oct 30, 2007
Adrian Hope-Bailie
<p>Cool, will have a look.<br />
How long does it normally take for these proposals to get from here to Core?</p>
Oct 31, 2007
Darby Felton
<ac:macro ac:name="note"><ac:parameter ac:name="title">Zend Comments</ac:parameter><ac:rich-text-body>
<p>This proposal has been approved by the Zend team for incubator development. We'll address specific implementation issues, unit tests and code coverage, and documentation during the incubation period.</p>
<p>Simon, many thanks for leading this proposal! <ac:emoticon ac:name="smile" /> Please see me with any questions about the unit testing facilities for Zend_Db, and we can work through them together.</p></ac:rich-text-body></ac:macro>
Feb 16, 2008
Svetoslav Marinov
<p> I am using Classes that extend Zend_Db_Table object and it is very limiting if you can't to supply select() object in fetchAll() and other similar methods.</p>
<p>the docs state</p>
<ac:macro ac:name="noformat"><ac:plain-text-body><![CDATA[
Zend_Db_Table_Abstract
]]></ac:plain-text-body></ac:macro>
<p>fetchAll (line 999)</p>
<p>Fetches all rows.</p>
<p>Honors the Zend_Db_Adapter fetch mode.</p>
<ul>
<li>return: The row results per the Zend_Db_Adapter fetch mode.</li>
<li>access: publicZend_Db_Table_Rowset_AbstractfetchAll([string], [string|array $order = null], [int $count = null], [int $offset = null])</li>
<li>string|array|Zend_Db_Table_Select$where: OPTIONAL An SQL WHERE clause or Zend_Db_Table_Select object.</li>
<li>string|array$order: OPTIONAL An SQL ORDER clause.</li>
<li>int$count: OPTIONAL An SQL LIMIT count.</li>
<li>int$offset: OPTIONAL An SQL LIMIT offset.</li>
</ul>
Feb 16, 2008
Simon Mundy
<p>But you can! Well, at least a Zend_Db_Table_Select object, which extends Zend_Db_Select.</p>
<p>How do you execute your queries? If you're using the above examples they should be working as-is.</p>