Skip to end of metadata
Go to start of metadata

<ac:macro ac:name="unmigrated-inline-wiki-markup"><ac:plain-text-body><![CDATA[

<ac:macro ac:name="unmigrated-inline-wiki-markup"><ac:plain-text-body><![CDATA[

Zend Framework: Zend_Queue Component Proposal

Proposed Component Name Zend_Queue
Developer Notes http://framework.zend.com/wiki/display/ZFDEV/Zend_Queue
Proposers Justin Plock
Daniel Lo
Revision 1.0 - 3 March 2008: Initial revision
1.1 - 27 February 2009: Updated proposal
1.2 - 9 April 2009: Code complete (wiki revision: 15)

Table of Contents

1. Overview

Zend_Queue is a standardized interface for dealing with a variety of queuing systems. Proposed systems include: simple array access, Zend_Cache, Zend Platform Job Queue, Amazon's Simple Queue Service (SQS). It should support creating queues, determining the number of messages in a queue, retrieving messages from a queue (all or specific number), submitting messages to a queue, and removing queues.

2. References

3. Component Requirements, Constraints, and Acceptance Criteria

  • This component should support a variety of backend services (Zend_Db, Zend Platform Job Queue, Arrays, MemcacheQ, etc).
  • This component will support the full Amazon SQS API
  • This component will support creating new queues
  • This component will support deleting existing queues
  • This component will support counting the number of a messages in a queue
  • This component will support retrieving messages from a queue (all or a specific count)
  • This component will support submitting messages to a queue

4. Dependencies on Other Framework Components

  • Zend_Http_Client
  • Zend_Log
  • Zend_Db

5. Theory of Operation

Zend_Queue will be a factory class for to create a specific queue interface class. Simple "send" and "receive" methods will be employed by each backend to interact with queues. "create", "getCount" and "delete" methods will be used to manage queues.

6. Milestones / Tasks

  • Milestone 1: [DONE] Design notes will be published here
  • Milestone 2: [DONE] Working prototype checked into the incubator supporting use cases
  • Milestone 3: [DONE] Unit tests exist, work, and are checked into SVN.
  • Milestone 4: [DONE] Initial documentation exists.

7. Class Index

  • Zend_Queue_Exception
  • Zend_Queue (factory class)
  • Zend_Queue_Message
  • Zend_Queue_Message_Iterator
  • Zend_Queue_Adapter_Abstract
  • Zend_Queue_Adapter_Exception
  • Zend_Queue_Adapter_Amazon (interface for Amazon SQS)
  • Zend_Queue_Adapter_Db (interface for Zend_Db)
  • Zend_Queue_Adapter_Memcacheq (interface for MemcacheQ)
  • Zend_Queue_Adapter_Null
  • Zend_Queue_Adapter_Stomp
  • Zend_Queue_Adapter_Stomp_IO
  • Zend_Queue_Adapter_Array

8. Use Cases

UC-01

9. Class Skeletons

]]></ac:plain-text-body></ac:macro>

]]></ac:plain-text-body></ac:macro>

Labels:
None
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.
  1. May 12, 2008

    <p>1. I think Zend_Queue_Message_Iterator is a typo of Zend_Queue_Adapter_Array, or vise verse. Could you please also provide skeleton of this class?</p>

    <p>2. For method Zend_Queue_Message::setFromArray(), probably the input array should be stored in $this->_data.</p>
    <ac:macro ac:name="code"><ac:default-parameter>php</ac:default-parameter><ac:plain-text-body><![CDATA[
    public function setFromArray(array $data)
    {
    foreach ($data as $columnName => $value)

    Unknown macro: { $this->_data[$columnName] = $value; }

    return $this;
    }]]></ac:plain-text-body></ac:macro>

    <p>Very nice proposal. Thanks.</p>

  2. Jun 02, 2008

    <ac:macro ac:name="note"><ac:parameter ac:name="title">Zend Comments</ac:parameter><ac:rich-text-body>
    <p>The Zend Framework team approves this proposal for immediate development in the<br />
    standard incubator, with the following requirements/suggestions:</p>
    <ul>
    <li>Check if queue adapters should have additional abstraction level for queues with transactions support.</li>
    <li>Add method (or option) for retrieving message from queue as atomic operation. It may be important for some cases (e.g. taking jobs from queue by several executors).</li>
    <li>Check if offered API conforms common messaging models (including models with message recieving notifications).</li>
    </ul>
    </ac:rich-text-body></ac:macro>

  3. Aug 01, 2008

    <p>Hi,</p>

    <p>I'm very interested in your proposal, since we implemented a basic Queue Service ourselves using a DB as a backend. If we unify the development it would be better. We may need a STOMP or AMQP Backend for our next iteration, we would provide that.</p>

    <p>I think it is very important that all backend adapters be concurrency-safe. E.g. if 5 clients are requesting the next message from the queue, the Adapter needs to make sure each message is returned only once. We do this by</p>

    <p>$sQuery = 'UPDATE '.Hitmeister_Db_Tables::get('system_messages').<br />
    ' SET '.$this->aDbFields<ac:link><ri:page ri:content-title="'status_message'" /></ac:link>.' = \'READ\''.<br />
    ' WHERE '.$this->aDbFields<ac:link><ri:page ri:content-title="'id_message'" /></ac:link>.' = '.$oMessage->id_message.<br />
    ' AND '.$this->aDbFields<ac:link><ri:page ri:content-title="'status_message'" /></ac:link>.' = \'NEW\'';</p>

    <p>//-- execute the statement<br />
    $oStatement = $this->oDBH->prepare($sQuery);<br />
    $oStatement->execute();<br />
    if(!$oStatement)<br />

    Unknown macro: { throw new Hits_Abstract_Service_Perform_Exception('@client}

    <br />
    //-- check if was not able to update<br />
    if ($oStatement->rowCount() == 0)<br />
    { try

    Unknown macro: { return System_Messages}

    <br />
    catch (Exception $e)<br />

    Unknown macro: { throw $e; }

    <br />
    }<br />
    This is more efficient than locking the table, at least in our use scenario.</p>

    <p>One more thing we found: if you reduce the strict FIFO requirement to "more or less FIFO" you can get more performance out of the components. Perhaps make it variable in the next version?</p>

  4. Feb 17, 2009

    <p>What needs to be done for this proposal to move ahead. It seems to have been very little movement since august.</p>

    1. Feb 17, 2009

      <p>The code is pretty much done.  As with my S3 proposal, this needs unit tests and documentation.  I committed a MemcacheQ adapter this morning.  Any help would be appreciated as I'm not to sure how to write documentation for the manual.</p>

      1. Feb 17, 2009

        <p>I'm going to see if I can't get the unit test done first and get it accepted in for 1.8 and then bust out the doc's. Take a look at the other documentation as it just uses the doc book format.</p>

  5. Feb 28, 2009

    <p>I think a component named that unspecific should be as unspecific as it's name is. How would you integrate Apache ActiveMQ or Rabbit MQ? Message Queues are Queues as well.</p>

    <p>I'm currently working on a library for the Stomp protocol using various Message Queues which I designed with maximum flexibility in mind. You might want to take a look at how many parts are exchangeable in such a system (and that's Stomp specific!):</p>

    <p><a class="external-link" href="http://github.com/felixjendrusch/stomp/tree/master/">http://github.com/felixjendrusch/stomp/tree/master/</a></p>

    <p>I'm still writing the documentation because I had to get it working as fast as possible for a customer, sorry for that. Some components are not finished yet but it's working.</p>

  6. Mar 19, 2009

    <p>I started creating a queue using Zend_Queue_Db and have come across an issue. It appears that the send and receive methods of Zend_Queue_Db expect the active queue to be set by queue_id instead of name. </p>

    <p>Zend_Queue_Db::create() will set the active queue by ID, so send and receive will work in that case. But, when using setActiveQueue() by name like in UC-01, send and receive will both fail because it is trying to set the queue_id to the queue name. </p>

    <p>Is this the expected behavior or is it an issue with Zend_Queue_Db?</p>

    1. Mar 19, 2009

      <p>Also, wouldn't it be a good idea for the queue name to be unique in the DB?</p>

  7. Apr 06, 2009

    <p>I still do not feel very confident with the name of this component. Zend_Queue is to general for this... what about message queues (Apache ActiveMQ, RabbitMQ, ...)? Will they get their own component (Zend_MessageQueue?)?</p>

    1. Apr 06, 2009

      <p>ActiveMQ and RabbitMQ are supported using the Zend_Queue_Adapter_Stomp adapter.</p>

      1. Apr 06, 2009

        <p>Stomp is a protocol. Apache ActiveMQ and RabbitMQ are message brokers supporting various protocols. Saying that Apache ActivqMQ and RabbitMQ are supported using a protocol is just wrong.</p>

        <p>Supporting the Stomp protocol is nice but in reality just supporting its definitions is not enough (e.g. RabbitMQ uses a different frame delimiter).</p>

        1. Apr 09, 2009

          <p>Feel free to contribute the code for ActiveMQ and RabbitMQ.</p>

          <p> -Justin </p>

  8. Apr 24, 2009

    <p>Hi Justin,</p>

    <p>Can you rename setMsgClass(), setMsgsetClass(), and so on to setMessageClass() and setMessageSetClass()? Randomly abbreviating words like that can make an API difficult to remember.</p>

    <p>I would also suggest that Stomp receive first-level support as Zend_Stomp (this is the case with other protocols, such as HTTP, Wildfire, etc.). Zend_Queue_Adapter_ActiveMQ and Zend_Queue_Adapter_RabbitMQ can then use that component. Perhaps the frame delimiter could be configurable via a method in Zend_Stomp, and ActiveMQ and RabbitMQ classes could simply set up the delimiter and then proxy requests to Zend_Stomp.</p>

    <p>Is there any support for filtering based on certain criteria? This is a very common use case.</p>

    <p>Otherwise, looking forward to seeing this in trunk!</p>

    <p>-Matt</p>

  9. May 01, 2009

    <p>Are you planing to implement "gearman" ?<br />
    It's an upcomming job system, which is easy to use and not bound to any language (you can add jobs to the queue from php, and get them from java).</p>

    <p><a class="external-link" href="http://www.danga.com/gearman/">http://www.danga.com/gearman/</a>
    <a class="external-link" href="http://www.gearman.org/doku.php">http://www.gearman.org/doku.php</a></p>

    1. May 18, 2009

      <p>I don't think it makes sense to implement Gearman as part of a unified Queue interface. Gearman is a job queue which is fundamentally different from a message queue. Looking through the Zend_Queue_Adapter_Abstract, the only method that would be supported would be send(). recieve() is for a worker not a client, and deleteMessage() is automatic with Gearman.</p>

      <p>There needs to be a separate proposal for Zend_Queue_Job or Zend_JobQueue that would support Gearman and Zend Platform's JobQueue. It also makes sense to start a Zend_Gearman proposal that ports in the Gearman Pear class, supports the Pear class and the Gearman PHP extension, sort of like Zend_Json does.</p>

  10. Jun 23, 2009

    <p>I'd also like to see support for the SplQueue and SplPriorityQueue classes being added to the SPL in PHP 5.3. <ac:emoticon ac:name="smile" /></p>