Skip to end of metadata
Go to start of metadata

<ac:macro ac:name="unmigrated-inline-wiki-markup"><ac:plain-text-body><![CDATA[]]></ac:plain-text-body></ac:macro>
<ac:macro ac:name="unmigrated-inline-wiki-markup"><ac:plain-text-body><![CDATA[

<ac:macro ac:name="unmigrated-inline-wiki-markup"><ac:plain-text-body><![CDATA[

Zend Framework: Zend_Service_Amazon_S3 Component Proposal

Proposed Component Name Zend_Service_Amazon_S3
Developer Notes http://framework.zend.com/wiki/display/ZFDEV/Zend_Service_Amazon_S3
Proposers Justin Plock
Alexander Veremyev (Zend liaison)
Stanislav Malyshev
Revision 1.0 - 10 March 2008: Initial revision.
1.1 - 31 December 2008: Revised code committed into laboratory SVN
2.0 - 25 February 2009: Revised the API and finalized the laboratory implementation (wiki revision: 15)

Table of Contents

1. Overview

Zend_Service_Amazon_S3 is an implementation of the service API and PHP user-stream wrapper for Amazon's Simple Storage Service (S3)

2. References

3. Component Requirements, Constraints, and Acceptance Criteria

  • This component will implement the Amazon S3 API
  • This component will use Zend_Http_Client to communicate with S3
  • This component will provide the API to use Amazon S3 directly
  • This component will implement a user-based stream wrapper for S3 (s3://)
  • This component will allow using multiple S3 accounts

4. Dependencies on Other Framework Components

  • Zend_Http_Client

5. Theory of Operation

This component will provide Amazon S3 API allowing to store and retrieve data from the Amazon S3 service.

Also, this component will be utilized using the stream_wrapper_register() PHP method to register a new user-based stream interface to S3. This allows the user to utilize existing fread(), fwrite(), fclose(), mkdir(), rmdir(), and stat() methods to directly interface with S3 buckets and objects.

6. Milestones / Tasks

  • Milestone 1: [DONE] Design notes will be published here
  • Milestone 2: [DONE] Working prototype checked into the laboratory supporting use cases
  • Milestone 3: [DONE] Unit tests exist, work, and are checked into SVN.
  • Milestone 4: [DONE] Initial documentation exists.
  • Milestone 5: Proposal accepted and moved to incubator
  • Milestone 6: The component is accepted as part of ZF core

7. Class Index

  • Zend_Service_Amazon_S3
  • Zend_Service_Amazon_S3_Stream

8. Use Cases

9. Class Skeletons

]]></ac:plain-text-body></ac:macro>

]]></ac:plain-text-body></ac:macro>

Labels:
amazon amazon Delete
s3 s3 Delete
proposal proposal Delete
proposals proposals Delete
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.
  1. Mar 10, 2008

    <p>At some point, I'd like to take out the kind of hacky self::$debug stuff and integrate with Zend_Log. I just found it useful to see how the stream was actually being called to help with building the methods. This class attempts to support GET Ranges as well because user-based streams limit to 8192 bytes being returned, even though it seemed Amazon was returning the full object size and ignoring my Range headers. Not too sure what was going on, but the class works regardless.</p>

  2. Apr 21, 2008

    <p>+1 (great work!)</p>

  3. Jun 02, 2008

    <ac:macro ac:name="note"><ac:parameter ac:name="title">Zend Comments</ac:parameter><ac:rich-text-body>
    <p>This proposal has been approved for laboratory development with the following suggestion:</p>
    <ul>
    <li>Watch use of static members, underscores in variable names and class members, hardcoded behavior for mime-type detection based on filename, opportunity for generalized filesystem storage component (e.g., with Nirvanix and S3 as available backends), and integration with Zend_Log.</li>
    </ul>
    </ac:rich-text-body></ac:macro>

    1. Jun 05, 2008

      <p>Once Zend_Mime becomes available, I'd like to replace my getMimeType() method with something from that. I've changed all of the variable names to be camelCase. I'm not quite sure how we would generalize this as a filesystem storage component considering it's intended usage is as a PHP stream wrapper. Unless a large portion of it is made static, it's sort of meant to be a standalone class. What are your thoughts? How do I migrate this class from the laboratory to the standard incubator? Thanks.</p>

  4. Sep 22, 2008

    <p>Hi,</p>

    <p>first of all I'd like to say I appreciated your work on this component and I'll use it as soon as it made it to the core. Anyhow I would prefer an API close to the S3-SOAP-Interface because it would be much easier to port existing code to use Zend_Service_Amazon_S3 and I don't like to create aliases for already existing method names (like mkdir vs. putBucket). I can only hope you reconsider the interface of your class <ac:emoticon ac:name="smile" /></p>

    1. Dec 11, 2008

      <p>I just refactored the stream class to Zend_Service_Amazon_S3_Stream which interfaces with a more standardized interface on Zend_Service_Amazon_S3.  It's missing some of the functionality that Amazon provides (like copying objects, interfacing with CloudFront, etc), but it should be a good base.</p>

  5. Nov 27, 2008

    <p>hey, EXCELLENT WORK. Why is this not on the zend framework ? Everybody needs this !</p>

    1. Nov 27, 2008

      <p>It's in the incubator. It is still missing test cases and documentation. I haven't had much time to contribute that yet.  I also haven't heard from Zend much in regards to what is the future of this class.  Thanks for the support though.</p>

  6. Dec 29, 2008

    <p>Hey, It's just bad to let this in the incubator. I've posted a few mails in the mailing list in order to get this project going. There are 2 proposals for an s3 component and dozens of messages arround from guys asking for this. <br />
    You've done an excellent work and it's been nearly a year since it's there lying in the incubator. It's not fair !</p>

    1. Dec 30, 2008

      <p>Tell me about it.  I wouldn't mind some help finishing up the component (documentation, unit tests, more refactoring, etc).  I just don't know who to talk to at Zend who can assist with granting more privileges on the laboratory SVN tree, but I'd be more than interested in having some help.  I've personally been really busy lately and also don't like seeing this component linger.</p>

      <p>Thanks for the support. </p>

      1. Dec 31, 2008

        <p>Justing, just ping me if you need additional svn privilegies.</p>

        <p>The component is in the Ready for Recommendation state now, so it should be additionally reviewed by Zend Team really soon. </p>

  7. Jan 09, 2009

    <p>Amazon S3 support is a great idea! I think this service is much needed, and people have been asking for it. But I have concerns for the proposed API.</p>

    <p>I am a massive proponent for not creating OO interface for OO's sake, but I am also a massive proponent of consistent API's. To me, the primary interface for this service component should consistent with the rest of the framework. This means that there should be accessors & mutators (setters & getters), as well as verb'd action names that "do work".</p>

    <p>I am all about using php's strengths, and I think a stream wrapper should definitely be part of this component, but not as the primary api.</p>

    <p>I feel like the primary class Zend_Service_Amazon_S3 should look and feel like the other service classes with respect to the api, and <strong>stay true to the concepts exposed by the API</strong>:</p>

    <ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
    class Zend_Service_Amazon_S3

    Unknown macro: { public createBucket(); public getBucket($name); public getData($bucket); .... // etc }

    ]]></ac:plain-text-body></ac:macro>

    <p>Beyond that, I think its perfectly acceptable to create a stream wrapper class to expose this service as a custom stream:</p>

    <ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
    $s3 = new Zend_Service_Amazon_S3();
    // stream name should be dynamic so that multiple accounts are possible
    $s3->registerAsStream($streamName);
    ]]></ac:plain-text-body></ac:macro>

    <p>As you can see, the stream wrapper class (Zend_Service_Amazon_S3_StreamWrapper) would be used to be the "proxy" Between the actual s3 object and php's stream internals.</p>

    <p>I guess thats all for now <ac:emoticon ac:name="smile" /></p>

    <p>-ralph</p>

    1. Feb 06, 2009

      <p>Regarding streams - I like this idea, but it's not as easy to do as one would think. The problem is that stream_open doesn't pass you enough context to find your object. It may be worked around with some creative play with stream contexts, etc. maybe but I don't see any API that binds contexts to wrappers - though API implies it in some places but it doesn't specifically says you can create context that would always be linked to certain wrapper. Maybe with more research it's possible to find the right magic.</p>

  8. Jan 19, 2009

    <p>Hi Guys,</p>

    <p>Congratulations to everyone for their work on this component so far. I'd have to say though that I agree with Ralph. I feel the stream wrapper is definitely important - in fact required for this component, but it should not be the primary API. Also as Ralph mentioned the $s3->registerAsStream method would allow for multiple accounts, which in itself may be fairly important for some users as well.</p>

    <p>Looking at the repository it seems we still need to finish the tests?</p>

    <p>Cheers,</p>

    <p>Rob</p>

  9. Jan 19, 2009

    <p>I'd really like someone to take this over and finish it up.  I just feel bad delaying this even more due to my schedule.  Any takers?</p>

    <p>Thanks.</p>

    <p>-Justin </p>

    1. Feb 06, 2009

      <p>I think I could help with it.</p>

    2. Feb 13, 2009

      <p>Justin,</p>

      <p>Ralph pinged me about helping with this. What is left to do?</p>

      <p>Jon</p>

      1. Feb 13, 2009

        <p>tests and documentation mostly.</p>

        <p>Thanks for helping out.</p>

        <p>-Justin </p>

        1. Feb 13, 2009

          <p>Justin,</p>

          <p>Will look at this weekend then.</p>

          <p>Jon</p>

  10. Feb 15, 2009

    <p>Justin,</p>

    <p>I have just created a proposal for Zend_Service_Amazon_Abstract so that all the Amazon Services can have the same key handling code.</p>

    <p>After talking with a few people in the chat room i have moved this to a JIRA issue.</p>

    <p><a class="external-link" href="http://framework.zend.com/issues/browse/ZF-5791">http://framework.zend.com/issues/browse/ZF-5791</a></p>

    <p>Let me know what you think.</p>

    <p>I have the unit test code done and checked into my sqs repo on gethub</p>

    <p><a class="external-link" href="http://github.com/sidhighwind/zend_service_amazon_sqs/tree/master">http://github.com/sidhighwind/zend_service_amazon_sqs/tree/master</a></p>

    <p>Jon</p>

  11. Mar 13, 2009

    <p>It would be nice, to add method like this:</p>

    <p>getQueryString($object, $expire);</p>

    <p>to return url like this:<br />
    ''http://bucket.s3.amazonaws.com/object?AWSAccessKeyId=44CF9590006BF252F707&Expires=1177363698&Signature=vjbyPxybdZaNmGa%2ByT272YEAiv4%3D''</p>

    <p>or</p>

    <p>''http://s3.amazonaws.com/bucket/object?AWSAccessKeyId=44CF9590006BF252F707&Expires=1177363698&Signature=vjbyPxybdZaNmGa%2ByT272YEAiv4%3D''</p>

    <p>link to doc:<br />
    <a href="http://docs.amazonwebservices.com/AmazonS3/latest/S3_QSAuth.html">http://docs.amazonwebservices.com/AmazonS3/latest/S3_QSAuth.html</a></p>

    <p>pch</p>

  12. Mar 18, 2009

    <p>I have problem with upload big files to s3. Maybe good idea will be add method to do POST request?</p>

    1. Mar 18, 2009

      <p>putObject is using PUT, which should be working fine. </p>

      1. Mar 18, 2009

        <p>Yes, it is using PUT, but for this object is read to memory. It's a problem for very large files.</p>

  13. Mar 18, 2009

    <ac:macro ac:name="note"><ac:parameter ac:name="title">Zend Comments</ac:parameter><ac:rich-text-body>
    <p>This proposal has been approved for standard library.</p></ac:rich-text-body></ac:macro>