Zend Framework: Zend_Service_Amazon_S3 Component Proposal
| Proposed Component Name | Zend_Service_Amazon_S3 |
|---|---|
| Developer Notes | http://framework.zend.com/wiki/display/ZFDEV/Zend_Service_Amazon_S3 |
| Proposers | Justin Plock Alexander Veremyev (Zend liaison) Stanislav Malyshev |
| Revision | 1.0 - 10 March 2008: Initial revision. 1.1 - 31 December 2008: Revised code committed into laboratory SVN 2.0 - 25 February 2009: Revised the API and finalized the laboratory implementation (wiki revision: 14) |
Table of Contents
1. Overview
Zend_Service_Amazon_S3 is an implementation of the service API and PHP user-stream wrapper for Amazon's Simple Storage Service (S3)
2. References
3. Component Requirements, Constraints, and Acceptance Criteria
- This component will implement the Amazon S3 API
- This component will use Zend_Http_Client to communicate with S3
- This component will provide the API to use Amazon S3 directly
- This component will implement a user-based stream wrapper for S3 (s3://)
- This component will allow using multiple S3 accounts
4. Dependencies on Other Framework Components
- Zend_Http_Client
5. Theory of Operation
This component will provide Amazon S3 API allowing to store and retrieve data from the Amazon S3 service.
Also, this component will be utilized using the stream_wrapper_register() PHP method to register a new user-based stream interface to S3. This allows the user to utilize existing fread(), fwrite(), fclose(), mkdir(), rmdir(), and stat() methods to directly interface with S3 buckets and objects.
6. Milestones / Tasks
- Milestone 1: [DONE] Design notes will be published here
- Milestone 2: [DONE] Working prototype checked into the laboratory supporting use cases
- Milestone 3: [DONE] Unit tests exist, work, and are checked into SVN.
- Milestone 4: [DONE] Initial documentation exists.
- Milestone 5: Proposal accepted and moved to incubator
- Milestone 6: The component is accepted as part of ZF core
7. Class Index
- Zend_Service_Amazon_S3
- Zend_Service_Amazon_S3_Stream
8. Use Cases
9. Class Skeletons
| Zend Comments This proposal has been approved for laboratory development with the following suggestion:
|
Once Zend_Mime becomes available, I'd like to replace my getMimeType() method with something from that. I've changed all of the variable names to be camelCase. I'm not quite sure how we would generalize this as a filesystem storage component considering it's intended usage is as a PHP stream wrapper. Unless a large portion of it is made static, it's sort of meant to be a standalone class. What are your thoughts? How do I migrate this class from the laboratory to the standard incubator? Thanks.
Hi,
first of all I'd like to say I appreciated your work on this component and I'll use it as soon as it made it to the core. Anyhow I would prefer an API close to the S3-SOAP-Interface because it would be much easier to port existing code to use Zend_Service_Amazon_S3 and I don't like to create aliases for already existing method names (like mkdir vs. putBucket). I can only hope you reconsider the interface of your class ![]()
hey, EXCELLENT WORK. Why is this not on the zend framework ? Everybody needs this !
Hey, It's just bad to let this in the incubator. I've posted a few mails in the mailing list in order to get this project going. There are 2 proposals for an s3 component and dozens of messages arround from guys asking for this.
You've done an excellent work and it's been nearly a year since it's there lying in the incubator. It's not fair !
Tell me about it. I wouldn't mind some help finishing up the component (documentation, unit tests, more refactoring, etc). I just don't know who to talk to at Zend who can assist with granting more privileges on the laboratory SVN tree, but I'd be more than interested in having some help. I've personally been really busy lately and also don't like seeing this component linger.
Thanks for the support.
Amazon S3 support is a great idea! I think this service is much needed, and people have been asking for it. But I have concerns for the proposed API.
I am a massive proponent for not creating OO interface for OO's sake, but I am also a massive proponent of consistent API's. To me, the primary interface for this service component should consistent with the rest of the framework. This means that there should be accessors & mutators (setters & getters), as well as verb'd action names that "do work".
I am all about using php's strengths, and I think a stream wrapper should definitely be part of this component, but not as the primary api.
I feel like the primary class Zend_Service_Amazon_S3 should look and feel like the other service classes with respect to the api, and stay true to the concepts exposed by the API:
Beyond that, I think its perfectly acceptable to create a stream wrapper class to expose this service as a custom stream:
As you can see, the stream wrapper class (Zend_Service_Amazon_S3_StreamWrapper) would be used to be the "proxy" Between the actual s3 object and php's stream internals.
I guess thats all for now ![]()
-ralph
Regarding streams - I like this idea, but it's not as easy to do as one would think. The problem is that stream_open doesn't pass you enough context to find your object. It may be worked around with some creative play with stream contexts, etc. maybe but I don't see any API that binds contexts to wrappers - though API implies it in some places but it doesn't specifically says you can create context that would always be linked to certain wrapper. Maybe with more research it's possible to find the right magic.
Hi Guys,
Congratulations to everyone for their work on this component so far. I'd have to say though that I agree with Ralph. I feel the stream wrapper is definitely important - in fact required for this component, but it should not be the primary API. Also as Ralph mentioned the $s3->registerAsStream method would allow for multiple accounts, which in itself may be fairly important for some users as well.
Looking at the repository it seems we still need to finish the tests?
Cheers,
Rob
I'd really like someone to take this over and finish it up. I just feel bad delaying this even more due to my schedule. Any takers?
Thanks.
-Justin
Justin,
I have just created a proposal for Zend_Service_Amazon_Abstract so that all the Amazon Services can have the same key handling code.
After talking with a few people in the chat room i have moved this to a JIRA issue.
http://framework.zend.com/issues/browse/ZF-5791
Let me know what you think.
I have the unit test code done and checked into my sqs repo on gethub
http://github.com/sidhighwind/zend_service_amazon_sqs/tree/master
Jon
It would be nice, to add method like this:
getQueryString($object, $expire);
to return url like this:
''http://bucket.s3.amazonaws.com/object?AWSAccessKeyId=44CF9590006BF252F707&Expires=1177363698&Signature=vjbyPxybdZaNmGa%2ByT272YEAiv4%3D''
or
''http://s3.amazonaws.com/bucket/object?AWSAccessKeyId=44CF9590006BF252F707&Expires=1177363698&Signature=vjbyPxybdZaNmGa%2ByT272YEAiv4%3D''
link to doc:
http://docs.amazonwebservices.com/AmazonS3/latest/S3_QSAuth.html
pch
I have problem with upload big files to s3. Maybe good idea will be add method to do POST request?
ZF Home Page
Code Browser
Wiki Dashboard
At some point, I'd like to take out the kind of hacky self::$debug stuff and integrate with Zend_Log. I just found it useful to see how the stream was actually being called to help with building the methods. This class attempts to support GET Ranges as well because user-based streams limit to 8192 bytes being returned, even though it seemed Amazon was returning the full object size and ignoring my Range headers. Not too sure what was going on, but the class works regardless.