Added by Thomas Weidner, last edited by Matthew Weier O'Phinney on Aug 05, 2008  (view change) show comment

Labels

 
(None)

Zend Framework: Zend_File_Transfer Component Proposal

Proposed Component Name Zend_File_Transfer
Developer Notes http://framework.zend.com/wiki/display/ZFDEV/Zend_File_Transfer
Proposers Thomas Weidner
Matthew Weier O'Phinney (Zend Liaison)
Revision 1.0 - 27 Apr 2007: Initial release
1.1 - 03 May 2007: Small reworks to the API
1.2 - 05 May 2007: Added hook for check as discussed
2.0 - 06 Aug 2007: Renamed to Zend_Http_Upload
3.0 - 22 Dec 2007: Renamed to Zend_File_Uploader and completly
reworked the API
4.0 - 20 Jan 2008: Renamed to Zend_File_Transfer, API reworked (wiki revision: 27)

Table of Contents

1. Overview

Zend_File_Transfer is a component to handle File Up- and Downloads within the Zend Framework in a standardized way.

2. References

3. Component Requirements, Constraints, and Acceptance Criteria

No requirements...

But to receive upload-processing informations a pecl extension and minimum PHP 5.2 is needed.
Otherwise the class can not provide the processing information.

4. Dependencies on Other Framework Components

  • Zend_Exception

5. Theory of Operation

This component is meant to handle file up- and downloads within the Zend Framework. Is should provide an simple and generic way for all users. Several tasks are possible:

  • Upload multiple files
  • Set validators
  • Set filters
  • Set paths
  • Rename files
  • Provide a way to get the download progress data for the handled
    files (could be limited to 5.2)
  • Wrapper for downloads
  • Different adapters for HTTP POST, WEBDAV, FTP, AJAX and so on...

6. Milestones / Tasks

  • Milestone 1: [FINISHED] Proposal finished
  • Milestone 2: [FINISHED] Proposal accepted
  • Milestone 3: Working release
  • Milestone 4: Unit tests
  • Milestone 5: Documentation
  • Milestone 6: Future enhancements

7. Class Index

  • Zend_File_Transfer_Exception
  • Zend_File_Transfer
  • Zend_File_Transfer_Protocol
  • Zend_File_Transfer_Protocol_Http
  • Zend_File_Transfer_Procotol_Ftp
  • Zend_File_Transfer_Procotol_WebDav
  • Zend_File_Transfer_Procotol_Ajax

8. Use Cases

Upload defined file

Default for transfer is http download. This code shows the simplest way to have downloads integrated.

Set validators

Validators can be used to check several options of files which are loaded. Validators could possibly be Zend_Validate validators... but this would be checked at time of integration. Not sure if this would fit all needs. But Zend_File_Transfer will integrate several own validators like FileSize, FileExtension, MimeType and so on...

Validators for single files/fields

Validators can also be set for single files, or for all. Also isUploaded can check for all or single files. Of course the API supports fluid interface where applicable.

Change content with filters

With filters it is possible to change content of downloaded files before they are stored. Doublicate transfered files to a second directory, or change content with self written filters in this example change linebreaks of textfiles from unix to windows. Several filters are thinkable.

HTTP PUT Uploads

Also other adapters can be used. In our example http put, but also other adapters like FTP, WEBDAV or AJAX will be integrated. Within all adapters the API will be the same.

FTP Downloads

The API can also be used for downloads. It will work as wrapper so the user does not see where the original files is located and ZF will send the file to the user.

WebDav Uploads

All adapters use the same API... here WebDav

Ajax Uploads

There is also an idea of creating an Ajax adapter which would make it possible to upload files in a form while the user has not to wait until the upload is finished and can work on the form or make other things while the file is uploaded in background with ajax. Also the status (amount of downloaded files, progress and so on) is avaiable to the user while the upload is in progress and can be displayed to him as additional information.

Actually there is no code because this will also have to be integrated in the form or the view the user get's displayed.

9. Class Skeletons

a) How is this easier than using the native functions?
b) When I have multiple file uploads, often I want to move them to different locations
c) If this were to be adopted, shouldn't it maybe be Zend_File_Upload instead?

a) Until now I often saw people creating own classes for handling file uploads... I just wanted to give a standard way for ZF users... as with all other ZF classes... you dont need to use them if you do not want to... My opinion is that a standard way is better than no way with everybody cooking his own soup.

b) This is supported... see setDestination/move... for each file or a collection of files a destination can be set

c) It was your name-cousine (Matthew-Weier-O'Phinney) who choose the name in january... see ZF-818
Zend_File_Upload expects the user to have other file manipulating classes like _Download _Compress _Copy and so on... I am not related to Zend_Update... I just wanted to have a simple, small name and this one was already written several times ago within the mailing list

Re: c) Matthew's opinion isn't a divine writ. In any event, he said "Zend_Upload (or similarly named)".

I anticipate other Zend_File-related classes (in fact, I am developing some now in conjunction with my as-yet unproposed Zend_Io). Makes sense to plan for the future instead of having related classes all over the place.

I already mentioned that I am not related to Zend_Upload...
I just want to serve the functionallity to the Zend Framework.

But what you wrote is quite confusing to me...
On one side you wrote "shouldn't it be Zend_File_Upload instead"
and on the other side you wrote "you anticipate other Zend_File related classes".
What about Zend_Http_Client_Form_Download_Multiple_Files_With_This_Class ?

But however we name it, are there any neg's for providing a standardized way of handling file uploads ?? (and maybe file downloads in the future)

Don't get me wrong; I'm in favor of this class. It would replace my own file upload class. It's just been said again and again that classes that wrap PHP functions without adding value (beyond object-orientation) won't be approved for inclusion. If this is approved, this class pushes the door open for the inclusion of other wrapper classes.

Thomas, I like it as is. No more further suggestions.

One comment:

Zend_Upload's constructor is still PHP-4 style (using the class name), but that's a minor issue __construct would be more suitable.

Of course you are right
Blame on me

I have ever made my own upload class to extend Zend Framework and I use it like this (very similar than you):

I can manage if extention are accepted or not not accepted...

Also, why not Zend_Http_Upload ?

a) I would prefer Zend_Http_Upload too..

b) Did you took file-arrays in consideration? "<input type="file" name="name[]" />" ..

c) Maybe it's possible to use Zend's Filters and Validators for file-name-validation - but on the other hand this might bloat the class..

c) I like your current ideas for methods, but I would prefer a bit more control over file-handling. Hence I would propose a more object-style structure:

  • Zend_Http_Upload (or whatever the final package will be)
  • Zend_Http_Upload_Group
    Primary used to group files with same options to make validation easier. As example: You want an upload-form with 20 files, where 10 should be .doc with a maximum filesize of 10kb and 10 should be images with a max. filesize of 100kb, you can use this class to make 2 groups with the specific settings.
  • Zend_Http_Upload_File
    Like "Group", but on a per-file base. This class can be passed to "Upload" and "Upload_Group" instead of field-names. The developer can define some settings (max filesize, allowed filetypes) here, as well as in "Upload" and "Upload_Group".
    This class could also be extended, to include support for some more specific cases - e.g. "Upload_File_Image" could be easily written to validate images (their types, their size, etc.).
    For this, there will be the need to return some usefull error-information to developers, so they know why a file is not valid.

Thus there will be three layers of validations. On global scope ("Upload"), on a per-group scope ("Upload_Group") and on a per-file scope ("Upload_File").

This would be really complicated and bloated if you simply want to check if a file is within a specific filesize, so it should be possible to use just the first layer (see examples below).

I'm currently not a registered developer here, but if these ideas turn out to be useful I could help, if help is needed.

Some examples

Simplest validation-example:

A bit more complicated example:

hi,

this is a good idea I think...

Do you check the file type on the file extension, or on the mime type?
(does this question make sense?)

vincent

If your comment was a reply to mine:

Since the methods name is "setValidExtensions(..)", the method just checks the file-extension. Because the mime-type can be changed on the browser-side, it's maybe a better idea to rely on an extension-check (http://www.addict3d.org/news/1075/.html, http://de2.php.net/manual/en/features.file-upload.php). The extension might be changed as well, but on a normal server (which just parse .php as PHP) a changed extension will also stop the server to parse the file. Are there other security issues which should be taken into consideration?

Greetings, Florian

1) In my opinion validating a file name by its extension is not the safest way to handle file uploads.

2) I must agree that Matthew Ratzloff is write better rename it Zend_File_Upload because probably in the future many operations with files will be implemented (for sure archiving would be a good idea).

3) The third thing that i don't understand is the if chains:

if($photoValid == UPLOAD_ERROR_IMG_MINSIZE) {
// error no. 1
} else if($photoValid == UPLOAD_ERROR_IMG_MAXSIZE) {
// error no. 2
} else if($photoValid != UPLOAD_ERROR_NONE) {
// general error
} else {
// proccess upload..
}

Can this be avoided? 

to 1)
This is just one way to limit given files.

In the proposal is also another way with the "addCheck" function which acts as callback to an self-defined checking function.

How the upload is checked depends on the checks you apply to the instance.

to 2)
No... better is "Zend_Http_Upload" as already stated before. As this class is not meant to handle FTP or UDP or other fileuploads...
It's designed as extension to Zend_Http.

to 3)
First... never use if-chains... we have a "switch - case" statement within php
Second... in my proposal I never stated that I will return a "UPLOAD_XXX_ERROR_CONSTANT"...

What you will get is a "true" or "false" on the isxxx functions.
Or an exception when processing the files.

And in my opinion an exception is the right way for the framework.

Another thing, i saw that $files->setOptions will receive an array, this may create a confusion because there are to many parameters and you are forcing someone to remember all, in my opinion the method should set the parameters but should receive only two values, key and value. It would be easier to check for consistency.

No, I dont agree with you, because almost all classes within the framework use arrays as options parameter.

See Zend_Date, Zend_Db, Zend_Filter, Zend_Session, Zend_Translate and many other classes.

Having scalar values as input would mean that you have to call the same method several times to set all wished options.

With Arrays you can have all your options stored within an config array (Zend_Config) and simply give this array.

I agree with this idea, to make a Zend Upload object.
I belive is better to rename it Zend_Http_Upload, cuz we need to remember that maybe will be a Zend_Ftp_Upload.

Nicolae,
Your 1) topic is very interesting, I must agree with that.

As already written to Nicolae...
What is checked depends on the restrictions you give for the upload.

Restrictions can for example be:

  • Name
  • Type
  • Size (max, min)
  • Mime
  • If it already exists
  • ...

You can even apply an selfwritten check with "addCheck" if you for example have to check if your image includes a watermark

If you dont restrict the filetype it will not be checked.

Yes, there are a lot of ways to restrict the archive.

I don't understand this "addCheck". You are saying that you can includes a watermark in the uploaded image?
But, this is more work then Zend_Http_Upload tasks. I think that one Zend_Image can do that?

By the way, when Zend_Http_Upload will be integrate to the ZF?

You can think of the addCheck Method as a hook where you can plug in self created checker-functions.

This could be a watermark-check, a regex-based filename check or something completly different. It depends on what you want to check...
Maybe it will also provide the possibility to change the content of the file before storing it to the server... I haven't finished thinking of all pro's and con's.

Related to "when it will be integrated"...

Until now I had not the time to finish my thoughts about this proposal.
Any I am not sure if it will be accepted in the actual state.
This is related to Matthews comment at 2.May.2007 about adding enough value.

Hi,

Is this good coding practice? This code in progress...

Actually this proposal is reviewed by the Dev-Team.

I tend to begin coding when an proposal has approved because I dont want to have more work than nessesary. Therefor I did not include code for now...

The problem is that when everyone uses your code before the proposal has approved and there is a change then all users have the problem that their code would not work anymore.

Because they believe that you are the author which is not true

Actually there are enough "pre-classes" out there in several forums that it is not nessesary to include unofficial code here in my opinon.

If this proposal is accepted I would be able to have my code included within the incubator.

Related to your code:
There are only some small issues, for example defining constants which are not in use, or the constructor.

So I am waiting for approvement from the dev team and I have not stopped working on this one.
Please do not add code here or use this code. You will expect problems after this class is officially finished and the API does not match.

PS:
It would be better to force the dev-team to accept this proposal or give an statement about possible problems.

Hello,

I am wondering if you are going to make it so that different back ends can be set with the file upload as in my case I don't need to save it to the local file system as we have a holding server that holds all the upload that get a approved.

So what I do after I process the file to make sure it's valid is use ftp to move it to my holding server. Would this be a possibility to do? I think it would add a lot to the component.

I don't quite understand what you mean or where you think that a problem with the API is...

Because when PHP receives a fileupload it is stored internally within the specified temporary path.
And to get it out of there you have to move it.

And within the API you can see that you can specify the destination for each file with the move() method.

Hi Thomas,

I like your proposal and hope it's accepted, since it seems useful and ZF needs a component like this.

I'm trying to implement it as I already need it for an application, and got some questions:

1. Do you agree Marcin Lewandowski's implementation of files and options? (not talking about the code, but about the schema) I was thinking about another property for "general" options:

2. About the 'move' method, is it supposed to call 'isUploaded' before trying to move?

3. More about 'move'. What is the first argument used for?

vs.

4. Must the 'isUploaded' method check for restrictions? Or does it only check 'isset($_FILES[$file])'?

5. What will the 'check' method do? (if you thought about it) Checking restrictions and calling the 'check function' if set? What about the callback function?

6. What about moving/copying uploaded file/s to multiple destinations? For example, you may need several copies of an image or maybe replicating the file into another server...

Thank you very much!

I like your proposal and hope it's accepted, since it seems useful and ZF needs a component like this.

Hopefully I will get response from the dev-team until new year.

I'm trying to implement it as I already need it for an application, and got some questions:

There are a few implementations out there, but non from me because I didn't want to anticipate the decision from the dev-team.

Do you agree Marcin Lewandowski's implementation of files and options? (not talking about the code, but about the schema) I was thinking about another property for "general" options:

Yes and no...
There will be an internal options array, and it will be protected.
But it will not look like Marcin's one.

Keep in mind:
Options in this place will be class-wide options, and not single-file options !

About the 'move' method, is it supposed to call 'isUploaded' before trying to move?

There will be several checks before we really "move" the file.
One of it is the isUploaded method.

More about 'move'. What is the first argument used for?

You can define that only special files instead of all are moved.
And you can define a new location for these files....

f.e.
// all other files are ignored or can be moved in a later move operation
$load->move(array('file1' => 'C:\mylocation\filex', 'file2' => 'C:\mylocation\filey'));
// define a new location for all files
$load->move('C:\mylocation');

Must the 'isUploaded' method check for restrictions? Or does it only check 'isset($_FILES[$file])'?

isUploaded is only one of the checks which will be avaiable...
And as the name says, it does only check if the file has been completely uploaded.

What will the 'check' method do? (if you thought about it) Checking restrictions and calling the 'check function' if set? What about the callback function?

It adds common checks like filesize, extension, and so on...
And gives the ability to work as callback to own functions which check or even change content of uploaded files.

Until now I am not sure if I will add several functions one for each check or if I collect them together and run them through options.

What about moving/copying uploaded file/s to multiple destinations? For example, you may need several copies of an image or maybe replicating the file into another server...

See move()...
How the method works depends on the set options.

Just keep in mind that the real implementation can be different from what you think, and you may need to adopt your code afterwards.

Greetings
Thomas
I18N Team Leader

Thank you, Thomas.

About the first argument of 'move', will it be possible to be called like this?

I mean, destination will be taken from the options.

Only if 'file1' is a downloaded file or a identifier.
Otherwise it would be treated as new target.

Maybe switchable through generic options.

Hi Thomas,

I like the API and I think the framework needs this class. The API is simple and consistent with other classes. It would nice if you could specify a protocol, and allow that object to handle the moving of the file. Because users might need to upload files using http (from local drive to server 1) or using ftp (from server 1 to server 2).

Something like this for example:

In the example below $upload->file holds the protocol object and handles the adding and copying of the files.

Posted by fc at Dec 17, 2007 16:56

So if the protocol object handles the moving of the file, you can pass a directory or a connection array as parameter, eg:

Posted by fc at Dec 17, 2007 17:01

See my generic reply...

Related to your testcode I see some problems which I would not solve like you:

  • A downloadclass for each mime type - too complicated
  • Set the protocol by initiating new class - too complicated
  • File adding / source - destination - has to be simplified
  • Move with ftp connection settings - problematic, only one server allowed

My implementation will add more usability and would be simpler in it's handling.

It would be no problem also to support other mechanism than HTTP.
HTTP, FTP, FILE, SOCKET,... whatever.

BUT:
I was told that Zend_Upload is not a proper name so I had to rename it to Zend_Http_Upload.
This does also mean that in this case other protocols are not supported.

IF we allow also to support other transport machanism, we MUST rename it back to Zend_Upload as I proposed original.

It would be great if we could add a generic uploader class, not only supporting HTTP but also other mechanism. But the general decision is on the dev-team, and I did not hear anything since several months.

Yes, I agree. If it's just for HTTP uploads, then Zend_Http_Upload is fine. And what about having specific methods based on the file type, for example:

That way, you could add a _convertLineBreaks() method to Zend_Http_Upload_Document, in case you want to convert Mac and/or PC line breaks to UNIX, or any other method specific to the file type that the user is uploading.

From the usability point of view, like I said before, your API is simple and easy-to-use. And that's always a plus.

Posted by fc at Dec 18, 2007 04:22

Basically this is a decision of the community.
If there is a need for other protocols, I could also implement Zend_Upload.

We will clearify this in the next days.

Related to Document/Image and so on...
First: This is no good seperation, because there are several document types out there and each one has to be handled different. This should depend on the mime typ or a user-selection.

Second: As I wrote in a comment before it is no problem to have own checks implemented. A check does not only mean to proove something, it could also mean to change something. So maybe the name "check" is not the right, but the mechanism of a self defined callback function should be clear.

Third: I am not so sure if we should implement file-changing functionality within an uploader class. Why should Zend_Upload change the content, for example strip the first three lines, and store the file elsewhere and say "it's uploaded"...

Of course it sounds a good idea, and it's already been discussed in the past.

Still there is no solution:
Pro: New amazing functionality
Con: No one would expect "upload" to change content

Hi Thomas, yes, good point.

About the Document/Image/Video separation. Is more like a grouping of mime-types. Yes, basically you are separating them into groups. Well, at least that's how I'm doing it. Each group defines its mime-types and adds its own options based on that. For example:

All the generic options are in Zend_Http_Upload_Abstract, and the mime-type specific options in Zend_Http_Upload_

Unknown macro: {type}
.

I'm not saying that this is right or wrong, I'm just brainstorming some ideas.

Posted by fc at Dec 18, 2007 12:49
View the rest of this thread. Most recent comment: Dec 19, 2007
4 more comments by: fc, Thomas Weidner

Why do you have a move() method?

The name 'move' might be confusing for people who don't know how PHP's builtin file upload works. And it makes it look like you're just wrapping the precedure of the lower level interface.

Why not call it accept()?

Because move() is able to do exactly that...
It is able to move the file elsewhere... or even duplicate it and move it to two locations.

Therefor accept() would not be the right name.

Thomas,

The user doesn't need to know anything about moving files. The whole idea of moving files comes from the procedure surrounding using $_FILES. So you're just wrapping a procedure which does not take advantage of OOP at all. You're excluding a lot of useful subclassing scenarios.

Along those lines, I also agree with Matthew that the concept behind this class is files and not something specific to the HTTP protocol or the act of uploading. Therefore the class should probably be Zend_File_Something and not Zend_Http_Something or Zend_Upload_Something. The focus is on files. Therefore, if the focus is on files, then move() does make sense. But I still something more abstract like processFiles() or accept() would be better. A lot more can happen in accept() like validation, filtering the filename, etc than just moving the file. And who's to say you're really going to move the file at all - maybe the file is just processed and discarded.

For example (just brainstorming here) you could have accept() call validate() and then maybe filterFilename() and then move(). That way someone can subclass and provide their own move (e.g. FTP the files to a remote server), do their own filename mangling, etc.

I don't see you putting OOP to work. You're just wrapping an existing procedure.

Mike

So, just because I did not write out the filter, validator and maybe protocol subclasses you say that this proposal is useless ?
And just because the code is not working you think that it would not have advantages for the user ?

Sorry, but this is nonsense... I saw much people coding such a class and I was told from several other people that such a functionality is a must for this framework.

Of course I can also cancel this whole proposal, and people would still be frustrated that there is no standardised way and they would have to code their own class as before.

View the rest of this thread. Most recent comment: Dec 19, 2007
2 more comments by: Michael B Allen, Thomas Weidner

Is there any way to override how the filename is filtered?

Perhaps you should have a filterFilename() method? It could have a default behavior but also allow a subclass to redefine it's bahavior. For example, files may be stored on the filesystem by MD5 hash whereas the completely unfiltered name is stored in a database. That guarantees that no funky characters can obstruct any code that might operate on those files.

Or perhaps a callback would be better?

As already said several times in the past...
There is a callback implemented. It's not called filterFilename() it's called setCheck().

There is a callback implemented. It's not called filterFilename() it's called setCheck().

Ah, ok. It wasn't clear to me how setCheck could be used to change the filename.

Hi Thomas,

1. Looking at it again, I want to make sure that people aren't misled. This isn't a class for uploading anything; there's no upload() method. It's for handling uploaded files. Therefore, Zend_File_UploadHandler seems like the best name. Whether it's HTTP or FTP is irrelevant to the functionality, right?

2. It needs an addFileType() method.

3. setCheck() would be more in line with ZF terminology as setValidator(). Similarly, check() would then be validate().

Otherwise, looks good.

I agree with Matt, Zend_File is where you'd expect to find it. I'm not 100% sure about mixing all those classes: CookieJar.php, Cookie.php, Response.php and Upload.php in the same directory. It's confusing. And like Matt said, I'm not sure about Zend_Upload either, it's a symbolic name, but doesn't represent the purpose of the component.

Posted by fc at Dec 19, 2007 12:57

We already changed the name two times in the past.
We should come to an conclusion related to the naming.

Zend_Upload
Zend_File_Upload
Zend_File_Uploader
Zend_Http_Upload
Zend_Http_Uploader
...

Zend_File_UploadHandler... I don't know...

Btw: File would us not be related to Http, so we could also add other protocols like Ftp for example.
We could also add functionality for creating the proper form elements. A related View Helper or something. They could work closed together.

I guess I'm not sure how this directly interacts with HTTP or FTP. It seems like it just picks up when whatever transfer method was used is complete. Can you explain a bit how you see this tying into those? Even if that's the case, it seems like it might be something where it would rely on Zend_Http and an inevitable Zend_Ftp class.

"Zend_File_UploadHandler" is somewhat unwieldy, but it's the most accurate description of its purpose.

Btw: What would a upload() function do ? Maybe we can add such functionality if it's API conform ?

That's exactly my point: there's nothing for that function to do because this class is not handling file uploads, it's handling what happens after the file is transferred from a client to a server.

View the rest of this thread. Most recent comment: Dec 20, 2007
2 more comments by: Laurent Melmoux, Thomas Weidner

1. I just remember your second reply to this proposal...

  • I anticipate other Zend_File-related classes

And in my opinion Zend_File_UploadHandler is a little bit unhandy in it's naming...
Btw: What would a upload() function do ? Maybe we can add such functionality if it's API conform ?

2. Of course you're right... there are several other functionalities which will need additional methods or subclasses like the standard file validators.

3. Good point with validator.
I am not full closed with the API naming... the method names are not fixed and I think they will slightly change if needed when I implement it. There will be a amount of time where we are able to fix this...

To be sure... this class will not be implemented within the next release... it will stay in the incubator until we cleared all things which will cost us several weeks.
As always I am open to any idea which makes sense.

I think this class should be top priority. I mean, this is something we all use on a daily basis. And the functionality of uploading files is a standard requirement in almost every project.

You have a solid API, so it would be nice if this class is given top priority, so you can focus 100% on developing it.

Posted by fc at Dec 19, 2007 14:21

Not all people are thinking this way.

Maybe that's the reason why, until now, this proposal was not read, commented and accepted.

I will change the API as discussed before. Some things I had already in mind, but I was too lazy to write down the whole thing. Give me 1-2 days to change the API and then we can go further...

I will send an notification when I've finished the rework.

Nice one Thomas (that was quick), I'll take this new API as your Christmas present. Looks great

Posted by fc at Dec 19, 2007 18:19

Thomas, the reason you're getting so much push-back and confusion about this proposal is because it's not clear what you intend to do with it.

Why don't you flesh out your protocol adapters and show some use cases of how you intend to use them? Until then, I don't see any reason for them to exist over a separate Zend_Ftp class with its own Zend_Ftp_Client, etc. I also can't tell if the class should be called Zend_File_Uploader (with an -er) or Zend_File_UploadHandler.

Okay, so now we've got a better idea of what you intend for the class.

This class understands HTTP POST and PUT requests. The client is sending a file to the server, and the server (via this class) is handling the file--in which case, I think Zend_File_UploadHandler is the most accurate name for the class, because it leaves little room for confusion. It's handling uploaded files, not doing the uploading itself; the client and the web server do that.

But then there's FTP in the last use case, and you're providing connection information. Huh? In this case, either you mean:

a) The server is acting as a client and sending files to a remote server (in which case, it is uploading files, not handling uploaded files), or
b) The server is connecting to a remote server and initiating a request to receive files (in which case, it is downloading files)

Do you see what I'm getting at? In both cases, the function of the class is in question because it's doing two completely different things. If you're wanting a class to do both uploading and downloading, you might call it Zend_File_Transfer. Build it on Zend_Http and coerce someone to create Zend_Ftp. You'd probably want your methods named things like send() and receive() in that case.

For HTTP PUT you have to do the downloading... the Client and the Web server do not download the files. It's quite the same as FTP... you just don't have to give a connection information.

I think it's a question of direction...
UPLOAD means in my eyes that you want to GET files onto your server from the client. Direction is Client to Server...

DOWNLOAD means in my eyes that you want to send files from your server to the client.
Direction is Server to Client...
This is a own proposal, where I want to add wrapper technology so the client does not know where the file originally was located for example.

It's always the point of view. This terms are normally seen from the client, even if the server act's on them. From the server's view all is switched of course.

We could also integrate both ways into one class, Downloading and Uploading, and create a Zend_File_Transfer. But I don't know if that is wished and until now I didn't have thought of pro's and con's of this. Also if I would wait for someone creating Zend_Ftp I would be waiting forever as with the environment defaults for ZF which I had integrated into Zend_Locale and the other propsal was not done. I would not want to do this once more as it's irritating to the user.

So the general question now is:
Do we want to have a generic class handling both, upload and download or do we need/want to seperate them.

Having both in one class would of course have impact on the API but on the other hand several methods could be used together which we would benefit from.

Hey Thomas, it's recommended not to use method names like process(), they are to generic.

Here are some naming guidelines from Jeff Moore...

  • Keep names pronounceable.
  • Abbreviate consistently. Don't abbreviate to save only one character.
  • 8 to 20 characters is best. Global and rarely used names should be longer.
  • avoid names with similar meanings, that sound similar, that are different by only one or two characters, that use numerals, or are intentionally misspelled to be shorter. (Hilite vs. Hilight)
  • Use opposites. (show & hide, open & close, insert & delete)
  • A variable or type name should refer to a real world problem rather than a programming language solution, should fully and accurately describe what the variable represents and should express what, not how.
  • Put computed qualifiers at the end of a variable name. (Total, Sum, Count)
  • Use meaningful control variable names (not i, j, or k) if the function is more than a couple lines long or the loop is nested.
  • Use meaningful names for temporary variables (SalesTotal instead of nTemp).
  • Boolean variable names should imply true or false. Avoid using not in the variable name. Prefix with is or has.
  • Use strong verbs in function names. Functions should describe their result. The name should describe everything the routine does. Consider breaking up the routine if this is not possible.
  • Avoid generic or wishy-washy verbs in function names (handle or process).
Posted by fc at Dec 24, 2007 11:59

Also, shouldn't Zend_File_Uploader_Protocol be renamed to Zend_File_Uploader_Protocol_Abstract? If I'm not wrong, interfaces ended with *_Interface and abstract classes with *_Abstract in ZF.

I still think this class should be top priority, I know that the dev-team is prioritizing Zend_Service_* components, but a lot of developers out there keep asking how come ZF doesn't ship with the basic components for web development, such as Zend_File_Uploader, Zend_File_Utility, Zend_Image and Zend_Ftp.

Posted by fc at Dec 24, 2007 12:06

There is no guideline for Abstract within ZF.
This is only true for Interfaces.

Also, what you see within this proposal is not the code but only a visual interface for how the implementation should work. I am sure it will change a little bit through the implementation itself.

Re: naming abstract classes "Abstract"

Maybe not, but it's pretty standard practice, right?

I don't know Jeff Moore...
Actually the naming guidelines do not recomment the above rules nor have I read them before within the ZF.

And if all I am not allowed to use any name which should I use then instead ?
upload - not allowed
process - not allowed
move - not allowed
download - not allowed
start - not allowed
go - not allowed

I really don't know any name to use instead...
Also to mention that the API is not fleshed in stone. It will change a little bit when we realize the implementation.
So the only benefit of having all negotated is that this class will not be accepted

Just a word about security upload rules throught HTTP POST enctyped as multipart/form-data, anyone should read this very interesting document : http://www.scanit.be/uploads/php-file-upload.pdf

Ilia Alshanetsky also demonstrates others methods to taint all the $_FILES array in the "PHPArchitect's Guide to PHP Security" book.

 PECL fileinfo extension is to be a good native C solution : it should be merged in PHP 5.3's core

So you propose to use the PECL fileinfo extension instead of creating a generic standardized uploader class...

2 Negs:
First: The minimum requirements are 5.1.4.
Second: Until now the extension is not in the core and until then everyone would have to cook his own soup as today.

Target of this proposal is to give a generic and standard way of file up-/downloading.
And I think most of ZF-users would not buy this book you mentioned just for downloading their files to make their own class.

Hy interested ones,

I reworked the API as requested and changed the name to Zend_File_Transfer.
Generally the class can now handle

  • File uploads
  • File downloads
  • File validations
  • File filters, to change the content
  • Work with different adapters like http, webdav or ajax. Could be extended in future

Waiting for approvment or another change request...

If this proposal includes file downloads it should cover http forced downloads, which allow a PHP script to send a file without revealing its actual directory and allows placing files in directories that are not directly accessible. This allows sending file based on permissions and other security benefits like preventing direct linking and so forth.
Documentation on performing this caveat is rather sparse on the web, but it basically involves sending the correct http headers.

Something like:

if(is_file($file)){
header("Pragma: public");
header("Expires: 0");
header("Cache-Control: must-revalidate, post-check=0, pre-check=0");
header("Cache-Control: private",false);
header("Content-type: application/force-download");
header("Content-Transfer-Encoding: Binary");
header("Content-length: ".filesize($file));
header("Content-disposition: attachment; filename=\"".basename($file)."\"");
readfile("$file");
}

Please read the proposal carefully...

This is already covered as downloading files works ALWAYS as Wrapper describes as Wrapper for Downloads. This means files must only be accessable by the php process but not to the public.

Due to my own stupidity I forgot to use the wiki tags.

Too complicated for normal users...

We will integrate a way to use a default set of headers and/or to set them via defined methods. Users should never set headers manually because most users don't know which header to set for downloads so they work properly.

We should make this as simple as possible.

Of course not, you missed my point - This type of implementation should be abstracted by the File_Transfer class so normal users can have this functionality without knowing the specifics. I was just suggesting an implementation of the forced download method.

I didn't see such functionality in the class skeletons provided, and their documentation isn't overly descriptive... I didn't understand your reference to Wrapper, and a text search only revealed an instance of the wording in one the use cases.

|Too complicated for normal users...

Of course not

Sorry, but in my opinion many users dont know the exact headers to use... so your code is too complicated for many users.

This type of implementation should be abstracted by the File_Transfer class

It will not be abstracted by the transfer class itself, but within the specific adapter. Otherwise we could not have it seperated by protocol.

I was just suggesting an implementation of the forced download method.

How could we implement this simpler as with only one send() command ??

I didn't see such functionality in the class skeletons provided

A small description in the usecase for FTP download... would be implemented quite the same with http... only headers would be set automatically, the usecase it not right at this point.

A wrapper is a code which hides what is done behind it... the wrapper has access to the file, he sends it to the user... this means the wrapper acts as download portal, and the users must not have access to the file he wants to download. This way you can also verify if the user has the right for this file by acl or a similar solution.

Anyhow... if I have to document all things which can be done with this class we will have it ready for ZF 4.0 and this proposal will have 20 pages useless documentation.
There is no proposal where ALL things are written. I waited now almost one year and now I am near at forgetting this proposal and delete it because until now I only got negative responses and no positive ones. That's really frustrating.

View the rest of this thread. Most recent comment: Jan 28, 2008
6 more comments by: Eran Galperin, Richard Vítek, Thomas Weidner, Matthew Ratzloff

Hi,

I think defining multiple file types should be made in a different way, because in my opinion something like 'jpg|pdf' isn't very cute.

Maybe this should be made trough some classes or at least a multi dimensional array.

Maybe it should be also be possible to exclude just some kind of files and allowing all others.

As stated within the proposal there will be "Validators".

And they are extendable... this means you can also write your own or extend existing validators to fit your needs.

One of these validators will be used to validate if a fileextension fit's or does not... another could be to exclude files or extensions. This will grow in future depending on needs and usefullness.

But the whole proposal is not accepted for now and I will not make implementation details without the basics accepted.

There's "too many things" going on with this proposal.

a) There should be a Zend_File_Collection / or Zend_Files which implements a Collection

$col = new Zend_Files;
$col->addFile('foo.img');
$col->filter(...);
$col->etc....

b) You can then upload this collection of files (through some adapter)

$tr = new Zend_File_Transfer; // Uses some default adapter
$tr->setTransport('WEBDAV', array(..options..));
$tr->add('foo.txt', $col); // Can add a single file or collection of files...
$tr->put('somePath/');

...

The filtering and validation should exist with the 'collection of data' - ideally similar to java or .net

There's "too many things" going on with this proposal.

To provide a generic way for transferring files there is much work to do...
So what's your clue ? To erase the proposal because it aims to be too generic ?

a) There should be a Zend_File_Collection / or Zend_Files which implements a Collection
b) You can then upload this collection of files (through some adapter)

This is not part of this proposal and will not be implemented... but it could be a feature enhencement or when it really has to be a new class then it must be a new proposal.

ideally similar to java or .net

We are not building java or net... we are collecting ideas from all over and try to make it simpler and possibly better.

Hi Thomas,

Could you please tell me what the current status of this class is?
I thought I could locate it in the Incubator, but I haven't found it.

I second the opinion that this class should get a higher priority at the "decision board"

Best regards,

Patrick

As you can see this proposal is still under review by the dev team.
It is not allowed to commit not accepted classes in the incubator.

As long as there is no decision that the API is ok, or that there are changes necessary to the API recommended by the dev team, I am not able to do any work. It would not be a good decision to do some coding and then throw all work away because the API is not accepted.

So we both have to wait some more...
I am sure that with the release of 1.5 (or the next RC) the dev team has little more time to do futher reviews as for now there are much proposals which have to be reviewed in the queue.

Greetings
Thomas, I18N Team Leader

I also think the class should be included rapidly. Imho Zend_Form will be much useful if it will be possible to manage file upload at the same way as textareas and inputs, without third party classes.

Hi Thomas,

Thanks for putting together this proposal. I think this is a pretty useful component, and it's looking great so far!

I'd like to ask whether it's planned to support files array notation in this component?

Zend_Form supports array notation, so I'd imagine that if it would rely on Zend_File_Transfer for file uploading (something I believe Zend_Form's author is planning) then support for array notation in Zend_File_Transfer would be a must. Also, unfortunately, PHP's array notation for $_FILES is a bit weird (different than that of $_POST at least. Ref: http://www.php.net/manual/en/features.file-upload.multiple.php), so it would make for a great potential area to improve.

Thanks, Thomas!

Kind Regards,

Yes, Zend_Form will be coupled with Zend_File_Transfer for file uploads and his name is Matthew

And yes, array notation will be supported. As you see, you can add files with the addFiles method which takes an array of any notation. It will support $_FILES, $_POST of defined structure and also sort of self defined array. And of course there will be a way of automatism, so it will handle any detected source of files automatically.

Usability and Simplicity are a must criteria for me and ZF. So you can be sure that it will be improving the actual way of PHP.

Zend Comments

The Zend Framework team approves this component for immediate development in the standard incubator, with the following suggestions:

  • Consider extending Zend_Form_Element. There is a lot of overlap with Zend_Form_Element, particularly in terms of using validation and filter chains and use of isValid(). Our recommendation is that it extend Zend_Form_Element; that way it can be used both standalone and as an element. A targetted Zend_Form_Element_File could simply extend it to ensure it can be found by Zend_Form automatically.
  • Ajax Uploads. These could be done via a decorator, if Zend_Form_Element is used as a base class.
  • Validators. We recommend building some file-specific validators for checking extensions, mime type, file size, upload status, etc. These could then be used for other purposes outside the Zend_File_Transfer component.

For me the proposal code does contains too many "magic strings". Replacing them with class-constants or object would make the code more secure against programmer faults, at least when they do use an IDE.

What do you mean with "magic strings"? Actually there are no magic strings at all.
Have you looked at the actual implementation?
Have you looked in the working examples I already provided in my blog?
I would not know how to make this simpler as it's actually implemented.

If you see any problems it would be better if you give a detailed description on the problems you see with the actual implementation.

This is no critics on you, it's just one thing I would do different in code so I suggested it and I might even agree if you have different opinion. But I might be a bit more specific as "magic strings" seems to be no common known concept. "magic strings" are the same thing to strings as "magic numbers" are to numberic values.

"Magic Strings" are constants represented trough a string, this makes the code harder to use because the "Magic Strings" are hardly to document, a type error can happen very fast and code complition of a IDE can not help.

Some Ideas how you could remove the Magic Strings from the examples above (but I know this suggestion does come very late, as the code is allready finished):

  • $files->addValidator(new Zend_Validator_MaxSize(2000)); instead of $files->addValidator('MaxSize', 2000);
  • $files = new Zend_File_Transfer(Zend_File_Transfer::WEBDAV, array('user' => 'adam''pwd' => 'sandler''server' => 'myserver.com'));  instead of $files = new Zend_File_Transfer('WEBDAV', array('user' => 'adam''pwd' => 'sandler''server' => 'myserver.com'));

I havn't seen this as critism, but it's always better to discuss with examples when people don't have the same motherlanguage.

What you see as "Magic String" is not implemented as you may expect.
Let's look at the validators... there are no constants or other variables declared. When you give a string, the class looks if there is a proper validator available... so if you give 'Size' it will look for 'Zend/Validate/File/Size'... if not found it will try to load Size as it is, maybe with autoloader or whatever you may have declared as include path. So to be clear... there is no constant defined. So if you try to load 'MyValidators_File_Size' it will also be loaded.

You said that you want to solve this a different way... well, which way would be the right one in your eyes ?

You can still initiate the validators manually your own but you will then have more overhead to do yourself. And you can use the validator class also in combination with the adapter directly. But there is no bonus with this way, except that you will have to write more...
new Zend_Validator_File_Size instead of 'Size'.

Maybe you should really first take a look at the existing code.

Please keep in mind that anything shown in this proposal is just a point of proove and no finished implementation. Because of this details which become clear as soon as we have working code, the end-version behaves slightly different.

I tend always to keep things as simple as possible for users.
Btw: You are declaring two different styles in your two example lines...
First you say to use class declaration... method(new class(...))...
And second you say to use class constants, which you declare are "magic strings", and which shall not be used ??? For me you negotate yourself.

So to get to the point:
Things are much simpler as you may expect... there are no "magic strings", as you declare them, implemented. You can eighter wait until the official implementation is ready to see this yourself, or look at the actual code and see yourself that all works like you wrote it should. Or you simply wait for 1.6RC2 to be available.

View the rest of this thread. Most recent comment: Jul 29, 2008
2 more comments by: Leo Büttiker, Matthew Weier O'Phinney

You need to do some reading on Zend_Loader_PluginLoader. The purpose of using short strings when passed to the validators and filters is to allow the developer the ability to define their own replacements for them. The PluginLoader will only attempt to load classes based on the prefixes registered with it – so, by default, these will only be within the standard defined validators and filters. They are not constants in any way.

The PluginLoader is being used in a variety of places within ZF, including Zend_View and Zend_Form – for the exact same purpose.

This is to save on typing. Zend_Db does this as well for the same reason.

Good morning every single one!

I have the need to download some files via FTP. Zend Framework offers File/Transfer/Adapter/Http which seems a bit not-working for my ftp-protocol.

Since here did none post something for about one year - I want to discuss the need (at least mine) to have a ftp-adapter.

I would suggest(and try) to copy the Http.php to Ftp.php and see how far I can implement a slightly more ftp-like functionality.

After some quick research I figured out, that at least a function "get file listing" is not yet available in the abstract adapter - and has to be implemented "somehow" "somewehere" (in the Ftp class?)

So - feel free to join the discussion about this, or if possible send me your Ftp-adapter

It looks like this has stagnated a bit - the manual states that:

Note: Limitation
The current implementation of Zend_File_Transfer is limited to HTTP Post Uploads. Other adapters supporting downloads and other protocols will be added in future releases.

Is there any work on the downloader?

There were many changes and additions which are used by all elements.

Additionally I worked on a PUT extension and made some pre-works and definitions for WEBDAV and FTP. But these are not official and therefor not available to public for now.

Additionally Downloads can already be done (limited to http) by using Zend_Http_Client.