Added by Wil Sinclair, last edited by Matthew Weier O'Phinney on Aug 20, 2008  (view change) show comment

Labels

 
(None)

Zend Framework: Zend Framework Default Project Structure Component Proposal

Proposed Component Name Zend Framework Default Project Structure
Developer Notes http://framework.zend.com/wiki/display/ZFDEV/Zend Framework Default Project Structure
Proposers Wil Sinclair
Liaison: Ralph Schindler
Revision 0.1 - January 21, 2008: First draft.
1.0 - January 30, 2008: Final draft for review. (wiki revision: 14)

Table of Contents

1. Overview

This proposals covers the Zend Framework default project structure. This is the project structure generated when no project profile is specified during project skeleton creation. It will also serve as our recommended project structure for basic projects that do not call for more specialized project organization and in environments where the developer has appropriate file system/web server admin privileges.

2. References

3. Component Requirements, Constraints, and Acceptance Criteria

  • This project structure must support all project requirement for core components in an environment where the developer has appropriate file system and web server admin privileges.
  • This project structure will not support all project requirements or constrained environments where, for example, the web server document root can not be configured.
  • This project structure must reflect all PHP application security best practices, including minimizing code under the web server document root.
  • This project structure will require that url rewrite is enabled on the web server to work properly and properly secure the application.
  • This project structure must be extensible/expandable to handle new requirements.
  • This project structure should minimize the complexity of the project directory for the given project. For example, if the project does not use Zend_Search_Lucene, an index folder should not be created.
  • This project structure must allow modules to be added to a non-modular application without having to manually update the project directory structure.
  • This project structure should be consistent in best-practices recommendations with the project structures that have been documented in the reference guide.

4. Dependencies on Other Framework Components

  • Zend_Build

5. Theory of Operation

Many developers seek guidance on the best project structure for a Zend Framework project in a relatively flexible environment. A 'flexible' environment is one in which the developer can manipulate their file systems and web server configurations as needed to achieve the most ideal project structure to run and secure their application. The default project structure will assume that the developer has such flexibility at their disposal.
The following file/directory structure is designed to be maximally extensible for complex projects, while providing a simple subset of folder and files for project with simpler requirements. This structure also works without alteration for both modular and non-modular ZF applications. The .htaccess files require url-rewrite functionality in the web server as described on in this Rewrite Configuration Guide.
It is not the intention that this project structure will support all possible ZF project requirements. The default project profile used by Zend_Build will reflect this project structure, but application with requirements that this structure does not facilitate should use an alternative- possibly custom- project profile. The default project structure is provided below along with its corresponding base project profile.

6. Milestones / Tasks

  • Milestone 1: Gather community feedback and finalize project structure.
  • Milestone 2: Complete work for the Zend_Build proposal.
  • Milestone 3: Implement project profile for this project structure and check in to SVN.
  • Milestone 4: Document the project structure in reference guide.

7. Class Index

No classes are required to implement this proposal. Please see Zend_Build for the required classes.

8. Use Cases

These use cases will necessarily be very high level.
UC-01

The users creates a project without specifying a project profile.
The system finds the default project profile and creates the components of the project structure below necessay for a basic MVC ZF application.

UC-02

The users creates an index for Zend_Search_Lucene in his/her existing project.
The system creates the required index directory for the Lucene indexes and creates necessary files to facilitate index building.

9. Class Skeletons

The following reflects the file system structure of the proposed default project structure:

Justifications for the above.


application/ - This directory is the meat and bones of what you are developing. It will house the MVC system, as well as conifgurations, api's used, and your bootstrap file.

apis/ - This directory is for your application specific web-service api files that are consumed and/or provided by your application (through a controller presumabely).

config/ - The application wide configuration directory

controllers/models/views/ - These directories serve as the 'default' controller/model/view directories. Having these three things inside the application directory provides the best layout for starting a simple project as well as starting a "modular" project that has a concept of "global" controllers/models/views.

controllers/helpers/ - These directories will contain action helpers. Action helpers will be namespaced either as "Controller_Helper_" for the default module or "<Module>Controller_Helper" in other modules.

data/ - The goal of this directory is to provide a place to store application data that is volatile and rumtime. The interruption of some data inside this directory might cause the application to fail on any one request. Also, the information in this directory may or may not make its way from/into a subversion repository. Examples of things contained in this directory are session files, cache files, sqlite databases, logs and indexes to name a few.

layouts/ - This layout directory is for MCV based layouts. Since Zend_Layout is capable of MVC and non-mvc layouts, this location choice for this directory signifies that layouts are not on a 1-to-1 relationship with controllers and are independent of templates within views/.

bootstrap.php - This file represents the entry point into your application. This first could be procedural php or could contain a Bootstrap class. Either way, the purpose for this file is to bootstrap the application – not PHP settings – and make components aware of their comsumption by starting or configuring them. This file would not call dispatch() on the front controller.

modules/ - The idea of modules allows a developer to group a set of related controllers (that might be growing to be too large for any one controller repository) into a logically orgainized group. The sub structure of the modules directory would mimic the entire structure of the application directory minus the bootstrap.php file.
docs/ - This directory exists for documentation, either generated (think PHPDoc) or hand written

library/ - This directory houses the library/Zend/* folder which is the Zend Framework. In theory, you could also use other include_path-able not named Zend/* in here

scripts/ - This directory is intended for maintenance, and/or build scripts. Typically, these might include command line, cron, or phing build scripts that are executed independently of the web application itself.

temp/ - The temp folder (typically set in php.ini as /tmp on unix), is set aside for non- or semi- volatile application data. Roughly speaking, the contents of temp would typically be generated by an application and used during its lifetime of requests. This information (typically), would not be seen inside the applications svn repository. Also, it should be noted that usually, if temp/ were cleared out, the application should be able to continue running without much of a hiccup.

tests/ - This directory serves as a purpose for containing application tests. These could be hand written, phpunit tests, selenium based tests or (insert your testing framework here). By default, library code could be tested my mimicing the directory structure of your library/ directory. Additionally, functional tests for your application could be written mimicing the application/ directory structure (including the application subdirectory). This directory would contain a PHPUnit configuration file to assure all functional tests would work correctly.

public/ - This directory would contain all public files for your application, including index.php. index.php would setup the PHP environment, include the bootstrap.php file from your application/ directory, and finally dispatch() the front controller.

This project structure would be specified in the following project profile:

Please note that the user would not have to create or modify this project profile unless he or she would like to use a custom project profile in Zend Framework tools such as the CLI. This project structure will also server as our default recommendation even if the user is not using Zend_Build.

Nice

We could add ?:

I would put layout/ under application/

I agree with you on these.

It's important to note that while these directory locations are specified in the project layout, unused items dont need to exist. For exmaple, if you application doesnt use web-services, the api's directory is not needed. Same with modules, and modules/config.

Hi Wil, I have a few feedbacks:

1. I would move /config from /application folder to root.
It seems more appropriate because there are some project wide configuration that not necessary could find a logic place under /application.

2. What about using /task instead of /jobs?

3. What is the meaning/purpose of /extras and /application/apis?

I am not sure I agree on the config being at the project level, considering everything inside the /config directory applies to things within the application directory.

Can you supply a use case that supports an instance of something that exists outside the application directory that needs configuring?

For instance, Rails and Symfony usually stores in /root/config database schema, /vendor settings (propel settings for Symfony) and CLI overall configuration.

/root/config might include as well project constants or environment files.

I like the idea, however some things are missing:

  • vendor folder, used for abstractions of the Zf; Eg ProjectName_Controller_Action (i suggest to put it in /vendor)
  • folder for build scripts (i suggest /<project-root>/build-tools)
  • i'd personally rename the "config" folder to "etc"
  • the <module name> section should also have an extra folder "locale" for locales per module

vendor folder, used for abstractions of the Zf; Eg ProjectName_Controller_Action (i suggest to put it in /vendor)

I agree with you.
In my previous comment I requested the purpose of /extras folder just because I guess /extras does what /vendor should do.
I agree with /vendor.

folder for build scripts (i suggest /<project-root>/build-tools)

Same as before.
I suppose Wil called this folder /jobs.

I would suggest to extend this proposal with a short description of each folder purpose.
Many of us already has experience with other MVC frameworks and folder name without a small explanation might cause a kind of misunderstanding.

Indeed, 'extras' is our version of 'vendor'. I would need to check with some business folks about the naming since Zend has true 'vendors' and we wouldn't want to cause confusion with ZF partners.

In that case, how does the idea of a "vendor" plug into the FrontController system, specifically with respect to the current module paradigm?

Also, wouldn't locale data be site wide? Does a good place for locales not seem like it would be data/locales?

Hi Wil,

My own comments!

1. The proposal suggests a default directory structure, but doesn't offer any justification for the decision on a case by case basis. From a feedback point of view - how can commentors debate the unjustified?

2. What are extras, apis?

3. config should be present in root if we're allowing for other top level directories besides public.

4. bootstrap.php I suggest would be better as Bootstrap.php reflecting the PEAR convention to a degree since it likely contains a class.

5. The proposal doesn't note the default content for bootstrap.php, index.php, and .htaccess. No doubt out of this proposal's context but I think the goal should be to generate an initial structure which responds almost immediately to requests (afterall who cares where it's put on a devel server). Over to Zend_Build then...

6. My favourite question: What can be removed as unnecessary? My thoughts are with apis, extras, data, jobs

7. I think either more should be moved to application, or move more to the root. Leave me less navigation clicks through PDT

8. Where is my tmp directory for sessions and other temporary data ops?

Best,
Paddy

2. What are extras, apis?

I agree, this needs to be explained.

4. bootstrap.php I suggest would be better as Bootstrap.php reflecting the PEAR convention to a degree since it likely contains a class.

+1

8. Where is my tmp directory for sessions and other temporary data ops?

In my first comment I proposed to rename "data/" to "var/", and "tmp/" could either go in the project's root, or (to follow the naming conventions on my Linux/Unix systems) "var/tmp/".

Wil,

I 2 pretty imporant ones: tests, *tmp. I suggest to put this one in <project-root>/tests, <project-root>/tmp.

I also wrote a short description once on folder names (not all of them are covered yet), but its a start:

The controllers/ directory
The controllers/ directory contains all the controllers for the application.

The etc/ directory
Configuration and database schema files will go here.

The jobs/ directory
Cron jobs, database schema migration scripts or import scripts to be written. These are run using the PHP CLI executable and are stored here.

The models/ directory
Holds model classes, some of which are derived from database abstraction layer

The scripts/ directory
This contains a sub-folder for each controller (named after the controller) and also some layout template files. Within each sub-directory there is a template file named after the action and possibly some additional helper templates which I prefix with 2 underscores.

The application.php/bootstrap.php file
This file will contain all the different stages of the application, such as loading configuration data, setting up a database connection, etc.

The library/ Directory
Any third party library is stored in here. Most of the time, this directory is maintained by svn:externals.

The public_html/ Directory
This one is pretty obvious I think ...

The tests/ directory
Unit tests (mostly for models) will go here.

The tmp/ directory
This is the base folder for all sort of files that can be deleted! Anything transient is stored here.

I'd rename "data/" to "var/".

"config/" should be in the project root, and also renamed to "etc/".

For the "document_root", I'm also voting for "public_html/", or "htdocs/".

Reason for renaming - all of them are (more) well known, and since most people will run on Linux/Unix and most likely have Apache it fits a scheme they are used to.

What I am personally missing in this proposal is support for different instances of an application.

Let's say the application is a content management system that serves several sites. Each site is administered by a different person. So there have to be at least several "public"-directories. But what if I want to separate the cache, the data directory etc. per site?

Imagine having to give the site admins ftp-access to their site-directories, where they can upload images, templates, data etc, but preventing them from breaking the application or accessing some other peoples' site.

Maybe this is too advanced a request for this proposal, but I am already working on two projects with multiple instances so this might be worthwile to consider. It makes things a heck more complicated though - i.e. if some sites have access to certain modules that others don't have or you support custom modules that clients develop themselves and plug into my application.

This seems like a good candidate for a custom project profile. It is a use case I hadn't considered up to this point, so I'll have to think about this WRT Zend_Build, but it certainly is not common enough to complicate the default project structure, which must be as simple as possible for most projects.

Haha, I was in the middle of building a Python app that would do the same thing.

I think this is an excellent idea and please stick with XML for building the project profile, primarily because my apps use a very different structure than the default one proposed and I would like it to be non-troublesome for me to edit the profile accordingly.

I don't want to double post, but, here is my directory structure for those that might find it useful. This is just an example of what mine looks like - I use XSLT transforms instead of .phtml views to handle rendering of the page; but adaptation shouldn't be an issue if you find anything in this that seems suitable.

Support for Yaml wouldn't hurt either. Since it's pretty "hip".

Reviews, and lack of Review updates to Proposals aside, from the "review" I received from someone at Zend in autumn YAML support is not being currently considered. The reasoning was basically use ext/syck from PECL and wait for it to support YAML 1.1. I was a bit busy at the time so I don't recall all the details, suggested a lighter wrapper for the syck-less, and went back to my overtime and stress running into December .

So effectively not going to happen.

Well, I wouldn't say that we have it in for YAML. But we are considering not doing it as Zend_Config data at all because of the limitations that come along with it. Ultimately XML is a very good match as a data format for project profiles: the data is highly structured and hierarchical, it can be read in most runtime environments if anyone wants to create their own tools in other languages (I know this is becoming the case with YAML, but I think XML still has an edge here), and it's not going to be transferred over the wire so size isn't much of a concern.
Ultimately, the YAML issue is orthogonal to this debate, and we probably have enough to consider already without opening up the whole YAML issue. Let's assume that project profiles will definitely support XML, and possibly support all formats supported by Zend_Config.

Just reiterating - having a directory for jobs/tasks/build. Does it meet the 80/20 needs level only? I seriously doubt it. I really think the default should be minimal with most of the application level directories moved to root (or vice versa is application specific). If we're putting in everything plus the kitchen sink we're far more likely to thread on toes where people end up with a billion directories they either never ever use, or whose naming they inevitably disagree on. If it's not essential to a new application, or is not inevitable of a new application, it probably isn't needed...

If you look at this directory structure as placeholders, wouldn't be as minimal or maximum as your application needs?

For example, if an application does not consume or supply any web-services, it seems like to me that the api's directory simply wouldn't be there until its needed. Like wise with data/sessions modules/ etc.

Its not an all or nothing per-se, but when a feature is used, the cli tool would create the necessary directory/directories based on need.

  • On layout directory:
    • I too think ./layouts/ should be under application. The reason is that Zend_Layout could not only support global site layouts (which i think is what you are referring to here), but also module specific layouts.
    • Think in terms of a situation where you have dropped in a blog MODULE that has a blog layout (think buttons for creating new posts, editing, controls, etc) and it in turn needs to be part of a much larger global layout. Zend_Layout supports this notion, which is why it probably would fit better in ./application/.
  • On public:
    • Do other names make more sense here? httpdocs seems pretty common across the board.
  • On bootstrap.php
    • I am on the fence for the necessity for a bootstrap.php file. Do we have a standard interface for this? What does it look like.
    • In previous projects httpdocs/index.php has sufficed as the bootstrap file, with the reasoning that your main(..) executable code is essentially boostrapping code.
    • With the addition of a bootstrap.php file, our index.php file essentially looks like this

I agree on the need for app level layouts.

I think public is just fine. I've never seen a httpdocs directory before, and moving to htdocs as some have suggested is Apache oriented. Not everyone uses Apache remember?

We need a bootstrap file. Bootstrapping has gone beyond a procedural spaghetti exercise and needs to have a little structure in a class. It needs to be on an application level since it contains all the setup code, and not public, since public is likely in a completely different location at the end of the day to the app code (with obvious outs). Finally it's a simple index.php include of Bootstrap.php - in a simpler deployment scenario, index.php becomes the marginal symlink to different app versions with different Bootstrapping code.

There's also the need for grouping bootstraps with application code for testing - one possible example is using a Bootstrap class to split discrete bootstrapping steps into callable methods, where several can be reused by external tools. Obvious one springing to mind is bootstrapping a Front Controller via a callback method taken from Bootstrap - a bit simpler than going completely into homebrewed testing solutions with a lot of typing overhead. I'm sure someone more imaginative than I can find other uses...

On the whole I think it's pretty good. Like everyone here I have a slightly diverged structure - I'm not precious about it and I'm open to changing it, but here's a few differences/notes:-

  • Is there going to be a template specific to hosted environments that don't give write access to any other folder than httpdocs (yeah, Plesk, I'm talking about YOU!). Could thought be given to creationg a 'framework' or 'zf' folder with a ready-made .htaccess file (containing 'deny' directives)?
  • My experience is that the application folder works quite well with:-
    /application/default - contains 'default' module (controllers, views, config, layout, models)
    /application/module1 - same as above but developer-designed module
    /application/module2... etc
    Any reason why the 'default' module folders aren't similarly organised?
  • I'm all for a 'default' bootstrap that we can overwrite. To me it would look pretty similar in structure to an '.ini' file (in terms of having pre-defined examples for developers to uncomment and use as-is). This would be called from 'index.php' similarly to how Ralph describes it above.
  • I'm happy with 'httpdocs', 'public_html', 'document_root' or 'www' for placing all public files
  • Contrary to 'var' and 'etc' I'd prefer to stick with plain-english folder descriptions - simply because they require less explanation to non-Linux types.
  • I also agree that 'apis' should be optional (first time I read it I was a little confused)
  • I like 'tasks' better than 'jobs' (only because I'm a snob )

Will look forward to giving this a try-out!

Reading back I realise my minimalise comment was unnecessary since some of these directories are only created if required. Presumably than a default project, wouldn't have them. Any chance we can clarify the required from the optional in the directory snapshot?

Why not put all directorys under the application directory - since they are ALL application related! I do not see the benefit to put some directorys IN the application dir and others don't.

Either public/ directory outside the application directory or inside the application directory - in this example it is outside the application dir.
(For this example i copied Wils proposal directory structure and modified it - so do not care about the names)

Tiny comment to note that /views/layouts is not a current Zend_Layout default (defaults to having a layout.phtml in views/scripts). Might be worth leaving out a layout directory to avoid confusion. I've noticed the documentation for Zend_Layout appears to not mention this anywhere - something I'll report as a documentation improvement.

After read all the post, i can't figure out what is the prefered directory structure, for my point of view all dirs are application specific, and i use for long years a structure that i changed to make it as standard as posible. I think that familiar common names will help the people to identify easy the mean of a directory.

etc/   (Global application configuration, datasources, log, front controller config, etc.)
lib/   (Libraries required for the application)
bin/   (Scripts to run from the CLI or via crond)
docs/ (Application documentation specific)
tests/ (Tests for the application code)
var/    (Data that changes frequently) 
    cache/    (store cached resources here)
    tmp/    (temporal  store)
    log/    (directory to store the logs generated by the application)
modules/ (Bundles or plugins of the application)
   <module_name>/
      controllers/
      models/
      views/
      locale/
public/    (public content)
    images/
    js/
    css/
    .htaccess
    index.php

I think that some dirs can have mode deep structure, for example images under public can have a locale directory structure if images for diferent locales are required, some times certain modules need specific configuration this can go in their module directory, so it's decision of developer to create them.

Other directory that comes in place is build, that of cource will be a directory for development stage that will be left in final product.

I think that the best aproach is make the application always modular if the developer don't wnat modules or it's a simple application only one module will be created. 

Will be great that where any place to see the proposal structure updates based on the feedback so people can see the form that the structure are take. 

Though Thomas Fritz nearly breaks the subject, I didn't see anyone else ask the following directly:

How will this work with shared hosting, where the web server root is the only directory into which customers can upload files?

There's always room for community contributions with different setups. These could/should be easy plugable.

Imagine:
php init-project.php http://framework.zend.com/setup/modular-setup-for-shared-hosting.xml

Where for instance Zend could host a few standard setups, or make them downloadable.

With the recent addition of forms they too will need a place in the structure. I posted a question about where on Nabble (http://www.nabble.com/Zend_Form%3A-Directory-for-subclass%27d-forms--tc15812272s16154.html) and Matthew Weier O'Phinney said:

"We haven't decided on a location for forms in the directory hierarchy yet. My prediction would be in application/forms/, and prefix them with 'Form_'; in modules, they'd then be prefixed with '<Module>Form'."

What about inside models?

Models, generally speaking, are environment agnostic classes containing the business logic of your application.

While forms do speak towards the view and some controller and some model aspects. I can see the argument for keeping forms at the top level. We should revisit this after some best practices emerge with respect to forms and applications.

Also, I would see it as "<Module>Form" to keep it consistent with what people do with Models and Controller prefixing.

-ralph

Proposal has been updated to take into account all of the above comments as well as Wil's responses and further clarificaitons.

-ralph