Zend Framework has grown substantially and organically throughout the 1.X series of releases.
The result has been a framework that is immensely powerful, broadly appealing, and enormous in scope – but also one that is cumbersome and difficult to learn, distribute, and extend, and one that is becoming less and less performant. Additionally, since Zend Framework has experienced large adoption, this has led us to re-evaluate a number of design decisions – but the inability to break backwards compatibility has made it impossible or difficult to create better implementations.
The primary thrust of ZF 2.0 is to make a more consistent, well-documented product, improving developer productivity and runtime performance.
The following document serves as a narrative detailing the various goals of Zend Framework 2.0. We will be gradually filling in an implementation plan for each set of goals so that we can measure the success of the effort.
- Ease the learning curve
- Make extending the framework trivially simple
- Improve baseline performance of the framework
- Simplify maintenance of the framework for the core team and contributors
- Be an exemplar of PHP 5.3 usage
- Provide mechanisms for using just the parts of the framework needed
In late 2009, we did a survey of framework users to determine what they use,
what environments they use, and what their needs are. The top issue, bar none,
was the difficulty of learning the framework. Some of these issues include:
- Difficulty in the "first hour" with the framework. Many of these questions have been addressed in the revised "Quick Start" shipped with 1.10, but additional issues remain: how to setup vhosts on non-Apache web servers, how to follow the quick start without a dedicated vhost, and problems arising from differences in operating environments.
- Uncertainty about the "next steps" following the quick start. Developers are unsure how to go about using different components to create a cohesive application. Again, first steps at alleviating this concern have been taken with 1.10 with the "Learning Zend Framework" section of the manual. However, these are largely disparate tutorials, and don't show full development on a single application.
- Inconsistent APIs in the source code itself. One component may use "plugins," another "helpers," and yet another "filters." One component may utilize fluent interfaces, another not. One component may allow MixedCase plugin names, while another only allows Titlecased. One component may use a broker for accessing helpers, while another delegates to them via __call(). One component may allow camelCasedOptions, while another only allows underscore_separated. One component may allow passing an array of options only, while others allow passing an array or a Zend_Config object. The list can go on for quite some time.
- Uncertainty about where extension points exist, and how to program for them.
- Confusion over whether they can use Zend Framework only as an MVC stack or as individual components.
- MUST create two separate quick start tutorials: one that covers using individual components of Zend Framework, and another covering creation of a basic MVC application. In both cases, the tutorials:
- MUST explicitly indicate the expectations of developer knowledge. E.g., "developer should be familiar with PHP 5, OOP, and have a basic understanding of autoloading; additional, some knowledge of Apache and configuring virtual hosts is assumed."
- MUST have reproducible steps that, if followed, will return the same results. Minor variations for common operating system environments should be covered (e.g., differences between linux, mac, and windows environments).
- MUST clearly identify the next steps a developer should take, and link to related references.
- Development of a set of cohesive tutorials covering intermediate and advanced Zend Framework topics. To achieve this we:
- MUST identify common development problems for Zend Framework, and the most common patterns used to address them.
- MUST write a sample application that illustrates these problems and solves them using the patterns identified.
- MUST have a clear narrative showing step-by-step construction of the application, one problem/pattern at a time.
- The reference manual:
- MUST clearly state the assumptions and use cases for each component. As an example, the various url() helpers currently allow passing a named route as an option. However, many developers do not understand that if none is provided, then it will use the currently matched route. This leads to a difference between developer expectations and code expectations, which has led to a number of issues in the tracker. Simply documenting these, as well as concise use cases, should alleviate the issues.
- MUST have consistent formatting for each component, for predictability.
- SHOULD provide an overview
- SHOULD provide expected arguments and return values, as well as exception conditions
- SHOULD provide one or more usage examples
- COULD provide narrative, but does not need to
- Address the various API consistency issues:
- MUST standardize option casing
- MUST standardize language and APIs surrounding the various extension points
- SHOULD attempt to standardize APIs for passing options to components
- SHOULD attempt to identify and implement common API patterns throughout the framework
Extending Zend Framework is relatively easy already. However, there are a few notable cases where it is difficult, and some of these cases are found in many places in the framework.
- Singletons. Many singletons have a lot of behavior coded into them, making it difficult, if not impossible, to provide alternate implementations. Examples include Zend_Controller_Front, Zend_Auth, and Zend_Session.
- Use of abstract classes instead of interfaces. Many ZF components define rich abstract classes with a plethora of behaviors. While these may be extended and alternate implementations provided, doing so often requires accepting functionality you don't need, or needing to overwrite large sets of functionality you don't want. In many cases, the consuming classes only rely on a small subset of behaviors.
- Hard-coded dependencies. In many cases, components have hard-coded dependencies, making it impossible to alter behavior without extending the class to add your own dependency or to allow injecting the dpendency.
- Failure to identify processes that could benefit from collaborators. As an example, many of Zend_Db's methods could benefit from user extension – for instance, to allow checking for and writing to a cache, normalizing values prior to insertion, etc. In order to do this now, users must extend the existing classes, leading to difficult to traverse hierarchies and reducing re-usability. A plugin/filter system could alleviate these issues easily.
- MUST Eliminate singletons where possible; when not possible, push behavior into collaborating objects.
- MUST identify the essential interfaces for each component and subcomponent, and typehint on these instead of the existing abstract classes.
- MUST remove hard-code dependencies and allow for dependency injection.
- MUST examine each component and address the need for collaborators.
- SHOULD add a section to the documetnation for each component, showing in a consistent structure the component's support for extension, plugins, and hooks.
Baseline performance of Zend Framework applications has been getting worse with almost every release; even when gains are made, new processes added in new releases lead to degradation. While ZF started as a fairly nimble framework, it is now among the slower alternatives available, when using the supplied MVC stack.
The goal of ZF 2.0 is to identify causes of performance issues and implement code changes to mitigate them until ZF is competitive with any other PHP framework of similar capabilities.
- MUST improve the baseline performance by 200-300% over the 1.x series, and competitive with the more agile full-stack frameworks currently available, including Solar and Symfony 2.
- MUST provide deployment-time tools for optimizing performance in production
- MUST provide documentation, tutorials, and examples to help developers follow best practices for performance.
As of the time of this writing, Zend Framework consists of over 2 million lines of code, including over 14k unit tests. The internal team at Zend consists of a project lead and two engineers, while actual contibutors number in the dozens. Contributors come and go, largely based on their interest in specific components and the time they have to devote to the project. High turnover of maintainers and lack of availability from maintainers, both internally and amongst contributors, has led to many orphaned components and/or components with large numbers of issues filed against them.
Additionally, with the growing complexity of many of the components, often due to differing expectations between different end-users and the original designs, we often get internal conflicts within components, where resolving one issue or addressing one feature requests can spawn regressions for existing functionality.
Another barrier has been actually getting contributions into the framework. Currently, we only offer SVN access on a case-by-case basis, which leads to developers posting patches as comments or attachments to the issue tracker. Often, the best way to really evaluate these patches is to apply them and run unit tests – and this method of obtaining the patches is inefficient.
When it comes to writing patches, many developers are unsure how to do so, or where in the tree to generate the diffs. As a result, often the patches submitted need to be doctored in order to be usable. Subversion does not help here, as svn diff can be done at arbitrary levels, and will not diff from the tree root.
Collaborating on complex components or component development is also often difficult, as there are no good conventions for maintaining separate branches in the ZF repository. As a result, many developers will do feature/component development in their own version control systems, and point developers to these to test. This can lead to overhead if they use a different class prefix during development versus once the component is pushed into the ZF repository, as well as if the original development is done in a different version control system.
Finally, there are a number of components for which 3rd party code offers more comprehensive solutions. Users may turn to these, or compare ZF components with them when evaluating solutions to their application needs.
- SHOULD migrate to a distributed version control system
- SHOULD be a system many ZF developers are already familiar and/or *comfortable with
- SHOULD have the ability to specify multiple remotes per local repository
- SHOULD create patches from the repository root by default
- SHOULD support some method of verifying committer identity, to allow easy verification of CLA status when merging to the canonical repository
- SHOULD include well-defined interoperability and compatibility with existing third-party libraries and frameworks.
- This COULD be via consuming these third-party solutions
- This SHOULD include helping define and follow interoperability standards, such as the common naming schema to allow common autoloading capabilities.
One goal of Zend Framework was to advance the usage of PHP 5 and establish best practices surrounding the usage of PHP 5. To a large extent, ZF has achieved this goal (along with a handful of other projects).
PHP 5.3 was released mid-2009, and with it a myriad of new features were introduced: closures/lambdas, namespaces, late static binding, goto, and more. Many of these new features were introduced at the insistence of framework developers, as they can assist in making code more maintainable, concise, and readable.
Simply stated, Zend Framework 2.0 MUST be an exemplar of PHP 5.3 usage.
- MUST use namespaces throughout the framework, to mitigate naming collisions with userland code and third-party code.
- MUST remove all create_function() constructs and replace them with closures
- MUST evaluate the framework to identify areas that could benefit from other new language features.
- MUST NOT use new language features just for the sake of using them. They should be utilized only where they make sense.
- SHOULD be a showcase for improving code through refactoring, which is itself a strength of PHP.
One criticism often levied against Zend Framework is that it is "bloated." In most cases, the accusers are critical of the shear breadth of the framework – they do not want to install the entire code base when they only need one or a handful of components from it.
Additionally, many developers are using individual Zend Framework components within larger applications, sometimes written in other frameworks. ZF should support and encourage such use cases, and make it trivially easy for developers to fetch the components they wish to integrate.
On a related note, when pushing an application to production, often there are huge swaths of the framework that must be pushed that will never, ever be used, which wastes bandwidth, time, and disk space.
We have traditionally shipped the entire framework because it makes sense to have the entire framework available; if you need a component, you don't need to fetch it before using it. However, with the shear breadth of the framework, and the many uses to which developers are putting it, the time has come to enable developers to select individual comonents.
- MUST provide code infrastructure to support packaging
- MUST use namespaces and good component design to ensure components are properly encapsulated
- MUST write tools that can automatically determine dependencies for components
- SHOULD write tools for automatically creating packaging information (e.g., files necessary, component name, release version, etc.)
- SHOULD create a PEAR and/or Pyrum channel for distributing packages
- SHOULD create a number of meta-packages or bundles for common functionality; e.g., "MVC", "I18n", etc.
- Programming by Contract
- Favor the explicit over the magical
Zend Framework is often accused of being "bloated". When pressed, those who claim this will point to two factors: overall size of download, and the fact that most ZF components try to cover absolutely every use case possible, while specializing in none. The first factor is a straw-man; with a comprehensive framework such as Zend Framework, one should expect the download size to be fairly large.
The second factor, however, is very much accurate. It is also the source of our large number of issue reports. Developers consistently want a component to cover "just one more" use case, and adding support for it may then lead to inconsistent or undesired behavior in other use cases, leading to a spiral increase in reports.
Additionally, we have maintained a number of "hacks" and structures within the framework that can either be eliminated or replaced with language features available in PHP 5.3.
Our goal with 2.0 should be to identify the essential use cases for each component, and what specific problems need to be solved. Components should be designed in such a way as to allow injection of alternate implementations, either through extension or implementation of interfaces. Any additional use cases can be covered either in alternate implementations shipped in the framework, or in custom, userland implementations outside the framework. The focus on maintenance should be on the core functionality of the component (and framework), no more, no less.
Additionally, we should examine components carefully to determine if a need for the component still exists, and whether the component is still under active maintenance. If the answer to either question is "no," we should consider removing the component.
One of the "rules" of Zend Framework 1.X development was that all code should have vendor and component prefixes. In fact, a 1:1 relationship between the classname and the filename has been enforced. This is a good practice overall, but with the use of "vendor prefixes", it has led to some verbose and difficult to read code.
For instance, to refer to any given class name, you must use the full classname. While this is not problematic for top-level classes such as Zend_Loader and Zend_Registry, once you get much deeper, the classes become unwieldy, and the prefixes tend to distract from the actual function of the given class: e.g., Zend_Controller_Action_HelperBroker seems like overkill for a class that is simply a "Broker".
Additionally, this leads to over-long lines. If I need to reference more than one classname in a given line, I'm almost guaranteed to exceed the recommended line length and/or split the line into multiple lines. This leads to reduced comprehension for those trying to read and understand the code. Class names, method names, and variable names should be brief. Namespaces in PHP 5.3 can help make this a reality for classnames.
Finally, there is inconsistency in the framework with how code is formatted. While we have a published Coding Standard, we do not always follow it. Additionally, some practices are considered "optimal", but not "required" – practices such as alignment of operators, keeping variable names meaningful yet short, consistent indentation, etc. For some detailed examples and discussions of why, read the Seven Pillars of Pretty Code. These details need to be made required, and enforced via peer code reviews or pre- and/or post- commit hooks utilizing a utility such as PHP_CodeSniffer.
Within the framework, we have a variety of common practices that have slight variations. As examples:
- Both action controllers and views can make use of "helpers" – but the mechanisms of how these work differ (action controllers use a "broker"; view objects call on helpers via method overloading).
- View objects allow post-filtering of rendered contents – but the filters themselves do not follow the same interface as those found within the Filter component.
Having differences between APIs makes learning the framework more difficult. We must identify those areas that are functionally similar, and refactor them to use the same common basis. This will ensure consistency across the framework, which will make learning and mastering the framework easier.
- MUST remove obsolete, deprecated, or unmaintained components
- MUST rely on PHPUnit features to automate and simplify test suite maintenance
- data providers
- @group annotations
- TestAsset namespaces
- MUST make code more consistent
- Consistency in naming conventions
- Classes are nouns, methods are verbs
- Interfaces are nound or adverbs
- Abstract classes are predictably named
- Interfaces for any component that could have multiple strategies
- MUST establish well-defined guidelines regarding code formatting to enforce readable code
- MUST migrate code to use namespaces
- MUST utilize a code sniffer, and define rules that correspond to our coding standards
- MUST define and enforce consistent APIs; as examples:
- Leverage language features when possible to assist (__invoke(), closures, etc)
- Consistency in naming conventions
Programming by Contract asks
developers and architects to create solid interfaces and answer the following
questions during development:
- What does it expect?
- What does it guarantee?
- What does it maintain?
(bullet points copied verbatim from the wikipedia article)
Basically, any time an object uses collaborators, we should be defining
interfaces to allow developers to inject their own implementations. These
interfaces must be well-designed to ensure that we accurately capture the
requirements of collaborators.
As an example, in the 1.X series of Zend_Controller, we have abstract Request and Response objects. These have grown over time to include additional functionality useful to developers, but the core functionality used within the ZF comonents themselves is quite minimal. Ideally, we would create lean interfaces capturing the behavior necessary for Request and Response objects to work in the ZF MVC – thus making it easier for developers to slip-stream in their own implementations. Any functionality beyond what is defined in the interface would be relegated to individual implementations, and likely through collaborator objects to ensure that alternate implementations don't break additional functionality.
- SHOULD adhere to SOLID principles.
- All components MUST allow for dependency injection.
- When collaborators are defined for a component, collaborators SHOULD follow a defined interface; if an interface does not exist, one SHOULD be defined.
- COULD implement a Dependency Injection container or utilize one from a third party source (such as Symfony Components).
One obstacle new ZF developers face is identifying the source of functionality within the framework. One prime example is view helpers. As an illustration, consider the following example code from a view script:
Developers using an IDE can determine quickly that $this refers to the view object, but, unfortunately, most IDEs cannot determine the source of the function headStyle() – because it does not exist in the view object. The new developer may then turn to the API documentation – and fail to locate the method. If they're familiar with the concept of overloading, they may look at the __call() method finally, and discover that it proxies to view helper objects, but then they need to determine how view helpers are loaded and executed. All-in-all, it adds a ton of overhead when trying to learn the framework.
Additionally, "magic" methods such as _call(), get(), and _set() have performance implications; they take roughly 6x longer to execute than simply calling a standard method. By favoring explicit calls, we can likely radically improve the baseline performance of the framework.
In the above example of view helpers, how might we change the design to be more explicit? One potential solution would be to introduce a "helper broker", and use it to access helpers. Some use cases might look like this:
In all such cases the user can easily traverse the API and also get assistance from the IDE; all calls (except the last example, using invocation) use explicit methods, with clearly-defined return values.
This explicit approach is slightly more verbose than using the magic methods, but should ensure that code is simpler to understand, more consistent to use, and more easily maintained.
- Evaluate all uses of __magic_method() calls, and determine if they may be refactored to remove them.
- Where we determine magic methods are necessary, their usage should be well-documented