Skip to end of metadata
Go to start of metadata

<ac:macro ac:name="unmigrated-inline-wiki-markup"><ac:plain-text-body><![CDATA[]]></ac:plain-text-body></ac:macro>

<ac:macro ac:name="unmigrated-inline-wiki-markup"><ac:plain-text-body><![CDATA[

<ac:macro ac:name="unmigrated-inline-wiki-markup"><ac:plain-text-body><![CDATA[

Zend Framework: Zend_Build Component Proposal

Proposed Component Name Zend_Build
Developer Notes http://framework.zend.com/wiki/display/ZFDEV/Zend_Build
Proposers Wil Sinclair
Revision 1.0 - 29 November 2007: Proposal created.
1.1 - 21 December 2007: Proposal updated.
1.2 - 17 January 2008: Proposal updated after component responsibilities finalized. (wiki revision: 41)

Table of Contents

1. Overview

Zend_Build provides a project management infrastructure for Zend Framework. This system introduces the concepts of 'actions' and 'resources' defined in both core and external components that can be used in managing and building a Zend Framework project. Resources can be nested to create project profiles that can be stored on the filesystem in Zend_Config's XML format to provide metadata to CLI, Rich UI, and other tools for project introspection and management. All actions and resources must be declared in build metadata files, which include metadata such as short name, help, and command line options.

2. References

3. Component Requirements, Constraints, and Acceptance Criteria

  • The build infrastructure must support the requirements of actions and resources used to perform common tasks on Zend Framework projects, including, but not limited to, project generation/deletion, scaffolding CRUD actions, and unit test CRUD actions. Note that this project will not deliver these specific capabilities, but will provide an infrastructure upon which to implement them easily. Please refer to Zend_Build for Core for specific implementations of actions and resources.
  • Must allow all actions and resources to be constructed using Zend_Config instances.
  • Resources must be nestable in both configurations and object graphs.
  • Actions and resources must work intuitively within the context provided by a project profile.
  • Resources must be serializable to XML fragments and these XML fragments must be composable to form valid XML 1.0/Zend_Config XML project profile documents.
  • All Zend_Config objects used in Zend_Build must be serializable to the filesystem in XML format. The functionality for rendering and writing these documents in valid Zend_Config_Xml format will be implemented in the Zend_Build component for now because it is TBD whether this would promote best practices if added to Zend_Config.
  • All Zend_Config_Xml files should support XML attributes in both reading and writing for enhanced readability of project profiles.
  • Actions and resources must provide full usage documentation in the build manifest files.
  • Each action and resource must have its own build manifest file with file names of the form '<action or resource classname>-ZFManifest.*'.
  • Zend_Build must support template files used in creating files for project creation, etc. These files will be valid php files that output the required file.
  • Actions should be atomic where possible.
  • It would be nice to have the ability of rewinding actions, including deletes.
  • This component should be able to read and write project profiles in any format supported by Zend_Config including INI.
  • It would be nice if the component stored MD5 checksums for all files to detect file changes between operations.
  • This componet must verify that an application profile is valid before using it for any operations.
  • Any instance of Zend_Build_Resource must be instantiable using an instance of Zend_Config.

4. Dependencies on Other Framework Components

  • Zend_Exception
  • Zend_Config
  • Zend_Loader
  • Zend_Log

5. Theory of Operation

The Zend_Build component will provide a basic project management system for Zend Framework. This component introduces the concepts of 'actions', 'resources', and 'project profiles'. In general an action is always required to perform an operation on the project, and some actions perform these operations on or with a specified resource. Project profiles are nested compositions of resources that make up a project. Resources- and, by extension, project profiles- genrally contain metadata about project resources. That is to say, a project profile is useful in creating a project skeleton or operating on an existing project, but generally does not contain the code and can not be used to fully re-generate a project after the user has modified the system-generated project skeleton.
This component will deliver basic actions and resources to support basic filesystem operations- such as a 'delete' action and a file 'resource' to support deleted a file- but more specific actions and resources will be developed in the component to which they pertain (see Zend_Build for Core). This component will also provide resources and actions that pertain to most or all projects, such as library directory and public directory. The Zend_Build component will generally follow ZF's 'use at will' architecture and may be used to describe and manage projects that don't use all ZF core components- or any core components besides Zend_Build's direct and indirect dependencies.
Actions and resources can be added to the library by first implementing Zend_Build_Action_Interface and Zend_Build_Resource_Interface, respectively. Next, the developer must add one manifest file per action or resource to his/her component with the name '<classname>-ZFManifest.*'. These manifest files can be any file type supported by Zend_Config, but must contain all information about the resource for building the project and any other functionality that requires metadata, such as that outlined in [Zend_Console]. Zend_Build requires the 'type' and 'name' attributes and other attributes or elements may be used by Zend_Build. The manifest format is extensible, however, so the developer defining the action or resource may add metadata required by other components unknown to Zend_Build. All such metadata will simply be ignored by Zend_Build. The following is an example of such a file in the xml format defining a resource that may be used by Zend_Console:

A project profile contains information about the resources in a project with a tree-like structure. This project profile may be serialized to the XML 1.0 dialect defined by Zend_Config and used to store any metadata about the project that is useful to Zend_Build actions or resources. As a rule, no PHP code will be serialized to the project profile, although components may be able to generate code using a combination of the data in a project profile and separate templates stored in the Zend Framework components to which they pertain. The following is an example of a project profile for a basic MVC application:

Since Zend_Config objects are used heavily in this component, all actions and resources will be instantiable from valid Zend_Config objects. A Configurable interface is provided to bring consistency to such instantiations. If the Zend_Config_Configurable is approved for core in the 1.5 release, this component will use the Zend_Config_Configurable.

6. Milestones / Tasks

  • Milestone 1: [DONE] Draft proposal submitted for community review
  • Milestone 2: Working prototype checked in to incubator for community evaluation
  • Milestone 3: Proposal finalized
  • Milestone 4: All unit test passing with coverage at 90% or higher for all code
  • Milestone 5: Documentation finished in English

7. Class Index

  • Zend_Build_Configurable
  • Zend_Build_AbstractConfigurable
  • Zend_Build_Exception
  • Zend_Build_Manifest
  • Zend_Build_Manifest_NameConflictException
  • Zend_Build_FileTemplate
  • Zend_Build_Action
  • Zend_Build_Action_Interface
  • Zend_Build_Action_Abstract
  • Zend_Build_Action_Create
  • Zend_Build_Action_Delete
  • Zend_Build_Resource_Interface
  • Zend_Build_Resource_Abstract
  • Zend_Build_Resource_Factory
  • Zend_Build_Resource_ConfigXmlWriter
  • Zend_Build_Resource_File
  • Zend_Build_Resource_Directory
  • Zend_Build_Resource_ApplicationDirectory
  • Zend_Build_Resource_ControllerDirectory
  • Zend_Build_Resource_DataDirectory
  • Zend_Build_Resource_LibraryDirectory
  • Zend_Build_Resource_ModelDirectory
  • Zend_Build_Resource_ModuleDirectory
  • Zend_Build_Resource_Project
  • Zend_Build_Resource_PublicDirectory
  • Zend_Build_Resource_TempDirectory
  • Zend_Build_Resource_TrashDirectory
  • Zend_Build_Resource_ViewDirectory

8. Use Cases

This component is primarily consumed by Zend_Console and other build interfaces. The use cases below show examples of defining an action and a resource.

UC-01

The command 'zf say_something dummy_file' on the command line (assuming Zend_Console is installed) would now return 'duh'.

9. Class Skeletons

Sample project profile for MVC application:

The same sample profile for MVC application rendered as an XML configuration file:

]]></ac:plain-text-body></ac:macro>

]]></ac:plain-text-body></ac:macro>

Labels:
None
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.
  1. Dec 04, 2007

    <p>The proposal seems to be in the beginning stages of development, but here is my feedback so far, which mostly includes questions to be addressed by the proposal contents.</p>

    <p>First, we should have clear definitions, including examples, for the following:</p>
    <ul>
    <li>Command</li>
    <li>Task</li>
    <li>Resource</li>
    </ul>

    <p>Regarding generation with Commands and Resources, the following should also be defined and illustrated:</p>
    <ul>
    <li>project</li>
    <li>project resource</li>
    <li>scaffolding</li>
    </ul>

    <p>Regarding the scope of unit test execution functionality:</p>
    <ul>
    <li>How will this work?</li>
    <li>What are the system requirements?</li>
    </ul>

    <p>Describe "database migrations" in terms of example usage and scope of functionality.</p>

    <p>"General code generations" should be better described:</p>
    <ul>
    <li>Administrative interfaces - To/For what? Examples?</li>
    <li>Login forms - To/For what? Examples?</li>
    </ul>

    <p>In the requirements section, "Note that this project will not deliver these specific capabilities, but will provide a framework upon which to implement them easily." It is not clear to me, then, which capabilities will be provided, or how the proposed component provides support for these features.</p>

    <p>"Resources must be serializable to XML fragments"</p>
    <ul>
    <li>Depends on definition of Resource</li>
    <li>Why is XML the best format for all possible Resources?</li>
    <li>For what purposes are these XML fragments created?</li>
    <li>How would the XML be used outside the context of the proposed components?</li>
    </ul>

    <p>I could not understand the exact meaning of the following statements, which should be illustrated by explanation and providing use case examples:</p>
    <ul>
    <li>"Resources... must be composable to form valid XML 1.0 project profile documents"</li>
    <li>"Commands and resources must be fully self-documenting to all tools, including CLI."</li>
    <li>"Commands and resources must have intuitive single-word names with short aliases for the CLI."</li>
    <li>"Must dependencies among commands and other tasks."
    <ul>
    <li>Rephrase?</li>
    <li>What kind(s) of dependencies?</li>
    </ul>
    </li>
    <li>"System must not run identical tasks multiple times if 2 or more paths lead to the same task in the dependencies graph."
    <ul>
    <li>Why not?</li>
    <li>If this behavior is needed how will it be accomplished?</li>
    </ul>
    </li>
    <li>"Commands should be atomic where possible."
    <ul>
    <li>Does the same idea also apply to Tasks?</li>
    </ul>
    </li>
    <li>"All created user interfaces should support configuration/overriding-based and code-generation-based extensibility."</li>
    <li>"It would be nice to have the capabililty of rewinding commands, including deletes."
    <ul>
    <li>What about Tasks?</li>
    <li>How would this look in code?</li>
    </ul>
    </li>
    <li>"Pluralization of resource types on the command line would be nice to have to improve readability of commands for English speakers, but singular syntax will always be valid."</li>
    <li>"Commands, which are special kinds of tasks that are intended to be called from the command line and are self-documenting"
    <ul>
    <li>What is "self-documenting" in this context?</li>
    </ul>
    </li>
    </ul>

    <p>Why is a tree structure most appropriate for modeling all of a project's resources? Why not a forest, or a DAG, or a general graph (e.g., cyclic workflow)?</p>

    <p>"This file will be editable, but the system will detect if a profile it generated or previously manipulated has been edited outside of the system and produce a warning. A warning will also be produced before taking destructive action on any resource that has been edited since it was generated. All checksums necessary for such warnings will be contained in the project profile."</p>
    <ul>
    <li>How will the system know either situation is the case when edited outside the system, given that checksums are stored in the edited project profile?</li>
    <li>Are the profiles impossible to reverse engineer, and if so, how, and, perhaps moreover, why?</li>
    </ul>

    <p>I would also recommend using Zend_Config or another approach to make the implementation independent of the resource serialization format (XML).</p>

    <p>Finally, it's all well and good but use cases are the most important aspect of this proposal (as well as others). Showing what the component will do via how it will be used is essential to understanding and prioritizing the work to be involved in developing the component. The class skeletons section should also begin filling with the results of the API from the use cases.</p>

    <p>I am also interested to see how the implementation will be separated from the interface so that CLI is not the only possible way to use the component (not all PHP environments have CLI available to customers). An example showing how to call the core functionality with PHP (e.g., for a web interface) would be most helpful.</p>

  2. Dec 22, 2007

    <p>What about providing integrating this scaffolding system into <a href="http://phing.info">phing</a>?</p>

    1. Dec 26, 2007

      <p>There is no reason someone can't build phing tasks to integrate this build system in to phing. I don't see phing as a good solution for a build system in PHP and ZF in particular, however. I know a lot of people will disagree with this, but allow me to make my argument:</p>

      <p>1) Build systems are really not much more than scripting systems with special facilities added for problems that come up routinely in building projects. One of these is dependencies. Another might be a set of common tasks (which are really not much more than a set of functions). So, IMO a good build system is best built out of a good scripting system. PHP has a good scripting system- PHP-CLI. So why aren't Phing build scripts created using PHP? As far as I can tell, this is because they wanted to emulate Ant. But the only reasons Ant was built using an XML dialect as a 'scripting language' was because Java is such a poor one, and XML happened to be the 'solution for all problems' at the time Ant was first conceived. I think if you look at both Ant and Phing build scripts, you'll see that XML is not a particularly good scripting language either (not surprising, it's a DATA format, people! <ac:emoticon ac:name="smile" />. So, AFAIC see, Phing is a design driven by another implementation language's limitations. I know that's a harsh criticism, and I know that people have been productive with Phing. But I encourage everyone to look at Phing and Ant, then take a look at what I consider I much more sane build system that happens to be written in a good scripting language: rake for ruby. Come to you own conclusions and post them here. <ac:emoticon ac:name="smile" /></p>

      <p>2) Build systems often become overly complex when they are designed to solve all problems for everybody. I think there is some benefit to building up an action/resource infrastructure on a bare-bones build system that address problems in ZF projects. Of course, basic stuff like copying files, etc. should just be handled by PHP itself. As with all ZF components, the intention is to provide a build system on top of which any build system functionality can be built, but develop the most common use cases that come up in ZF usage.</p>

      <p>From what I've seen, build system are just starting to get some serious attention in the PHP world, but I've searched around and haven't yet found a reasonably good one. This presents a fairly good opportunity for ZF to present best practices and tools to the community. Please keep in mind, this is coming from someone who has spent most of his career in the Java world- where build systems have been around from the get-go since you have that pesky compilation step to deal with <ac:emoticon ac:name="wink" />. I used, but hardly loved, both Make and Ant extensively. Then I switched to rake and realized I wasn't the only one feeling the pain. Has anyone had experience with other build systems that might inform us in this discussion?</p>

      1. Dec 27, 2007

        fc

        <p>I agree with your opinion about Ant. I'm glad you see it that way, and that you are building this component. This is something that ZF needs and will benefit from. I'm using shell scripts to build projects, they do the job, but they are difficult to extend and maintain. Here's an example of one of the scripts I'm using:</p>

        <ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
        $ ./addmodule -h

        Usage: addmodule [-hacvmsl] -n <module_name> -N <class_name>
        Options:
        -a, create actions/ directory
        -c, create config/ directory
        -v, create views/ directory
        -m, create models/ directory
        -s, create setup/ directory
        -n, name of the module (lower-case)
        -N, name of the class (camel-case)
        -h, usage and options (this help)
        -l, see this script
        ]]></ac:plain-text-body></ac:macro>

        1. Dec 27, 2007

          <p>BTW, I'd imagine you'll find the CLI tools relatively intuitive based on the interface above. Take a look- <ac:link />. The actual build scripting part of the build system is in this (totally incomplete) <ac:link><ri:page ri:content-title="proposal" /><ac:link-body>Zend_Build_Script - Wil Sinclair</ac:link-body></ac:link> and may or may not make it in to 1.5. There are some potentially 'controversial' implementation details in that build script system that we'll have to discuss as a community, so check back on the proposal soon!</p>

  3. Dec 23, 2007

    <p>Hi!</p>

    <p>What exactly will this proposal provide?<br />
    code generator or is it a build Tool like phing? In case of YABT (Yet Another Build Tool) - Why not provide build functionality for the Zend Framework on top of Phing (as Lars said already)? Afaik, phing is very modular OO and i think, specific Zend Framework Tasks would not the real problem. Also, phing is already approved and many developers are familiar with it.</p>

    <p>There is also a proposal (Zend_Db_Schema_Manager) for managing DB changes in the Framework. How will this (or other components, doing the same) fit into this build tool? I think it would be good to be able to use this in an existing build tool. So it would be nice to think about that too.</p>

    1. Dec 26, 2007

      <p>Both. <ac:emoticon ac:name="smile" /><br />
      See above comment RE: Phing.<br />
      RE: Zend_Db_Schema_Manager- this seems to be an implementation of db migrations from rails in PHP, more or less. It could work in much the same way migrations work in rake. 'zf migrate schema schema-name -version 3' for example, where 'migrate' is the action and 'schema' is the resource. But this proposal is subject to the same questions that rails migrations raise, which I'll get in to in comments there. Also, I believe Cake has a migrations implementation; I think we should be asking questions about how we can make a powerful system that handles less 'conventional schemas'. This is the area where we can excel as a framework and achieve true longevity and enterprise adoption; we can leave Rails-clone functionality to Cake AFAIC. <ac:emoticon ac:name="wink" /></p>