Skip to end of metadata
Go to start of metadata

<ac:macro ac:name="unmigrated-inline-wiki-markup"><ac:plain-text-body><![CDATA[

<ac:macro ac:name="unmigrated-inline-wiki-markup"><ac:plain-text-body><![CDATA[

Zend Framework: PHK Component Proposal

Proposed Component Name PHK
Developer Notes http://framework.zend.com/wiki/display/ZFDEV/PHK
Proposers Francois Laupretre
Darby Felton, Zend liaison
Revision 0.1 - 22 Jan 2007: Creation (wiki revision: 15)

Table of Contents

1. Overview

PHK is a new PHP-oriented package software. Its purpose is to distribute and run a software/library as a single package file (you may think of it as a very distant descendant of PHAR or a PHP-oriented jar-like package format). Distributing the Zend Framework libraries and documentations as PHK packages would be easy to implement and would bring many benefits.

I built some demonstration packages from the Zend framework original distribution:

  • 2 library packages (core and incubator)
  • 2 API documentation packages (core and incubator)
  • 12 End-user documentation packages: 11 for core (1 per language) and 1 for incubator (english)

Here are the main benefits I see :

  • keeping a library or any functional unit as a single file makes it more robust, consistent, easy to distribute, manage, remove, upgrade.
  • The most important, IMHO, is consistency. It is critical for software users to ensure that what they use is exactly what it is supposed to. The main problem I see with most PHP software is that it is provided as a file tree. With a PHK package, it is impossible to inadvertently remove a source file, there cannot be access modes problems, everything is done to detect data corruption... The point here is that a software is an entity, it is consistent and must remain exactly this way. Keeping the software as a single file is a good starting point but PHK also provides additional features as CRC checking and digital signatures.
  • PHK provides a 'webinfo' mode which gives users a standard way to access package information, license, a documentation page, and other information you may customize at will. As every other feature, all these information are served from the package as it is, keeping it consistent.
  • Embedded unit tests allow to keep unit tests with the code. There is no performance impact as the test code is included in a sub-package, which is ignored until you explicitely run the test procedure.
  • As noted above, PHK includes a digital signature feature. It is trivial to note how important it can be for corporate users... I think that it can be a very interesting feature for the Zend Framework. Please note that, unlike the jar signature feature, PHK signatures are verified offline and don't have any impact on runtime performance.
  • PHK includes an embedded autolad manager. It means that, in the case of 100% OO libraries like the Zend framework, it allows to remove every include/require/load_class from the user code. This feature is transparent and extracts the symbol names from the PHP source code at package creation time (no map to maintain by hand). IMHO, this one is very interesting because it provides a sort of 'linker' feature to the library with every benefits of just-in-time loading. Of course it replaces the current 'class name to file path' mapping mechanism, making you free to locate your classes and interfaces anywhere you want. As a side effect, it allows to put more than one class/interface in a single source file. I don't say it is always better but it removes the previous constraint.
  • A minor one is the ability to specify the list of PHP extensions needed by the package. I personnally hate receiving an ugly error message just because some needed extension is not loaded. This happens today with the Zend Framework when the iconv extension, for instance, is unavailable. Using the PHK 'needed_extensions' option, if the extension cannot be loaded when the package starts its execution, an exception is thrown with an explicit message giving the list of missing extensions. Such a small feature allows to spare much support time and user frustration !
  • This one is very personal: The Framework is split in 2 parts but I personally would prefer a more layered approach. For instance, I don't see why the Cache backends, Db adapters, or services are not implemented as optional plugins. PHK could help for this, especially since the 0.4.4 release, which comes with a new plugin facility. Now, it can easily be the basis for an architecture where single-file components communicate with the core through standard interfaces and check their mutual dependencies through a core package manager. OK, that's all, you all know what a plugin approach provides. But, once again, it is just a personal view, no flame for this, please ...

2. References

3. Component Requirements, Constraints, and Acceptance Criteria

  • Needs PHP 5.1.0 or newer.

4. Dependencies on Other Framework Components

In the Zend framework example packages, as in the original version, the unit tests need PHPUnit to run. It could be included in the packages (just a matter of choice). Also, as in the original non-packaged version, the incubator unit tests need the core library.

5. Theory of Operation

Look at the Zend Framework building kit downloadable from http://phk.tekwire.net. It is quite simple and shows how the packages are generated.

6. Milestones / Tasks

As noted above, demonstration packages are available. They are not perfectly functional because some of the unit tests still fail, but take these packages as proofs of concept: their purpose is to demonstrate that it is possible to adapt the libraries and documentations to PHK with minimal efforts. So, for instance, I didn't spend more time trying to fix the 3 unit tests which keep failing in the core libraries because I considered that it was more important to keep the building kit easy to understand. In this respect, I must conclude that, considering the minimal filter which are applied to the PHP sources before building the package, failing only 5 unit tests out of 775 can be considered as a success (it is even beyond my initial expectations .

Please note that, except for these unit tests, all the tests/demos I tried using the Zend framework packages worked exactly the same as with the original libraries.

Although it was not PHK's primary goal, I also discovered that PHK can be a very interesting tool to build documentation packages, e.g. static or pseudo-static content. Once again, I didn't have that in mind when I started the project but, if you have a look at the PHK documentation demo packages, you will see that, for instance, distributing and serving the API documentations as PHK single-file packages can bring several benefits like more consistency, on-the-fly compression, an embedded mime resolution table (independant of web server config)...

The only data I don't have is the performance overhead to consider. I tried to get timings but the tests and demos included in the Zend framework are too small and I didn't get anything reliable. One can note that it is intrinsically hard to measure because the overhead essentially depends on disk access times. With a fast Linux server (Dell 6850), I even measured that the Pdf demo always ran slightly faster when using the library as a package instead of original separate source files! It is just a matter of disk cache and access/transfer speed. So, I think it will be very hard to determine an average overhead value. Until somebody finds a better value, I keep considering it is small because, as I noted above, in the tests I did, I could not measure any difference.
<Edit 26-MAR-2007> I just started an 'accelerator' C extension for PHK. The 'accelerator' approach was chosen because the PHK runtime spends more than 70% of its time in less than 10% of the code. This CPU-intensive code is now well identified and the corresponding C accelerator extension will be available soon. Please note that the accelerator will always remain optional: if it is present, it will be used. If not, the result will be the same, just slower.

As you want milestones, please note that the PHK project today represents more than 9,000 lines of code and about 700 hours of work.

PHK status: PHK is considered alpha version because it is not used by enough people yet, but it is fully functional and just lacks testing. If the Zend Framework, for instance, decides to use it, it can go to production release very fast.

7. Class Index

PHK brings an extensive PHP API. In the case of the Zend Framework, I didn't have to use any API call.

8. Use Cases

Case 1: Running the core unit tests

1. Download the PHPUnit demonstration package and save the file in your include path.
2. Download the 'Zend Framework (core library)' demonstration package and save it in your current directory.
3. Run: php zfw_core.phk @check
4. You can do the same with the incubator unit tests but they also need the core library, which means that they need to find the zfw_core.phk file in the include path.

Case 2: Running the PDF demo

1. Download the 'Zend Framework (core library)' demonstration package and save the file in your include path.
2. Download the Zend Framework and extract the 'demos/Zend/pdf' directory.
3. In this directory, edit the 'demo.php' file :

  • Remove the line starting with 'set_include_path'.
  • replace the line containing
    require_once 'Zend/Pdf.php'
    by
    require_once 'zfw_core.phk'.
    4. Ensure your PHP contains the GD extension (with jpeg support).
    5. Run 'php demo.php test.pdf result.pdf'

9. Class Skeletons

None

]]></ac:plain-text-body></ac:macro>

]]></ac:plain-text-body></ac:macro>

Labels:
None
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.
  1. Feb 22, 2007

    <p>Why would someone use this over Phing or PEAR Installer?</p>

    1. Feb 26, 2007

      <p>1. About phing : PHK is a <span style="text-decoration: underline;">package format</span> (like jar), not a build system. An example : I wanted to keep the PHK demonstration packages (<a class="external-link" href="http://phk.tekwire.net">http://phk.tekwire.net</a>) easy to understand. So, I used Makefiles to build them, but, ideally, I would have used Phing. So, there is no choice to do between PHK and phing.</p>

      <p>2. About the PEAR installer :</p>
      <ul>
      <li>There is nothing incompatible between PHK and the pear installer. Today, PEAR packages are distributed as a tarball containing PHP source files, but it could also be a tarball containing PHK archives. They would be installed in the PEAR directory, included as any other source file, and there would be nothing to modify.</li>
      <li>The PEAR installer, as RPM and most other packages formats, allows to <span style="text-decoration: underline;">distribute</span> a software as a single file. But, in order to <span style="text-decoration: underline;">use</span> it, you must expand the package to a file tree. PHK keeps a software as a single file for the rest of its life. After being saved on the target host, it can be run, unit tested, renamed, deleted, analyzed... <span style="text-decoration: underline;">as a single file</span>.</li>
      </ul>

      <p>It is a question of <span style="text-decoration: underline;">conceptual unit</span> here: as we consider a 'software' as an immutable conceptual unit, we provide a tool to enforce this constraint. And, on the file systems we currently use, the easiest basic unit to use is the file. So, PHK allows to manage a software as a single file.<br />
      For people who don't see the benefits they can get from managing one file instead of a whole file tree, please ask yourself why java applications and libraries are distributed as jar files. It is not just a question of class resolution, as it could be done through java's equivalent of the include_path, it is a question of keeping libraries as identifiable, manageable units.<br />
       I know that people have been using PEAR packages for years, and some can find PHK uselessly complex, but, as soon as some important projects (like ZFW) start using it, I am sure they will understand the benefits it brings.</p>

      1. Mar 12, 2007

        <p>Thanks for the detailed answer. I'm a bit concerned by this line in your use case:</p>

        <ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
        require_once 'zfw_core.phk';
        ]]></ac:plain-text-body></ac:macro>

        <p>That's not including the entire Zend Framework, is it? :-O</p>

        <p>Can you give a basic explanation of how PHK works?</p>

        1. Mar 17, 2007

          <p>Of course, PHK is not a bare concatenation of source files <ac:emoticon ac:name="smile" />. Including a PHK package just declares and registers what it will need later to access its internal files, especially the autoload mechanism. And this 'bootstrap' code is designed to be as small and fast as possible. But you're right asking this question as it seems so trivial to me that, maybe, I was not clear enough in the documentation...</p>

          <p>To make it very basic, including a PHK package declares a new autoloader, creates a 'phk://' stream wrapper and registers the PHK package in the stream wrapper internal data. Then, the internal files are referenced through a 'phk://<mount point>/<virtual path>' URI. But, for an OO software like ZF, we don't care about the URIs because everything is resolved by the integrated autoloader: the needed (virtual) source files are autoloaded when the symbols they define are referenced somewhere.</p>

          <p> If you want a comparison, start by imagining the features of 'jar' applied to the PHP world. Or imagine it as a sort of PHP-oriented 'linker' tool as it can be compared to that too: A typical linker like 'ld' can be used to build a library or a program, it resolves the symbol references (PHK does it at runtime), and it produces a single file from multiple object files and/or libraries. It is a good comparison to start with. For more information please refer to the PHK documentation and to the ZF demonstration package at <a class="external-link" href="http://phk.tekwire.net">http://phk.tekwire.net</a>.</p>

          <p>Note: If you look at the ZF demonstration package, you will see that including zfw_core.phk automatically executes its (slightly modified) internal Zend.php virtual file, as it is explicitely defined this way in the package options. But, by default, including a package does not run anything from the library.</p>

  2. Apr 11, 2007

    <p>Could you explain the main differences between this and the PHAR format? It's mentioned as the "distant descendant", but you don't explain what's new/better than PHAR.</p>

    1. May 15, 2007

      <p>The PHK FAQ has a chapter named 'How does PHK differ from PHAR ?' at <a class="external-link" href="http://www.tekwire.net/joomla/products/phk/faq#p2">http://www.tekwire.net/joomla/products/phk/faq#p2</a></p>

  3. Jul 13, 2007

    <p>Though I would bet that PHK is a fine choice for some software, I do not think that this is the case for Zend Framework:</p>

    <ul>
    <li>Our user base should have come to expect the structure we have today; classes are easily located in the filesystem based on their names.</li>
    <li>There must be complications in debugging ZF-powered applications if the framework is packaged thusly.</li>
    <li>Code review and patches will not be as readily feasible with users' local framework copy as a PHK.</li>
    <li>It appears that the entire framework would have to be located in a single file... surely this is suboptimal as the framework grows. How does this scale when the framework is 2x, 5x, or 10x the current size?</li>
    <li>Our user base is not likely as familiar with PHK as they are with standard archiving of PHP software (tar.gz, zip).</li>
    <li>All things considered, I doubt that using this as our primary release mechanism meets our goal of extremely simple, though nothing prevents community members from redistributing this way if it suits their needs.</li>
    </ul>

    <p>If I have made invalid assumptions here, please correct me.</p>

    1. Aug 06, 2007

      <p>I have to agree with Darby Felton. The idea is good for some libraries but i think not for ZF.</p>

  4. Sep 28, 2007

    <p>Hi.</p>

    <p>I think, that will be a nice idea as additional offer, but then i would prefer Phar, because there are rumors, that it will be come with the next PHP-release 6. I have tested Phar with ZF (as my test victim <ac:emoticon ac:name="wink" />) and on this way i reduce the whole Framework from 22MB (about 100MB because the filesystem...) to one 10MB file uncompressed (without comments and docblocks) and 1.8MB gzip-compressed (also without comments and docblocks). So this might be an alternative usage on production environment, but surely its slower then many uncompressed files.<br />
    It is not able to load files within a phar by using the include path, so additionally all 'require's in the components has to be changed and the loader has also to be changed to allow "auto"loading classes within a phar-file, but i think its possible.</p>

    1. Sep 29, 2007

      <p>I dont think that we should release the ZF itself in any other format than plain.</p>

      <p>But for Phar it would be no problem to add a page to the wiki where it describes how to reduce ZF with Phar.</p>

      <p>I think the download size of the framework is no problem nowadays... even if it grows 10 times from now on.</p>

      <p>Phar is a good choice for those who have limited size on their webspace due to their provider. So these people will normally create their application on their servers and before loading it to their provider phar'ing it.</p>

      <p>Adding a "how to do" would be the best choice.</p>

      1. Dec 31, 2007

        <p>I must agree with Thomas that adding a "HOWTO"-style document, to the wiki, for example, would be the best choice.</p>

        <p>Pending objection and re-parenting by the proposer or someone else, I'll go ahead and archive this proposal. Of course, the community can move forward with PHK packaging of ZF independently.</p>

  5. Oct 30, 2007

    <p>I have been playing with the idea of an "installer" a bit, a quick example can be see with</p>

    <p><a class="external-link" href="http://zend.cyberlot.net/zf_dirty_installer.phps">http://zend.cyberlot.net/zf_dirty_installer.phps</a></p>

    <p>I agree I don't think size is an issue, is delivery..</p>

    <p>How can I download, install and get up and running as fast as possible? For that we need more then just a package we need some dedicated cli "RoR" style tools.</p>

    <p>The current tarball is not really friendly in this aspect due to its structure so I am still playing with a couple ideas</p>