Hi Ralph, I really like this work, just saw the webinar and I see tons of potential in this component, I'm worry about something, I've been playing around with code generation for a while and I've found the best way you can generate anything is by using templates, I see you are using procedural code for generate stuff, it does the job when you need to generate something which is not big, but when things go big, to find a line where you need to make a change can be really messy, have you consider to implement something like Zend_View as a templating system for Zend_Tool?

I think it could be better have a template like:

well, obviously it could vary some, but IMHO, it is easier to read than writing code inside a method, btw, you said in the webinar django and after checking it out, it uses templates, but it has a 'flaw', which I've already experienced, I used to create my templates using some xml tags for common operations like, foreach, print, etc, but templates went bigger and bigger, to use Zend_View as templating system is to use the very same PHP as template language, which is what I'm using right now for templating (not Zend_View, PHP).

Hope it helps,

Aldemar

Specifics:

• Putting this in the Zend_Tool namespace indicates that it will execute in the tooling execution environment. This is an unnecessary limitation. If we decide to implement Dojo integration that provided client-side validation, for example, we might reuse this code to generate the appropriate JavaScript. In that case, it would be awkward if this code resided in this package. I suggest Zend_CodeGenerator- or even Zend_Generate or Zend_Generator, depending on whether you like verbs or nouns in your namespaces. This will also reduce namespace bloat.

• I like the simple Zend_Tool_CodeGenerator_Abstract interface, but have a few reservations. First, I'm not sure there's enough here to justify an abstract class. Plus, the options functionality is peripheral to the main abstraction here, and it could be argued that this form of options handling is not ideal for some systems; maybe an interface would be better with an abstract base class for the code generators that we provide as part of this proposal. Secondly, I'd consider method names like 'generate()' and 'reflect()', or maybe 'serialize()' and 'deserialize()' to avoid name collision with the very semantically fixed __toString() method.

• I'd like to see the Apace config file generation soon, since it is a very different type of file from the php files that you go in to detail on. Also consider naming it 'ApacheConfig' or something similar.

• What does "It will take cue from Reflection in what parts and bits of can become represented in object oriented notation" mean?

• How would this class be instantiated if you wanted to modify an already existing class? Will it work with Zend_Loader? Will (can?) it work for plugins?

• I agree with Aldemar about the templates and why we'd need them, and I think it's interesting that he reused PHP as we had discussed in the past.

General:

• Again with the confusing typos.

• Nurse, we need use cases, stat.

I agree with Ralph on the Zend_Tool namespace meaning. The namespace contains tools to aid with project development and provides tools that applications can use themselves. I do think it is important to define a few aspects of code contained in Zend_Tool:

• It is typically focused on functionality, not performance
• It should not use the project config/context but rather have it supplied explicitly if called from the application (See my comment on Zend_Tool_Rpc proposal for more details)
• It should not be executed by default in production environments but be limited to debug/development activities as well as specific uses such as generating code for dojo. In this case the result should be cached and the generation should only happen if required.

I love this component (along with Zend_Reflection) and believe it is a critical component for furthering meta-driven development. My development work focuses on building frameworks that are meta-driven and extensible. Most frameworks define an architecture which is abstracted to admin tools to different extents but require the developer to build business logic and other application-specific code by hand. Some frameworks can generate scaffolding code to get you started fast, but once the code is modified it falls outside of the framework's ability to update it. On one hand you have frameworks that provide a lot of end-user functionality out of the box but typically strap the developer to an architecture that does not lend itself well for extensive customization. One the other hand you have frameworks that provide components but require the developer to wire everything together cuasing lots of repetitive work. The challenge is to marry the two with the aim of reducing grunt work and at the same time maintain flexibility.

The only way I see this happening is through code reflection, parsing and generating. This involves several phases of abstraction and development:

1. Be able to parse any file into an object structure and save it again
2. Be able to isolate patterns in the object structure
3. Be able to compare and map the object structure to defined standards
4. Be able to manipulate the object structure in an intelligent way
5. Be able to manipulate the object structure based on meta-data
6. Be able to test the resulting file using unit testing driven by the mate-data

This will provide the required functionality to:

• generate scaffolding
• generate completely functional modules
• enable the developer to modify code while still being able to maintain it automatically
• provide detailed information to advanced IDE's that can write code-blocks based on meta programming
• enable the developer to swap out sections of generated code for custom code that is faster, different etc... without breaking automatic maintainability of the entire module.

Generating code is easy compared to parsing and re-saving it with some changes without changing previous formatting. Who is to say we need to preserve formatting? Standards exist for a reason. They allow two or more people to easily work on the same code interchangeably. Thus we need standards for writing PHP code, .htaccess files, apache configs etc... It will enable the developer to stay in sync with the system and not get frustrated when things get reformatted.

To me code parsing is just as important as code generation. I would love to see the parsing functionality be built at the same time as the generation functionality to ensure a seamless flow. It would also make unit testing of generated code much easier.

I think this component is so important that it warrants a separate project with a C library that can be used by any programming language. We need to start somewhere and ZF is a great place, but I only see the concept reaching it's true potential if we can get support from other camps who can provide parsers and generators to and from a generic (XML?) format. They can at the same time define strict formatting conventions for their files which will help everyone.

The Zend_Tool_CodeGenerator name is great for the "generating" aspect but I think the parsing should be put into Zend_Tool_CodeParser and the two components should use the same underlying data structure for representing the code in tokenized form similar to DOM. If CodeGenerator is too specific we could call it something like StructuredParser and StructuredGenerator implying that it parses and generated data in a structured way based on standards.

The possibilities for such a system are endless and I would love to see ZF pioneer it for all structured data a typical PHP system encounters. Maybe a Zend_Structured namespace would be best as I see many uses of this functionality within applications. One application would be the parsing of the XML returned from Service calls via a chain of parsers and filters to generate desired object structures that can be used directly in other ZF components.

One important consideration: a lot of code generation may be required for bootstrapping, which is currently pretty free form in ZF. If we add a bit of predictable structure, as the Zend_Application component does, it will be much easier to manipulate the bootstrapping process without having to parse and modify the body of a large bootstrapping method. I would be satisfied with support of the latter, assuming Zend_Application is flexible enough to serve most, if not all, bootstrapping requirements.

,Wil

Why proposal is "Initial Draft"?

It already in 1.8 release.

I wrote a nearly feature-complete implementation of CSS 2.1 code generation, extending this component... I don´t really have the time to get a CLA account working for me... But I would like to share it and see it in maybe a future release..

Anyone interested, please let me know!

Archiving this proposal, feel free to recover it when you want to work on it again. For more details see this email.