Skip to end of metadata
Go to start of metadata

<ac:macro ac:name="unmigrated-inline-wiki-markup"><ac:plain-text-body><![CDATA[

<ac:macro ac:name="unmigrated-inline-wiki-markup"><ac:plain-text-body><![CDATA[

Zend Framework: Zend_Cache Refactoring Component Proposal

Proposed Component Name Zend_Cache Refactoring
Developer Notes http://framework.zend.com/wiki/display/ZFDEV/Zend_Cache Refactoring
Proposers Marc Bennewitz
Zend Liaison TBD
Revision 0.1 - 6 Jun 2009: Initial Draft.
0.2 - 8 Jun 2009: Finished Proposal.
0.3 - 23 Jun 2009: Only one namespace option.
0.4 - 23 Jun 2009: Handling of multiple cache ids.
0.5 - 20 Jul 2009: Added how to instantiate without factory. (UC: Own Backend)
0.6 - 30 Jul 2009: Added namespace use case
0.7 - 18 Sep 2009: Added functions replace & add
0.8 - 08 Oct 2009: Added function optimize and moved automatic optimization from sqlite backend to core fontend
0.9 - 23 Oct 2009: Added Zend_Serializer proposal on references
0.10 - 23 Oct 2009: Added increment/decrement functionality
0.11 - 21 Jan 2010: Multiple Interface & getDelayed/fetch interface 6 backwards compatibility (wiki revision: 63)

Table of Contents

1. Overview

Refactoring Zend_Cache for ease of use, more functionality and optimized performance.

2. References

Source Code (In Development)
svn://marc-bennewitz.de/zf-proposals

3. Component Requirements, Constraints, and Acceptance Criteria

  • The Zend_Cache factory will use a configurable plugin loader to load frontends and backends.
  • The Zend_Cache factory will only accept one argument for options.
  • The frontend will implement tagging if it isn't done by backend.
  • The frontend will no longer serialize data. This must be done by backend.
  • The Zend_Cache can handle an array of cache ids to get/set/remove multiple cache ids.
  • The compontent will accept all php datatypes (incl. NULL) to cache them.
  • Matching modes should be joined with the binary OR (|) operator for method clean and getIds.
  • Only one namespace/prefix option will be used.
  • The sqlite backend will support differents php extensions and versions: php_sqlite, sqlite, sqlite3.
  • The backends will have just one interface to implement.
  • The core frontend will be moved to the frontend folder -> Zend_Cache_Frontend_Core.
  • The Test backend will be removed and the tests have to use the Array backend.

4. Dependencies on Other Framework Components

  • Zend_Cache
  • Zend_Exception
  • Zend_Loader_PluginLoader
  • Zend_Serializer

5. Theory of Operation

Serialization:

  • The serialization has been moved to backend.
  • Backends that only supports to handle strings there is a option named "serializer".
    This can be an instance of Zend_Serializer_Adapter_AdapterInterface or a string used to instantiate the serializer with Zend_Serializer::factory.
    Additionally you can disable serialization by set this option to NULL.
  • Backends supporting different datatypes doesn't implement this option but memcached.
    Memcached has own serializer selectable with Memcache::SERIALIZER_* constants. To use the build-in serializer you can set the serializer option to one of the constants or you can handle this options same as other backends.

deprecated methods:

  • Zend_Cache_Core::setLifetime - Please use the option "ttl" instead.
  • Zend_Cache_Core::load - Please use get instead.
    • Old: load($id, $doNotTestCacheValidity = false, $doNotUnserialize = false)
    • New: get($id = null, array $opts = array())
  • Zend_Cache_Core::test - Please use exist or info instead.
    • Old: test($id = null) : return mtime or false
    • New: exist($id = null, array $opts = array()) : return boolean
    • To get mtime use info($id, array $opts=array()) : return array with "mtime" key
  • Zend_Cache_Core::save - Please use set/add/replace instead.
    • Old: save($data, $id = null, $tags = array(), $specificLifetime = false, $priority = 8)
    • New: set($data, $id = null, array $opts=array())
  • Zend_Cache_Core::clean - Please use clear instead.
    • Old: clean($mode = Zend_Cache::CLEANING_MODE_ALL, array $tags = array())
    • New: clear($mode = Zend_Cache::MATCHING_ALL, array $opts = array())
  • Zend_Cache_Core::getIdsMatchingTags - Please use find instead.
    • Use find(Zend_Cache::MATCHING_ACTIVE | Zend_Cache::MATCHING_TAGS, array('tags' => $tags))
  • Zend_Cache_Core::getIdsNotMatchingTags - Please use find instead.
    • Use find(Zend_Cache::MATCHING_ACTIVE | Zend_Cache::MATCHING_NO_TAGS, array('tags' => $tags))
  • Zend_Cache_Core::getIds - Please use find instead.
    • Use find(Zend_Cache::MATCHING_ACTIVE)
  • Zend_Cache_Core::getFillingPercentage - Please use status instead.
    • Use status() - return array('total' => <number>, 'free' => <number>)
  • Zend_Cache_Core::getTags - Please use info instead.
  • Zend_Cache_Core::getMetadatas - Please use info instead.
    • Use info($id) - return array('mtime' => <int>, 'tags' => <array>, 'ttl' => <int>[, + backend spezific information])

Bc of all deprecated methods are only implemented on core frontend which is normally the base class.

deprecated core options:

  • cache_id_prefix - Please use "namespace" instead.
  • automatic_serialization - The option "automatic_serialization" has been removed. There is no more usage of it.
    • Now backends who doesn't support different datatypes implement new option "serializer" (see Zend_Serializer)
  • automatic_cleaning_factor - Please use "automatic_clearing_factor" instead.
    • Because the method name has changed
  • lifetime - Please use "ttl" instead.
    • Zend_Cache was the only caching system who named it lifetime. On all other systems it is named ttl

deprecated file backend options:

  • read_control_type - Please use "read_control_algo" instead.
  • hashed_directory_level - Please use "directory_level" instead.
  • hashed_directory_umask - Please use "directory_perm" or "directory_umask" instead.
    • hashed_directory_umask wasn't a really umask it was a permission value
    • The new directory_perm set permissions by the same value
    • The new directory_umask set permissions by using umask
  • file_name_prefix - Please use "namespace" option of core frontend instead.
  • cache_file_umask - Please use "file_perm" or "file_umask" instead.
    • Same as hashed_directory_umask
  • metadatas_array_max_size - Option completely removed

deprecated sqlite backend options:

  • cache_db_complete_path - Please use "dsn" instead.
  • automatic_vacuum_factor - Please use "automatic_optimizing_factor" of core frontend instead.
    • Normalized backend optimization interface

Changes on memcache(d) backend:

  • Renamed Zend_Cache_Backend_Memcached to Zend_Cache_Backend_Memcache (uses memcache php extension)
  • added Zend_Cache_Backend_Memcached (uses memcached php extension)
    • TODO: How can bc done with this naming change ???

deprecated class frontend options:

  • cached_methods - Please use "cache_callbacks" instead.
  • non_cached_methods - Please use "non_cache_callbacks" instead.
    • Now the class frontend extends the new callback frontend

deprecated function frontend options:

  • cache_functions - Please use "cache_callbacks" instead.
  • non_cache_functions - Please use "non_cache_callbacks" instead.
    • Now the class frontend extends the new callback frontend

6. Milestones / Tasks

  • Milestone 1 [DONE]: Finish proposal
  • Milestone 2 [DONE]: Working prototype
  • Milestone 3: Prototype checked into the incubator
  • Milestone 4: Unit tests exist finished and component is working
  • Milestone 5: Initial documentation exists
  • Milestone 6: Changed related components
  • Milestone 7: Moved to core.

7. Class Index

  • Zend_Cache
  • Zend_Cache_Exception
  • Zend_Cache_Frontend_Core renamed
  • Zend_Cache_Frontend_Callback new
  • Zend_Cache_Frontend_Function (extends Zend_Cache_Frontend_Callback)
  • Zend_Cache_Frontend_Class (extends Zend_Cache_Frontend_Callback)
  • Zend_Cache_Frontend_File
  • Zend_Cache_Frontend_Output
  • Zend_Cache_Frontend_Page
  • Zend_Cache_Backend_Interface
  • Zend_Cache_Backend_Abstract renamed
  • Zend_Cache_Backend_Array new
  • Zend_Cache_Backend_Apc
  • Zend_Cache_Backend_File
  • Zend_Cache_Backend_Memcache renamed
  • Zend_Cache_Backend_Memcached new
  • Zend_Cache_Backend_Xcache
  • Zend_Cache_Backend_Sqlite
  • Zend_Cache_Backend_SqliteAdapter_Abstract new
  • Zend_Cache_Backend_SqliteAdapter_Sqlite new
  • Zend_Cache_Backend_SqliteAdapter_Sqlite3 new
  • Zend_Cache_Backend_SqliteAdapter_PdoSqlite new
  • Zend_Cache_Backend_ZendPlatform
  • Zend_Cache_Backend_ZendServer
  • Zend_Cache_Backend_ZendServer_Disk
  • Zend_Cache_Backend_ZendServer_ShMem
  • Zend_Cache_Backend_TwoLevels

8. Use Cases

9. Class Skeletons

Please take a look to my development repository to see all classes.

]]></ac:plain-text-body></ac:macro>

]]></ac:plain-text-body></ac:macro>

Labels:
zend_cache zend_cache Delete
cache cache Delete
refactor refactor Delete
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.
  1. Jun 10, 2009

    <p>There is a german forum thread:
    <a class="external-link" href="http://www.zfforum.de/showthread.php?t=4193">http://www.zfforum.de/showthread.php?t=4193</a></p>

    1. Jun 18, 2009

      <p>Update Discussions from the German thread:</p>

      <p><strong>1. The new file backend calculate the expire time on the last modification time of cache file.</strong><br />
      This could be a problem if the file system doesn't set/update the last modification time.</p>

      <p><strong>2. Add functionality to handle a list of cache ids by one cache call and if the backend has implemented this functionality (like php_memcahced) this can give a performance boost.</strong><br />
      e.g:</p>
      <ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
      // cache status:
      // id1 -> data1 -> not expired
      // id2 -> data2 -> expired

      // multiple load
      $rs = $cache->load(array('id1', 'id2', 'id3'));
      // -> array('id1' => 'data1')

      // multiple load without validation
      $rs = $cache->load(array('id1', 'id2', 'id3'), array('validate' => false));
      // -> array('id1' => 'data1', 'id2' => 'data2')

      // multiple test
      $rs = $cache->test(array('id1', 'id2', 'id3'));
      // -> array('id1' => [mtime], 'id2' => false, 'id3' => false)

      // multiple test without validation
      $rs = $cache->test(array('id1', 'id2', 'id3'), array('validate' => false));
      // -> array('id1' => [mtime], 'id2' => [mtime], 'id3' => false)
      ]]></ac:plain-text-body></ac:macro>

      <p><strong>3. Use a configurable adapter to handle tags by frontend.</strong><br />
      This is now implemented please see the "Frontend Tagging" example.</p>

      <p><strong>4. normalize namespaces/prefix options:</strong><br />
      Add the frontend option "namespace" (Zend_Cache_Frontend_Core).<br />
      This is the default namespace and the frontend forwards this namespace to the backend.</p>
      <ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
      $cache = Zend_Cache::factory('Core', 'File', array('namespace' => 'test'));

      // The file backend use the prefix as file prefix
      $cache->load('id1');

      // On change backend the namespace option will be obtained
      $cache->setBackend(new Zend_Cache_Backend_ZendServer_Disk());

      // The ZendServer backend use the namespace as cache id prefix separated by "::"
      $cache->load('id1');

      // All other backends use the namespace as cache id prefix separated by "-"
      $cache->setBackend(new Zend_Cache_Backend_Apc());
      $cache->load('id1');

      // You can change/set a namespace on call
      $cache->load('id1', array('namespace' => 'mynamespace'));
      ]]></ac:plain-text-body></ac:macro>

      <p>Remove the following options:</p>
      <ul>
      <li>"cache_id_prefix" (Zend_Cache_Frontend_Core)</li>
      <li>"file_prefix" (Zend_Cache_Backend_File)</li>
      <li>"namespace" (Zend_Cache_Backend_ZendServer)</li>
      </ul>

  2. Jun 23, 2009

    <p>"The compontent will accept all php datatypes (incl. NULL) to cache them."</p>

    <p>How do you plan to distinguish between "key not found" and boolean false value?</p>

    1. Jun 24, 2009

      <p>If your cached value can be NULL you have to test this record:</p>

      <ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
      if (($data = $cache->load('id')) !== null || $cache->test('id') !== false) {
      echo 'Cached data (' . gettype($data) . '): ' . $data;
      } else {
      echo 'Data not cached';
      }
      ]]></ac:plain-text-body></ac:macro>

      <p>EDIT:<br />
      The component return null (instead of false) if cache doesn't hit.</p>

      1. Jun 25, 2009

        <p>Hi,</p>

        <p>the problem here is that if you allow caching all data-types how do distinguish between "key not found" and "key not found but is null". </p>

        <p>I think returning false if key is not found makes sense since this seems to be the way that every other caching solution for PHP works.</p>

        1. Jun 25, 2009

          <p>1. I think there is no problem to differentiate between "key not found" and "key not found but is null" because you can call the test method to get this information.</p>

          <p>2. I return null instead of false because it can be more often to store boolean values as to store a NULL value.</p>

          <p>3. There are some caching extensions which are allow to store any php type (except for resources) to store but the most of them can't differentiate between failure false and valid return false.<br />
          I think in the new memcached extension this is solved good:</p>

          <p><a class="external-link" href="http://de3.php.net/manual/en/memcached.get.php">http://de3.php.net/manual/en/memcached.get.php</a><br />
          Returns the value stored in the cache or FALSE otherwise. The Memcached::getResultCode will return Memcached::RES_NOTFOUND if the key does not exist. </p>

          <p>-> This can be adjusted with the test method and there is only need if the load method returns NULL.</p>

          1. Jun 25, 2009

            <p>> 1. I think there is no problem to differentiate between "key not found" and "key not found but<br />
            > is null" because you can call the test method to get this information.</p>

            <p>> 2. I return null instead of false because it can be more often to store boolean values as to <br />
            > store a NULL value.</p>

            <p>How would you handle this with backends such as APC and Memcache? If the serialization is delegated to the backend the only information you would get back is boolean false or the value. This includes 'test' method which is not much more than a ->get() with the current Memcache backend.</p>

            <p>APC, Memcache and Memcached all return false if key is not found. I am not sure if it is good idea not to follow established conventions.</p>

            1. Jun 25, 2009

              <p>> How would you handle this with backends such as APC and Memcache? If the serialization is delegated to the backend the only information you would get back is boolean false or the value. This includes 'test' method which is not much more than a ->get() with the current Memcache backend.</p>

              <p>On this backends more than the single data information will be stored as an array (added mtime & tags). Than the backend have to return an array. If not the cache isn't hit for the zend cache backends.</p>

              <p>> APC, Memcache and Memcached all return false if key is not found. I am not sure if it is good idea not to follow established conventions.<br />
              This calls for a community discussion, I have explained my position.</p>

              1. Jun 25, 2009

                <p>> On this backends more than the single data information will be stored as an array (added mtime > & tags). Than the backend have to return an array. If not the cache isn't hit for the zend <br />
                > cache backends.</p>

                <p>I don't think it is really that space usage friendly/performance friendly to force array on simple data-types. </p>

                <p>Also a major annoyance is that things like increment / decrement will never work on those values. I think most of the backends could support simple incr/decr operations.</p>

                <p>I don't understand how the tags would be queried with for example memcached in this scenario. getIdsMatchingTags would need to step through every key and compare the tags?</p>

                1. Jun 25, 2009

                  <p>Sorry this was a little bit wrong.</p>

                  <p>In addition to the data I stores the current time and the ttl as an array.<br />
                  This informations are needed for the test method and the metadata method.</p>

                  <ul class="alternate">
                  <li>The test method returns the last modification time if id hits.</li>
                  <li>The metadata array returns an array with last modification time and ttl.</li>
                  </ul>

                  <p>It is not a new feature. This functionality is based on the current implementation of Zend_Cache.</p>

  3. Jun 25, 2009

    <p>Hey Marc,</p>

    <p>Would really appreciate if you could also consider some of the changes I proposed in my own Zend_Cache proposals - mainly the minor refactoring I added so I could implement a page cache. Most of this was, if I remember accurately, the limitations on the format of a cache id which prevented passing URLs (which are necessary IDs for a page cache that need to remain compatible with non-PHP caching solutions like an Nginx frontend).</p>

    1. Jun 25, 2009

      <p>Hi Pádraic,</p>

      <p>to remove the limitations for cache ids is an interesting think.<br />
      The problem is that the most backends have some restrictions on keys.<br />
      (eg. File is limited by file system, ZendServer is limited by zend data cache logic, ....)</p>

      <p>This means if the frontend doesn't limit the ids all backends have to handle its restrictions.</p>

      <p>But a MUST restriction is that cache ids are only strings.</p>

      1. Jun 26, 2009

        <p>Yeah, that's the main issue with such a change - backends are given the responsibility of validating the ID rather than having a single restrictive check in the frontend that all backends have to live with. It's also the BC breaking behaviour that is questionable - how would it effect existing custom backends relying on frontend validation?</p>

        <p>Definitely all id's should be strings.</p>

        <p>It's the benefits that made me suggest it for, at least, ZF 2.0 eventually. With the current restrictions it's messy to work around it, and the restrictions prevent making cache ids that are portable and easy to use outside of PHP. For example, a page cache using URL keys can see the PHP side cache the page in memcached (using a URL based key), and an Nginx frontend proxy query the memcached server with a rewritten URL to retrieve a cached page without a pass through to Apache+PHP. These caching mechanism blow the current Zend_Cache "Page" system out of the water - it frees the cache from sucking up RAM since it never needs to invoke PHP - Nginx can grab it from memcached there and then.</p>

        <p>Just something to consider <ac:emoticon ac:name="wink" />. Not sure whether the BC breakage is acceptable in your proposal or not.</p>

        1. Jul 30, 2009

          <p>Hi Pádraic</p>

          <p>I have some trouble with this functionality because the backends doesn't support this by default.<br />
          Every backend supports different characters <ac:emoticon ac:name="sad" /></p>

          <p>My current way to solve this is to encode the cache id to a base64 string because I need have to get the raw id by the stored id.</p>

          <p>Do you have a better idea or could this better implement in frontend if it is needed ?</p>

  4. Jun 30, 2009

    <p>The current implementation allows to store all php types (inc. boolean & NULL) in all backend caches<br />
    and the result of method test is the last modification time if the cache hit.</p>

    <p>Thereby on simple cache backends like apc and memcache(d) more than the raw data have to store in the cache to get the last modification time on test. This is much more overhead as needed.</p>

    <p>If only the raw data have to store in the cache the boolean false can't store in the cache because some php extensions return false if the cache isn't hit.</p>

    <p><strong>opinion poll:</strong><br />
    1. Can the method "test" only return true if the cache hit?<br />
        Or have it to return the last modification time ?<br />
    2. Is it enough if all php data types will be cached beside boolean and NULL ?<br />
        Or should cached all data beside FALSE values?<br />
        Or actually all datatypes should be stored in cache?</p>

    1. Jul 20, 2009

      <p>1. I think the basic interface should not be changed from the current. So in my opinion test() should return the last modification time when the cache is hit like it does currently.<br />
      PHP is not a strict language so it's more than natural to the language this way. And in some cases you really want to have a lastModification time of the cache hit. Doing an extra call and keep metadata somewhere of all cache sounds more than overhead to me than the little array that is currently used to pack some metadata with the cache.<br />
      The currently used is nothing more than:</p>
      <ac:macro ac:name="code"><ac:default-parameter>php</ac:default-parameter><ac:plain-text-body><![CDATA[
      array($data, time());
      ]]></ac:plain-text-body></ac:macro>
      <p>Does not really look like a major overhead to me.</p>

      <p>2. In my opinion all datatypes should be stored in cache, as long as it can be serialized.</p>

      <p>To bad the confluence wiki vote macro isn't working correctly <ac:emoticon ac:name="sad" /></p>
      <ac:macro ac:name="unmigrated-inline-wiki-markup"><ac:plain-text-body><![CDATA[

      What should the method "test" return if the cache hit? (Log In to vote.)
      Choices Your Vote

      boolean true (BC breakage)

      string lastModificationTime (like in the current implementation)

      ]]></ac:plain-text-body></ac:macro>
      <ac:macro ac:name="unmigrated-inline-wiki-markup"><ac:plain-text-body><![CDATA[
      What php data types should be cacheable? (Log In to vote.)
      Choices Your Vote

      Everything (serializable?) besides boolean and NULL (BC breakage)

      Everything (serializable?) besides boolean false (BC breakage)

      Everything serializable (like in the current implementation)

      ]
      ]></ac:plain-text-body></ac:macro>

      <p>ps. @Marc Bennewitz, I cannot access you svn repository, getting a timeout. I would really like to take a look and see if I can help developing this proposal code/tests/etc.</p>

      1. Jul 21, 2009

        <p>Hi Patrick</p>

        <p>Thank you for your interest.</p>

        <p>Yes, it's a pity that the vote macro isn't working <ac:emoticon ac:name="sad" /> but it's nice to look.</p>

        <p>I don't have trouble with the svn respository on two differnet computers (without authorisation).</p>
        <ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
        svn co "svn://marc-bennewitz.de/php/zfcache" "/tmp/zfcache"

        1. traceroute have to find "lvps87-230-85-172.dedicated.hosteurope.de (87.230.85.172)"
          traceroute marc-bennewitz.de
          tracert marc-bennewitz.de
        2. check simple svn connection
          telnet marc-bennewitz.de 3690
          ]]></ac:plain-text-body></ac:macro>
          <p>-> If the problem continues exists please let me know if you can't find the the domain or if the svn deamon doesn't response.</p>
        1. Jul 21, 2009

          <p>Thanks, seems to work now. I was checking at work.. seems that the svn protocol port is blocked in the internal network at work <ac:emoticon ac:name="sad" /></p>

          <p>I'll check out the source at home <ac:emoticon ac:name="laugh" /></p>

      2. Jul 22, 2009

        <p>> Does not really look like a major overhead to me.</p>

        <p>The overhead comes from additional serialize/unserialize call for each value. This is also against interoperability between languages since the data is going to be PHP serialized. In addition it renders native increment/decrement operations impossible on the values.</p>

        1. Jul 22, 2009

          <blockquote>
          <p>This is also against interoperability between languages since the data is going to be PHP serialized</p></blockquote>
          <p>Is it even possible to use PHP cache for other languages? (just an interest question)<br />
          If so, do you have some more info resources about this?</p>

          <blockquote>
          <p>In addition it renders native increment/decrement operations impossible on the values.</p></blockquote>
          <p>Can you provide more info on this? What it is and how it works? (or just provide links to resources <ac:emoticon ac:name="smile" />)</p>

          <blockquote>
          <p>The overhead comes from additional serialize/unserialize call for each value.</p></blockquote>
          <p>What do you think as a good solution for saving the lastModifiedTime? Where do you think the lastModifiedTime should be saved?<br />
          Using another cache item for saving metadata has the same or even more overhead than the 2 serializations that need to be done for the current solution</p>

          1. Jul 22, 2009

            <p>> Is it even possible to use PHP cache for other languages? (just an interest question)<br />
            > If so, do you have some more info resources about this?</p>

            <p>Alot of the backends can be used from other languages: memcached, sqlite and I guess files.</p>

            <p>>> In addition it renders native increment/decrement operations impossible on the values.<br />
            > Can you provide more info on this? What it is and how it works? (or just provide links to resources )</p>

            <p><a class="external-link" href="http://uk2.php.net/manual/en/memcached.increment.php">http://uk2.php.net/manual/en/memcached.increment.php</a>
            <a class="external-link" href="http://xcache.lighttpd.net/wiki/XcacheApi#SimpleCounter">http://xcache.lighttpd.net/wiki/XcacheApi#SimpleCounter</a></p>

            <p>APC has similar functionality coming.</p>

            <p>> What do you think as a good solution for saving the lastModifiedTime? Where do you think the > lastModifiedTime should be saved?<br />
            > Using another cache item for saving metadata has the same or even more overhead than the 2 <br />
            > serializations that need to be done for the current solution</p>

            <p>I don't really think last modified is something cache should support for all the keys, it is something that can be done optionally or in the application logic.</p>

            1. Jul 22, 2009

              <p>Good points, thanks for the info <ac:emoticon ac:name="smile" /></p>

        2. Oct 09, 2009

          <p>The additional array isn't needed if no mtime or ttl have to be saved.<br />
          To Check boolean values if a boolean false was cached can done by the most backends:</p>
          <ul class="alternate">
          <li>array</li>
          <li>memcached (Memcached::getResultCode)</li>
          <li>apc (using second parameter on apc_fetch)</li>
          <li>xcache (xcache_isset)</li>
          <li>ZendPlatform (output_cache_exists)</li>
          <li>unserialize (can be check direct by the serialized value like this $serialized === 'b:0')<br />
          -> unserialize is used by file, sqlite and memcache backend</li>
          </ul>

          <p>Only on ZendServer_[ShMem] (return false) I can't check a direct false value <ac:emoticon ac:name="sad" /><br />
          On this backend I have to store more than the raw data like "array($data)" else I can't check a valid false value with raw data.</p>

          <p>EDIT:<br />
          Or I don't accept to cache boolean and NULL values.</p>

  5. Jul 22, 2009

    <p>I think that there also should be a way to add global tags to the cache. Tags that need to be set on all caches, something like a namespace tag. Each application uses it's own global tags which then are used with all cache actions within that application. It still must be possible to set specific tags at the save and clean functions like it's currently possible. Only thing that changes is the implementation, where the global and specific tags are simply merged before use by the backends.</p>

    <p>A situation that I'm currently looking at is the following:</p>
    <blockquote>
    <p>A caching backend that is used/shared over multiple applications, like Zend Platform or memcached. Each application needs a global tag for all it's cache, so all cache for that specific application can be cleaned. And more specific cache can be cleaned by setting extra tags to cleanup.</p></blockquote>

    <p>The cache_id_prefix provides for seperating cache id's, but this cannot be used by a cleaning action of the cache. Using tags would be the best fit for this.</p>

    <p>I'm not sure what would be the best place to set this global cache tags, as close to the backends as possible would be best. Or maybe this is not the correct proposal to place this and the <a href="http://framework.zend.com/wiki/pages/viewpage.action?pageId=9437207">Zend_Cache_Manager</a> proposal would be a better fit for this. </p>

    1. Jul 30, 2009

      <p>Hi Patrick,</p>

      <p>I removed the frontend option "cache_id_prefix" because there was more similar options on within some backends that I removed, too. ("file_name_prefix" on file backend and "namespace" on ZendServer)</p>

      <p>Instead of this I added the frontend option "namespace" which can defined as default or set on each method. The namespace will be passed to the backend which have to handle this.</p>

      <p>You can use this option on the following methods:<br />
      save, load, test, remove, touch, clean, getIds, getMetadata</p>

      <p>Please read the namespace use case I currently added.</p>

  6. Jul 28, 2009

    <p>Hi Marc,</p>

    <p>I've started to look into your proposal in more depth than previously since I'm pushing my own Cache proposal onwards for a review. I think Zend_Cache definitely needs a refactoring, but more than a technical refactoring it needs improvements on a few levels. As I read this I'm reminded you should probably rename it to avoid using the term "refactoring". I would call it something like Zend_Cache Improvements or Zend_Cache Enhanced since it actually adds to the pot.</p>

    <p>I would also like to request you look at my own Zend_Cache proposals in case I'm doing something that could cause your proposal any problems. If so, let me know so we can discuss how to resolve that.</p>

    1. Jul 30, 2009

      <p>Hi Pádraic</p>

      <p>In my opinion refactoring is correct because the basic functionality is the same but there are a lot of changes inclusive BC break to get a more simpler code by additional features and performance improvements <ac:emoticon ac:name="wink" /> - But it can be debatable.</p>

      <p>I read you proposals:</p>

      <p>Zend_Cache_Manager:</p>
      <ul class="alternate">
      <li>there are no propblems</li>
      <li>only some options have changed</li>
      </ul>

      <p>Zend_Cache_Backend_Static:</p>
      <ul class="alternate">
      <li>There are not more problems as the current implementation</li>
      <li>I have some problems with the cache id to allow all characters but this is for previous comment above</li>
      </ul>

  7. Aug 19, 2009

    <p>Hello Marc,</p>

    <p>Thanks for the proposal.</p>

    <p>I have checked out the prototype and it is promising, when do you expect these codes can get the into core?</p>

    1. Aug 19, 2009

      <p>hi tsz</p>

      <p>because some bc break it could be to the core for 2.0 but this have to be recommend by the zend team before, and we have to solve some problems and not all classes are working (or implemented) for current.</p>

  8. Sep 18, 2009

    <ul class="alternate">
    <li><strong>new function replace(mixed $data, string $id[, array $options])</strong> — Replace the item under an existing id<br />
    Replace data under under an existing id. If id doesn't exist this generates an error.<br />
    -> Some backends support this and have a better performance with this combination.</li>
    </ul>

    <ul class="alternate">
    <li><strong>new function add(mixed $data, string $id[, array $options])</strong> - Add an item under a new id<br />
    Only adds data for a new id. If the id already exist this generates an error.<br />
    -> Some backends support this and have a better performance with this combination.</li>
    </ul>

    <p>edit: changed argument order</p>

  9. Sep 18, 2009

    <p>rem</p>

  10. Oct 09, 2009

    <p>When this component will be moved to 'ready for recommendation'?</p>

  11. Oct 09, 2009

    <p>Currently I need some reviews of the following behaviors i found:</p>

    <p>1. Zend_Cache_Frontend_Class & Zend_Cache_Frontend_Function uses the magic method __call to forward and cache reqested functions/methods. This doesn't work with all functions/methods because the requested names could be implemented by one of the extended frontend class and not are not caught by __call <ac:emoticon ac:name="sad" /><br />
    -> To solve this either these frontends may not be extends core frontend. Than they are no more a cache frontend (like Zend_Cache_Function / Zend_Cache_Class) or I remove this from zend cache and all users have to implement similar on their own code using Zend_Cache_Frontend_Callback?</p>

    <p>2. I assayed to accept all strings of cache ids (no more character restriction) to solve an issue with the <a href="http://framework.zend.com/wiki/pages/viewpage.action?pageId=8947732">Zend_Cache_Backend_Static</a> proposal but my idea to encode/decode ids in base64 would create some other problems and ate to many performance (In my mind).<br />
    -> Have someone a better idea or is it fair to solve this by frontend?</p>

    <p>3. Sone backends implemented a increment/decrement functionality.<br />
    -> I'm currently working to add this as a basic functionality by keeping backend usage if it is implemented by backend</p>

    1. Oct 23, 2009

      <p>-> 1.<br />
      My Idea to solve this would be really big changes on splitting frontend/backend structure:</p>
      <ul>
      <li>rename Zend_Cache_Backend_* -> Zend_Cache_Adapter_*</li>
      <li>marge Zend_Cache & Zend_Cache_Frontend_Core -> Zend_Cache</li>
      <li>Zend_Cache::factory only instantiate Zend_Cache using only core and adapter options</li>
      <li>move Zend_Cache_Frontend_* -> Zend_Cache_* and doesn't extends Zend_Cache anymore<br />
      simple use case:
      <ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
      $cacheOpts = array('caching' => true, 'ttl' => 3600, 'cache_dir' => '/tmp');
      $cache = Zend_Cache::factory('File', $cacheOpts);
      $pageCacheOpts = array('http_etag' => true);
      $pageCache = new Zend_Cache_Page($cache, $pageCacheOpts);
      $pageCache->start();
      ]]></ac:plain-text-body></ac:macro></li>
      </ul>

      <p>-> 2.<br />
      I added following methods:</p>
      <ul>
      <li>Zend_Cache_Frontend_Core::increment($value=1, $id=null, array $opts=array())</li>
      <li>Zend_Cache_Frontend_Core::decrement($value=1, $id=null, array $opts=array())</li>
      <li>Zend_Cache_Backend_Abstract::increment($value, $id, array $opts=array())</li>
      <li>Zend_Cache_Backend_Abstract::decrement($value, $id, array $opts=array())<br />
      If a backend supports this funxtionality it can overwrite these methods</li>
      </ul>

  12. Oct 29, 2009

    <p>I have a few questions/comments.</p>

    <ul>
    <li>Cache IDs. Since the problem is typically on the <strong>backend</strong> in terms of ID restrictions, my inclination is that we allow <strong>anything</strong> as an identifier, but put the onus on the backend to normalize the identifier for its own use. This makes for a <em>much</em> easier story for users.</li>
    <li>Why are all options, frontend and backend, in the same array? While this works in terms of the signature, it is confusing on two levels:
    <ul>
    <li>First, for those who know the method signature, it will look like no backend options are being defined</li>
    <li>Second, it's difficult to determine which options are related to the backend, and which to the front.</li>
    <li>I'd recommend instead having a multi-dimensional array (or Zend_Config object) with keys for frontend and backend. This makes it easier for developers to understand which options are for which component.</li>
    </ul>
    </li>
    <li>Caching PHP types should remain as-is. Certainly, you <strong>can</strong> cache null, false, or otherwise empty values, but the implicit assumption should be that you're caching <strong>something</strong> – and thus an empty value will require re-running the operation that generates the cached data.</li>
    <li>test() should also remain as is. It returns false if no hit is made, and a timestamp otherwise. As such, it can be used for simple existence tests, as well as to give an idea of cache age. Potentially, an alias such as "cacheAge()" might be good to have.</li>
    <li>Old config options/prefixes such as "file_name_prefix", "cache_id_prefix", etc. should continue to be accepted, but translated internally to the new representation. Potentially, detection of them can be used to raise a deprecation notice.</li>
    </ul>

    <p>Please also work with Paddy regarding his Zend_Cache_Manager proposal to ensure (a) compatibility between the two, and (b) no overlap between them.</p>

    1. Oct 31, 2009

      <p>Hi Matthew,</p>
      <ul class="alternate">
      <li><strong>Cache IDs</strong>: Yes the problem is on <strong>backends</strong>. For example the file backend saves cached data in files "<namespace><separator><cache id>.[cache]". The problem isn't that it is infeasible but it is slower to encode/decode not allowed characters on every test/load/save call. (Hashing doesn't work because I have to reconvert the value to the original id.)</li>
      </ul>

      <ul class="alternate">
      <li><strong>Options Array</strong>: The idea for this is based by options which is not clear if this is a backend or a frontend option (like ttl (lifetime), caching or serializer)
      <ul>
      <li>The main functionality isn't on backend, it is the core frontend and additional functionality is made by the other frontends. Out of it my previous comment - the idea to split frontends and backends to a base class with an adapter pattern and additional functionality with own class structures. (but this breaks backward compatibility)</li>
      <li>On the described api on top bc can be implemented</li>
      </ul>
      </li>
      </ul>

      <ul class="alternate">
      <li><strong>test()</strong>: If test have to return the last mtime - backend features can never supported completely.
      <ul>
      <li>For example increment/decrement is supported by apc and memcache(d) and xcache but it supposes that cached data are stored unchanged. To return the last mtime I have to store the mtime because there is no way to get the last mtime on these backends. I think it is better to only return last modification time on getMetadata only if the backend supports this else it could stored manually by frontends which needs this.</li>
      <li>Can I implement both ways with Zend_Cache::$compatibilityMode like Zend_Locale, for zf 1.10 and completely change it in 2.0 ?</li>
      <li>In 2.0 it could solve with the adapter idea and a new class like this:
      <ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
      // Zend_Cache_Mtime extends Zend_Cache and stores mtime for all backends $cache = Zend_Cache_Mtime::factory('memcache', array()); $cache->getMTime($cacheId);
      ]]></ac:plain-text-body></ac:macro></li>
      </ul>
      </li>
      </ul>

      <ul class="alternate">
      <li><strong>Old config options/prefixes</strong>: It's no problem to implement bc for this.</li>
      </ul>

      1. Oct 30, 2009

        <ul>
        <li><strong>Cache IDs:</strong> How much slower are we talking? I'd hazard a few milliseconds. I'm not necessarily suggesting hashing, but instead normalization – strip all characters that are not allowed. This can be done with str_replace() or preg_replace() – both of which are quite fast.</li>
        <li><strong>Options array:</strong> I'm okay with the change <em>so long as the original functionality continues to work</em> – i.e., if somebody passes both frontend options and backend options, that should also work. As another possibility, we could introduce a new factory that uses merged options. Whatever is done, the existing behavior must be maintained for the 1.x series.</li>
        <li><strong>test():</strong> let's introduce a different method for retrieving the TTL, or use getMetadata(). test() will work as it does currently, but now raise a notice indicating that it should be used in the future simply to test for a cache hit; in 2.0, we can then remove the notice, and implement that behavior.</li>
        </ul>

        1. Oct 31, 2009

          <p><strong>Cache IDs</strong>: ok before I use base64_[encode]. Now I tested it with a simple strtr for the file backend. I replaced all control characters (0x00 - 0x1f + 0x7f), characters not allowed on filenames (<,>,?,",:,|,\,/,*), all non ASCII characters and the characters ~, + with "+<HEXCODE>"<br />
          My PC needs 1,4s to encode a 40 digit string and 1,6s to decode it with 100000 repeats.</p>
          <ac:macro ac:name="code"><ac:plain-text-body><![CDATA[
          $id = chr(1).chr(0).'!"#$%&\'()*+,-./%09:;%<=>?@AZ[
          ]^_`az

          Unknown macro: {|}

          ~';
          echo strtr($id, $replace); // +01+00!+22#$%&'()+2A+2B,-.+2F%09+3A;%+3C=+3E+3F@AZ[+5C]^_`az{+7C}+7E
          ]]></ac:plain-text-body></ac:macro>
          <p>Now I think it looks like ok but I also think that escape character have to be configurable.</p>

          <p><strong>Options array:</strong> ok</p>

          <p><strong>test():</strong> If I add a new method this a have to store the mtime too <ac:emoticon ac:name="sad" /> . What do you think about the idea to add a new class which implements the mtime functionality but only if this new class is used else mtime is only available on Metadata if backend supports this innately.<br />
          How can I raise the notice, on every test call or at documentation ?</p>

          <p>What do you think about the idea to split the frontend/backend structure to a main core class within a simple adapter pattern and complete self-explanatory frontends. If you like this too or if I don't articulate I give you more detailed information or create a new proposal for it.<br />
          I think this can solve some more problems. For example cachable components (like Zend_Locale) only accept an instance of the main cache class and can configure the frontend by self it it makes sence. (Zend_Locale can use the file frontend to automatic reparse files if it has been changed)<br />
          (For current the components accept all frontends but only use base functionality of core.)<br />
          -> I'm sorry but I get the idea to late.</p>

          <p>EDIT: Problems on show multiline code <ac:emoticon ac:name="sad" /></p>

          1. Feb 13, 2010

            <p>test():</p>

            <p>I am heavily against storing mtime as this renders memcached/other backends native increment/decrement/add etc functionality useless. With the backends I care about (memcached mainly and I guess it's the same with APC) the test() method fetches the whole data array but just checks one item from it so it does not really save anything compared to just loading the data.</p>

            <p>Something like (transparent) in-process cache might be a good addition so that accesses to the same key from same backend during a request would not cause a round-trip to backend unless configured to do so.</p>

    2. Oct 31, 2009

      <p><strong>Caching PHP types should remain as-is</strong>:<br />
      Currently only strings can cached. The only way cache non string values is to enable "automatic_serialization" in frontend which serialize all data even if the backend supports caching of different datatypes. To disallow empty values like "", null, false is ok but all other types/values (expect for not serializable types like resources) have to be cachable by default on backend.</p>

      <p>Is the following restriction ok:? if (!$value && $value !== '0') throw new Zend_Cache_Exception('Empty values are not allowed');<br />
      (If this results in an exception on frontend is configurable by "throw_exceptions")</p>