When cleaning caches based on tags or expiry (manual or automated), if there are a lot of caches with the same prefix it will take too long and in some cases timeout requests.

I propose that tags and expiries be stored in central index files for each prefix, that way Zend_Cache_Backend_File doesn't have to open every single cache metadata file to figure out which ones need to be cleaned, instead it could open just one file and loop through an array.

This could be made more efficient not only by storing an array of all caches, but also storing arrays for each tag with all caches containing that tag. This will minimise memory/processor usage.

I came across this problem when a highly load balanced would "randomly" take 30-60 seconds or more on some requests. It turned out the default automatic cleanup of 1/10 was occuring often and i had hundreds of thousands of caches with the same prefix (equivalent to the number of rows in a table). Each time it would loop through hundreds of thousands of files over NFS just to see if they were due to expire.

More details here: http://www.yewchube.com/?p=58 http://www.yewchube.com/?p=62