Issues

ZF-563: Limit log file size

Description

We need an option to limit the file-size to keep it under.

TIA

Comments

I think a better solution would be something like built-in Apache rotatelogs functionality.

Agreed. It's not a good idea to put a hard limit on log file size, because after the log reaches that size limit, no new entries can be written to the log. Effectively, you start losing data unless someone is proactive about emptying the log files.

The better solution, used by many other software products, is to provide some schedule-driven mechanism for archiving logs and letting the service write to a new, empty log. Then you can decide how long you need to keep logs around, or delete them, etc.

Apache's rotatelogs is one such solution.

Perhaps Zend_Log could provide an optional decorator around whatever logging backend is chosen, to provide rotatelogs-like functionality.

Assigning to Gavin.

I've just browsed through the code on Zend_Log_Adapter_File class and noticed the code actually uses fopen, fwrite, fclose and other low-level functions and nothing is really automatic. I don't understand why no new entries could be written after the file size reaches its set limit as this is all in the 'control' of the Zend_Log_Adapter_File class. I mean - while writing back to the file, it could be a matter of reading the old content from the file if any, appending (suffix) the old content to the new content, truncating the string to whatever the limit was set as (approx) discarding from the end of the string (old content) and writing the limited string to the file. Bingo - the file size always stays in limit without any manual intervention! If the limit was set as 20k, the length of the string would be approximately 20480 chars - doesn't really need to be accurate.

It gets annoying on Windows Notepad to open up a 1.5+ MB log file because the response it too slow even on a local machine, opening the file, moving the cursor, selecting text, copying text, pasting text, finding text... It takes for ever for each of these actions.

Regards,

Understood. Many small appliances offer exactly the feature you requested Shekar, including my DSL router/firewall at home. We will take this feature request into consideration.

Scheduling for release 0.8.0.

I'm interested in coding this feature. After a quick glance of the architecture of the log component, we need to address the following:

  • When a log is full, we have the options of *# Rotating the logs (works file files, but not necessarily databases) *# Refusing to write anything more (not sure how practical this is) *# Removing early entries and writing at end *# Deleting the existing log and starting fresh *# Other actions depending on unknown adapter type
  • Do we support each ability?
  • If so, what is the most efficient way to handle it? Different log adapters are different beasts. As said, rotating is only really applicable to files. Deleting earlier entries is much easier with a database. We could always just add a prune() method to the adapter interface and go from there. We could also code up a new interface that provides the abilities to prune (delete first N entries, etc) and provide the ability for each log adapter to register a custom prune interface.

I'd like to hear other's opinions on what the best method to accomplish this is.

Will just two features/adapters cover max functionality for most people (80/20)?

  1. using files
  2. using database tables

with configuration flags to blow off entries on FIFO (based on file size, number or rows). In case of the database adapter, it could have variables for the DSN (or an existing connection), table name, column names, etc. so we do not have to extend the adapter - just invoke its methods to configure the adapter. This could keep its footprint small, simple add to performance.

Regards,

Assign to Bill Karwin for verification that these issues are resolved by the new Zend_Log redesign.

ZF-263