We need an option to limit the file-size to keep it under.
TIA
Comments
Posted by Matthew Ratzloff (mratzloff) on 2006-11-18T12:41:49.000+0000
I think a better solution would be something like built-in Apache rotatelogs functionality.
Posted by Bill Karwin (bkarwin) on 2006-11-18T12:49:04.000+0000
Agreed. It's not a good idea to put a hard limit on log file size, because after the log reaches that size limit, no new entries can be written to the log. Effectively, you start losing data unless someone is proactive about emptying the log files.
The better solution, used by many other software products, is to provide some schedule-driven mechanism for archiving logs and letting the service write to a new, empty log. Then you can decide how long you need to keep logs around, or delete them, etc.
Apache's rotatelogs is one such solution.
Perhaps Zend_Log could provide an optional decorator around whatever logging backend is chosen, to provide rotatelogs-like functionality.
Posted by Bill Karwin (bkarwin) on 2006-11-18T12:49:29.000+0000
Assigning to Gavin.
Posted by Shekar Reddy (zendfw) on 2006-11-20T01:37:54.000+0000
I've just browsed through the code on Zend_Log_Adapter_File class and noticed the code actually uses fopen, fwrite, fclose and other low-level functions and nothing is really automatic. I don't understand why no new entries could be written after the file size reaches its set limit as this is all in the 'control' of the Zend_Log_Adapter_File class. I mean - while writing back to the file, it could be a matter of reading the old content from the file if any, appending (suffix) the old content to the new content, truncating the string to whatever the limit was set as (approx) discarding from the end of the string (old content) and writing the limited string to the file. Bingo - the file size always stays in limit without any manual intervention! If the limit was set as 20k, the length of the string would be approximately 20480 chars - doesn't really need to be accurate.
It gets annoying on Windows Notepad to open up a 1.5+ MB log file because the response it too slow even on a local machine, opening the file, moving the cursor, selecting text, copying text, pasting text, finding text... It takes for ever for each of these actions.
Regards,
Posted by Gavin (gavin) on 2006-11-20T11:07:52.000+0000
Understood. Many small appliances offer exactly the feature you requested Shekar, including my DSL router/firewall at home. We will take this feature request into consideration.
Posted by Bill Karwin (bkarwin) on 2006-11-28T17:54:53.000+0000
Scheduling for release 0.8.0.
Posted by Gregory Szorc (indygreg) on 2007-02-15T00:30:48.000+0000
I'm interested in coding this feature. After a quick glance of the architecture of the log component, we need to address the following:
When a log is full, we have the options of
*# Rotating the logs (works file files, but not necessarily databases)
*# Refusing to write anything more (not sure how practical this is)
*# Removing early entries and writing at end
*# Deleting the existing log and starting fresh
*# Other actions depending on unknown adapter type
Do we support each ability?
If so, what is the most efficient way to handle it? Different log adapters are different beasts. As said, rotating is only really applicable to files. Deleting earlier entries is much easier with a database. We could always just add a prune() method to the adapter interface and go from there. We could also code up a new interface that provides the abilities to prune (delete first N entries, etc) and provide the ability for each log adapter to register a custom prune interface.
I'd like to hear other's opinions on what the best method to accomplish this is.
Posted by Shekar Reddy (zendfw) on 2007-02-15T08:16:50.000+0000
Will just two features/adapters cover max functionality for most people (80/20)?
using files
using database tables
with configuration flags to blow off entries on FIFO (based on file size, number or rows). In case of the database adapter, it could have variables for the DSN (or an existing connection), table name, column names, etc. so we do not have to extend the adapter - just invoke its methods to configure the adapter. This could keep its footprint small, simple add to performance.
Regards,
Posted by Bill Karwin (bkarwin) on 2007-03-19T16:00:18.000+0000
Assign to Bill Karwin for verification that these issues are resolved by the new Zend_Log redesign.
Posted by Mike Naberezny (mike) on 2007-04-11T12:59:35.000+0000
Comments
Posted by Matthew Ratzloff (mratzloff) on 2006-11-18T12:41:49.000+0000
I think a better solution would be something like built-in Apache rotatelogs functionality.
Posted by Bill Karwin (bkarwin) on 2006-11-18T12:49:04.000+0000
Agreed. It's not a good idea to put a hard limit on log file size, because after the log reaches that size limit, no new entries can be written to the log. Effectively, you start losing data unless someone is proactive about emptying the log files.
The better solution, used by many other software products, is to provide some schedule-driven mechanism for archiving logs and letting the service write to a new, empty log. Then you can decide how long you need to keep logs around, or delete them, etc.
Apache's rotatelogs is one such solution.
Perhaps Zend_Log could provide an optional decorator around whatever logging backend is chosen, to provide rotatelogs-like functionality.
Posted by Bill Karwin (bkarwin) on 2006-11-18T12:49:29.000+0000
Assigning to Gavin.
Posted by Shekar Reddy (zendfw) on 2006-11-20T01:37:54.000+0000
I've just browsed through the code on Zend_Log_Adapter_File class and noticed the code actually uses fopen, fwrite, fclose and other low-level functions and nothing is really automatic. I don't understand why no new entries could be written after the file size reaches its set limit as this is all in the 'control' of the Zend_Log_Adapter_File class. I mean - while writing back to the file, it could be a matter of reading the old content from the file if any, appending (suffix) the old content to the new content, truncating the string to whatever the limit was set as (approx) discarding from the end of the string (old content) and writing the limited string to the file. Bingo - the file size always stays in limit without any manual intervention! If the limit was set as 20k, the length of the string would be approximately 20480 chars - doesn't really need to be accurate.
It gets annoying on Windows Notepad to open up a 1.5+ MB log file because the response it too slow even on a local machine, opening the file, moving the cursor, selecting text, copying text, pasting text, finding text... It takes for ever for each of these actions.
Regards,
Posted by Gavin (gavin) on 2006-11-20T11:07:52.000+0000
Understood. Many small appliances offer exactly the feature you requested Shekar, including my DSL router/firewall at home. We will take this feature request into consideration.
Posted by Bill Karwin (bkarwin) on 2006-11-28T17:54:53.000+0000
Scheduling for release 0.8.0.
Posted by Gregory Szorc (indygreg) on 2007-02-15T00:30:48.000+0000
I'm interested in coding this feature. After a quick glance of the architecture of the log component, we need to address the following:
I'd like to hear other's opinions on what the best method to accomplish this is.
Posted by Shekar Reddy (zendfw) on 2007-02-15T08:16:50.000+0000
Will just two features/adapters cover max functionality for most people (80/20)?
with configuration flags to blow off entries on FIFO (based on file size, number or rows). In case of the database adapter, it could have variables for the DSN (or an existing connection), table name, column names, etc. so we do not have to extend the adapter - just invoke its methods to configure the adapter. This could keep its footprint small, simple add to performance.
Regards,
Posted by Bill Karwin (bkarwin) on 2007-03-19T16:00:18.000+0000
Assign to Bill Karwin for verification that these issues are resolved by the new Zend_Log redesign.
Posted by Mike Naberezny (mike) on 2007-04-11T12:59:35.000+0000
ZF-263