Dumping content to files using Log::Dispatch::Dir

Logging frameworks like Log::Log4perl and Log::Dispatch are great. They relieve you of the burden of reinventing your own (which, admit it, will probably suck more), and they decouple your code from logging details (which can be changed later independently). You just say I want to log "something something" in the code, and you can later configure whether those messages are actually written to the logs, where those logs are, how the messages are formatted, etc without changing your log-using code.

Logging can be used, to some extent, to replace debugging or to aid it. Each log message is usually only a single line, like "Starting foo ...", "Ended foo ...", "The value of foo is $val", although we can also log large data structure dumps or file contents.

When writing web robots like HTML scrapers or interfaces to online banking sites, which are particularly fragile, it is often convenient to save each server's full response into a separate file so you can easily check each step by opening the saved file in a browser.

If you want to use Log::Log4perl or Log::Dispatch for this, you can too, using Log::Dispatch::Dir. This module will write each log message to a separate file in a specified log directory.

An example, in Finance::Bank::ID::Base I have code like this:

$self->logger_dump->trace(
    "<!-- result of mech request #$i ($method ".Dump($args)."):\n".
    $mech->response->status_line."\n".
    $mech->response->headers->as_string."\n".
    "-->\n".
    $mech->content
);

where $mech is a WWW::Mechanize object. I use $self->logger for "normal" log messages and $self->logger_dump specifically for dumping contents. Both the logger and logger_dump attributes can be supplied by the module user, e.g.:

my $ibank = Finance::Bank::ID::BCA->new(
    ...
    logger      => Log::Log4perl->get_logger("Messages"),
    logger_dump => Log::Log4perl->get_logger("Dumps"),
);

and the Log::Log4perl configuration is something like this:

log4perl.logger.Messages=TRACE, SCREEN, LOGFILE
log4perl.logger.Dumps=TRACE, LOGDIR

log4perl.appender.SCREEN=Log::Log4perl::Appender::ScreenColoredLevels
log4perl.appender.SCREEN.layout=PatternLayout
log4perl.appender.SCREEN.layout.ConversionPattern=[\%r] %m%n

log4perl.appender.LOGFILE=Log::Log4perl::Appender::File
log4perl.appender.LOGFILE.filename=/path/to/logs/main.log
log4perl.appender.LOGFILE.layout=PatternLayout
log4perl.appender.LOGFILE.layout.ConversionPattern=[\%d] %m%n

log4perl.appender.LOGDIR=Log::Dispatch::Dir
log4perl.appender.LOGDIR.dirname=/path/to/logs/dumps
log4perl.appender.LOGDIR.layout=PatternLayout
log4perl.appender.LOGDIR.layout.ConversionPattern=%m

This is convenient enough for me, but in the future I want to do some MIME checking to the log messages, so Log::Dispatch::Dir can automatically add a suitable file extension e.g. .html, .txt, .jpg, etc.