Parsing logs in sed efficiently

Posted on

Problem

I have an application which log file has most lines start with a time/date, followed by a debugging level and the actual log message. Often I want to watch the output live and colors make it a lot better readable. This is what I use at the moment for piping the logs:

while read line; do
echo "$line" |
sed -e "s,^.* DEBUG,$(tput setaf 2)&$(tput sgr0)," 
    -e "s,^.* INFO,$(tput setaf 2)&$(tput sgr0)," 
    -e "s,^.* WARN,$(tput setaf 3)&$(tput sgr0)," 
    -e "s,^.* ERROR,$(tput setaf 1)&$(tput sgr0)," 
    -e "s,^.* SEVERE,$(tput setaf 1)$(tput bold)&$(tput sgr0),"
done

The call on the command line looks like this:

tail -fn 200 /path/server.log | ./logparser

This all works as expected but when many lines are printed at once the parser lags behind. I think this is because I run sed once for every single line. Is there a better solution?

Solution

Thanks to itsbruce this runs much faster:

sed -e "s,^.* DEBUG,$(tput setaf 2)&$(tput sgr0)," 
    -e "s,^.* INFO,$(tput setaf 2)&$(tput sgr0)," 
    -e "s,^.* WARN,$(tput setaf 3)&$(tput sgr0)," 
    -e "s,^.* ERROR,$(tput setaf 1)&$(tput sgr0)," 
    -e "s,^.* SEVERE,$(tput setaf 1)$(tput bold)&$(tput sgr0),"

Note: first two and the last line are removed.

Since sed takes the STDIN, this basically starts one sed instance for whole stream in contrast to one per line.

Leave a Reply

Your email address will not be published. Required fields are marked *