Shell tips: Finally colorized stderr, stdout!

Shell tips: Finally colorized stderr, stdout!

Something that's bothered me for years is an inability to colorize or highlight mixed stderr and stdout in a command. Particularly when using commands like strace, which dumps all execution to stderr, it's difficult to distinguish actual stderr output from stdout. Take strace for example:

$ strace dig google.com

If I'm having DNS issues or missing dependency issues, how can I sort out where my problem is in this mess? Since strace (or any binary I'm debugging) dumps its useful out to stderr (2) I can either dump it to another file and try to sort it out or grep everything here. Wouldn't it be nice to see stderr inline and highlighted to give some continuity? There is an older utility in the wild called "hilite" that you may be able to find on github but requires building from C source. I've been using a Q&D cheat sheet workaround with process substitution. Step one: I create a short coloring script inside my ~/bin dir (included in PATH).

#!/bin/bash#colorize  by JohnnyB - boeroboy@gmail.com
RED=`echo -e "\e[31m"`
RESET=`echo -e "\e[39m"`

while read line
do
  echo"$RED$line$RESET"
done < "${1:-/dev/stdin}"

Then I'm able to easily colorize stderr on any command. Trying again, I put the above colorize script into my PATH and then I can do this handy process substitution on output 2 aka stderr (bash shell):

bash$ strace dig google.com 2> >(colorize)

or this way (fish shell):

fish$ strace dig google.com 2>| colorize

I hope this helps anybody out when they're digging through a pile of debug info. Note that you can also modify colorize script or parameterize it or use sed to highlight certain selections different colors. Very handy.

JohnnyB


John, Cool and helpful idea

Like
Reply
Kevin Barth

Principal UNIX HA/DR Consultant, KJB Software Services

7y

Hi John -- interesting use of process substition there. Here's what I think: A much more portable way to introduce color is to use the tput utility to get the color-strings for the environment dynamically -- I often use the following in my scripts: # # Use the tput(1) command to obtain color-codes for the current terminal # declare -x tputReset=$(tput sgr0) declare -x tputRed=$(tput setaf 1) # ERROR-level messages declare -x tputBold=$(tput bold) declare -x tputYellow=$(tput setaf 3) # WARNING-level messages declare -x tputGreen=$(tput setaf 2) Then, instead of your line: echo -e "$RED$line$RESET" I go with: echo -e "${tputRed}$line${tputReset}" BUT, and this is a big "BUT" for me, using your idea to make the stderr messages show up in red by invoking commands like: # ls aNonExistentFile aFileThatDoesExist 2> >(colorize) Does not work so well because they often out in wrong order, which can be confusing in more realistic examples. That is becuase of line buffering by the redirection of stderr to your colorize process. Basically, according to what I have been able to understand on this (only minimal research), the shell (or something else) will NOT buffer lines when they are to the normal stdout and stderr, but, as soon as you redirect, something changes there and the output is buffered (as per standard UNIX I/O bufferinf), which causes the stdout and stderr messages to end up in the wrong order (though it does seem a bit random at times). What is interesting is that your example does not reveal this, and, to be honest, using your method, where the color codes are "hard-coded" as 'RED="\033[0;31m" '; the order did seem to be more correct, but not always. BTW: all my 20 minutes of testing was on OS X 10.11.6 ( kerel release 15.6.0 ) and BASH 4.3.42. I just haven't tried in under Linux nor Solaris yet (sorry). Anyway, thanks for that very interesting way to redirect stderr to a process -- that I did not know, though I use the other form, ' < <(someProcess)', all the time in my while loops, etc... All the best good buddy, Kevin

Like
Reply

Cool idea. Thanks for sharing. Your script didn't work for me out of the gate (stupid OS/X). Here is my version: #!/bin/bash #colorize by JohnnyB - boeroboy@gmail.com RED="\033[0;31m" RESET="\033[0m" while read line do echo -e "$RED$line$RESET" done < "${1:-/dev/stdin}" But then I thought it could be a touch simpler with perl #!/usr/bin/env perl #colorize by JohnnyB - boeroboy@gmail.com $e=chr(27); while (<>) { print "${e}[0;31m$_${e}[0m"; }

Sam Bracke

Cloud Architect at Descartes Systems Group

7y

First time in a long while I've seen some actual job related content on this so-called "professional" networking platform. Nice one John!

To view or add a comment, sign in

Insights from the community

Explore topics