out: an output utility

Intro

Whilst writing the tests for the Job Logging feature in Upstart, I discovered that I needed some very specific job behaviour that was not easy to produce using any simple existing utility I could find.

The Problem

What I wanted was a utility that would produce specified amounts of data to either standard output, standard error or direct to the terminal without using shell redirection. The reason for eschewing the shell being that Upstart is clever and intelligently determines whether a shell should be involved. From a user perspective, that's fantastic, but for testing purposes, I needed to force Upstart down particular code paths where it would not for example automatically pass the job through a shell. In fact the constraint was even more restrictive; what I really wanted to do was this:
Produce output (including null bytes) to stdout, stderr or the terminal without using any shell meta characters.
Unless I used some extremely esoteric shell, that effectively meant "no shells" ;-) ksh and zsh actually came close as their printf/print shell built-ins allow output to a specified file descriptor, but not to the terminal. Anyway, by this stage, I'd decided I wanted a generic utility, not some specific shell feature.

The Pragmatic Solution

For the Upstart tests, I ended up using various different command-line utilities for different tests that could produce the data I wanted. However, I couldn't help thinking there should be a better way. Yes, I could have written a bespoke C program or Python/Perl script to achieve my goal, but in the spirit of Larry Walls view that "Easy things should be easy", I thought that there should be a simpler method.

The Improved Solution


In the end this niggling problem got the better of me so I decided to write a tool to satisfy the above constraint. What I came up with is a small utility which sits somewhere between echo(1) and printf(1) (with just a dash of seq(1) thrown in) which, since it produces output, I've called simply "out".

 Here's an example of how you could print "hello" direct to the terminal (the redirection here is just to show that you still get output even though we've discarded stdout and stderr output):
$ out -t hello >/dev/null 2>&1
hello


This command below will echo "hello" to stdout, the terminal and stderr:


$ out hello -t -r 1 -e -r 1

Along with allowing output redirection to any file descriptor or the terminal without the support of shell redirection, it supports the standard C-style escape sequences (like '\n' and '\a'). It also allows you to redefine the escape character.



Additionally, out supports printf(1)-style Unicode escape sequences such as '\uFFFFFFFF' . After adding the Unicode support, I thought it would be rather fun to add ranges (similar to bash "sequence expressions") so out also supports some new sequences that allow sets of characters to be generated. These are great fun for exploring Unicode. Here's an example of generating the alphabet in lower-case:


$ out '\{a..z}\n'
abcdefghijklmnopqrstuvwxyz


How about some Greek?


$ out "\{α..ω}\n"
αβγδεζηθικλμνξοπρςστυφχψω




Or we could specify this using Unicode characters to get the same result:




$ out "\{\u03b1..\u03c9}\n"
αβγδεζηθικλμνξοπρςστυφχψω


And here's how you could generate the Unicode braille block:


$ out '\{\u2800..\u28FF}\n'
⠀⠁⠂⠃⠄⠅⠆⠇⠈⠉⠊⠋⠌⠍⠎⠏⠐⠑⠒⠓⠔⠕⠖⠗⠘⠙⠚⠛⠜⠝⠞⠟⠠⠡⠢⠣⠤⠥⠦⠧⠨⠩⠪⠫⠬⠭⠮⠯⠰⠱⠲⠳⠴⠵⠶⠷⠸⠹⠺⠻⠼⠽⠾⠿⡀⡁⡂⡃⡄⡅⡆⡇⡈⡉⡊⡋⡌⡍⡎⡏⡐⡑⡒⡓⡔⡕⡖⡗⡘⡙⡚⡛⡜⡝⡞⡟⡠⡡⡢⡣⡤⡥⡦⡧⡨⡩⡪⡫⡬⡭⡮⡯⡰⡱⡲⡳⡴⡵⡶⡷⡸⡹⡺⡻⡼⡽⡾⡿⢀⢁⢂⢃⢄⢅⢆⢇⢈⢉⢊⢋⢌⢍⢎⢏⢐⢑⢒⢓⢔⢕⢖⢗⢘⢙⢚⢛⢜⢝⢞⢟⢠⢡⢢⢣⢤⢥⢦⢧⢨⢩⢪⢫⢬⢭⢮⢯⢰⢱⢲⢳⢴⢵⢶⢷⢸⢹⢺⢻⢼⢽⢾⢿⣀⣁⣂⣃⣄⣅⣆⣇⣈⣉⣊⣋⣌⣍⣎⣏⣐⣑⣒⣓⣔⣕⣖⣗⣘⣙⣚⣛⣜⣝⣞⣟⣠⣡⣢⣣⣤⣥⣦⣧⣨⣩⣪⣫⣬⣭⣮⣯⣰⣱⣲⣳⣴⣵⣶⣷⣸⣹⣺⣻⣼⣽⣾⣿


out also allows arbitrary delays to be introduced, repeats, basic random character generation and a few other tidbits. Here is the man page including a lot of examples:






OUT(1)         OUT(1)



NAME
       out - manual page for out


SYNOPSIS
       out [OPTION]... [STRING]...

DESCRIPTION
       Echo strings to specified output stream(s).

OPTIONS
       -a, --intra-char=<char>
       Insert  specified  character  (which may be a 1-character escape
       character) between all output characters.

       -b, --intra-pause=<delay>
       Pause between writing each character.

       -e, --stderr
       Write subsequent strings to standard error (file descriptor 2).

       -h, --help
       This help text.

       -i, --interpret
       Interpret escape characters (default).

       -l, --literal
       Write literal strings only (disable escape characters).

       -o, --stdout
       Write subsequent strings to standard output (file descriptor 1).

       -p, --prefix=<prefix>
       Use <prefix> as escape prefix (default='\').

       -r, --repeat=<repeat>
       Repeat previous value <repeat> times.

       -s, --sleep=<delay>
       Sleep for <delay> amount of time.

       -t, --terminal
       Write subsequent strings directly to terminal.

       -u, --file-descriptor=<fd> Write to specified file descriptor.

       -x, --exit=<num>
       Exit with value <num>.

ESCAPE CHARACTERS
       out recognises C-style escape sequences as  used  by  printf(1) .   By
       default an  escape  sequence  is introduced by the backslash character
       ('\'), however this may be changed with the -p option.  out  also  sup‐
       ports some additional sequences:

       \0     - nul byte (hex value 0x00)

       \a     - alert (bell)

       \b     - backspace

       \c     - no further output

       \e     - escape character (used for changing terminal attributes)

       \f     - form feed

       \g     - generate pseudo-random printable character

       \n     - newline

       \oNNN  - byte with octal value NNN (1 to 3 digits)

       \r     - carriage return

       \t     - horizontal tab

       \uNNNN -  2  byte Unicode (ISO/IEC 10646) character with hex value NNNN
       (4 digits)

       \UNNNNNNNN
       - 4 byte Unicode  (ISO/IEC  10646)  character  with  hex  value
       NNNNNNNN (8 digits)

       \v     - vertical tab

       \xNN   - byte with hexadecimal value NN (1 to 2 digits)

RANGE ESCAPES
       out also supports range escapes which allow a range of characters to be
       specified in a compact format.

       \{N..N}
       - specify a range by two 1-byte literal characters.

       \{oNNN..oNNN}
       - specify a range by two 3-byte octal values.

       \{uNNNN..uNNNN}
       - specify a range by two 2-byte Unicode values.

       \{UNNNNNNNN..UNNNNNNNN}
       - specify a range by two 4-byte Unicode values.

       \{xNN..xNN}
       - specify a range by two 2-byte hex values.

       Note that ranges take two values of the same type and the maximum width
       for that type must be specified.

NOTES
       ·   Arguments are processed in order.

       ·   With the exception of '-x', arguments may be repeated any number of
    times.

       ·   All output will be sent to standard output until  an  output  redi‐
    rection  option is specified that changes the output stream (namely
    -e or -t (or their  long-option  equivalents),  or  if  output  has
    already been redirected -o (or its long-option equivalent)).

       ·   If  <str>  is  the empty string ("" or '') it will be treated as \0
    such that a nul byte will be displayed.

       ·   To cancel the effect of -a, specify a null string: -a ''.

       ·   If <repeat> is '-1', repeat forever.

       ·   Replace the 'Z' in the range formats  above with  the  appropriate
    characters.

       ·   Ranges can be either ascending or descending.

       ·   <delay>  can  take  the  following  forms where <num> is a positive
    integer:

    <num>ns : nano-seconds (1/1,000,000,000 second)
    <num>us : micro-seconds (1/1,000,000 second)
    <num>ms : milli-seconds (1/1,000 second)
    <num>cs : centi-seconds (1/100 second)
    <num>ds : deci-seconds (1/10 second)
    <num>s  : seconds
    <num>m  : minutes
    <num>h  : hours
    <num>h  : days
    <num>   : seconds

    If <num> is -1, wait until any signal is received.
    If signal is SIGNUM continue, else exit immediately.

       ·   Generated printable random characters may not  display  unless  you
    are using an appropriate font.

EXAMPLES
 # Print "foofoofoo" to stderr, followed by "barbar" to stdout.
 out "foo" -r 2 -o "bar" -r 1

 # Write 50 nul bytes direct to the terminal.
 out -t "" -r 49

 # Write continuous stream of nul bytes direct to the terminal,
 # 1 per second.
 out -b 1s -t '' -r -1

 # Display a greeting slowly (as a human might type)
 out -b 20cs "Hello, $USER.\n"

 # Display a "spinner" that loops 4 times.
 out -b 20cs -p % "%r|%r/%r-%r\%r" -r 3

 # Display all digits between zero and nine with a trailing
 # newline.
 out "\{0..9}\n"

 # Display slowly the lower-case letters of the alphabet,
 # backwards without a newline.
 out -b 1ds "\{z..a}"

 # Display upper-case 'ABC' with newline.
 out '\u0041\u0042\u0043\n'

 # Display 'foo' with newline.
 out '\o146\u006f\x6F\n'

 # Clear the screen.
 out '\n' -r $LINES

 # Write hello to stdout, stderr and the terminal.
 out 'hello' -t -r 1 -e -r 1

 # Display upper-case letters of the alphabet using octal
 # notation, plus a newline.
 out "\{\o101..\o132}"

 # Display 'h.e.l.l.o.' followed by a newline.
 out -a . "hello" -a '' "\n"

 # Display upper-case and lower-case letters of the alphabet
 # including the characters in-between, with a trailing newline.
 out "\{A..z}\n"

 # Display lower-case alphabet followed by reversed lower-case alphabet
 # with the digits zero to nine, then nine to zero on the next line.
 out "\{a..z}\{z..a}\n\{0..9}\{9..0}\n"

 # Display lower-case Greek letters of the alphabet.
 out "\{α..ω}"

 # Display cyrillic characters.
 out "\{Ѐ..ӿ}"

 # Display all printable ASCII characters using hex range:
 out "\{\x21..\x7e}"

 # Display all printable ASCII characters using 2-byte UTF-8 range:
 out "\{\u0021..\u007e}"

 # Display all printable ASCII characters using 4-byte UTF-8 range:
 out "\{\U00000021..\U0000007e}"

 # Display all braille characters.
 out "\{\u2800..\u28FF}"

 # Display 'WARNING' in white on red background.
 out '\e[37;41mWARNING\e[0m\n'

 # Generate 10 random characters.
 out '\g' -r 9


AUTHOR
       Written by James Hunt <james.hunt@ubuntu.com>

COPYRIGHT
       Copyright © 2012 James Hunt <james.hunt@ubuntu.com>

LICENSE
       GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
       This  is  free  software:  you  are free to change and redistribute it.
       There is NO WARRANTY, to the extent permitted by law.

SEE ALSO
       echo(1) printf(1)



User Commands     2012-09-10    OUT(1)




Ideas



The TODO file in the distribution contains a number of ideas to enhance outs abilities.





Code




See Also















































Comments











Post a Comment



















Popular posts from this blog









Byobu Bling with Unicode Custom Indicators




















Image







 Introduction  In these heady times of almost daily advances in UI pizzazz, you could be forgiven for thinking the terminal is somewhat of a dead-zone when it comes to excitement. Oh how wrong you would be though...!   If you've never used  Byobu ,  Dustin's  awesome text-based window manage, sudo apt-get install  it without delay to be bathed in pure terminal goodness. As shown below, byobu comes with a whole host of standard indicators, allowing the display of useful snippets of information.       The indicators are minimal out of terminal-width real estate necessity, but that's also their strength - they provide just the details you really care about in a tight ASCII format. And they're cute of course :-)   However, byobu has a well-kept secret: when running with the tmux  backend, it supports Unicode indicators. So with the proviso that you are running " byobu-tmux " rather than " byobu-screen ", you can squeeze down those indicators even further...








Read more










Procenv 0.46 - now with more platform goodness





















 I have just released procenv version 0.46 . Although this is a very minor release for the existing platforms (essentially 1 bug fix), this release now introduces support for a new platform...    Darwin  Yup - OS X now joins the ranks of supported platforms.   Although adding support for Darwin was made significantly easier as a result of the recent internal restructure of the procenv  code, it did present a challenge: I don't own any Apple hardware. I could  have borrowed a Macbook, but instead I decided to see this as a challenge:    Could I port procenv  to Darwin without actually having a local Apple system?    Well, you've just read the answer, but how did I do this?    Stage 1: Docker   Whilst surfing around I came across this interesting docker image:    https://hub.docker.com/r/andrewd/osxcross/     It provides a Darwin toolchain that I could run under Linux. It didn't take very long to follow my own instructions on porting procenv to a new platform . But although I...








Read more










Coverity static analysis for C, C++ and Java code





















 It's a well known principle of software engineering that the earlier bugs can be caught, the lower the overall cost. As such, testing needs to happen at every  level. Once your project is at the coding stage, the earliest form of testing is on the code itself , not on the binaries the compiler produces.   We run a variety of tools over critical codebases such as Upstart and Whoopsie  regularly to identify issues well before they "escape into the wild". These tools include Coverity Scan  (see the list of projects already using it ).   If you really care about your code and you are involved with a C, C++ or Java project, I'd strongly encourage you to take a look at this awesome tool. If you aren't  directly involved in such projects, try contacting those running them and suggesting they use Coverity.   The Coverity Scan service is entirely free for OSS projects. You will need to register to obtain an account and then download the client analysis tool. Once setup, a...








Read more