Monday, 14 October 2013

Procenv 0.27 released

Procenv 0.27 has been released. This release introduces a raft of new features...


 It is now possible to display details of the following IPC mechanisms:

  • message queues
  • semaphores
  • shared memory
Alas, this feature is not available on BSD's as yet partly since there appears to be no documented way to query these mechanisms.

Output Categories

The introduction of the IPC categories brings the total number of output categories to 32:
  • meta
  • arguments
  • capabilities
  • cgroups
  • clocks
  • compiler
  • confstr
  • environment
  • file descriptors
  • libraries
  • limits
  • locale
  • misc
  • message queues
  • mounts
  • network
  • oom
  • platform
  • process
  • ranges
  • rusage
  • semaphores
  • shared memory
  • signals
  • sizeof
  • stat
  • sysconf
  • threads
  • time
  • timezone
  • tty
  • uname

Highly-Structured Output

The code that handles procenv has been completely rewritten so that all output is highly structured. So, rather than displaying file descriptors like this:

$ procenv --fds
  fd 0: terminal=yes ('/dev/pts/12')
  fd 1: terminal=no
  fd 2: terminal=yes ('/dev/pts/12')
fds (linux/proc):
  '/proc/self/fd/0' -> '/dev/pts/12' (terminal=yes, valid=yes)
  '/proc/self/fd/1' -> 'procenv.log' (terminal=no, valid=yes)
  '/proc/self/fd/2' -> '/dev/pts/12' (terminal=yes, valid=yes)

... they are now displayed like this:

$ procenv --fds
file descriptors:
    terminal: yes
    valid: yes
    device: /dev/pts/25
    terminal: no
    valid: yes
    device: procenv.log
    terminal: yes
    valid: yes
    device: /dev/pts/25

Note that not only is the output more structured, but for file-descriptor output there is now only a single section combining details of both the generic and Linux-specific details.

Format Version

Since it may be necessary to modify the output format, procenv now includes a format-version field in the meta section. This value (currently 1), will be incremented for each output format change. The format-version combined with the version of procenv itself should ease any version transition issues.

Tweaking the Output

If the default space indenting offends your sensibilities, you'll be glad to know that you can now change it to use tabs instead (or in fact any other character you choose):


Maybe you'd prefer 3 tab indents:

--indent-char="\t" --indent=3

Output Formats

The new highly-structured text output is in fact a side-effect of the fact that procenv can now produce output in other formats. The default is still text, but it can now produce JSON and XML. This even works when running --version:

$ procenv --version

$ procenv --format=json --version
  "version" : {
    "name" : "procenv",
    "version" : "0.27",
    "author" : "James Hunt <>"

$ procenv --format=xml --version
<?xml version="1.0" encoding="UTF-8"?>
<procenv version="0.27" package_string="procenv 0.27" mode="non-privileged" format_version="1">
  <section name="version">
    <entry name="name">procenv</entry>
    <entry name="version">0.27</entry>
    <entry name="author">James Hunt &lt;;</entry>

This makes procenv output much more consumable by standard tools.


However, the new highly-structured output introduces a problem. Before its introduction, it was easy to use grep(1) to extract particular values. But now, some of those values are on different lines. Yes, you can use the magic grep(1) options -A, -B and -C, but if that isn't convenient for you, procenv does offer an alternative in the form of an output format I've called "crumb" (short for "breadcrumb"). Here's the file descriptor output in breadcrumb output:

$ procenv --format=crumb --fds
file descriptors:0:terminal:yes
file descriptors:0:valid:yes
file descriptors:0:device:/dev/pts/25
file descriptors:1:terminal:yes
file descriptors:1:valid:yes
file descriptors:1:device:/dev/pts/25
file descriptors:2:terminal:yes
file descriptors:2:valid:yes
file descriptors:2:device:/dev/pts/25

As you can see, "crumb mode" shows every value with the appropriate sections the particular value belongs prior to the value. So the unique path to each value is shown by way of a list of "breadcrumbs" or headings. Thus, to extract all details of stdin (aka file descriptor zero):

$ procenv --format=crumb --fds | grep "^file descriptors:0"
file descriptors:0:terminal:yes
file descriptors:0:valid:yes
file descriptors:0:device:/dev/pts/25

It's also really easy to generate CSV data if you wanted to import the output into a spreadsheet:

$ procenv --format=crumb --crumb-separator=',' --separator=','

Download it from:

Tuesday, 27 August 2013

procenv 0.26 released

Version 0.26 of the procenv utility is now available.


  • Check to determine if running on a console now works for FreeBSD/kFreeBSD too.
  • Added ability to show all arguments (-A/--arguments)
    (useful when using --exec).
  • Added ability to display network details (-N/--network).
  • Added BSD/Hurd-specific signals.
  • Corrected output sort order.
  • Mount details now include block, inode and fsck details.
There are now 29 categories of environment information displayed when procenv is run (on Linux).

Grab it from:

This update should appear in Debian, Ubuntu and FreeBSD soon...

Friday, 23 August 2013

Upstart 1.10 released

Lots of goodness in this release (explanatory posts to follow):

  • upstart-local-bridge: New bridge for starting jobs on local socket connections.
  • upstart-dconf-bridge: New bridge for Session Inits to react to dconf/gsettings changes.
  • upstart-dbus-bridge: New '--bus-name' option to allow bus name variable to be included in dbus-event(7).
  • New "reload signal" stanza to allow jobs to specify a custom signal that will be sent to the main process (rather than the default SIGHUP).
  • Inclusion of Session Init sample jobs.
  • Re-exec fixes for handling chroot sessions.
  • Shutdown fix for Session Inits.
  • New python3 module and accompanying integration test suite for testing Upstart running as PID 1 and as a Session Init (privileged and non-privileged).

The Upstart cookbook has been updated for this release.

Saturday, 10 August 2013

Upstart Cookbook now updated for Upstart in Debian

I've started a long-overdue update to the Upstart Cookbook for Upstart on Debian. Look out for the Debian swirl! :-)

procenv 0.25 now builds on Android

Procenv 0.25 now builds natively under Android:

It also fixes an interesting issue that was causing it to fail to generate FreeBSD build logs. However, that is now fixed so you can see the FreeBSD build environment now too by looking at the port logs:

More logs to come - the FreeBSD ports seem to build rather slowly...)

Monday, 5 August 2013

Counting down to Ubuntu and Debian releases with distro-info

A while back I though, "wouldn't it be great if the Ubuntu release schedule could be accessed as some form of web service?" That would allow all sorts of fun scripts to be written. Yes, you could write code to parse the wiki page, but I think the technical term for that is "gross".

However, there is now a much simpler alternative in the form of distro-info. If you've never used this awesome tool written by Benjamin Drung you've been missing out! It can tell you all sorts of interesting information. A few examples...

Examples of how to use distro-info

  • What is the current development release on the system I am running on?

    $ distro-info --devel
  • What is the current Debian development release?

    $ debian-distro-info --devel
  • What are the currently supported Ubuntu releases?

    $ ubuntu-distro-info --supported
  • What's the latest Ubuntu LTS release?
    $ ubuntu-distro-info --lts
  • What's the current stable Debian release?

    $ debian-distro-info --stable

distro-info... Now with Added Milestone Goodness

I've been working with Benjamin recently to add a new feature which landed late last week in both Debian unstable and Ubuntu Saucy. So, we can now ask distro-info when a particular release milestone will happen (or has already happened):

  • How many days until Ubuntu Saucy is released?

    $ ubuntu-distro-info --devel --days
  • How many days since the Ubuntu Saucy release was "created"?

    $ ubuntu-distro-info --devel --days=created
  • How many days until Ubuntu Precise goes "end-of-life" (desktop only!)

    $ ubuntu-distro-info --lts --days=eol
  • How many days since Debian sid was created?

    $ debian-distro-info --devel --days=created

A Scripted Example

We can now write simple little scripts like this to show a popup when getting close to a release:

set -- $(distro-info --devel --codename --days=release)


# Only warn close to a release
[ "$days" -gt 30 ] && exit 0

zenity --info --text "$codename is released in $days days!"

Wait! Let's do that with Upstart!

In fact, you could turn the script above into an Upstart session job rather easily:
cat >>$HOME/.config/upstart/release-reminder.conf<<EOT
start on desktop-start

  set -- $(distro-info --devel --codename --days=release)


  # Only warn close to a release
  [ "$days" -gt 30 ] && exit 0

  zenity --info --text "$codename is released in $days days!"
end script
Now, this script will run automatically every time you login, assuming you are running either:
The currently recognised milestones are "created", "release", "eol" and for Ubuntu LTS releases only "eol-server".

Further Ideas

We could conceivably enhance this feature further by adding in support for querying additional milestones such as "alpha-1" and "feature-freeze".

Want to know more?

For full details, read distro-info(1).

I'm going to DebConf13

Going to DebConf13! Vorlon and I will be giving a talk on Upstart ('Upstart in Debian'). So if you'd like to learn more about the awesome next-gen event-based init system, please drop by!

Friday, 17 May 2013

A simple two-player QML game for Ubuntu Touch using the Ubuntu SDK: noughts and crosses (aka tic-tac-toe)!

Inspired by Rick's recent blog posts, and keen to write a blog post with a ridiculously long title, I've been reading up on QML recently. Still bearing the scars from the XML horrors of working with J2EE in the early days, that 3-byte acronym ending in "ML" initially subconsciously somewhat filled me with trepidation. However, as soon as I actually saw some QML, I could see these fears were unfounded (! :-)  And in fact I now love QML. It's clean, elegant, powerful, declarative and (OMG YAY!) you can even "%-bounce" on the braces in vim! :-) That said, the qtcreator IDE is extremely good, managing to provide just enough of what you want without requiring endless additional configuration.

But it doesn't stop there. The Design Team have done some incredible work in creating the Ubuntu SDK components: not only do they look fantastic (if you have the ubuntu-ui-toolkit-examples package installed, try running /usr/lib/ubuntu-ui-toolkit/demos/launch_componentshowcase), they are also extremely flexible and powerful.

As Rick has mentioned, it does take a while to grok the "QML-ish" way of doing things. And if like me you spend most of your time writing in imperative languages, initially you just think "all this QML is wonderful, but where do I actually put the code?". But then you have the epiphany moment when you realise you're already writing "the code" - in many cases, you don't need anything beyond the declarative QML itself.

I Need an Itch to Scratch

The only real way to learn a new language is to use it. But what to do? I wanted to code something simple and fun, like a game. There are already few games on the Collections page so I needed to think of a really simple one that is also fun to play. How about a game that even children can appreciate? Of course - Noughts and Crosses (aka tic-tac-toe)!
Note that the code is pretty rudimentary right now, but it's just about usable ;-)


This is a simple game so we only need a few objects: Cell, Game and MainView.

The MainView is the container for the application and includes a Page and the actual Game object. The only property we specify for the game is the boardSize of 3 giving us a 3x3 board. Technically, we don't actually even need to specify this since -- as we're about to see -- 3x3 is the default board size anyway. So, the Game object could be specified minimally as "Game {}". However, I've left it specified as a reminder to myself that ultimately I'd like to pass a variable to allow the board size to be specified at game creation time.

Here is a slightly simplified version of the MainView (noughts-and-crosses.qml):

import QtQuick 2.0
import Ubuntu.Components 0.1

MainView {
    Page {
        title: "Noughts and Crosses"

        id: page

        Game {
            // change this to whatever value you want for an NxN-sized board
            boardSize: 3

The Game object is a Column and comprises a Label, to show some text relating to the game, and a Grid to actually represent the game. There is some magic going on in the grid as it uses the very cool Repeater object to make laying out the grid easy: for a 3x3 board it creates 9 Cell objects and packs them into the grid. Here's a cut-down version of the Game object:

Column {

    property alias boardSize: gameGrid.boardSize

    Label {
        id: text
        text: "Noughts goes first"

    Grid {
        id: gameGrid

        // Default to a 3x3 board (we only support square boards).
        property real boardSize: 3

        // toggled between "O" and "X". The value specified below denotes
        // which side goes first.
        property string player: "O"

        columns: boardSize
        rows: boardSize

        // layout the appropriate number of cells for the board size
        Repeater {
            id: gridRepeater
            model: boardSize * boardSize

            Cell {
                width: 100
                height: width

Note the property alias for boardSize in the Column object - it exposes a boardSize variable which is just a way to access the real variable of the same name within the Grid object. Note too that we tell the Grid object its dimensions by setting its columns and rows properties.

The Game object also contains a chunk of Javascript in the form of the checkForWin() function to determine whether a move resulted in the game being won.

The Cell object is the most interesting object. A Cell represents an individual location on the board. It is constructed from a Rectangle and comprises a Text value. The text value is either a middle-dot (to denote the cell has not yet been selected), a "O" or a "X". It also includes a MouseArea that specifies the new cell state to apply when the cell is clicked. Initially, the state is middle-dot but when the cell is clicked, the state is changed to the value of the parent (Game) objects player property. The Cell object specifies 3 states to represent every possible value a Cell can display. What's neat here is that changing the cells state also toggles the parent (Game) objects player property which allows the game to proceed with each player taking a turn. Clicking a cell also calls the checkForWin()function to determine if a particular turn results in the game being won. Here's the complete Cell object:

Rectangle {
    id: cell
    state: gameGrid.defaultText

    property alias textColor: cellText.color

    Text {
        id: cellText
        text: parent.state
        color: "silver"
        anchors.horizontalCenter: parent.horizontalCenter
        anchors.verticalCenter: parent.verticalCenter
        font.pointSize: 48

    states: [
        State {
            name: cell.parent.defaultText
            PropertyChanges { target: gameGrid; player: "" }
        State {
            name: "O"
            PropertyChanges { target: gameGrid; player: cell.state }
        State {
            name: "X"
            PropertyChanges { target: gameGrid; player: cell.state }

    // when clicked,
    MouseArea {
        anchors.fill: parent
        onClicked: {
            cell.state = (gameGrid.player == "O" ? "X" : "O");
            gameGrid.numberTurns += 1

Winning Algorithm

The approach I've taken is very simplistic: just scan each row, column and diagonal looking for a winning run. This isn't particularly efficient (we're scanning the board multiple times) but that's not a problem for small board sizes. However, it has two fairly compelling attributes:

  • It's simple to understand
  • It works for arbitrary-sized boards.

My favourite alternative algorithm is to make use of the properties of Magic Squares. Using these, you can scan the board a single time to determine if a player has one. This is achieved by determining if a cell has been selected by a player and if so incrementing their counter based on the magic square value for that index. For a 3x3 board, if a players total equals 15, they win!


So, what does it look like at this early beta stage...?

Start of a new game:

We have a winner!

Another winner on a 7x7 board (the person playing crosses needs more practice me thinks :-):

What's Next

  • The javascript code is currently horrid and needs to be refactored with dynamite.
  • Add ability to play "the computer".
  • Config option to allow variable-sided playing grids.
  • Once the game is stopped, we need to disallow further board clicks.
  • Leverage more QML facilities to simplify the code further.
  • Visual improvements (animation for a winning run maybe?)
  • Ability to change player that starts.
  • Score-keeping and "best of 'n' games " support (particularly useful when the kids beat you repeatedly ;-)
  • Menu to start new game.
The code is on github, so get forking!

In Conclusion

My "clean-room" implementation is far from perfect at the moment, but it's been a fantastic learning exercise so far and a lot of fun!

There are of course other QML noughts-and-crosses games out there. They come with varying licenses, some use C++ for the game logic, and most -- if not all -- are hard-coded to produce a 3x3 board only. Additionally, they generally use graphical representations for the noughts and crosses whereas here, I'm just using styled text. If you're interested, compare my github code with, for example, the Qt version to see the different approaches taken:

See Also

Thursday, 18 April 2013

Observing the initial LXC environment using procenv

If you're interested in seeing the initial environment inside an LXC container, here's how:
  1. Install procenv inside the container:

    $ sudo apt-get install -y procenv

  2. Shutdown the container:

    $ sudo shutdown -h now
  3. Boot the container (mine is called "raring" in this example) like this:

    $ sudo lxc-start -n raring --console-log /tmp/lxc-console.log \
        -- /usr/bin/procenv --file=/dev/console --exec \
        -- /sbin/init --debug
  4. View the /tmp/lxc-console.log logfile in the host environment.
Note that those two sets of non-option double-dashes are required; the first tells LXC to treat what follows as a command to run after the container starts (in this case procenv), and the second set tells procenv to treat what follows as a command to run after it has finished running (in this case Upstart (/sbin/init :-))!

Monday, 15 April 2013

Byobu Bling with Unicode Custom Indicators


In these heady times of almost daily advances in UI pizzazz, you could be forgiven for thinking the terminal is somewhat of a dead-zone when it comes to excitement. Oh how wrong you would be though...!

If you've never used ByobuDustin's awesome text-based window manage, sudo apt-get install it without delay to be bathed in pure terminal goodness. As shown below, byobu comes with a whole host of standard indicators, allowing the display of useful snippets of information.

The indicators are minimal out of terminal-width real estate necessity, but that's also their strength - they provide just the details you really care about in a tight ASCII format. And they're cute of course :-)

However, byobu has a well-kept secret: when running with the tmux backend, it supports Unicode indicators. So with the proviso that you are running "byobu-tmux" rather than "byobu-screen", you can squeeze down those indicators even further, saving precious onscreen characters and get extra bling at the same time! Intrigued? Then read on...


A few years ago (yes, it's taken that long to get around to writing this up! ;-), I was playing around with Unicode and ended up writing a very simple C program to display the time in "binary" using the Braille Unicode Block. Assuming you were only displaying hours and minutes, that meant you could display any time in 2 onscreen characters - nice and compact :) I wanted to use this program as a byobu custom indicator but at the time byobu was only using GNU screen and alas even the latest version of screen did not have full UTF-8 support (for its status line atleast). However, tmux was fast becoming the worthy successor to screen and of course byobu now supports both backends. I ended up rewriting the C program as a shell script to fit in with the other byobu indicators. That shell script became the time_binary indicator shipped with byobu.

Here it is displaying the time (21:04) in 24-hour format:


Admittedly, it's kinda terse and somewhat difficult to read but that's half the fun right? :-) time_binary is slightly mis-named as it can also display the time in hexadecimal and octal.

But why stop there? I now wanted all sorts of Unicode indicators showing the level of things like sound, wifi signal strength, left/right speaker volumes, memory used, system load, etc. However, since UIs are very personal, I wanted to be able to write short custom indicators for each of the above but allow the choice of unicode indicators to be changed with minimal trouble as desired.

Enter byobu-ulevel

What I ended up doing is trawling through some of those myriad Unicode symbols looking for suitable groups and creating a small utility called byobu-ulevel. This takes a numeric value and converts it into an appropriate "level" symbol. This all sounds a bit vague, so here's an example:

$ byobu-ulevel 100

The above example has converted the value 100 to its default 100% value using the default theme. And here's the default 30% value:

$ byobu-ulevel 30

Starting to make sense? If your data source doesn't provide values from 0% to 100%, fear not. Let's say your data source ranges from -123 to +456 and your value to display is 333.0:

$ byobu-ulevel -c 333 -m -123.45 -x +678.90

This utility is generally only useful if your data has some specifiable bounded range. However, there is a "cheat mode" - if you are happy for values to overflow/underflow silently, you can specify "-p":

$ byobu-ulevel -c 999 -x 100
ERROR: value (999) > maximum (100)
$ byobu-ulevel -c 999 -x 100 -p

Template for a Byobu Unicode Custom Notification

We can create a Unicode custom notification:

$ mkdir -p ~/.byobu/bin
$ cd ~/.byobu/bin
$ cat >5_utest<<EOT

# XXX: rather than hard-coding the value, put your logic here
# XXX: to determine an actual value from some data source.

printf "U[%s]\n" $(byobu-ulevel $value)
$ chmod 755 5_utest

In fact, as you'll see once you finish this article, the example notification above is really a template for most of your Unicode notifications.

Once you've activated the custom notifications in byobu (F9 → Toggle status notifications → check the "custom" option), and waited for a few seconds (5 in the example above), you'll see the following on your byobu status line:


I've added a "U" to denote my Unicode test indicator and delimited the actual indicator value glyph between square brackets, but of course you can use whatever ornamentation you wish.


So, we now have an indicator that, assuming you add in some appropriate logic to set "$value" dynamically, will show a 1-character wide bar whose height varies depending on the magnitude of $value.

But what if you don't like that bar much? No problem because byobu-ulevel comes with a set of built-in "themes" and since we haven't specified one, we get the default (vbars_8). Let's list the available themes by asking byobu-ulevel to display them:

$ byobu-ulevel -l

So now we know the theme names, let's take a look at them:

$ for theme in $(byobu-ulevel -l); do byobu-ulevel -l -t "$theme"; done
Listing theme 'circles_2'

○ ● 

Listing theme 'diamonds_2'

◇ ◆ 

Listing theme 'flags_2'

⚐ ⚑ 

Listing theme 'hearts_2'

♡ ♥ 

Listing theme 'squares_2'

◻ ◼ 

Listing theme 'squares_small_2'

◽ ◾ 

Listing theme 'stars_2'

☆ ★ 

Listing theme 'vdots_thick_4'

⣀ ⣤ ⣶ ⣿ 

Listing theme 'vdots_thin_4'

⢀ ⢠ ⢰ ⢸ 

Listing theme 'fractions_4'

¼ ½ ¾ ¹ 

Listing theme 'quadrants_4'

◔ ◑ ◕ ● 

Listing theme 'shades_4'

░ ▒ ▓ █ 

Listing theme 'circles_5'

◦ ○ ◎ ◉ ● 

Listing theme 'dice_6'

⚀ ⚁ ⚂ ⚃ ⚄ ⚅ 

Listing theme 'hbars_8'

▏ ▎ ▍ ▌ ▋ ▊ ▉ █ 

Listing theme 'vbars_8'

▁ ▂ ▃ ▄ ▅ ▆ ▇ █ 

Listing theme 'circle_number_10'

➀ ➁ ➂ ➃ ➄ ➅ ➆ ➇ ➈ ➉ 

Listing theme 'solid_numbers_a_10'

➊ ➋ ➌ ➍ ➎ ➏ ➐ ➑ ➒ ➓ 

Listing theme 'solid_numbers_b_10'

❶ ❷ ❸ ❹ ❺ ❻ ❼ ❽ ❾ ❿ 

So how do you use these themes? Easy, just choose the one you like the look of and specify it using "-t $theme". Let's try out the stars_2 theme:

$ byobu-ulevel -c 60 -t stars_2

All themes ending with "_2" contain two glyphs and are called "rating themes" (for obvious reasons I hope). The first glyph in a rating theme represents the "unused" part of the range and the second glyph represents the "used" part. So, here we see that 60% (equating to 3 solid stars) is "used" since the value is 60 and the default range is 0-100%. Obviously, we can't display half glyphs so for this theme for example the value provided is "rounded to the nearest star" :-)

By default rating themes are displayed left-to-right. However, you can change that using the "-r" option:

$ byobu-ulevel -c 60 -t stars_2 -r

Or you can invert the theme such that the first glyph will represent the "used" part and the second the "unused" using the "-i" option:

$ byobu-ulevel -c 60 -t stars_2 -i

But maybe 5 stars isn't enough for you? No problem, use the width option ("-w"):

$ byobu-ulevel -c 60 -t stars_2  -w 10

Themes with more than 2 glyph values are different to rating themes in that rather than using a combination of the "used" and "unused" glyphs to show the value, only a single glyph is selected. For example, here's our 60% again using the circle_number_10 theme:

$ byobu-ulevel -c 60 -t circle_number_10

Since I'm bound to have missed out your favourite Unicode glyphs, you can of course make your own theme trivially too. Let's create a rating theme:

$ byobu-ulevel -c 60 -u "· ☢"

And now a normal theme with, in this example, 10 values:

$ byobu-ulevel -c 60 -u "a b c d e f g h i j"

You could even use utfout to construct your theme:

$ byobu-ulevel -c 45 -u "$(utfout -a ' ' '\{α..ω}')"


But what about accessibility? What about users who cannot display some of these characters? byobu-ulevel provides for these folk too: either specify "-a" or set the BYOBU_A11Y environment variable to any value, and byobu-ulevel will display purely numeric (floating point) values rather than Unicode bling. You can specify the number of decimal places too with the "-e" option.

Multiple Values

Great, but byobu-ulevel only handles a single value - what if you want to display multiple values? For example a range of values over time? No problem because along with byobu-ulevel we have byobu-ugraph which takes a series of values and uses byobu-ulevel to display them. The canonical example being the load average display. Let's create a rolling load average showing 5 data points by creating ~/.byobu/bin/5_uload and making it executable:

awk '{print $1}' /proc/loadavg >> $file
printf "L[%s]\n" $(byobu-ugraph -p 5 -x 3 -f $file)

Leave it to run for a few iterations (to generate the 5 data points we've specified we want and we get something like this:


Like byobu-ulevel, byobu-ugraph assumes all data values range from 0 to 100 so assuming your system load is somewhat more manageable than that, you'll want to specify a "reasonable" maximum value as I have with "-x 3" to allow byobu-ugraph to scale the values appropriately.

Again, you can specify a theme to pass through to byobu-ulevel. Here's the same data using the solid_numbers_b_10 theme:


... And the same data again this time using the vdots_thick_4 theme:


Can console bliss get any better than this? Of course it can - we haven't introduced colours yet!

This part is still rather a work-in-progress as my current unreleased colour utility is rather basic. I'm hoping to expand it and potentially merge it into byobu-ulevel "Real Soon Now". However, my latest versions of byobu-ulevel and byobu-ugraph now accept a "-y" option to enable colour. This allow us to bling up that original load example rather trivially:

awk '{print $1}' /proc/loadavg >> $file
printf "L[%s]\n" $(byobu-ugraph -p 5 -x 3 -f $file -y)

All I've done is add that "-y" to produce:


The colour script currently takes current, minimum and maximum values but also allows thresholds to be specified along with colours for each threshold. By default, two thresholds are specified: "low" (0-15%) and "high" (85-100%). If the value is within the low threshold, it will be coloured red, if in the high threshold it gets coloured green, and yellow if in the middle. The threshold colours can also be inverted so that for example an indicator showing 95% disk usage will default to displaying in red when inverted (as that's a bad scenario), whilst an indicator showing 95% free memory will show in green when not inverted (as that's good :-)

Obligatory Screenshot

Here's an example of the sorts of indicators you can create with just a few lines of shell script for your custom indicators:

As you can see, it's pretty sparse - just the way I like it.

The indicators, from left to right (ignoring the "0:*" byobu window number) are:

  • Time (23:01) in binary ("T[...]")
  • Free disk ("D[...]")
  • Some sort of rating (you decide :-) using hearts ("R[...]")
  • Free memory ("M[...]")
  • Another rating this time using stars ("R[...]")
  • Load average with 5 values ("L[...]")
  • Wifi signal strengh ("W[...]")

Tips and Observations

  • Sometimes, you may find that an indicator doesn't appear. This appears to be an issue with either tmux or the terminal emulator you are using (gnome-terminal, rxvt, urxvt, xterm, etc). The solution seems to be to exit byobu, close the tab/window and restart byobu.
  • When writing custom indicators, ensure that the output ends with a newline character ('\n') - without it, byobu won't display the indicator.
  • Performance can be an issue if you specify a large (10+ say) number of values to byobu-ugraph or if you have more than a few byobu-ulevel and/or byobu-ugraph indicators on-screen at once.
  • byobu-ulevel and byobu-ugraph can run in a terminal as long as you don't enable colour mode (which outputs not colour escape sequences but tmux colour commands).
  • For extra fun, you could have your indicator script call byobu-ulevel specifying a random theme. Here's an indicator which will use a random theme to display the value (80) each time it is called:
  • # change this to by the value you want to display
    count=$(byobu-ulevel -l | wc -l)
    selected=$(((RANDOM % count)+1))
    theme=$(byobu-ulevel -l | sed -n ${selected}p)
    printf "X[%s]\n" $(byobu-ulevel -c $num -t $theme)

Planned Future Work

  • Finish the colour script and potentially merge with byobu-ulevel. New features that could be added to the colour script:
    • ability to specify arbitrary thresholds with arbitrary colours and attributes (bold, etc).
    • ability to invert the colour applied to a value from a colour theme.
  • Improve performance (probably by rewriting in C).
  • Improve accessibility support.
  • Finish another utility I've half-written that shows sound levels (with individual left and right speaker channels all in a single on-screen character!)


There are lots of other Unicode possibilities. Just browse through the wikipedia Unicode page for inspiration.
For example, I'd love it for example if someone would write a Unicode weather app that downloaded the current weather for my location and then displayed it using suitable glyphs like: ⁂ ※ ⁑   ⁕ ⁖  ⁜ ⁛ ☀ ☁ ☂ ☔  ☃.
Find an excuse to get involved!


With minimal effort, you can now add custom Unicode indicators to your byobu session saving you precious onscreen space whilst also offering a dash of visual spice.

Stateful re-exec for Upstart Sessions? You bet!

As of Upstart 1.6, restarting PID 1 is supported without losing state. So as soon as you get an update to either Upstart itself or one of its dependent libraries in Ubuntu, you'll be running the new version automatically, without the need for a reboot.

But what about User Sessions? Are they "second-class citizens"? Of course not! When PID 1 restarts, all the running Session Inits will automatically restart too. You won't notice this of course as your desktop session will continue to run and be fully functional but you too will be running the latest version of Upstart.

How does this work? When an instance of Upstart has restarted, it signals this fact over D-Bus. From the user perspective, the upstart-event-bridge, listening to the system instance of Upstart running as PID 1, will generate a ":sys:restarted" event when it detects the restart. This in turn will cause the session job /usr/share/upstart/sessions/re-exec.conf to run. And that job requests the Session Init restarts. Since all Session Inits will have the re-exec job registered, all Session Init instances will restart as soon as PID 1 does.

To convince yourself, let's create a session job that will start when your Session Init restarts. We'll do this by having the job start when the re-exec job has finished:

$ cat >$HOME/.config/upstart/restart.conf<<EOT
start on stopped re-exec
exec xclock

Now, let's force Upstart running as the system init daemon to restart:

$ sudo telinit u

You should now see xclock running!

Note that the job could have specified any of the following start on conditions...

  • start on starting re-exec
  • start on started re-exec
  • start on stopping re-exec
  • start on stopped re-exec
  • start on :sys:restarted
... however, I chose "start on stopped re-exec" because I want xclock to run just after the re-exec job has finished.

Friday, 12 April 2013

Upstart User Sessions in Ubuntu Raring


Ubuntu Raring now includes Upstart 1.8. Upstart 1.7 and 1.8 combined mark a major milestone since they bring the proven power of Upstart to the user as never before: not only is Upstart now managing the system, but it is also capable of managing the default Ubuntu user's desktop sessions too.


Question: Why reuse a system facility at the user level in this way?

The modern Linux desktop environment is a very dynamic one: users start and stop applications, switch workspaces, search the dash, adjust personal settings in the panel, connect to different networks, hot-plug USB devices and so on. All of these activities can be represented by "events".

Stepping back a second, recall that Upstart was written from the outset to take advantage of the dynamic nature of a modern Linux system. Long gone are the days when a system booted serially. A modern Linux system abounds with "events" from all sorts of different sources:
  • The user plugs or unplugs a peripheral device.
  • A process changes state.
  • A file gets created.
  • A file is deleted.
  • A D-Bus signal is emitted.
  • et cetera.
In recognition of this, Upstarts core concept is that of an event. At its heart, Upstart is an event-based process supervisor. So what better than to use a proven facility such as Upstart at the session level too? Upstarts design fits perfectly with the requirements for an init daemon but also fits the requirements for a session supervisor!

Question: Yes but isn't having Upstart manage users sessions overkill?

Not really - Upstart is not a big application: it was written to be small, fast, safe and reliable, all of which are highly desirable attributes for controlling sessions.

Aside: Upstart has in fact been able to run as a non-privileged user for a couple of years now (since version 1.3) although the facility was added initially for testing and then later for the init-checkconf(8) utility.

This cycle, the Ubuntu developers have gone to great lengths to squash down the default image making the overall system smaller, leaner and faster. By allowing Upstart to manage user sessions, even greater gains can be achieved going forwards since although we've now added a small number of extra (albeit small) user processes, these allow us to convert some of the historically "long-running" user applications to "on-demand" ones. Like the corresponding Upstart system jobs, by default these won't run at all: Upstart will start them if and when they need to run.

Initial targets are apps like update-notifier and update-manager. However, for the "S" cycle we'll be looking even more closely at what is running by default to see if we can start more of those processes on demand.

Some of the changes recently introduced to Upstart also mean that additional system services can also benefit further from on-demand startup. For example, the plan is for the whoopsie daemon to only start on-demand.

This will be an ongoing project, but the foundations have been laid and the concept proved.


The default Ubuntu Desktop (*) in Raring is now fully capable of being managed by Upstart. However, since all the changes landed relatively late in the cycle, the decision was taken not to enable this feature by default.

(*) - we'd like to offer this facility to all the others desktop environments so please contact us if you wish to get involved in making the remaining Xsession scripts Upstart-aware.

However, the good news is two-fold:
  • Upstart User Sessions for the default desktop will be enabled by default for the "S" cycle.
  • Enabling Upstart User Sessions for the default Ubuntu Raring desktop is extremely easy!
If you wish to try out User Sessions (and I'd like to encourage as many folks as possible to) and you use the default Ubuntu desktop, you just need to change a single file, then logout and back in again:

To enable, simply uncomment "ubuntu" in file /etc/upstart-xsessions like this:

sudo sed -i 's/^#ubuntu/ubuntu/' /etc/upstart-xsessions

To disable, simply comment-out "ubuntu" again by running the following:

sudo sed -i 's/^ubuntu/#ubuntu/' /etc/upstart-xsessions

Show me my Session

When you've enabled sessions and re-logged in, everything will look the same but you'll be running within an Upstart environment. You can see the session you're running in by looking at the value of the $UPSTART_SESSION environment variable:


To list all running sessions for your user:

$ initctl list-sessions
3066 unix:abstract=/com/ubuntu/upstart-session/1000/3066

The format of the list-sessions output in case you haven't guessed it is:


Proof that we are running Upstart:

$ ps -fp 3066
james     3066  2884  0 07:58 ?        00:00:00 init --user

Have I got any jobs running?


$ initctl list
xsession-init stop/waiting
dbus start/running, process 3304
firefox stop/waiting
procenv stop/waiting
term start/running, process 3289
gnome-session start/running, process 3343
logrotate stop/waiting
thunderbird start/running, process 3339
mumble stop/waiting
gnome-settings-daemon start/running, process 3309
emacs start/running, process 4324
re-exec stop/waiting
upstart-event-bridge start/running, process 3305

What's interesting is that some of those jobs are "built-ins" from /usr/share/upstart/sessions/ whilst some are jobs I've created myself (thunderbird, emacs, term, procenv and mumble).

Note that if you're already familiar with Upstart, there is a subtle behavioural change here - since you are running within an Upstart session, "initctl list" shows jobs in that session. To see system jobs, you can run either of the following:

$ sudo initctl list
$ initctl --system list

Writing your first user job

Let's create a job that pops up xclock when it's started. Upstart 1.7 can read configuration files from multiple locations, but the main location for user jobs should be $XDG_CONFIG_HOME/upstart/. For  most folk, this equates to $HOME/.config/upstart/:

$ conf_dir=${XDG_CONFIG_HOME:-$HOME/.config}/upstart
$ mkdir -p $conf_dir

(Note that if $conf_dir did not exist initially, you will need to logout and back in for jobs in that directory to take effect - a one-off task)

Now, create file $conf_dir/xclock.conf containing:

start on desktop-start
stop on desktop-end
exec xclock -update 1

That's it. But to be sure we have specified the syntax correctly, let's check that Upstart is happy with it first:

$ init-checkconf $conf_dir/xclock.conf
File /home/james/.config/upstart/xclock.conf: syntax ok

I'd recommend always using init-checkconf(8) to sanity-check your jobs. So, Upstart is happy with this job. Let's start it:

$ start xclock

And you should now see the wonderfully retro xclock running. To stop the job simply run:

$ stop xclock

Note that since we specified a "start on" and "stop on" stanza in the xclock.conf job configuration file, this job will now run every time you login and stop just before you logout. If you don't want that to happen, you have a few choices:
  • Delete xclock.conf - the job is then gone.
  • Add "manual" anywhere after the start on line in xclock.conf such that the job will not be auto-started.
  • Create a new file called xclock.override in the same directory that simply contains "manual". Again, this will stop the file from being auto-started but allows you to leave the existing .conf file as-is.
  • Delete (or simply comment out) the start on condition.

Let's create another job to start firefox which we shall call somewhat unimaginatively firefox.conf:

start on desktop-start
stop on desktop-end
exec firefox

Now, that is more useful - firefox will now start when we login and stop just before we logout!

But what if we want xclock to start after firefox starts and stop just before firefox stops? Simple, just change the start on and stop on stanzas in xclock.conf:

start on started firefox
stop on stopping firefox
exec xclock -update 1

The job events (events that are emitted by jobs in your "initctl list" output) are called starting, started, stopping and stopped but there are many more interesting ones.

Application Output

Error output from applications started from within a session (that is applications started from the Unity dash or launcher) still gets redirected to the usual $HOME/.xsession-errors file. However, all output from Upstart session jobs is redirected automatically to $XDG_CACHE_HOME/upstart/$job.log ,which equates to $HOME/.cache/upstart/$job.log by default.

Since gnome-session is now running as an Upstart job, it too gets it own log file. By each job having its own log file, you can now identify where messages are coming from (not always obvious when perusing $HOME/.xsession-errors).

We have included a logrotate job that automatically keeps the Upstart logs conveniently compressed and rotated. In fact, if you discover whilst testing a job that it is starting to use too much space, or if you simply want to shrink the amount of space your logs are taking, you can run this job whenever you wish like this:

$ start logrotate

The logrotate job (/usr/share/upstart/sessions/logrotate.conf) is rather interesting in that it changes its behavior based on whether it was started by the Session Init, or was invoked manually by a user. Study of this job could be instructive for those wishing to explore some of the possibilities provided by having Upstart manage your session.


To use the GUI, you'll first need to install it (as it is not part of the core upstart package):

$ sudo apt-get install upstart-monitor
$ upstart-monitor

If you have already enabled User Sessions, it will connect to your session. Otherwise, the default is to connect to the system Upstart (running as PID 1).

Auto-start an application on plugging a USB device

This is my favourite example of the power of user jobs. Let's create a job that auto-starts a VoIP client when you plug in your USB headset.

The skeleton of the job is simple. So, I'll create $HOME/.config/upstart/mumble.conf containing:

# start VoIP client of choice
exec mumble

It's a start, but it's not terribly impressive is it? However, we can now "start mumble" and "stop mumble" and Upstart will DTRT.

But what about the 'start on' condition? We need a way to tell Upstart when to start mumble. The simplest way to ascertain this for your particular USB headset is to fire up the upstart-monitor, plug in your headset and see what happens:

As you can see, I get quite a few (15!) events for my USB headset, partly because it is providing multiple sound devices (headphones and a microphone). But which event do I need? Luckily, the answer is simple for sounds devices: the sound-device-changed event. So, let's copy it...

... and paste it here:

:sys:sound-device-changed KERNEL='card2' DEVPATH='/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2/2-1.2:1.0/sound/card2' SUBSYSTEM='sound' ACTION='change' ID_BUS='usb' ID_FOR_SEAT='sound-pci-0000_00_1d_0-usb-0_1_2_1_0' ID_ID='usb-Logitech_Logitech_USB_Headset-00-Headset' ID_MODEL='Logitech_USB_Headset' ID_MODEL_ENC='Logitech\x20USB\x20Headset' ID_MODEL_FROM_DATABASE='ClearChat Pro USB' ID_MODEL_ID='0a0b' ID_PATH='pci-0000:00:1d.0-usb-0:1.2:1.0' ID_PATH_TAG='pci-0000_00_1d_0-usb-0_1_2_1_0' ID_REVISION='1013' ID_SERIAL='Logitech_Logitech_USB_Headset' ID_TYPE='audio' ID_USB_DRIVER='snd-usb-audio' ID_USB_INTERFACES=':010100:010200:030000:' ID_USB_INTERFACE_NUM='00' ID_VENDOR='Logitech' ID_VENDOR_ENC='Logitech' ID_VENDOR_FROM_DATABASE='Logitech, Inc.' ID_VENDOR_ID='046d' SEQNUM='2803' SOUND_FORM_FACTOR='headset' SOUND_INITIALIZED='1' TAGS=':seat:' UDEV_LOG='3' USEC_INITIALIZED='795038608'

As you can see, that's a lot of information! The most important part of this event and the reason we chose the sound-device-changed event in the first place is highlighted below:

:sys:sound-device-changed KERNEL='card2' DEVPATH='/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2/2-1.2:1.0/sound/card2' SUBSYSTEM='sound' ACTION='change' ID_BUS='usb' ID_FOR_SEAT='sound-pci-0000_00_1d_0-usb-0_1_2_1_0' ID_ID='usb-Logitech_Logitech_USB_Headset-00-Headset' ID_MODEL='Logitech_USB_Headset' ID_MODEL_ENC='Logitech\x20USB\x20Headset' ID_MODEL_FROM_DATABASE='ClearChat Pro USB' ID_MODEL_ID='0a0b' ID_PATH='pci-0000:00:1d.0-usb-0:1.2:1.0' ID_PATH_TAG='pci-0000_00_1d_0-usb-0_1_2_1_0' ID_REVISION='1013' ID_SERIAL='Logitech_Logitech_USB_Headset' ID_TYPE='audio' ID_USB_DRIVER='snd-usb-audio' ID_USB_INTERFACES=':010100:010200:030000:' ID_USB_INTERFACE_NUM='00' ID_VENDOR='Logitech' ID_VENDOR_ENC='Logitech' ID_VENDOR_FROM_DATABASE='Logitech, Inc.' ID_VENDOR_ID='046d' SEQNUM='2803' SOUND_FORM_FACTOR='headset' SOUND_INITIALIZED='1' TAGS=':seat:' UDEV_LOG='3' USEC_INITIALIZED='795038608'

The event containing SOUND_INITIALIZED='1' denotes that the "overall" headset device comprising multiple sound devices is "ready to use". So we have the correct type of event and we know that that event is only emitted when the device really is ready but we still need to identify the device uniquely. That might sound like an arduous task, but it really isn't so panic not! Since this is my workstation and since I know I've only got a single USB headset, all I have to do is select  some unique value (or  combination of values) from the variables in the above data.

:sys:sound-device-changed KERNEL='card2' DEVPATH='/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2/2-1.2:1.0/sound/card2' SUBSYSTEM='sound' ACTION='change' ID_BUS='usb' ID_FOR_SEAT='sound-pci-0000_00_1d_0-usb-0_1_2_1_0' ID_ID='usb-Logitech_Logitech_USB_Headset-00-Headset' ID_MODEL='Logitech_USB_Headset' ID_MODEL_ENC='Logitech\x20USB\x20Headset' ID_MODEL_FROM_DATABASE='ClearChat Pro USB' ID_MODEL_ID='0a0b' ID_PATH='pci-0000:00:1d.0-usb-0:1.2:1.0' ID_PATH_TAG='pci-0000_00_1d_0-usb-0_1_2_1_0' ID_REVISION='1013' ID_SERIAL='Logitech_Logitech_USB_Headset' ID_TYPE='audio' ID_USB_DRIVER='snd-usb-audio' ID_USB_INTERFACES=':010100:010200:030000:' ID_USB_INTERFACE_NUM='00' ID_VENDOR='Logitech' ID_VENDOR_ENC='Logitech' ID_VENDOR_FROM_DATABASE='Logitech, Inc.' ID_VENDOR_ID='046d' SEQNUM='2803' SOUND_FORM_FACTOR='headset' SOUND_INITIALIZED='1' TAGS=':seat:' UDEV_LOG='3' USEC_INITIALIZED='795038608'

There are some variables you do not want to select on:

  • KERNEL and DEVPATH: these refer to the physical location of the device so don't select these as that location will differ depending on which USB port you plug your device into.
  • DEVNAME and DEVNUM are incremented every time a new device is plugged in (even if it's the same device you plugged in before).

What you are looking for ideally is one more more variables that are:

  • Describe your device clearly.
  • Will always have the same value whenever and wherever you plug the device.

Your mileage may vary (for example if you have multiple different USB headsets). However, for me, ID_MODEL is sufficient and conveniently human-readable ;-)

Back to our $HOME/.config/upstart/mumble.conf session job. I can now specify the start on condition:

start on :sys:sound-device-changed ID_MODEL="Logitech_USB_Headset" SOUND_INITIALIZED="1"
stop on desktop-end

# start VoIP client of choice
exec mumble

I've added a stop on condition too such that Upstart will stop mumble just before the desktop session ends (in case I forget to stop it myself). And that's it. To try it out, simply plug in your USB headset!

An approximation of what is happening when the USB device is plugged in is:
  • The kernel detects the device and generates a uevent.
  • The upstart-udev-bridge, running at the system level, is monitoring the kernels netlink socket via libudev and therefore sees the event and emits a corresponding Upstart system event.
  • The users upstart-event-bridge sees the system event and proxies it down to the users Upstart session by emitting an Upstart user event tagged with a ":sys:" prefix.
  • The users Session Init compares the start on condition in mumble.conf with the Upstart event emitted by the upstart-event-bridge, finds that it matches and so starts the mumble job.

This example should give you a taste of the power you now have at your fingertips to harness system-level events in your own jobs :-)

A summary of well-known user-level and system-level upstart jobs is available in the  upstart-events(7) manual page.


The above will not work if the device is already plugged at boot time. The problem is that although the upstart-udev-bridge will emit the correct set of events for your device at boot time, your session won't have started at that point in time so there will be no upstart-event-bridge to listen for them. So by the time your session starts, those events will have gone.

The File Bridge

Wouldn't it be great if you could create a job that would only start when a particular file got created? Well, now you can....

Highly contrived, but let's start xclock when the X session errors file gets modified:

start on file FILE=~/.xsession-errors EVENT=modify
exec xclock

How about watching a directory for any activity?

start on file FILE=~/var/mydir/
exec xclock

Now, xclock will get started when files get created, modified or deleted in the directory $HOME/var/mydir/.

Or maybe a file glob might be useful to you?

start on file FILE=~/.cache/myapp/*.crash
exec handle-my-buggy-app

Here, the "handle-my-buggy-app" application will be started to do something useful when your app creates ".crash" files.

Understanding how jobs interact

Upstart provides a tool called initctl2dot which generates graphviz visualisations of jobs. Let's run it:

$ initctl2dot -o - | dot -Tpng -o upstart.png

Here's the result on my system:

  • The rectangular nodes represent jobs.
  • The diamonds denote events.
  • Blue lines represent start on conditions.
  • Red lines represent stop on conditions.
  • Green lines show the events that jobs may emit.
There's quite a lot happening in that diagram partly as we've added some useful "well-known" events that you can use for your jobs. In particular, note the desktop-start and desktop-end events. These are not the first events emitted, but if your job specifies the following, it is guaranteed to run in a fully-functional desktop environment:

start on desktop-start
stop on desktop-end

If you want your job to start as early as possible and end as late as possible, but don't care about whether the desktop is usable when the job starts, you can specify:

start on startup
stop on session-end

Note: The desktop-start and desktop-end events are emitted by the job that actually starts the users graphical desktop. Currently, only the default Ubuntu desktop is emitting these events, but all desktop environments that support for Session Inits need to emit these events.

The Future

There are already a lot of "well-known" events your jobs can make use of, but we plan to introduce even more to enrich the available palette next cycle. Ideas for event sources include:

  • Suspend/Resume events
  • Improved sound device events (for example, start a job when headphones plugged)
  • network events (for example, start/stop jobs when you connect to an open wifi network)
  • D-Bus signals (upstart-dbus-bridge)
  • dconf/gsettings changes (upstart-dconf-bridge)
  • cron-like temporal events (upstart-time-bridge)

Further Information