Upstart User Sessions in Ubuntu Raring

Overview

Ubuntu Raring now includes Upstart 1.8. Upstart 1.7 and 1.8 combined mark a major milestone since they bring the proven power of Upstart to the user as never before: not only is Upstart now managing the system, but it is also capable of managing the default Ubuntu user's desktop sessions too.

Why?

Question: Why reuse a system facility at the user level in this way?

The modern Linux desktop environment is a very dynamic one: users start and stop applications, switch workspaces, search the dash, adjust personal settings in the panel, connect to different networks, hot-plug USB devices and so on. All of these activities can be represented by "events".

Stepping back a second, recall that Upstart was written from the outset to take advantage of the dynamic nature of a modern Linux system. Long gone are the days when a system booted serially. A modern Linux system abounds with "events" from all sorts of different sources:

• The user plugs or unplugs a peripheral device.
• A process changes state.
• A file gets created.
• A file is deleted.
• A D-Bus signal is emitted.
• et cetera.

In recognition of this, Upstarts core concept is that of an event. At its heart, Upstart is an event-based process supervisor. So what better than to use a proven facility such as Upstart at the session level too? Upstarts design fits perfectly with the requirements for an init daemon but also fits the requirements for a session supervisor!

Question: Yes but isn't having Upstart manage users sessions overkill?

Not really - Upstart is not a big application: it was written to be small, fast, safe and reliable, all of which are highly desirable attributes for controlling sessions.

Aside: Upstart has in fact been able to run as a non-privileged user for a couple of years now (since version 1.3) although the facility was added initially for testing and then later for the init-checkconf(8) utility.

This cycle, the Ubuntu developers have gone to great lengths to squash down the default image making the overall system smaller, leaner and faster. By allowing Upstart to manage user sessions, even greater gains can be achieved going forwards since although we've now added a small number of extra (albeit small) user processes, these allow us to convert some of the historically "long-running" user applications to "on-demand" ones. Like the corresponding Upstart system jobs, by default these won't run at all: Upstart will start them if and when they need to run.

Initial targets are apps like update-notifier and update-manager. However, for the "S" cycle we'll be looking even more closely at what is running by default to see if we can start more of those processes on demand.

Some of the changes recently introduced to Upstart also mean that additional system services can also benefit further from on-demand startup. For example, the plan is for the whoopsie daemon to only start on-demand.

This will be an ongoing project, but the foundations have been laid and the concept proved.

Enabling

The default Ubuntu Desktop (*) in Raring is now fully capable of being managed by Upstart. However, since all the changes landed relatively late in the cycle, the decision was taken not to enable this feature by default.

(*) - we'd like to offer this facility to all the others desktop environments so please contact us if you wish to get involved in making the remaining Xsession scripts Upstart-aware.

However, the good news is two-fold:

• Upstart User Sessions for the default desktop will be enabled by default for the "S" cycle.
• Enabling Upstart User Sessions for the default Ubuntu Raring desktop is extremely easy!

If you wish to try out User Sessions (and I'd like to encourage as many folks as possible to) and you use the default Ubuntu desktop, you just need to change a single file, then logout and back in again:

To enable, simply uncomment "ubuntu" in file /etc/upstart-xsessions like this:

sudo sed -i 's/^#ubuntu/ubuntu/' /etc/upstart-xsessions

To disable, simply comment-out "ubuntu" again by running the following:

sudo sed -i 's/^ubuntu/#ubuntu/' /etc/upstart-xsessions

Show me my Session

When you've enabled sessions and re-logged in, everything will look the same but you'll be running within an Upstart environment. You can see the session you're running in by looking at the value of the $UPSTART_SESSION environment variable:

$ echo $UPSTART_SESSION
unix:abstract=/com/ubuntu/upstart-session/1000/3066

To list all running sessions for your user:

$ initctl list-sessions
3066 unix:abstract=/com/ubuntu/upstart-session/1000/3066

The format of the list-sessions output in case you haven't guessed it is:

$pid $UPSTART_SESSION

Proof that we are running Upstart:

$ ps -fp 3066
UID        PID  PPID  C STIME TTY          TIME CMD
james     3066  2884  0 07:58 ?        00:00:00 init --user

Have I got any jobs running?

Yes!

$ initctl list
xsession-init stop/waiting
dbus start/running, process 3304
firefox stop/waiting
procenv stop/waiting
term start/running, process 3289
gnome-session start/running, process 3343
logrotate stop/waiting
thunderbird start/running, process 3339
mumble stop/waiting
gnome-settings-daemon start/running, process 3309
emacs start/running, process 4324
re-exec stop/waiting
upstart-event-bridge start/running, process 3305

What's interesting is that some of those jobs are "built-ins" from /usr/share/upstart/sessions/ whilst some are jobs I've created myself (thunderbird, emacs, term, procenv and mumble).

Note that if you're already familiar with Upstart, there is a subtle behavioural change here - since you are running within an Upstart session, "initctl list" shows jobs in that session. To see system jobs, you can run either of the following:

$ sudo initctl list
$ initctl --system list

Writing your first user job

Let's create a job that pops up xclock when it's started. Upstart 1.7 can read configuration files from multiple locations, but the main location for user jobs should be $XDG_CONFIG_HOME/upstart/. For  most folk, this equates to $HOME/.config/upstart/:

$ conf_dir=${XDG_CONFIG_HOME:-$HOME/.config}/upstart
$ mkdir -p $conf_dir

(Note that if $conf_dir did not exist initially, you will need to logout and back in for jobs in that directory to take effect - a one-off task)

Now, create file $conf_dir/xclock.conf containing:

start on desktop-start
stop on desktop-end

exec xclock -update 1

That's it. But to be sure we have specified the syntax correctly, let's check that Upstart is happy with it first:

$ init-checkconf $conf_dir/xclock.conf
File /home/james/.config/upstart/xclock.conf: syntax ok

I'd recommend always using init-checkconf(8) to sanity-check your jobs. So, Upstart is happy with this job. Let's start it:

$ start xclock

And you should now see the wonderfully retro xclock running. To stop the job simply run:

$ stop xclock

Note that since we specified a "start on" and "stop on" stanza in the xclock.conf job configuration file, this job will now run every time you login and stop just before you logout. If you don't want that to happen, you have a few choices:

• Delete xclock.conf - the job is then gone.
• Add "manual" anywhere after the start on line in xclock.conf such that the job will not be auto-started.
• Create a new file called xclock.override in the same directory that simply contains "manual". Again, this will stop the file from being auto-started but allows you to leave the existing .conf file as-is.
• Delete (or simply comment out) the start on condition.

Let's create another job to start firefox which we shall call somewhat unimaginatively firefox.conf:

start on desktop-start
stop on desktop-end

exec firefox

Now, that is more useful - firefox will now start when we login and stop just before we logout!

But what if we want xclock to start after firefox starts and stop just before firefox stops? Simple, just change the start on and stop on stanzas in xclock.conf:

start on started firefox
stop on stopping firefox

exec xclock -update 1

The job events (events that are emitted by jobs in your "initctl list" output) are called starting, started, stopping and stopped but there are many more interesting ones.

Application Output

Error output from applications started from within a session (that is applications started from the Unity dash or launcher) still gets redirected to the usual $HOME/.xsession-errors file. However, all output from Upstart session jobs is redirected automatically to $XDG_CACHE_HOME/upstart/$job.log ,which equates to $HOME/.cache/upstart/$job.log by default.

Since gnome-session is now running as an Upstart job, it too gets it own log file. By each job having its own log file, you can now identify where messages are coming from (not always obvious when perusing $HOME/.xsession-errors).

We have included a logrotate job that automatically keeps the Upstart logs conveniently compressed and rotated. In fact, if you discover whilst testing a job that it is starting to use too much space, or if you simply want to shrink the amount of space your logs are taking, you can run this job whenever you wish like this:

$ start logrotate

The logrotate job (/usr/share/upstart/sessions/logrotate.conf) is rather interesting in that it changes its behavior based on whether it was started by the Session Init, or was invoked manually by a user. Study of this job could be instructive for those wishing to explore some of the possibilities provided by having Upstart manage your session.

upstart-monitor

To use the GUI, you'll first need to install it (as it is not part of the core upstart package):

$ sudo apt-get install upstart-monitor
$ upstart-monitor

If you have already enabled User Sessions, it will connect to your session. Otherwise, the default is to connect to the system Upstart (running as PID 1).

Auto-start an application on plugging a USB device

This is my favourite example of the power of user jobs. Let's create a job that auto-starts a VoIP client when you plug in your USB headset.

The skeleton of the job is simple. So, I'll create $HOME/.config/upstart/mumble.conf containing:

# start VoIP client of choice
exec mumble

It's a start, but it's not terribly impressive is it? However, we can now "start mumble" and "stop mumble" and Upstart will DTRT.

But what about the 'start on' condition? We need a way to tell Upstart when to start mumble. The simplest way to ascertain this for your particular USB headset is to fire up the upstart-monitor, plug in your headset and see what happens:

As you can see, I get quite a few (15!) events for my USB headset, partly because it is providing multiple sound devices (headphones and a microphone). But which event do I need? Luckily, the answer is simple for sounds devices: the sound-device-changed event. So, let's copy it...

... and paste it here:

:sys:sound-device-changed KERNEL='card2' DEVPATH='/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2/2-1.2:1.0/sound/card2' SUBSYSTEM='sound' ACTION='change' ID_BUS='usb' ID_FOR_SEAT='sound-pci-0000_00_1d_0-usb-0_1_2_1_0' ID_ID='usb-Logitech_Logitech_USB_Headset-00-Headset' ID_MODEL='Logitech_USB_Headset' ID_MODEL_ENC='Logitech\x20USB\x20Headset' ID_MODEL_FROM_DATABASE='ClearChat Pro USB' ID_MODEL_ID='0a0b' ID_PATH='pci-0000:00:1d.0-usb-0:1.2:1.0' ID_PATH_TAG='pci-0000_00_1d_0-usb-0_1_2_1_0' ID_REVISION='1013' ID_SERIAL='Logitech_Logitech_USB_Headset' ID_TYPE='audio' ID_USB_DRIVER='snd-usb-audio' ID_USB_INTERFACES=':010100:010200:030000:' ID_USB_INTERFACE_NUM='00' ID_VENDOR='Logitech' ID_VENDOR_ENC='Logitech' ID_VENDOR_FROM_DATABASE='Logitech, Inc.' ID_VENDOR_ID='046d' SEQNUM='2803' SOUND_FORM_FACTOR='headset' SOUND_INITIALIZED='1' TAGS=':seat:' UDEV_LOG='3' USEC_INITIALIZED='795038608'

As you can see, that's a lot of information! The most important part of this event and the reason we chose the sound-device-changed event in the first place is highlighted below:

:sys:sound-device-changed KERNEL='card2' DEVPATH='/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2/2-1.2:1.0/sound/card2' SUBSYSTEM='sound' ACTION='change' ID_BUS='usb' ID_FOR_SEAT='sound-pci-0000_00_1d_0-usb-0_1_2_1_0' ID_ID='usb-Logitech_Logitech_USB_Headset-00-Headset' ID_MODEL='Logitech_USB_Headset' ID_MODEL_ENC='Logitech\x20USB\x20Headset' ID_MODEL_FROM_DATABASE='ClearChat Pro USB' ID_MODEL_ID='0a0b' ID_PATH='pci-0000:00:1d.0-usb-0:1.2:1.0' ID_PATH_TAG='pci-0000_00_1d_0-usb-0_1_2_1_0' ID_REVISION='1013' ID_SERIAL='Logitech_Logitech_USB_Headset' ID_TYPE='audio' ID_USB_DRIVER='snd-usb-audio' ID_USB_INTERFACES=':010100:010200:030000:' ID_USB_INTERFACE_NUM='00' ID_VENDOR='Logitech' ID_VENDOR_ENC='Logitech' ID_VENDOR_FROM_DATABASE='Logitech, Inc.' ID_VENDOR_ID='046d' SEQNUM='2803' SOUND_FORM_FACTOR='headset' SOUND_INITIALIZED='1' TAGS=':seat:' UDEV_LOG='3' USEC_INITIALIZED='795038608'

The event containing SOUND_INITIALIZED='1' denotes that the "overall" headset device comprising multiple sound devices is "ready to use". So we have the correct type of event and we know that that event is only emitted when the device really is ready but we still need to identify the device uniquely. That might sound like an arduous task, but it really isn't so panic not! Since this is my workstation and since I know I've only got a single USB headset, all I have to do is select  some unique value (or  combination of values) from the variables in the above data.

:sys:sound-device-changed KERNEL='card2' DEVPATH='/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2/2-1.2:1.0/sound/card2' SUBSYSTEM='sound' ACTION='change' ID_BUS='usb' ID_FOR_SEAT='sound-pci-0000_00_1d_0-usb-0_1_2_1_0' ID_ID='usb-Logitech_Logitech_USB_Headset-00-Headset' ID_MODEL='Logitech_USB_Headset' ID_MODEL_ENC='Logitech\x20USB\x20Headset' ID_MODEL_FROM_DATABASE='ClearChat Pro USB' ID_MODEL_ID='0a0b' ID_PATH='pci-0000:00:1d.0-usb-0:1.2:1.0' ID_PATH_TAG='pci-0000_00_1d_0-usb-0_1_2_1_0' ID_REVISION='1013' ID_SERIAL='Logitech_Logitech_USB_Headset' ID_TYPE='audio' ID_USB_DRIVER='snd-usb-audio' ID_USB_INTERFACES=':010100:010200:030000:' ID_USB_INTERFACE_NUM='00' ID_VENDOR='Logitech' ID_VENDOR_ENC='Logitech' ID_VENDOR_FROM_DATABASE='Logitech, Inc.' ID_VENDOR_ID='046d' SEQNUM='2803' SOUND_FORM_FACTOR='headset' SOUND_INITIALIZED='1' TAGS=':seat:' UDEV_LOG='3' USEC_INITIALIZED='795038608'

There are some variables you do not want to select on:

• KERNEL and DEVPATH: these refer to the physical location of the device so don't select these as that location will differ depending on which USB port you plug your device into.
• DEVNAME and DEVNUM are incremented every time a new device is plugged in (even if it's the same device you plugged in before).
• UDEV_LOG, USEC_INITIALIZED, SEQNUM.

What you are looking for ideally is one more more variables that are:

• Describe your device clearly.
• Will always have the same value whenever and wherever you plug the device.

Your mileage may vary (for example if you have multiple different USB headsets). However, for me, ID_MODEL is sufficient and conveniently human-readable ;-)

Back to our $HOME/.config/upstart/mumble.conf session job. I can now specify the start on condition:

start on :sys:sound-device-changed ID_MODEL="Logitech_USB_Headset" SOUND_INITIALIZED="1"
stop on desktop-end

# start VoIP client of choice
exec mumble

I've added a stop on condition too such that Upstart will stop mumble just before the desktop session ends (in case I forget to stop it myself). And that's it. To try it out, simply plug in your USB headset!

An approximation of what is happening when the USB device is plugged in is:

• The kernel detects the device and generates a uevent.
• The upstart-udev-bridge, running at the system level, is monitoring the kernels netlink socket via libudev and therefore sees the event and emits a corresponding Upstart system event.
• The users upstart-event-bridge sees the system event and proxies it down to the users Upstart session by emitting an Upstart user event tagged with a ":sys:" prefix.
• The users Session Init compares the start on condition in mumble.conf with the Upstart event emitted by the upstart-event-bridge, finds that it matches and so starts the mumble job.

This example should give you a taste of the power you now have at your fingertips to harness system-level events in your own jobs :-)

A summary of well-known user-level and system-level upstart jobs is available in the  upstart-events(7) manual page.

Restrictions

The above will not work if the device is already plugged at boot time. The problem is that although the upstart-udev-bridge will emit the correct set of events for your device at boot time, your session won't have started at that point in time so there will be no upstart-event-bridge to listen for them. So by the time your session starts, those events will have gone.

The File Bridge

Wouldn't it be great if you could create a job that would only start when a particular file got created? Well, now you can....

Highly contrived, but let's start xclock when the X session errors file gets modified:

start on file FILE=~/.xsession-errors EVENT=modify
exec xclock

How about watching a directory for any activity?

start on file FILE=~/var/mydir/
exec xclock

Now, xclock will get started when files get created, modified or deleted in the directory $HOME/var/mydir/.

Or maybe a file glob might be useful to you?

start on file FILE=~/.cache/myapp/*.crash
exec handle-my-buggy-app

Here, the "handle-my-buggy-app" application will be started to do something useful when your app creates ".crash" files.

Understanding how jobs interact

Upstart provides a tool called initctl2dot which generates graphviz visualisations of jobs. Let's run it:

$ initctl2dot -o - | dot -Tpng -o upstart.png

Here's the result on my system:

• The rectangular nodes represent jobs.
• The diamonds denote events.
• Blue lines represent start on conditions.
• Red lines represent stop on conditions.
• Green lines show the events that jobs may emit.

There's quite a lot happening in that diagram partly as we've added some useful "well-known" events that you can use for your jobs. In particular, note the desktop-start and desktop-end events. These are not the first events emitted, but if your job specifies the following, it is guaranteed to run in a fully-functional desktop environment:

start on desktop-start
stop on desktop-end

If you want your job to start as early as possible and end as late as possible, but don't care about whether the desktop is usable when the job starts, you can specify:

start on startup
stop on session-end

Note: The desktop-start and desktop-end events are emitted by the job that actually starts the users graphical desktop. Currently, only the default Ubuntu desktop is emitting these events, but all desktop environments that support for Session Inits need to emit these events.

The Future

There are already a lot of "well-known" events your jobs can make use of, but we plan to introduce even more to enrich the available palette next cycle. Ideas for event sources include:

• Suspend/Resume events
• Improved sound device events (for example, start a job when headphones plugged)
• network events (for example, start/stop jobs when you connect to an open wifi network)
• D-Bus signals (upstart-dbus-bridge)
• dconf/gsettings changes (upstart-dconf-bridge)
• cron-like temporal events (upstart-time-bridge)

Further Information

• The Upstart Cookbook
• #upstart on freenode
• The upstart-devel mailing list.
• The all-important man pages:
• init(5)
• init(8)
• upstart-monitor(8)
• upstart-events(7)