Tuesday, March 25, 2014

procenv update

Earlier today I released procenv 0.34. Quite a bit has changed since version 0.27 including:
  • Recognises AARCH64, SuperH (fix), PPC64, PPCspe, PPC64LE, OpenRISC systems.
  • Added symbolic names in '--ranges'.
  • Displays Linux binary personality (and flags).
  • Improved '--capabilities' output showing not only bounding set, but also whether each capability is supported, permitted, effective and inheritable values.
  • Added '--memory' which shows NUMA memory details.
  • Added '--cpu' which displays CPU affinity details.
  • Added rpm spec file allowing it to build on RHEL5, Fedora, etc.
  • Improved '--sizeof' which now shows lots more standard types.
  • Displays FreeBSD Capsicum capabilities.
  • Lots of fixes.
Version 0.34 is now available in Debian sid whilst Ubuntu Trusty will be released with procenv 0.33 (which lacks the binary personality information).

Take a look at the links on the main procenv page to see the different environments that it is currently building in (the build logs show the build environments as procenv runs itself as part of its build):

Monday, October 14, 2013

Procenv 0.27 released

Procenv 0.27 has been released. This release introduces a raft of new features...

IPC

 It is now possible to display details of the following IPC mechanisms:

  • message queues
  • semaphores
  • shared memory
Alas, this feature is not available on BSD's as yet partly since there appears to be no documented way to query these mechanisms.

Output Categories


The introduction of the IPC categories brings the total number of output categories to 32:
  • meta
  • arguments
  • capabilities
  • cgroups
  • clocks
  • compiler
  • confstr
  • environment
  • file descriptors
  • libraries
  • limits
  • locale
  • misc
  • message queues
  • mounts
  • network
  • oom
  • platform
  • process
  • ranges
  • rusage
  • semaphores
  • shared memory
  • signals
  • sizeof
  • stat
  • sysconf
  • threads
  • time
  • timezone
  • tty
  • uname

Highly-Structured Output

The code that handles procenv has been completely rewritten so that all output is highly structured. So, rather than displaying file descriptors like this:

$ procenv --fds
fds:
  fd 0: terminal=yes ('/dev/pts/12')
  fd 1: terminal=no
  fd 2: terminal=yes ('/dev/pts/12')
fds (linux/proc):
  '/proc/self/fd/0' -> '/dev/pts/12' (terminal=yes, valid=yes)
  '/proc/self/fd/1' -> 'procenv.log' (terminal=no, valid=yes)
  '/proc/self/fd/2' -> '/dev/pts/12' (terminal=yes, valid=yes)

... they are now displayed like this:

$ procenv --fds
file descriptors:
  0:
    terminal: yes
    valid: yes
    device: /dev/pts/25
  1:
    terminal: no
    valid: yes
    device: procenv.log
  2:
    terminal: yes
    valid: yes
    device: /dev/pts/25

Note that not only is the output more structured, but for file-descriptor output there is now only a single section combining details of both the generic and Linux-specific details.

Format Version


Since it may be necessary to modify the output format, procenv now includes a format-version field in the meta section. This value (currently 1), will be incremented for each output format change. The format-version combined with the version of procenv itself should ease any version transition issues.

Tweaking the Output


If the default space indenting offends your sensibilities, you'll be glad to know that you can now change it to use tabs instead (or in fact any other character you choose):

--indent-char="\t"

Maybe you'd prefer 3 tab indents:

--indent-char="\t" --indent=3

Output Formats


The new highly-structured text output is in fact a side-effect of the fact that procenv can now produce output in other formats. The default is still text, but it can now produce JSON and XML. This even works when running --version:

$ procenv --version
0.27

$ procenv --format=json --version
{
  "version" : {
    "name" : "procenv",
    "version" : "0.27",
    "author" : "James Hunt <james.hunt@ubuntu.com>"
  }
}

$ procenv --format=xml --version
<?xml version="1.0" encoding="UTF-8"?>
<procenv version="0.27" package_string="procenv 0.27" mode="non-privileged" format_version="1">
  <section name="version">
    <entry name="name">procenv</entry>
    <entry name="version">0.27</entry>
    <entry name="author">James Hunt &lt;james.hunt@ubuntu.com&gt;</entry>
  </section>
</procenv>

This makes procenv output much more consumable by standard tools.

Crumbs!


However, the new highly-structured output introduces a problem. Before its introduction, it was easy to use grep(1) to extract particular values. But now, some of those values are on different lines. Yes, you can use the magic grep(1) options -A, -B and -C, but if that isn't convenient for you, procenv does offer an alternative in the form of an output format I've called "crumb" (short for "breadcrumb"). Here's the file descriptor output in breadcrumb output:

$ procenv --format=crumb --fds
file descriptors:0:terminal:yes
file descriptors:0:valid:yes
file descriptors:0:device:/dev/pts/25
file descriptors:1:terminal:yes
file descriptors:1:valid:yes
file descriptors:1:device:/dev/pts/25
file descriptors:2:terminal:yes
file descriptors:2:valid:yes
file descriptors:2:device:/dev/pts/25

As you can see, "crumb mode" shows every value with the appropriate sections the particular value belongs prior to the value. So the unique path to each value is shown by way of a list of "breadcrumbs" or headings. Thus, to extract all details of stdin (aka file descriptor zero):

$ procenv --format=crumb --fds | grep "^file descriptors:0"
file descriptors:0:terminal:yes
file descriptors:0:valid:yes
file descriptors:0:device:/dev/pts/25

It's also really easy to generate CSV data if you wanted to import the output into a spreadsheet:

$ procenv --format=crumb --crumb-separator=',' --separator=','

Download it from:

Tuesday, August 27, 2013

procenv 0.26 released


Version 0.26 of the procenv utility is now available.

Changes:

  • Check to determine if running on a console now works for FreeBSD/kFreeBSD too.
  • Added ability to show all arguments (-A/--arguments)
    (useful when using --exec).
  • Added ability to display network details (-N/--network).
  • Added BSD/Hurd-specific signals.
  • Corrected output sort order.
  • Mount details now include block, inode and fsck details.
There are now 29 categories of environment information displayed when procenv is run (on Linux).

Grab it from:

This update should appear in Debian, Ubuntu and FreeBSD soon...

Friday, August 23, 2013

Upstart 1.10 released

Lots of goodness in this release (explanatory posts to follow):

  • upstart-local-bridge: New bridge for starting jobs on local socket connections.
  • upstart-dconf-bridge: New bridge for Session Inits to react to dconf/gsettings changes.
  • upstart-dbus-bridge: New '--bus-name' option to allow bus name variable to be included in dbus-event(7).
  • New "reload signal" stanza to allow jobs to specify a custom signal that will be sent to the main process (rather than the default SIGHUP).
  • Inclusion of Session Init sample jobs.
  • Re-exec fixes for handling chroot sessions.
  • Shutdown fix for Session Inits.
  • New python3 module and accompanying integration test suite for testing Upstart running as PID 1 and as a Session Init (privileged and non-privileged).

The Upstart cookbook has been updated for this release.

Saturday, August 10, 2013

Upstart Cookbook now updated for Upstart in Debian

I've started a long-overdue update to the Upstart Cookbook for Upstart on Debian. Look out for the Debian swirl! :-)

procenv 0.25 now builds on Android

Procenv 0.25 now builds natively under Android:


It also fixes an interesting issue that was causing it to fail to generate FreeBSD build logs. However, that is now fixed so you can see the FreeBSD build environment now too by looking at the port logs:

More logs to come - the FreeBSD ports seem to build rather slowly...)

Monday, August 5, 2013

Counting down to Ubuntu and Debian releases with distro-info

A while back I though, "wouldn't it be great if the Ubuntu release schedule could be accessed as some form of web service?" That would allow all sorts of fun scripts to be written. Yes, you could write code to parse the wiki page, but I think the technical term for that is "gross".

However, there is now a much simpler alternative in the form of distro-info. If you've never used this awesome tool written by Benjamin Drung you've been missing out! It can tell you all sorts of interesting information. A few examples...

Examples of how to use distro-info


  • What is the current development release on the system I am running on?

    $ distro-info --devel
    saucy
  • What is the current Debian development release?

    $ debian-distro-info --devel
    sid
  • What are the currently supported Ubuntu releases?

    $ ubuntu-distro-info --supported
    lucid
    precise
    quantal
    raring
    saucy
    
  • What's the latest Ubuntu LTS release?
    $ ubuntu-distro-info --lts
    precise
    
  • What's the current stable Debian release?

    $ debian-distro-info --stable
    wheezy
    

distro-info... Now with Added Milestone Goodness

I've been working with Benjamin recently to add a new feature which landed late last week in both Debian unstable and Ubuntu Saucy. So, we can now ask distro-info when a particular release milestone will happen (or has already happened):

  • How many days until Ubuntu Saucy is released?

    $ ubuntu-distro-info --devel --days
    73
  • How many days since the Ubuntu Saucy release was "created"?

    $ ubuntu-distro-info --devel --days=created
    -102
  • How many days until Ubuntu Precise goes "end-of-life" (desktop only!)

    $ ubuntu-distro-info --lts --days=eol
    1360
  • How many days since Debian sid was created?

    $ debian-distro-info --devel --days=created
    -7294

A Scripted Example

We can now write simple little scripts like this to show a popup when getting close to a release:
#!/bin/sh

set -- $(distro-info --devel --codename --days=release)

codename="$1"
days="$2"

# Only warn close to a release
[ "$days" -gt 30 ] && exit 0

zenity --info --text "$codename is released in $days days!"

Wait! Let's do that with Upstart!

In fact, you could turn the script above into an Upstart session job rather easily:
cat >>$HOME/.config/upstart/release-reminder.conf<<EOT
start on desktop-start

task
script
  set -- $(distro-info --devel --codename --days=release)

  codename="$1"
  days="$2"

  # Only warn close to a release
  [ "$days" -gt 30 ] && exit 0

  zenity --info --text "$codename is released in $days days!"
end script
EOT
Now, this script will run automatically every time you login, assuming you are running either:
The currently recognised milestones are "created", "release", "eol" and for Ubuntu LTS releases only "eol-server".

Further Ideas


We could conceivably enhance this feature further by adding in support for querying additional milestones such as "alpha-1" and "feature-freeze".

Want to know more?


For full details, read distro-info(1).