tag:blogger.com,1999:blog-73494259058623006622024-03-18T03:44:42.221+00:00#ifdef linuxA life immersed in LinuxJames Hunthttp://www.blogger.com/profile/03416491632578860019noreply@blogger.comBlogger56125tag:blogger.com,1999:blog-7349425905862300662.post-78857383848046025682021-06-26T12:59:00.000+01:002021-06-26T12:59:15.965+01:00Procenv v0.58 releasedAnother <a href="https://github.com/jamesodhunt/procenv">procenv</a> <a href="https://github.com/jamesodhunt/procenv/releases">release</a> that is bringing the OSX version up to parity.
Also, thanks to <a href="https://github.com/harens">harens</a>, procenv is <a href="https://ports.macports.org/port/procenv/">now available in MacPorts</a>!
If anyone knows about querying IPC details on Darwin (via Mach?), <a href="https://github.com/jamesodhunt/procenv/issues/22">please comment on the GitHub issue</a> (or raise a PR :)
James Hunthttp://www.blogger.com/profile/03416491632578860019noreply@blogger.com2tag:blogger.com,1999:blog-7349425905862300662.post-20961746352614542122021-05-28T20:54:00.000+01:002021-05-28T20:54:06.105+01:00rout is out<p>I’ve just released the <code>rout</code> tool I mentioned in my last blog post about <a href="https://ifdeflinux.blogspot.com/2021/05/can-you-handle-argument.html">command-line parsing semantics</a>.</p>
<p><code>rout</code> is a simple tool, written in rust, that produces unicode utf-8 output in interesting ways. It uses the minimal command-line parsing crate <a href="https://crates.io/crates/ap"><code>ap</code></a>. It also uses a fancy <a href="https://pest.rs/">pest</a> parser for interpreting escape sequences and range syntax.</p>
<p>Either <a href="https://github.com/jamesodhunt/rout">grab the source</a>, or <a href="https://crates.io/crates/rout">install the crate</a>:</p>
<div class="sourceCode" id="cb1"><pre class="sourceCode bash"><code class="sourceCode bash"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true"></a>$ <span class="ex">cargo</span> install rout</span></code></pre></div>
<p>Full details (with lots of examples! ;) are on both sites:</p>
<ul>
<li><a href="https://crates.io/crates/rout">https://crates.io/crates/rout</a></li>
<li><a href="https://github.com/jamesodhunt/rout">https://github.com/jamesodhunt/rout</a></li>
</ul>James Hunthttp://www.blogger.com/profile/03416491632578860019noreply@blogger.com1tag:blogger.com,1999:blog-7349425905862300662.post-15369761245421038242021-05-09T09:53:00.001+01:002021-05-09T09:53:44.176+01:00Can you handle an argument?<h1 id="tldr">TL;DR</h1>
<p>This post explores some of the darker corners of command-line parsing that some may be unaware of.</p>
<p>You might want to grab a coffee.</p>
<h1 id="intro">Intro</h1>
<p>No, I’m not questioning your debating skills, I’m referring to parsing command-lines!</p>
<p>Parsing command-line option is something most programmers need to deal with at some point. Every language of note provides some sort of facility for handling command-line options. All a programmer needs to do is skim read the docs or grab the sample code, tweak to taste, <em>et voila!</em></p>
<p>But is it that simple? Do you <em>really</em> understand what is going on? I would suggest that most programmers really don’t think that much about it. Handling the parsing of command-line options is just something you bolt on to your codebase. And then you move onto the more interesting stuff. Yes, it really does tend to be that easy and everything just works… most of the time.</p>
<p><em>Most?</em> I hit an interesting issue recently which expanded in scope somewhat. It might raise an eyebrow for some or be a minor bomb-shell for others.</p>
<h1 id="back-story">Back-story</h1>
<h2 id="utfout">utfout</h2>
<p>Back in the mists of time (~2012), I wrote a simple CLI utility in C called <a href="https://github.com/jamesodhunt/utfout"><code>utfout</code></a>. <code>utfout</code> is a simple tool that basically produces output. It’s like <code>echo(1)</code> or <code>printf(3)</code>, but maybe slightly better ;)</p>
<p>Unsurprisingly, <code>utfout</code> uses the ubiquitous and venerable <code>getopt(3)</code> library function to parse the command-line. Specifically, <code>utfout</code> relies on <code>getopt(3)</code> to:</p>
<ul>
<li>Parse the command-line arguments <em>in strict order</em>.</li>
<li>Handle multiple identical options as and when they occur.</li>
<li>Handle positional (non-option) arguments.</li>
</ul>
<p>(Note: We’re going to come back to the term “in strict order” later. But, for now, let’s move on).</p>
<p>One interesting aspect of <code>utfout</code> is that it allows the specification of a repeat value so you can do something like this to display “hello” three times:</p>
<div class="sourceCode" id="cb1"><pre class="sourceCode bash"><code class="sourceCode bash"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true"></a>$ <span class="ex">utfout</span> <span class="st">"hello"</span> -r 2</span>
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true"></a><span class="ex">hellohellohello</span></span></code></pre></div>
<p>That <code>-r</code> repeat option takes an integer as the repeat value. But the integer can be specified as <code>-1</code> meaning “repeat forever”. Looking at such a command-line we have:</p>
<div class="sourceCode" id="cb2"><pre class="sourceCode bash"><code class="sourceCode bash"><span id="cb2-1"><a href="#cb2-1" aria-hidden="true"></a>$ <span class="ex">utfout</span> <span class="st">"hello\n"</span> -r -1</span></code></pre></div>
<p>We’re going to come back to examples like this later. For now, just remember that this option can accept a numeric (negative) value.</p>
<h2 id="rout">rout</h2>
<p>Recently, I decided to rewrite <code>utfout</code> in rust. Hence, <code>rout</code> was born (well, it’s almost been born: I’m currently writing a test suite for it, but I should be releasing it soon).</p>
<p>When I started working on <code>rout</code>, I looked for a rust command-line argument parsing crate that had the semantics of <code>getopt(3)</code>. Although there <em>are</em> <code>getopt()</code> clones, I wanted something a little more “rust-like”. The main contenders didn’t work out for various reasons (I’ll come back to this a little later), and since I was looking for an excuse to write some more rust, I decided, in the best tradition of basically every programmer ever, to reinvent the wheel and write my own. This was fun. But more than that, I uncovered some interesting behavioural points that may be unknown to many. More on this later.</p>
<p>I soon had some command-line argument parsing code and sine it ended up being useful to me, I ended up publishing it as my first rust crate. It’s called <a href="https://crates.io/crates/ap"><code>ap</code></a> for “argument parser”. Not a very creative name maybe, but succinct and simple, like the crate itself.</p>
<p>By this stage, the <code>rout</code> codebase was coming along nicely and it was time to add the CLI parsing. But when I added <code>ap</code> to <code>rout</code> and tried running the repeat command (<code>-r -1</code>), it failed. The problem? <code>ap</code> was assuming that <code>-1</code> was a command-line <em>option</em>, not an option <em>argument</em>. Silly bug right? Err, yes and no. Read on for an explanation!</p>
<h1 id="getopt-minutiae">getopt minutiae</h1>
<p>It may not common knowledge, but <code>getopt(3)</code>, and in fact most argument parsing packages, provide support for <em>numeric option names</em>. If you haven’t read the back-story, this means it supports options like <code>-7</code> which might be a short-hand for the long option <code>--enable-lucky-seven-mode</code> (whatever <em>that</em> means ;) And so to our first revelation:</p>
<blockquote>
<p><strong>Revelation #1:</strong></p>
<p><code>getopt(3)</code> supports <em>any</em> ASCII option name that is not <code>-</code>, <code>;</code> or <code>:</code>.</p>
</blockquote>
<p>In fact, it’s a little more subtle that that: although you can create an option called <code>+</code>, it cannot be the first character in <code>optstring</code>.</p>
<p>If you didn’t realise this, don’t feel too bad! You need to squint a bit when reading the man page to grasp this point, since it is almost painfully opaque on the topic of what constitutes a valid option character. Quoting verbatim from <code>getopt(3)</code>:</p>
<blockquote>
<p><u>optstring</u> is a string containing the legitimate option characters.</p>
</blockquote>
<p>Aside from the fact that the options are specified as a <code>const char *</code>, yep, that is your only clue! The <a href="https://www.freebsd.org/cgi/man.cgi?getopt(3)">FreeBSD man page</a> is slightly clearer, but I would still say not clear enough personally. Yes, you <em>could</em> read the source, but I’ll warn you now, it’s not pretty or easy to grok!</p>
<p>But let this sink in: you can use numeric option <em>names</em>…</p>
<p>The more astute reader may be hearing faint alarm bells ringing at this point. Not to worry if that’s not you as I’ll explain all later.</p>
<h2 id="an-easy-way-to-test-getopt-behaviour">An easy way to test getopt behaviour</h2>
<p>I’ve created a simple C program called <a href="https://github.com/jamesodhunt/test-getopt"><code>test_getopt.c</code></a> that allows you to play with <code>getopt(3)</code> without having to create lots of test programs, or recompile a single program constantly as you tweak it.</p>
<p>The program allows you to specify the <code>optstring</code> as the <em>first command-line argument</em> with all subsequent arguments being passed to <code>getopt(3)</code>.</p>
<p>See the README for some examples.</p>
<h1 id="real-world-evidence">Real-world evidence</h1>
<p>If you’ve ever run the <code>ss(1)</code> or <code>socat(1)</code> commands, you may have encountered numeric options as both commands accept <code>-4</code> and <code>-6</code> options to denote IPv4 and IPv6 respectively. I’m reasonably sure I’ve also seen a command use <code>-#</code> as an option but cannot remember which.</p>
<h1 id="the-ap-bug">The ap bug</h1>
<p>The real bug in <a href="https://crates.io/crates/ap"><code>ap</code></a> was that it was prioritising options over <em>argument order</em>: it was not parsing <em>“in strict order”</em>.</p>
<h1 id="parsing-arguments-in-strict-order">Parsing arguments in strict order</h1>
<p>Remember we mentioned parsing argument “in strict order” earlier? Well “in strict order” doesn’t just mean that arguments are parsed sequentially in the order presented (first, second, third, <em>etc</em>), it also means that option arguments will be consumed by the “parent” (aka previous) option, regardless of whether the option argument starts with a dash or not. It’s beautifully simple and logical and crucially results in zero ambiguity for <code>getopt(3)</code>.</p>
<p>To explain this, imagine your program calls <code>getopt()</code> like this:</p>
<div class="sourceCode" id="cb3"><pre class="sourceCode c"><code class="sourceCode c"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true"></a>getopt(argc, argv, <span class="st">"12:"</span>);</span></code></pre></div>
<p>The program could then be invoked in any of the following ways:</p>
<div class="sourceCode" id="cb4"><pre class="sourceCode bash"><code class="sourceCode bash"><span id="cb4-1"><a href="#cb4-1" aria-hidden="true"></a>$ <span class="ex">prog</span> -1</span>
<span id="cb4-2"><a href="#cb4-2" aria-hidden="true"></a>$ <span class="ex">prog</span> -2 foo</span></code></pre></div>
<p>But: it <em>could</em> also be called like <em>this</em>:</p>
<div class="sourceCode" id="cb5"><pre class="sourceCode bash"><code class="sourceCode bash"><span id="cb5-1"><a href="#cb5-1" aria-hidden="true"></a>$ <span class="ex">prog</span> -2 -1</span></code></pre></div>
<p><code>getopt(3)</code> parses this happily: there is no error and no ambiguity. As far as <code>getopt(3)</code> is concerned the user specified the <code>-2</code> <em>option</em> passing it the <em>value</em> of <code>-1</code>. To be clear, as far as <code>getopt(3)</code> is concerned, the <code>-1</code> <em>option</em> was not been specified!</p>
<blockquote>
<p><strong>Revelation #2:</strong></p>
<p>In argument parsing, “in strict order” means the parser considers each argument in sequential order <strong>and</strong> if a command-line argument is an option and that option requires an argument, the <em>next</em> command-line argument will become that options argument, regardless of whether it starts with dash or not!</p>
</blockquote>
<p>Going back to the revelation. Consuming the next argument after an option requiring a value is a brilliantly simple design. It’s also easy to implement. And since <code>getopt(3)</code> is part of the POSIX standard, it’s actually the behaviour you should be expecting from a command-line parser, atleast if you started out as a systems programmer. But since the details of this parsing behaviour have been somewhat shrouded in mystery, you may not be aware that you <em>should</em> be expecting such behaviour from other parsers!</p>
<p>But, alas, POSIX or not, this behaviour isn’t necessarily intuitive (see above) and indeed this is not how all command-line parsers work.</p>
<h1 id="summary-of-command-line-argument-parsers">Summary of command-line argument parsers</h1>
<p>As my curiosity was now piqued, I decided to do a quick survey of command-line parsing packages for a variety of languages. This is in no way complete and I’ve missed out many languages and packages. But it’s an interesting sample nonetheless.</p>
<p>The table below summarises the behaviour for various languages and parsing libraries:</p>
<table>
<thead>
<tr class="header">
<th>language</th>
<th>library/package</th>
<th>strict ordering?</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td>bash</td>
<td><code>getopts</code></td>
<td>Yes (uses <code>getopt(3)</code>)</td>
</tr>
<tr class="even">
<td>C/C++</td>
<td><a href="https://man7.org/linux/man-pages/man3/getopt.3.html"><code>getopt(3)</code></a></td>
<td>Yes (POSIX standard! ;)</td>
</tr>
<tr class="odd">
<td>Go</td>
<td><a href="https://github.com/urfave/cli"><code>cli</code></a></td>
<td>Yes</td>
</tr>
<tr class="even">
<td>java</td>
<td><a href="https://commons.apache.org/proper/commons-cli/"><code>apache-commons-cli</code></a></td>
<td>Yes</td>
</tr>
<tr class="odd">
<td>lua</td>
<td><a href="https://github.com/mpeterv/argparse"><code>argparse</code></a></td>
<td><strong>No</strong></td>
</tr>
<tr class="even">
<td>perl</td>
<td><a href="https://perldoc.perl.org/Getopt::Std"><code>Getopt::Std</code></a></td>
<td>Yes</td>
</tr>
<tr class="odd">
<td>python</td>
<td><a href="https://docs.python.org/3/library/argparse.html"><code>argparse</code></a></td>
<td><strong>No</strong></td>
</tr>
<tr class="even">
<td>ruby</td>
<td><a href="https://github.com/ruby/optparse"><code>optparse</code></a></td>
<td>Yes</td>
</tr>
<tr class="odd">
<td>rust</td>
<td><a href="https://crates.io/crates/ap"><code>ap</code></a></td>
<td>Yes</td>
</tr>
<tr class="even">
<td>rust</td>
<td><a href="https://crates.io/crates/clap"><code>clap</code></a> (v2+v3)</td>
<td><strong>No</strong></td>
</tr>
<tr class="odd">
<td>swift</td>
<td><a href="https://github.com/apple/swift-argument-parser"><code>swift-argument-parser</code></a></td>
<td><strong>No</strong></td>
</tr>
<tr class="even">
<td>zsh</td>
<td><code>getopts</code></td>
<td>Yes (uses <code>getopt(3)</code>)</td>
</tr>
</tbody>
</table>
<blockquote>
<p><strong>Note:</strong></p>
<p>The libraries that do not use strict ordering (aka “the <code>getopt</code> way”) are not <em>wrong</em> or broken, they just work slightly differently! As long as you are <em>aware</em> of the difference, there is no problem ;)</p>
</blockquote>
<h1 id="why-are-some-libraries-different">Why are some libraries different?</h1>
<p>It comes down to how the command-line arguments are parsed by the package.</p>
<p>Assume the library has just read an argument and determined definitively that it is an option and that the option requires a value. It then reads the next argument:</p>
<ul>
<li><p>If the library is like <code>getopt(3)</code>, it will just consume the argument as the value for the just-seen option (regardless of whether the argument starts with a dash or not).</p></li>
<li><p>Alternatively, if this new argument starts with a dash, the library will consider it an option and then error since the previous argument (the option) was expecting a value.</p>
<p>The subtlety here is that “<code>getopt()</code>-like” implementations allow option values to look like options, which may surprise you.</p></li>
</ul>
<h1 id="so-what">So what?</h1>
<p>We’ve had two revelations:</p>
<ol type="1">
<li>Most argument parsers support numeric option names.</li>
<li>Strict argument parsing means consuming the next argument, even if it starts with a dash.</li>
</ol>
<p>You may be envisaging some of the potential problems now:</p>
<blockquote>
<p>“What if my program accepts a numeric <em>option</em> and also has an option that accepts a numeric <em>argument</em>?”</p>
</blockquote>
<p>There is also the slightly more subtle issue:</p>
<blockquote>
<p>"What if my program has a flag option and also has an option that can accept a free-form string value?</p>
</blockquote>
<p>Indeed! Here be dragons! To make these problems clearer, we’re going to look at some examples.</p>
<h2 id="example-1-missile-control">Example 1: Missile control</h2>
<p>Imagine an evil and powerful tech-savvy despot asks his minions to write a CLI program for him to launch missiles. The program uses <code>getopt(3)</code> with an <code>optstring</code> of “<code>12n:</code>” so that he can launch a single missile (<code>-1</code>), two missiles (<code>-2</code>), or lots (<code>-n <count></code>):</p>
<p>Here’s how he could do his evil work:</p>
<div class="sourceCode" id="cb6"><pre class="sourceCode bash"><code class="sourceCode bash"><span id="cb6-1"><a href="#cb6-1" aria-hidden="true"></a>$ <span class="ex">fire-missile</span> -1</span>
<span id="cb6-2"><a href="#cb6-2" aria-hidden="true"></a><span class="ex">Firing</span> 1 missile!</span></code></pre></div>
<div class="sourceCode" id="cb7"><pre class="sourceCode bash"><code class="sourceCode bash"><span id="cb7-1"><a href="#cb7-1" aria-hidden="true"></a>$ <span class="ex">fire-missile</span> -2</span>
<span id="cb7-2"><a href="#cb7-2" aria-hidden="true"></a><span class="ex">Firing</span> 2 missiles!</span></code></pre></div>
<p>Unfortunately, the poor programmer who wrote this program didn’t check the inputs correctly. Here’s what happens when the despot decides to fire a single missile, but maybe in a drunken stupor / tab-complete fail, runs the following by mistake:</p>
<div class="sourceCode" id="cb8"><pre class="sourceCode bash"><code class="sourceCode bash"><span id="cb8-1"><a href="#cb8-1" aria-hidden="true"></a>$ <span class="ex">fire-missiles</span> -n -1</span>
<span id="cb8-2"><a href="#cb8-2" aria-hidden="true"></a><span class="ex">Firing</span> 4294967295 missiles!</span></code></pre></div>
<p>He’s meant to run <code>fire-missiles -1</code> (or indeed <code>fire-missiles -n 1</code>), but got confused and appears to have started Armageddon by mistake since the program parsed the <code>-n</code> option value as a <em>signed</em> integer.</p>
<h2 id="example-2-get-rich-quick-or-get-fired">Example 2: Get rich quick or get fired?</h2>
<p>Another example. Imagine a program used to transfer money between banks by allowing the admin to specify two IBAN (International Bank Account Number) numbers, an amount and a transaction summary field. Here are the arguments the program will accept:</p>
<ul>
<li><code>-f <IBAN></code>: Source account.</li>
<li><code>-t <IBAN></code>: Destination account.</li>
<li><code>-a <amount></code>: Amount of money to transfer (let’s ignore things like different currencies and exchange rates to keep it simple).</li>
<li><code>-s <text></code>: Human readable summary of the transaction.</li>
<li><code>-d</code>: Dry-run mode - don’t actually send the money, just show what would be done.</li>
</ul>
<p>We could use it to send 100 units of currency like this:</p>
<div class="sourceCode" id="cb9"><pre class="sourceCode bash"><code class="sourceCode bash"><span id="cb9-1"><a href="#cb9-1" aria-hidden="true"></a>$ <span class="ex">prog</span> -f me -t you -a 100 -s test</span></code></pre></div>
<p>For this program we specify a <code>getopt(3)</code> <code>optstring</code> of “<code>a:df:s:t:</code>”. Fine. But using strict ordering, if I run the program as follows, I’ll probably get fired!</p>
<div class="sourceCode" id="cb10"><pre class="sourceCode bash"><code class="sourceCode bash"><span id="cb10-1"><a href="#cb10-1" aria-hidden="true"></a>$ <span class="ex">prog</span> -f me -t you -a 10000000000 -s -d</span></code></pre></div>
<p>Oops! I meant to specify a summary, but I forgot. But hey, that’s fine as I specified to run this in dry-run mode using <code>-d</code>. Oh. Wait a second…</p>
<p>Yes, I’m in trouble because the money <em>was</em> sent as in fact I <em>didn’t</em> specify to run in dry-run mode: I specified a <em>summary</em> of “<code>-d</code>” due to the strict argument parsing semantics of <code>getopt(3)</code>.</p>
<h2 id="example-3-something-to-give-you-nightmares">Example 3: Something to give you nightmares</h2>
<p>Using the knowledge of the revelations, you can easily contrive some real horrors. Take, for example, the following abomination:</p>
<div class="sourceCode" id="cb11"><pre class="sourceCode bash"><code class="sourceCode bash"><span id="cb11-1"><a href="#cb11-1" aria-hidden="true"></a>$ <span class="ex">prog</span> -12 3 -4 -5 -67 8 -9</span></code></pre></div>
<p>How is that parsed? Is that first <code>-12</code> argument a simple negative number? Or is it actually a <code>-1</code> option with the option argument value <code>2</code>? Or is it a <code>-1</code> option and a <code>-2</code> option “bundled” together?</p>
<p>The answer of course depends on how you’ve defined the <code>optstring</code> value to <code>getopt(3)</code>. But please, <em>please</em> never write programs with interfaces like this! ;)</p>
<p>You can use the <a href="https://github.com/jamesodhunt/test-getopt"><code>test_getopt.c</code></a> program to test out various ways of parsing that horrid command-line. For example, one way to handle them might be like this:</p>
<div class="sourceCode" id="cb12"><pre class="sourceCode bash"><code class="sourceCode bash"><span id="cb12-1"><a href="#cb12-1" aria-hidden="true"></a>$ <span class="ex">test_getopt</span> <span class="st">"1::45:9"</span> -12 3 -4 -5 -67 8 -9</span>
<span id="cb12-2"><a href="#cb12-2" aria-hidden="true"></a><span class="ex">INFO</span>: getopt option: <span class="st">'1'</span> (optarg: <span class="st">'2'</span>, optind: 2, opterr: 1, optopt: 0)</span>
<span id="cb12-3"><a href="#cb12-3" aria-hidden="true"></a><span class="ex">INFO</span>: getopt option: <span class="st">'4'</span> (optarg: <span class="st">''</span>, optind: 4, opterr: 1, optopt: 0)</span>
<span id="cb12-4"><a href="#cb12-4" aria-hidden="true"></a><span class="ex">INFO</span>: getopt option: <span class="st">'5'</span> (optarg: <span class="st">'-67'</span>, optind: 6, opterr: 1, optopt: 0)</span>
<span id="cb12-5"><a href="#cb12-5" aria-hidden="true"></a><span class="ex">INFO</span>: getopt option: <span class="st">'9'</span> (optarg: <span class="st">''</span>, optind: 8, opterr: 1, optopt: 0)</span></code></pre></div>
<p>But alternatively, it could be parsed like this:</p>
<div class="sourceCode" id="cb13"><pre class="sourceCode bash"><code class="sourceCode bash"><span id="cb13-1"><a href="#cb13-1" aria-hidden="true"></a>$ <span class="ex">test_getopt</span> <span class="st">"12:4567:9"</span> -12 3 -4 -5 -67 8 -9</span>
<span id="cb13-2"><a href="#cb13-2" aria-hidden="true"></a><span class="ex">INFO</span>: getopt option: <span class="st">'1'</span> (optarg: <span class="st">''</span>, optind: 1, opterr: 1, optopt: 0)</span>
<span id="cb13-3"><a href="#cb13-3" aria-hidden="true"></a><span class="ex">INFO</span>: getopt option: <span class="st">'2'</span> (optarg: <span class="st">'3'</span>, optind: 3, opterr: 1, optopt: 0)</span>
<span id="cb13-4"><a href="#cb13-4" aria-hidden="true"></a><span class="ex">INFO</span>: getopt option: <span class="st">'4'</span> (optarg: <span class="st">''</span>, optind: 4, opterr: 1, optopt: 0)</span>
<span id="cb13-5"><a href="#cb13-5" aria-hidden="true"></a><span class="ex">INFO</span>: getopt option: <span class="st">'5'</span> (optarg: <span class="st">''</span>, optind: 5, opterr: 1, optopt: 0)</span>
<span id="cb13-6"><a href="#cb13-6" aria-hidden="true"></a><span class="ex">INFO</span>: getopt option: <span class="st">'6'</span> (optarg: <span class="st">''</span>, optind: 5, opterr: 1, optopt: 0)</span>
<span id="cb13-7"><a href="#cb13-7" aria-hidden="true"></a><span class="ex">INFO</span>: getopt option: <span class="st">'7'</span> (optarg: <span class="st">'8'</span>, optind: 7, opterr: 1, optopt: 0)</span>
<span id="cb13-8"><a href="#cb13-8" aria-hidden="true"></a><span class="ex">INFO</span>: getopt option: <span class="st">'9'</span> (optarg: <span class="st">''</span>, optind: 8, opterr: 1, optopt: 0)</span></code></pre></div>
<h3 id="aside">Aside</h3>
<p>Coincidentally, by combining <code>test_getopt</code> with <code>utfout</code>, you can prove Revelation #1 rather simply:</p>
<div class="sourceCode" id="cb14"><pre class="sourceCode bash"><code class="sourceCode bash"><span id="cb14-1"><a href="#cb14-1" aria-hidden="true"></a>$ <span class="kw">(</span><span class="ex">utfout</span> -a <span class="st">"\n"</span> <span class="st">"\{\x21..\x7e}"</span><span class="kw">;</span> <span class="bu">echo</span><span class="kw">)</span> <span class="kw">|\</span></span>
<span id="cb14-2"><a href="#cb14-2" aria-hidden="true"></a> <span class="kw">while</span> <span class="bu">read</span> <span class="va">char</span></span>
<span id="cb14-3"><a href="#cb14-3" aria-hidden="true"></a> <span class="kw">do</span></span>
<span id="cb14-4"><a href="#cb14-4" aria-hidden="true"></a> <span class="ex">test_getopt</span> <span class="st">"x</span><span class="va">$char</span><span class="st">"</span> -<span class="st">"</span><span class="va">$char</span><span class="st">"</span></span>
<span id="cb14-5"><a href="#cb14-5" aria-hidden="true"></a> <span class="kw">done</span></span></code></pre></div>
<blockquote>
<p><strong>Note:</strong> The leading “<code>x</code>” in the specified <code>optstring</code> argument is to avoid having to special case the string since the first character is “special” to getopt(3). See the man page for further details.</p>
</blockquote>
<h2 id="summary">Summary</h2>
<p>Admittedly, these are very contrived (and hopefully unrealistic!) examples. The missile control example is also a very poor use of <code>getopt(3)</code> since in this scenario, a simple check on <code>argv[1]</code> would be sufficient to determine how many missiles to fire. However, you can now see the potential pitfalls of numeric options and strict argument parsing.</p>
<h1 id="to-test-a-parser">To test a parser</h1>
<p>If you want to establish if your chosen command-line parsing library accepts numeric options and if it parses in strict order, create a program that:</p>
<ul>
<li><p>Accepts a <code>-1</code> flag option (an option that does not require an argument).</p></li>
<li><p>Accepts a <code>-2</code> argument option (that does accept an argument).</p></li>
<li><p>Run the program as follows:</p>
<div class="sourceCode" id="cb15"><pre class="sourceCode bash"><code class="sourceCode bash"><span id="cb15-1"><a href="#cb15-1" aria-hidden="true"></a>$ <span class="ex">prog</span> -2 -1</span></code></pre></div></li>
<li><p>If the program succeeds (and sets the value for your <code>-2</code> option to <code>-1</code>), your parser is “<code>getopt()</code>-like” (is parsing in strict order) and implicitly also supports numeric options.</p></li>
</ul>
<h1 id="conclusions">Conclusions</h1>
<p>Here’s what we’ve unearthed:</p>
<ul>
<li><p>The <code>getopt(3)</code> man page on Linux is currently ambiguous.</p>
<p><a href="https://github.com/alejandro-colomar/man-pages/commit/fde4d1608e1dfe207408d79dc8d054ea4001d029">I wrote a patch to resolve this</a>, and the patch has been accepted. Hopefully it will land in the next release of the man pages project.</p></li>
<li><p>All command-line parsing packages should document precisely how they consume arguments.</p>
<p>Unfortunately, most don’t say anything about it! However, <a href="https://crates.io/crates/ap"><code>ap</code></a> does. Specifically, see the documentation <a href="https://docs.rs/ap/*/ap/#handling-ambiguity">here</a>.</p></li>
<li><p><code>getopt(3)</code> doesn’t just support alphabetic option names: a name can be almost any ASCII character (<code>-3</code>, <code>-%</code>, <code>-^</code>, <code>-+</code>, <em>etc</em>).</p></li>
<li><p>Numeric options should be used with caution as they can lead to ambiguity; not for <code>getopt(3)</code> <em>et al</em>, but for the end user running the program. Worst case, there could be security implications.</p></li>
<li><p>Permitting negative numeric option values should also be considered carefully. Rather than supporting <code>-r -1</code>, it would be safer if <code>utfout</code> and <code>rout</code> required the repeat count to be <code>>= 1</code> and if the user wants to repeat forever, support <code>-r max</code> or <code>-r forever</code> rather than <code>-r -1</code>.</p></li>
<li><p>Some modern command-line parsers prioritise options over argument ordering (meaning they are not “<code>getopt()</code>-like”).</p></li>
<li><p>You should understand how your chosen parser works before using it.</p></li>
<li><p>Parsing arguments “in strict order” does not only mean “in sequential order”: it means the parser prioritises command-line arguments over option values.</p></li>
<li><p>If your chosen parsing package prioritises arguments over options (like <code>getopt(3)</code>, you need to take care if you use numeric options since arguments will be consumed “greedily” (and silently).</p></li>
<li><p>If your chosen parsing package prioritises options over arguments, you will probably be <em>safer</em> (since an incorrect command-line will generate an error), but you need to be aware that the package is not “<code>getopt()</code>-like”.</p></li>
<li><p>a CLI program <strong>must</strong> validate all command-line option values; command-line argument parsers provide a way for users to inject data into a program, so a wise programmer will always be paranoid!</p></li>
<li><p><em>The devil is in the detail</em> ;)</p></li>
</ul>James Hunthttp://www.blogger.com/profile/03416491632578860019noreply@blogger.com2tag:blogger.com,1999:blog-7349425905862300662.post-22618169480288760522021-05-06T19:56:00.001+01:002021-05-06T19:58:05.576+01:00(Lots of) new procenv release(s)<h1 id="lots-of-new-procenv-releases"></h1>
<h1 id="lots-of-new-procenv-releases">(Lots of) new procenv release(s)</h1><p><code> <br /></code></p><p><a href="https://github.com/jamesodhunt/procenv"><code>procenv</code></a> is now at version <a href="https://github.com/jamesodhunt/procenv/releases"><code>0.55</code></a>.</p>
<p>Significant changes since version <code>0.46</code>:</p>
<ul>
<li>FreeBSD and Hurd fixes.</li>
<li>Show if running in a VM.</li>
<li><code>--memory</code> now shows more memory details.</li>
<li>More capabilities.</li>
<li><code>PROCENV_EXEC</code> fixes.</li>
</ul>
<p>Further details on the <a href="https://github.com/jamesodhunt/procenv/releases">release page</a>.</p>James Hunthttp://www.blogger.com/profile/03416491632578860019noreply@blogger.com1tag:blogger.com,1999:blog-7349425905862300662.post-59828066257637021442016-05-28T20:44:00.001+01:002020-03-19T17:29:08.050+00:00Procenv 0.46 - now with more platform goodness<br />
I have just released <a href="https://github.com/jamesodhunt/procenv/releases/tag/0.46">procenv version 0.46</a>. Although this is a very minor release for the existing platforms (essentially 1 bug fix), this release now introduces support for a new platform...<br />
<br />
<h2>
Darwin</h2>
Yup - OS X now joins the ranks of supported platforms.<br />
<br />
Although adding support for Darwin was made significantly easier as a result of the recent internal restructure of the <span style="font-family: "courier new" , "courier" , monospace;">procenv</span> code, it did present a challenge: I don't own any Apple hardware. I <i>could</i> have borrowed a Macbook, but instead I decided to see this as a challenge:<br />
<br />
<ul>
<li>Could I port <span style="font-family: "courier new" , "courier" , monospace;">procenv</span> to Darwin without actually having a local Apple system?</li>
</ul>
Well, you've just read the answer, but how did I do this?<br />
<br />
<h2>
Stage 1: Docker </h2>
<br />
Whilst surfing around I came across this interesting docker image:<br />
<br />
<ul>
<li><a href="https://hub.docker.com/r/andrewd/osxcross/">https://hub.docker.com/r/andrewd/osxcross/</a> </li>
</ul>
<br />
It provides a Darwin toolchain that I could run under Linux. It didn't take very long to <a href="https://github.com/jamesodhunt/procenv/tree/master/src/platform">follow my own instructions on porting procenv to a new platform</a>. But although I ended up with a binary, I couldn't actually run it, partly because Darwin uses a different binary file format to Linux: rather than <a href="https://en.wikipedia.org/wiki/Executable_and_Linkable_Format">ELF</a>, it uses the <a href="https://en.wikipedia.org/wiki/Mach-O">Mach-O</a> format.<br />
<br />
<br />
<br />
<h2>
Stage 2: Travis</h2>
The final piece of the puzzle for me was solved by Travis. I'd read the <a href="https://docs.travis-ci.com/">very good documentation on their site</a>, but had initially assumed that you could only build Objective-C based projects on OSX with Travis. But a quick test proved my assumption to be incorrect: it didn't take much more than adding "<span style="font-family: "courier new" , "courier" , monospace;">osx</span>" to the <span style="font-family: "courier new" , "courier" , monospace;">os</span> list and "<span style="font-family: "courier new" , "courier" , monospace;">clang</span>" to the <span style="font-family: "courier new" , "courier" , monospace;">compiler</span> list in <span style="font-family: "courier new" , "courier" , monospace;">procenv</span>'s<span style="font-family: inherit;"> <span style="font-family: "courier new" , "courier" , monospace;">.travis.yml</span> </span>to have <span style="font-family: "courier new" , "courier" , monospace;">procenv</span> building <i><b>and running</b></i> (it runs itself as part of its build) on OSX under Travis!<br />
<br />
Essentially, the following <a href="https://en.wikipedia.org/wiki/YAML">YAML</a> snippet from <span style="font-family: "courier new" , "courier" , monospace;">procenv</span>'s <span style="font-family: "courier new" , "courier" , monospace;"><a href="https://github.com/jamesodhunt/procenv/blob/master/.travis.yml">.travis.yml</a></span> did most of the work:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">language: c<br />compiler:<br /> - gcc<br /> - clang<br />os:<br /> - linux<br /> - osx</span><br />
<br />
<br />
All that remained was to install the build-time dependencies to the same file with this additional snippet:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">before_install:</span><br />
<span style="font-family: "courier new" , "courier" , monospace;"> - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew update; fi<br /> - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then brew install expat check perl; fi</span><br />
<br />
(Note that it seems Travis is rather picky about <span style="font-family: "courier new" , "courier" , monospace;">before_install</span> - all code must be on a single line, hence the rather awkward-to-read "<span style="font-family: "courier new" , "courier" , monospace;">if; then ....; fi</span>" tests).<br />
<br />
<br />
<h2>
Summary</h2>
<br />
Although I've never personally run <span style="font-family: "courier new" , "courier" , monospace;">procenv</span> under OSX, I have got a good degree of confidence that it does actually work.<br />
<br />
That said, it <i>would</i> be useful if someone could independently verify this claim on a <i>real</i> system!) Feel free to raise bugs, send code (or even Apple hardware :-) my way!<br />
<br />
<br />
<br />James Hunthttp://www.blogger.com/profile/03416491632578860019noreply@blogger.com67tag:blogger.com,1999:blog-7349425905862300662.post-71090139685340172402016-03-22T22:05:00.000+00:002016-03-22T22:06:19.522+00:00Procenv 0.45 released<br />
Yesterday, I released <a href="https://github.com/jamesodhunt/procenv/releases/tag/0.45">procenv 0.45</a>.<br />
<br />
Although not a huge amount has changed on the outside in this release, rather<br />
dramatic changes have occurred on the inside, as evinced by the following:<br />
<span style="font-size: x-small;"><br /><span style="font-family: "courier new" , "courier" , monospace;">$ git diff 0.44..0.45 --stat|grep "src<span style="font-family: "courier new" , "courier" , monospace;">/<span style="font-family: "courier new" , "courier" , monospace;">.*\.[ch]</span></span>"<br /> src/messages.h | 41 +<br /> src/output.c | 2001 ++++++<br /> src/output.h | 179 +<br /> src/platform-headers.h | 243 +<br /> src/platform.h | 167 +<br /> src/platform/android/platform.c | 46 +<br /> src/platform/freebsd/platform-freebsd.h | 43 +<br /> src/platform/freebsd/platform.c | 479 ++<br /> src/platform/hurd/platform-hurd.h | 38 +<br /> src/platform/hurd/platform.c | 91 +<br /> src/platform/linux/platform-linux.h | 174 +<br /> src/platform/linux/platform.c | 2176 ++++++<br /> src/platform/minix/platform-minix.h | 25 +<br /> src/platform/minix/platform.c | 126 +<br /> src/platform/netbsd/platform-netbsd.h | 25 +<br /> src/platform/netbsd/platform.c | 237 +<br /> src/platform/openbsd/platform-openbsd.h | 25 +<br /> src/platform/openbsd/platform.c | 199 +<br /> src/platform/platform-generic.c | 922 +++<br /> src/platform/platform-generic.h | 66 +<br /> src/platform/unknown/platform-unknown.h | 25 +<br /> src/platform/unknown/platform.c | 89 +<br /> src/pr_list.c | 2 +-<br /> src/pr_list.h | 2 +-<br /> src/procenv.c | 11575 +++++++-----------------------<br /> src/procenv.h | 918 +--<br /> src/pstring.c | 309 +<br /> src/pstring.h | 55 +<br /> src/string-util.c | 543 ++<br /> src/string-util.h | 60 +<br /> src/types.h | 34 +<br /> src/util.c | 288 +<br /> src/util.h | 57 +</span></span><br />
<br />
Partly driven by "#ifdef hell", and part inspired by <a href="http://hisham.hm/htop/">Hisham's</a> Fosdem presentation on <a href="https://github.com/hishamhm/htop">htop</a>, "<a href="https://fosdem.org/2016/schedule/event/htop/attachments/slides/1167/export/events/attachments/htop/slides/1167/htop_talk.pdf)">Going cross-platform: how htop was made portable</a>", I decided to modularise the code and introduce platform drivers encapsulating the O/S specifics and allowing <span style="font-family: "courier new" , "courier" , monospace;">procenv.c</span> itself to be considerably simplified. This allows for easier maintenance and easier porting to new operating systems.<br />
<br />
To prove it, <span style="font-family: "courier new" , "courier" , monospace;">procenv</span> now runs on OpenBSD, NetBSD and Minix 3!<br />
<br />
There is still further refactoring to do, but the code is now in much better shape for others to work with.<br />
<br />
So, if you'd like to get involved in porting <span style="font-family: "courier new" , "courier" , monospace;">procenv</span> to [insert-your-favourite-as-yet-unsupported-OS-here], get involved! I even wrote a simple porting guide to help get your started:<br />
<br />
<ul>
<li><a href="https://github.com/jamesodhunt/procenv/blob/master/src/platform/README.rst">https://github.com/jamesodhunt/procenv/blob/master/src/platform/README.rst</a> </li>
</ul>
<br />James Hunthttp://www.blogger.com/profile/03416491632578860019noreply@blogger.com1tag:blogger.com,1999:blog-7349425905862300662.post-45398055562084046182015-11-24T19:29:00.000+00:002015-11-24T19:29:05.406+00:00Procenv 0.43 released(Cough... This should probably be called the "<i>I-almost-forgot-I-had-a-blog release</i>"! :-)<br />
<br />
Development has now moved to github:<br />
<br />
<ul>
<li><a href="https://github.com/jamesodhunt/procenv">https://github.com/jamesodhunt/procenv</a></li>
</ul>
So you can grab the release files here:<br />
<br />
<ul>
<li><a href="https://github.com/jamesodhunt/procenv/releases">https://github.com/jamesodhunt/procenv/releases</a></li>
</ul>
<br />
The move to github brings a few advantages, including Travis-CI integration:<br />
<br />
<ul>
<li><a href="https://travis-ci.org/jamesodhunt/procenv">https://travis-ci.org/jamesodhunt/procenv</a><br />(btw - does anyone know of a CI solution for FreeBSD?)</li>
</ul>
Even more awesome being that, thanks to the wonder of webhooks, <span style="font-family: "Courier New",Courier,monospace;">procenv</span> is now building for lots of distros. Take a look at:<br />
<br />
<ul>
<li><a href="https://build.opensuse.org/package/show/home:jamesodhunt:procenv/procenv">https://build.opensuse.org/package/show/home:jamesodhunt:procenv/procenv</a></li>
</ul>
So you get to see the environment those builds run in <i>and</i> OBS is also providing <span style="font-family: "Courier New",Courier,monospace;">procenv</span> packages!<br />
<br />
Here's the funky download page (just click your distro):<br />
<br />
<ul>
<li><a href="https://software.opensuse.org/download.html?project=home%3Ajamesodhunt%3Aprocenv&package=procenv">https://software.opensuse.org/download.html?project=home%3Ajamesodhunt%3Aprocenv&package=procenv</a></li>
</ul>
Caveat emptor: those packages are building off the tip, so are not necessarily releases - they are built from the latest commit! That said, since <span style="font-family: "Courier New",Courier,monospace;">procenv</span> runs a lot of tests at build-time, you should be reasonably safe.<br />
<br />
If you'd rather opt for official releases, the new version should be in Debian and Ubuntu Xenial soon. It should also arrive in <a href="https://clearlinux.org/">Clear Linux</a> tomorrow.<br />
<br />
As for what has changed since the last blog-post, just take a look at the NEWS file:<br />
<br />
<ul>
<li><a href="https://github.com/jamesodhunt/procenv/blob/master/NEWS">https://github.com/jamesodhunt/procenv/blob/master/NEWS</a></li>
</ul>
<br />
<br />James Hunthttp://www.blogger.com/profile/03416491632578860019noreply@blogger.com9tag:blogger.com,1999:blog-7349425905862300662.post-77918074427692604902014-03-25T19:21:00.001+00:002014-03-25T19:21:32.839+00:00procenv updateEarlier today I released <a href="https://launchpad.net/procenv/+download">procenv 0.34</a>. Quite a bit has changed since version 0.27 including:<br />
<ul>
<li>Recognises AARCH64, SuperH (fix), PPC64, PPCspe, PPC64LE, OpenRISC systems.</li>
<li>Added symbolic names in '<span style="font-family: "Courier New",Courier,monospace;">--ranges</span>'.</li>
<li>Displays Linux binary personality (and flags).</li>
<li>Improved '<span style="font-family: "Courier New",Courier,monospace;">--capabilities</span>' output showing not only bounding set, but also whether each capability is supported, permitted, effective and inheritable values.</li>
<li>Added '<span style="font-family: "Courier New",Courier,monospace;">--memory</span>' which shows NUMA memory details.</li>
<li>Added '<span style="font-family: "Courier New",Courier,monospace;">--cpu</span>' which displays CPU affinity details.</li>
<li>Added rpm spec file allowing it to build on RHEL5, Fedora, <i>etc</i>.</li>
<li>Improved '<span style="font-family: "Courier New",Courier,monospace;">--sizeof</span>' which now shows lots more standard types.</li>
<li>Displays FreeBSD Capsicum capabilities.</li>
<li>Lots of fixes. </li>
</ul>
Version 0.34 is now available in <a href="https://packages.debian.org/sid/procenv">Debian sid</a> whilst <a href="https://launchpad.net/ubuntu/+source/procenv">Ubuntu Trusty</a> will be released with procenv 0.33 (which lacks the binary personality information).<br />
<br />
Take a look at the links on the main procenv page to see the different environments that it is currently building in (the build logs show the build environments as procenv runs itself as part of its build):<br />
<ul>
<li><a href="https://launchpad.net/procenv/">https://launchpad.net/procenv/</a></li>
</ul>
James Hunthttp://www.blogger.com/profile/03416491632578860019noreply@blogger.com1tag:blogger.com,1999:blog-7349425905862300662.post-73795544530012311012013-10-14T20:15:00.003+01:002013-10-14T20:15:59.107+01:00Procenv 0.27 releasedProcenv 0.27 has been released. This release introduces a raft of new features...<br />
<br />
<h2>
IPC </h2>
It is now possible to display details of the following IPC mechanisms:<br />
<br />
<ul>
<li>message queues</li>
<li>semaphores </li>
<li>shared memory</li>
</ul>
Alas, this feature is not available on BSD's as yet partly since there appears to be no <i>documented</i> way to query these mechanisms.<br />
<br />
<h2>
Output Categories</h2>
<br />
The introduction of the IPC categories brings the total number of output categories to 32:<br />
<ul>
<li>meta</li>
<li>arguments</li>
<li>capabilities</li>
<li>cgroups</li>
<li>clocks</li>
<li>compiler</li>
<li>confstr</li>
<li>environment</li>
<li>file descriptors</li>
<li>libraries</li>
<li>limits</li>
<li>locale</li>
<li>misc</li>
<li>message queues</li>
<li>mounts</li>
<li>network</li>
<li>oom</li>
<li>platform</li>
<li>process</li>
<li>ranges</li>
<li>rusage</li>
<li>semaphores</li>
<li>shared memory</li>
<li>signals</li>
<li>sizeof</li>
<li>stat</li>
<li>sysconf</li>
<li>threads</li>
<li>time</li>
<li>timezone</li>
<li>tty</li>
<li>uname</li>
</ul>
<h2>
Highly-Structured Output</h2>
The code that handles <span style="font-family: "Courier New",Courier,monospace;">procenv</span> has been completely rewritten so that all output is highly structured.
So, rather than displaying file descriptors like this:<br />
<br />
<pre brush="text">$ procenv --fds
fds:
fd 0: terminal=yes ('/dev/pts/12')
fd 1: terminal=no
fd 2: terminal=yes ('/dev/pts/12')
fds (linux/proc):
'/proc/self/fd/0' -> '/dev/pts/12' (terminal=yes, valid=yes)
'/proc/self/fd/1' -> 'procenv.log' (terminal=no, valid=yes)
'/proc/self/fd/2' -> '/dev/pts/12' (terminal=yes, valid=yes)
</pre>
<br />
... they are now displayed like this:<br />
<br />
<pre brush="text">$ procenv --fds
file descriptors:
0:
terminal: yes
valid: yes
device: /dev/pts/25
1:
terminal: no
valid: yes
device: procenv.log
2:
terminal: yes
valid: yes
device: /dev/pts/25
</pre>
<br />
Note that not only is the output more structured, but for file-descriptor output there is now only a single section combining details of both the generic and Linux-specific details.<br />
<br />
<h2>
Format Version</h2>
<br />
Since it may be necessary to modify the output format, <span style="font-family: "Courier New",Courier,monospace;">procenv</span> now includes a <span style="font-family: "Courier New",Courier,monospace;">format-version</span> field in the <span style="font-family: "Courier New",Courier,monospace;">meta</span> section. This value (currently <span style="font-family: "Courier New",Courier,monospace;">1</span>), will be incremented for each output format change. The format-version combined with the version of <span style="font-family: "Courier New",Courier,monospace;">procenv</span> itself should ease any version transition issues.<br />
<h2>
Tweaking the Output</h2>
<br />
If the default space indenting offends your sensibilities, you'll be glad to know
that you can now change it to use tabs instead (or in fact
any other character you choose):<br />
<br />
<pre brush="text">--indent-char="\t"
</pre>
Maybe you'd prefer 3 tab indents:<br />
<br />
<pre brush="text">--indent-char="\t" --indent=3
</pre>
<h2>
Output Formats</h2>
<br />
The new highly-structured text output is in fact a side-effect of the fact that <span style="font-family: "Courier New",Courier,monospace;">procenv</span> can now produce output in other formats. The default is still text, but it can now produce JSON and XML. This even works when running <span style="font-family: "Courier New",Courier,monospace;">--version</span>:<br />
<br />
<pre brush="text">$ procenv --version
0.27
$ procenv --format=json --version
{
"version" : {
"name" : "procenv",
"version" : "0.27",
"author" : "James Hunt <james.hunt@ubuntu.com>"
}
}
$ procenv --format=xml --version
<?xml version="1.0" encoding="UTF-8"?>
<procenv version="0.27" package_string="procenv 0.27" mode="non-privileged" format_version="1">
<section name="version">
<entry name="name">procenv</entry>
<entry name="version">0.27</entry>
<entry name="author">James Hunt &lt;james.hunt@ubuntu.com&gt;</entry>
</section>
</procenv>
</pre>
<br />
This makes <span style="font-family: "Courier New",Courier,monospace;">procenv</span> output much more consumable by standard tools.<br />
<br />
<h2>
Crumbs!</h2>
<br />
However, the new highly-structured output introduces a problem. Before its introduction, it was easy to use <span style="font-family: "Courier New",Courier,monospace;">grep(1)</span> to extract particular values. But now, some of those values are on different lines. Yes, you can use the magic <span style="font-family: "Courier New",Courier,monospace;">grep(1)</span> options <span style="font-family: "Courier New",Courier,monospace;">-A</span>, <span style="font-family: "Courier New",Courier,monospace;">-B</span> and <span style="font-family: "Courier New",Courier,monospace;">-C</span>, but if that isn't convenient for you, <span style="font-family: "Courier New",Courier,monospace;">procenv</span> does offer an alternative in the form of an output format I've called "<span style="font-family: "Courier New",Courier,monospace;">crumb</span>" (short for "breadcrumb"). Here's the file descriptor output in breadcrumb output:<br />
<br />
<pre brush="text">$ procenv --format=crumb --fds
file descriptors:0:terminal:yes
file descriptors:0:valid:yes
file descriptors:0:device:/dev/pts/25
file descriptors:1:terminal:yes
file descriptors:1:valid:yes
file descriptors:1:device:/dev/pts/25
file descriptors:2:terminal:yes
file descriptors:2:valid:yes
file descriptors:2:device:/dev/pts/25
</pre>
<br />
As you can see, "crumb mode" shows every value with the appropriate sections the particular value belongs <i>prior</i> to the value. So the unique path to each value is shown by way of a list of "breadcrumbs" or headings. Thus, to extract all details of <span style="font-family: "Courier New",Courier,monospace;">stdin</span> (aka file descriptor zero):<br />
<br />
<pre brush="text">$ procenv --format=crumb --fds | grep "^file descriptors:0"
file descriptors:0:terminal:yes
file descriptors:0:valid:yes
file descriptors:0:device:/dev/pts/25
</pre>
<br />
It's also really easy to generate CSV data if you wanted to import the output into a spreadsheet:<br />
<br />
<pre brush="text">$ procenv --format=crumb --crumb-separator=',' --separator=','
</pre>
<br />
Download it from:<br />
<ul>
<li><a aiotarget="false" aiotitle="https://launchpad.net/procenv/" href="https://launchpad.net/procenv/">https://launchpad.net/procenv/</a> </li>
</ul>
James Hunthttp://www.blogger.com/profile/03416491632578860019noreply@blogger.com4tag:blogger.com,1999:blog-7349425905862300662.post-82818939327320133782013-08-27T23:33:00.000+01:002013-08-27T23:33:21.563+01:00procenv 0.26 released<br />
Version 0.26 of the <a href="http://ifdeflinux.blogspot.co.uk/2012/10/procenv-and-process-environment.html">procenv</a> utility is now available.<br />
<br />
Changes:<br />
<br />
<ul>
<li>Check to determine if running on a console now works for FreeBSD/kFreeBSD too.</li>
<li>Added ability to show all arguments (<span style="font-family: "Courier New",Courier,monospace;">-A</span>/<span style="font-family: "Courier New",Courier,monospace;">--arguments</span>)<br />(useful when using <span style="font-family: "Courier New",Courier,monospace;">--exec</span>).</li>
<li>Added ability to display network details (<span style="font-family: "Courier New",Courier,monospace;">-N</span>/<span style="font-family: "Courier New",Courier,monospace;">--network</span>).</li>
<li>Added BSD/Hurd-specific signals.</li>
<li>Corrected output sort order.</li>
<li>Mount details now include block, inode and fsck details.</li>
</ul>
There are now 29 categories of environment information displayed when <span style="font-family: "Courier New",Courier,monospace;">procenv</span> is run (on Linux).<br />
<br />
Grab it from:<br />
<br />
<ul>
<li><a href="https://launchpad.net/procenv/">https://launchpad.net/procenv/</a></li>
</ul>
This update should appear in Debian, Ubuntu and FreeBSD soon...<br />
<br />
James Hunthttp://www.blogger.com/profile/03416491632578860019noreply@blogger.com1tag:blogger.com,1999:blog-7349425905862300662.post-66621361787410367782013-08-23T16:52:00.000+01:002013-08-23T16:52:02.775+01:00Upstart 1.10 releasedLots of goodness in this release (explanatory posts to follow):<br />
<br />
<ul>
<li><span style="font-family: "Courier New",Courier,monospace;">upstart-local-bridge</span>: New bridge for starting jobs on local socket connections.</li>
<li><span style="font-family: "Courier New",Courier,monospace;">upstart-dconf-bridge</span>: New bridge for Session Inits to react to dconf/gsettings changes.</li>
<li><span style="font-family: "Courier New",Courier,monospace;">upstart-dbus-bridge</span>: New '<span style="font-family: "Courier New",Courier,monospace;">--bus-name</span>' option to allow bus name variable to be included in <span style="font-family: "Courier New",Courier,monospace;">dbus-event(7)</span>.</li>
<li>New "<span style="font-family: "Courier New",Courier,monospace;">reload signal</span>" stanza to allow jobs to specify a custom signal that will be sent to the main process (rather than the default <span style="font-family: "Courier New",Courier,monospace;">SIGHUP</span>).</li>
<li>Inclusion of Session Init sample jobs.</li>
<li>Re-exec fixes for handling chroot sessions.</li>
<li>Shutdown fix for Session Inits.</li>
<li>New python3 module and accompanying integration test suite for testing Upstart running as <span style="font-family: "Courier New",Courier,monospace;">PID 1</span> and as a Session Init (privileged and non-privileged). </li>
</ul>
<br />
The <a href="http://upstart.ubuntu.com/cookbook/">Upstart cookbook</a> has been updated for this release.James Hunthttp://www.blogger.com/profile/03416491632578860019noreply@blogger.com1tag:blogger.com,1999:blog-7349425905862300662.post-6707979982900666172013-08-10T12:00:00.001+01:002013-08-10T12:00:18.069+01:00Upstart Cookbook now updated for Upstart in DebianI've started a long-overdue update to the Upstart Cookbook for Upstart on Debian. Look out for the Debian swirl! :-)<br />
<br />
<ul>
<li><a href="http://upstart.ubuntu.com/cookbook">http://upstart.ubuntu.com/cookbook</a></li>
</ul>
James Hunthttp://www.blogger.com/profile/03416491632578860019noreply@blogger.com0tag:blogger.com,1999:blog-7349425905862300662.post-60450274652453465272013-08-10T11:56:00.001+01:002013-08-10T11:56:20.689+01:00procenv 0.25 now builds on AndroidProcenv 0.25 now builds natively under Android:<br />
<br />
<ul>
<li><a href="https://launchpad.net/procenv/">https://launchpad.net/procenv/</a></li>
</ul>
<br />
It also fixes an interesting issue that was causing it to fail to generate FreeBSD build logs. However, that is now fixed so you can see the FreeBSD build environment now too by looking at the port logs:<br />
<br />
<ul>
<li><a href="http://pointyhat.freebsd.org/errorlogs/i386-8-latest-logs/procenv-0.25.log">http://pointyhat.freebsd.org/errorlogs/i386-8-latest-logs/procenv-0.25.log</a></li>
<li><a href="http://pointyhat.freebsd.org/errorlogs/i386-9-latest-logs/procenv-0.25.log">http://pointyhat.freebsd.org/errorlogs/i386-9-latest-logs/procenv-0.25.log</a> </li>
</ul>
More logs to come - the FreeBSD ports seem to build rather slowly...)James Hunthttp://www.blogger.com/profile/03416491632578860019noreply@blogger.com1tag:blogger.com,1999:blog-7349425905862300662.post-34641741503266556212013-08-05T12:40:00.000+01:002013-08-05T12:40:22.969+01:00Counting down to Ubuntu and Debian releases with distro-infoA while back I though, "wouldn't it be great if the <a href="https://wiki.ubuntu.com/SaucySalamander/ReleaseSchedule">Ubuntu release schedule</a> could be accessed as some form of web service?" That would allow all sorts of fun scripts to be written. Yes, you <i>could</i> write code to parse the wiki page, but I think the technical term for that is "gross".<br />
<br />
However, there is now a much simpler alternative in the form of <span style="font-family: "Courier New",Courier,monospace;">distro-info</span>. If you've never used this awesome tool written by <a href="https://launchpad.net/~bdrung">Benjamin Drung</a> you've been missing out! It can tell you all sorts of interesting information. A few examples...<br />
<br />
<h2>
Examples of how to use distro-info</h2>
<br />
<ul>
<li>What is the current development release on the system I am running on?<br /><br /><pre code="shell">$ distro-info --devel
saucy</pre>
</li>
<li>What is the current Debian development release?
<br /><br /><pre code="shell">$ debian-distro-info --devel
sid</pre>
</li>
<li>What are the currently supported Ubuntu releases?
<br /><br /><pre code="shell">$ ubuntu-distro-info --supported
lucid
precise
quantal
raring
saucy
</pre>
</li>
<li>What's the latest Ubuntu LTS release?
<pre code="shell">$ ubuntu-distro-info --lts
precise
</pre>
</li>
<li>What's the current stable Debian release?
<br /><br /><pre code="shell">$ debian-distro-info --stable
wheezy
</pre>
</li>
</ul>
<br />
<h2>
distro-info... Now with Added Milestone Goodness</h2>
I've been working with Benjamin recently to add a new feature which landed late last week in both Debian unstable and Ubuntu Saucy. So, we can now ask <span style="font-family: "Courier New",Courier,monospace;">distro-info</span> when a <i>particular release milestone will happen</i> (or <i>has already happened</i>):<br />
<br />
<ul>
<li>How many days until Ubuntu Saucy is released?
<br /><br /><pre code="shell">$ ubuntu-distro-info --devel --days
73</pre>
</li>
<li>How many days since the Ubuntu Saucy release was "created"?
<br /><br /><pre code="shell">$ ubuntu-distro-info --devel --days=created
-102</pre>
</li>
<li>How many days until Ubuntu Precise goes "end-of-life" (desktop only!)
<br /><br /><pre code="shell">$ ubuntu-distro-info --lts --days=eol
1360</pre>
</li>
<li>How many days since Debian sid was created?
<br /><br /><pre code="shell">$ debian-distro-info --devel --days=created
-7294</pre>
</li>
</ul>
<h2>
A Scripted Example</h2>
We can now write simple little scripts like this to show a popup when getting close to a release:
<br />
<pre brush="shell">#!/bin/sh
set -- $(distro-info --devel --codename --days=release)
codename="$1"
days="$2"
# Only warn close to a release
[ "$days" -gt 30 ] && exit 0
zenity --info --text "$codename is released in $days days!"
</pre>
<br />
<h2>
Wait! Let's do that with Upstart!</h2>
In fact, you could turn the script above into an Upstart session job rather easily:
<br />
<pre brush="shell">cat >>$HOME/.config/upstart/release-reminder.conf<<EOT
start on desktop-start
task
script
set -- $(distro-info --devel --codename --days=release)
codename="$1"
days="$2"
# Only warn close to a release
[ "$days" -gt 30 ] && exit 0
zenity --info --text "$codename is released in $days days!"
end script
EOT
</pre>
Now, this script will run automatically every time you login, assuming you are running either:<br />
<ul>
<li>an <a href="http://ifdeflinux.blogspot.co.uk/2013/04/upstart-user-sessions-in-ubuntu-raring.html">Ubuntu Raring system with Upstart User sessions enabled</a>.</li>
<li>Ubuntu Saucy where Upstart User sessions are enabled by default.</li>
</ul>
The currently recognised milestones are "<span style="font-family: "Courier New",Courier,monospace;">created</span>", "<span style="font-family: "Courier New",Courier,monospace;">release</span>", "<span style="font-family: "Courier New",Courier,monospace;">eol</span>" and for Ubuntu LTS releases only "<span style="font-family: "Courier New",Courier,monospace;">eol-server</span>".<br />
<h2>
Further Ideas</h2>
<br />
We could conceivably enhance this feature further by adding in support for querying additional milestones such as "<span style="font-family: "Courier New",Courier,monospace;">alpha-1</span>" and "<span style="font-family: "Courier New",Courier,monospace;">feature-freeze</span>".<br />
<br />
<h2>
Want to know more?</h2>
<br />
For full details, read <span style="font-family: "Courier New",Courier,monospace;">distro-info(1)</span>.<br />
<br />
<br />
<br />James Hunthttp://www.blogger.com/profile/03416491632578860019noreply@blogger.com1tag:blogger.com,1999:blog-7349425905862300662.post-5644163035974098502013-08-05T08:48:00.000+01:002013-08-05T08:48:10.076+01:00I'm going to DebConf13<a href="http://debconf13.debconf.org/"><img alt="Going to DebConf13!" src="http://media.debconf.org/dc13/artwork/dc13-btn1-going-bg.png" style="border: 0px;" title="" /></a>
<a href="http://web.dodds.net/~vorlon/wiki/blog/">Vorlon</a> and I will be giving a talk on Upstart (<a href="http://penta.debconf.org/dc13_schedule/events/1027.en.html">'Upstart in Debian'</a>). So if you'd like to learn more about the awesome next-gen event-based init system, please drop by!James Hunthttp://www.blogger.com/profile/03416491632578860019noreply@blogger.com0tag:blogger.com,1999:blog-7349425905862300662.post-77088890365598976082013-05-17T19:37:00.000+01:002013-05-17T19:37:15.429+01:00A simple two-player QML game for Ubuntu Touch using the Ubuntu SDK: noughts and crosses (aka tic-tac-toe)!Inspired by <a href="http://theravingrick.blogspot.com/">Rick</a>'s recent blog posts, and keen to write a blog post with a ridiculously long title, I've been reading up on QML recently. Still bearing the scars from the XML horrors of working with J2EE in the early days, that 3-byte acronym ending in "ML" initially subconsciously somewhat filled me with trepidation. However, as soon as I actually <i>saw</i> some QML, I could see these fears were unfounded (<phew>! :-) And in fact I now <i>love</i> QML. It's clean, elegant, powerful, declarative and (OMG YAY!) you can even "%-bounce" on the braces in vim! :-) That said, the qtcreator IDE is extremely good, managing to provide just enough of what you want without requiring endless additional configuration.</phew><br />
<br />
But it doesn't stop there. The <a href="http://design.ubuntu.com/">Design Team</a> have done some <i>incredible</i> work in creating the <a href="http://developer.ubuntu.com/api/ubuntu-12.10/qml/mobile/overview-ubuntu-sdk.html">Ubuntu SDK</a> components: not only do they look fantastic (if you have the <span style="font-family: Courier New, Courier, monospace;">ubuntu-ui-toolkit-examples</span> package installed, try running <span style="font-family: Courier New, Courier, monospace;">/usr/lib/ubuntu-ui-toolkit/demos/launch_componentshowcase</span>), they are also extremely flexible and powerful.<br />
<br />
As Rick has mentioned, it does take a while to grok the "QML-ish" way of doing things. And if like me you spend most of your time writing in imperative languages, initially you just think "a<i>ll this QML is wonderful, but where do I actually put the code?</i>". But then you have the epiphany moment when you realise you're <i>already</i> writing "the code" - in many cases, you don't need anything beyond the declarative QML itself.<br />
<br />
<h2>
I Need an Itch to Scratch</h2>
<br />
The only real way to learn a new language is to use it. But what to do? I wanted to code something simple and fun, like a game. There are already few games on the <a href="https://wiki.ubuntu.com/Touch/Collection">Collections page</a> so I needed to think of a <i>really</i> simple one that is also fun to play. How about a game that even children can appreciate? Of course - <a href="http://en.wikipedia.org/wiki/Tic-tac-toe">Noughts and Crosses</a> (aka <i>tic-tac-toe</i>)!<br />
<ul>
<li><a href="https://github.com/jamesodhunt/qml-noughts-and-crosses">https://github.com/jamesodhunt/qml-noughts-and-crosses</a> (GPLv3)</li>
</ul>
Note that the code is pretty rudimentary right now, but it's just about usable ;-)<br />
<br />
<h2>
Design</h2>
This is a simple game so we only need a few objects: <span style="font-family: Courier New, Courier, monospace;">Cell</span>, <span style="font-family: Courier New, Courier, monospace;">Game</span> and <a href="http://developer.ubuntu.com/api/ubuntu-12.10/qml/mobile/qml-ubuntu-components0-mainview.html">MainView</a>.<br />
<br />
The <span style="font-family: Courier New, Courier, monospace;">MainView</span> is the container for the application and includes a <a href="http://developer.ubuntu.com/api/ubuntu-12.10/qml/mobile/qml-ubuntu-components0-page.html">Page</a> and the actual <span style="font-family: Courier New, Courier, monospace;">Game</span> object. The only property we specify for the game is the <span style="font-family: Courier New, Courier, monospace;">boardSize</span> of <span style="font-family: Courier New, Courier, monospace;">3</span> giving us a 3x3 board. Technically, we don't actually even need to specify this since -- as we're about to see -- 3x3 is the default board size anyway. So, the <span style="font-family: Courier New, Courier, monospace;">Game</span> object could be specified minimally as "<span style="font-family: Courier New, Courier, monospace;">Game {}</span>". However, I've left it specified as a reminder to myself that ultimately I'd like to pass a variable to allow the board size to be specified at game creation time.<br />
<br />
Here is a slightly simplified version of the <span style="font-family: Courier New, Courier, monospace;">MainView</span> (<span style="font-family: Courier New, Courier, monospace;">noughts-and-crosses.qml</span>):
<br />
<br />
<pre class="brush: javascript">import QtQuick 2.0
import Ubuntu.Components 0.1
MainView {
Page {
title: "Noughts and Crosses"
id: page
Game {
// change this to whatever value you want for an NxN-sized board
boardSize: 3
}
}
}
</pre>
<br />
<br />
The <span style="font-family: Courier New, Courier, monospace;">Game</span> object is a <a href="http://qt-project.org/doc/qt-5.0/qtquick/qml-qtquick2-column.html">Column</a> and comprises a <a href="http://developer.ubuntu.com/api/ubuntu-12.10/qml/mobile/qml-ubuntu-components0-label.html">Label</a>, to show some text relating to the game, and a <a href="http://qt-project.org/doc/qt-5.0/qtquick/qml-qtquick2-grid.html">Grid</a> to actually represent the game. There is some magic going on in the grid as it uses the very cool <a href="http://qt-project.org/doc/qt-5.0/qtquick/qml-qtquick2-repeater.html">Repeater</a> object to make laying out the grid easy: for a 3x3 board it creates 9 <span style="font-family: Courier New, Courier, monospace;">Cell</span> objects and packs them into the grid. Here's a cut-down version of the <span style="font-family: Courier New, Courier, monospace;">Game</span> object:<br />
<br />
<pre class="brush: javascript">Column {
property alias boardSize: gameGrid.boardSize
Label {
id: text
text: "Noughts goes first"
}
Grid {
id: gameGrid
// Default to a 3x3 board (we only support square boards).
property real boardSize: 3
// toggled between "O" and "X". The value specified below denotes
// which side goes first.
property string player: "O"
columns: boardSize
rows: boardSize
// layout the appropriate number of cells for the board size
Repeater {
id: gridRepeater
model: boardSize * boardSize
Cell {
width: 100
height: width
}
}
}
}
</pre>
<br />
Note the property alias for boardSize in the <span style="font-family: Courier New, Courier, monospace;">Column</span> object - it exposes a <span style="font-family: Courier New, Courier, monospace;">boardSize</span> variable which is just a way to access the real variable of the same name within the <span style="font-family: Courier New, Courier, monospace;">Grid</span> object. Note too that we tell the <span style="font-family: Courier New, Courier, monospace;">Grid</span> object its dimensions by setting its <span style="font-family: Courier New, Courier, monospace;">columns</span> and <span style="font-family: Courier New, Courier, monospace;">rows</span> properties.<br />
<br />
The <span style="font-family: Courier New, Courier, monospace;">Game</span> object also contains a chunk of Javascript in the form of the <span style="font-family: Courier New, Courier, monospace;">checkForWin()</span> function to determine whether a move resulted in the game being won.<br />
<br />
The <span style="font-family: Courier New, Courier, monospace;">Cell</span> object is the most interesting object. A <span style="font-family: Courier New, Courier, monospace;">Cell</span> represents an individual location on the board. It is constructed from a <a href="http://qt-project.org/doc/qt-5.0/qtquick/qml-qtquick2-rectangle.html">Rectangle</a> and comprises a <a href="http://qt-project.org/doc/qt-5.0/qtquick/qml-qtquick2-text.html">Text</a> value. The text value is either a middle-dot (to denote the cell has not yet been selected), a "<span style="font-family: Courier New, Courier, monospace;">O</span>" or a "<span style="font-family: Courier New, Courier, monospace;">X</span>". It also includes a <a href="http://qt-project.org/doc/qt-5.0/qtquick/qml-qtquick2-mousearea.html">MouseArea</a> that specifies the new cell state to apply when the cell is clicked. Initially, the state is middle-dot but when the cell is clicked, the state is changed to the value of the parent (<span style="font-family: Courier New, Courier, monospace;">Game</span>) objects <span style="font-family: Courier New, Courier, monospace;">player</span> property. The <span style="font-family: Courier New, Courier, monospace;">Cell</span> object specifies 3 <a href="http://qt-project.org/doc/qt-5.0/qtquick/qml-qtquick2-state.html">states</a> to represent every possible value a <span style="font-family: Courier New, Courier, monospace;">Cell</span> can display. What's neat here is that changing the cells state also toggles the parent (<span style="font-family: Courier New, Courier, monospace;">Game</span>) objects <span style="font-family: Courier New, Courier, monospace;">player</span> property which allows the game to proceed with each player taking a turn. Clicking a cell also calls the <span style="font-family: 'Courier New', Courier, monospace;">checkForWin()</span>function to determine if a particular turn results in the game being won. Here's the complete <span style="font-family: Courier New, Courier, monospace;">Cell</span> object:<br />
<br />
<pre class="brush: javascript">Rectangle {
id: cell
state: gameGrid.defaultText
property alias textColor: cellText.color
Text {
id: cellText
text: parent.state
color: "silver"
anchors.horizontalCenter: parent.horizontalCenter
anchors.verticalCenter: parent.verticalCenter
font.pointSize: 48
}
states: [
State {
name: cell.parent.defaultText
PropertyChanges { target: gameGrid; player: "" }
},
State {
name: "O"
PropertyChanges { target: gameGrid; player: cell.state }
},
State {
name: "X"
PropertyChanges { target: gameGrid; player: cell.state }
}
]
// when clicked,
MouseArea {
anchors.fill: parent
onClicked: {
cell.state = (gameGrid.player == "O" ? "X" : "O");
gameGrid.numberTurns += 1
gameGrid.checkForWin();
}
}
}
</pre>
<br />
<h2>
Winning Algorithm</h2>
The approach I've taken is very simplistic: just scan each row, column and diagonal looking for a winning run. This <i>isn't</i> particularly efficient (we're scanning the board multiple times) but that's not a problem for small board sizes. However, it has two fairly compelling attributes:<br />
<br />
<br />
<ul>
<li>It's simple to understand</li>
<li>It works for arbitrary-sized boards.</li>
</ul>
<br />
My favourite alternative algorithm is to make use of the properties of <a href="http://en.wikipedia.org/wiki/Magic_square">Magic Squares</a>. Using these, you can scan the board a single time to determine if a player has one. This is achieved by determining if a cell has been selected by a player and if so incrementing their counter based on the magic square value for that index. For a 3x3 board, if a players total equals 15, they win!<br />
<br />
<h2>
Screenshots</h2>
<div>
So, what does it look like at this early beta stage...?<br />
<br />
Start of a new game:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-rlPYWTJf7bU/UZYh2oMinnI/AAAAAAAAAYk/l_-z7PveGzw/s1600/noughts-and-crosses-new-board.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="http://1.bp.blogspot.com/-rlPYWTJf7bU/UZYh2oMinnI/AAAAAAAAAYk/l_-z7PveGzw/s320/noughts-and-crosses-new-board.png" width="258" /></a></div>
<br />
We have a winner!<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-Xhvlpn3Gseo/UZYh_OasCCI/AAAAAAAAAYs/JMys2ZUsgeA/s1600/noughts-and-crosses-win.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="http://2.bp.blogspot.com/-Xhvlpn3Gseo/UZYh_OasCCI/AAAAAAAAAYs/JMys2ZUsgeA/s320/noughts-and-crosses-win.png" width="258" /></a></div>
<br />
Another winner on a 7x7 board (the person playing crosses needs more practice me thinks :-):<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-e7My0PUMUOI/UZZRuvzcvpI/AAAAAAAAAY8/WkxrGC_Cwac/s1600/noughts-and-crosses-7x7-win.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="http://2.bp.blogspot.com/-e7My0PUMUOI/UZZRuvzcvpI/AAAAAAAAAY8/WkxrGC_Cwac/s320/noughts-and-crosses-7x7-win.png" width="278" /></a></div>
<br />
<br />
<h2>
What's Next</h2>
<div>
<ul>
<li>The javascript code is currently horrid and needs to be refactored with dynamite.</li>
<li>Add ability to play "the computer".</li>
<li>Config option to allow variable-sided playing grids.</li>
<li>Once the game is stopped, we need to disallow further board clicks.</li>
<li>Leverage more QML facilities to simplify the code further.</li>
<li>Visual improvements (animation for a winning run maybe?)</li>
<li>Ability to change player that starts.</li>
<li>Score-keeping and "best of '<i>n</i>' games <n>" support (particularly useful when the kids beat you repeatedly ;-)</n></li>
<li>Menu to start new game.</li>
</ul>
</div>
<div>
The code is on github, so get forking!<br />
<br />
<h2>
In Conclusion</h2>
My "clean-room" implementation is far from perfect at the moment, but it's been a fantastic learning exercise so far and a <i>lot</i> of fun!<br />
<br />
There are of course other QML noughts-and-crosses games out there. They come with varying licenses, some use C++ for the game logic, and most -- if not all -- are hard-coded to produce a 3x3 board only. Additionally, they generally use graphical representations for the noughts and crosses whereas here, I'm just using styled text. If you're interested, compare my github code with, for example, the Qt version to see the different approaches taken:<br />
<ul>
<li><a href="http://qt-project.org/doc/qt-4.8/declarative-toys-tic-tac-toe.html">http://qt-project.org/doc/qt-4.8/declarative-toys-tic-tac-toe.html</a></li>
</ul>
</div>
<h2>
See Also</h2>
<ul>
<li><a href="https://wiki.ubuntu.com/Touch/Collection">Ubuntu Touch Collections pag</a>e</li>
<li><a href="http://developer.ubuntu.com/api/ubuntu-12.10/qml/mobile/overview-ubuntu-sdk.html">Ubuntu SDK documentation</a></li>
<li><a href="https://qt-project.org/doc/">https://qt-project.org/doc/</a></li>
</ul>
<br />
<br />James Hunthttp://www.blogger.com/profile/03416491632578860019noreply@blogger.com5tag:blogger.com,1999:blog-7349425905862300662.post-27878692392568856002013-04-18T19:53:00.000+01:002013-04-18T19:53:03.082+01:00Observing the initial LXC environment using procenvIf you're interested in seeing the initial environment inside an LXC container, here's how:
<br />
<ol>
<li>Install <a href="http://ifdeflinux.blogspot.co.uk/2012/10/procenv-and-process-environment.html">procenv</a> inside the container:<br /><br />
<pre lang="shell">$ sudo apt-get install -y procenv</pre>
<br />
</li>
<li>Shutdown the container:<br /><br />
<pre lang="shell">$ sudo shutdown -h now
</pre>
</li>
<li>Boot the container (mine is called "<span style="font-family: Courier New, Courier, monospace;">raring</span>" in this example) like this:
<br /><br />
<pre lang="shell">$ sudo lxc-start -n raring --console-log /tmp/lxc-console.log \
-- /usr/bin/procenv --file=/dev/console --exec \
-- /sbin/init --debug
</pre>
</li>
<li>View the <span style="font-family: Courier New, Courier, monospace;">/tmp/lxc-console.log</span> logfile in the host environment.
</li>
</ol>
<div>
Note that those two sets of non-option double-dashes are required; the first tells LXC to treat what follows as a command to run after the container starts (in this case <span style="font-family: Courier New, Courier, monospace;">procenv</span>), and the <i>second</i> set tells <span style="font-family: Courier New, Courier, monospace;">procenv</span> to treat what follows as a command to run after it has <span style="font-family: inherit;"><i>finished running</i></span> (in this case Upstart (<span style="font-family: Courier New, Courier, monospace;">/sbin/init</span> :-))!</div>
James Hunthttp://www.blogger.com/profile/03416491632578860019noreply@blogger.com0tag:blogger.com,1999:blog-7349425905862300662.post-31320173248302223732013-04-15T23:36:00.001+01:002013-04-15T23:36:22.322+01:00Byobu Bling with Unicode Custom Indicators<h2>
Introduction</h2>
In these heady times of almost daily advances in UI pizzazz, you could be forgiven for thinking the terminal is somewhat of a dead-zone when it comes to excitement. Oh how wrong you would be though...!<br />
<br />
If you've never used <a href="http://byobu.co/index.html">Byobu</a>, <a href="http://blog.dustinkirkland.com/">Dustin's</a> awesome text-based window manage, <span style="font-family: Courier New, Courier, monospace;">sudo apt-get install</span> it without delay to be bathed in pure terminal goodness. As shown below, byobu comes with a whole host of standard indicators, allowing the display of useful snippets of information.<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-Ge4IvSQNyT4/UWxPtMOu8VI/AAAAAAAAAXo/-Fzx8mtX6js/s1600/byobu-welcome.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="192" src="http://2.bp.blogspot.com/-Ge4IvSQNyT4/UWxPtMOu8VI/AAAAAAAAAXo/-Fzx8mtX6js/s320/byobu-welcome.png" width="320" /></a></div>
<br />
The indicators are minimal out of terminal-width real estate necessity, but that's also their strength - they provide just the details you really care about in a tight ASCII format. And they're cute of course :-)<br />
<br />
However, byobu has a well-kept secret: when running with the <span style="font-family: Courier New, Courier, monospace;">tmux</span> backend, it supports Unicode indicators. So with the proviso that you are running "<span style="font-family: Courier New, Courier, monospace;">byobu-tmux</span>" rather than "<span style="font-family: Courier New, Courier, monospace;">byobu-screen</span>", you can squeeze down those indicators <i>even further</i>, saving precious onscreen characters <i>and</i> get extra bling at the same time! Intrigued? Then read on...<br />
<div>
<br />
<h2>
Background</h2>
<br />
A few years ago (yes, it's taken that long to get around to writing this up! ;-), I was playing around with Unicode and ended up writing a very simple C program to display the time in "binary" using the <a href="http://en.wikipedia.org/wiki/List_of_Unicode_characters#Braille_patterns">Braille Unicode Block</a>. Assuming you were only displaying hours and minutes, that meant you could display any time in 2 onscreen characters - nice and compact :) I wanted to use this program as a byobu custom indicator but at the time byobu was only using GNU screen and alas even the latest version of screen did not have full UTF-8 support (for its status line atleast). However, tmux was fast becoming the worthy successor to screen and of course byobu now supports both backends. I ended up rewriting the C program as a shell script to fit in with the other byobu indicators. That shell script became the <span style="font-family: Courier New, Courier, monospace;">time_binary</span> indicator shipped with byobu.<br />
<br /></div>
Here it is displaying the time (21:04) in 24-hour format:
<br />
<br />
<pre lang="shell">T[⢄⠐]
</pre>
<br />
Admittedly, it's kinda terse and somewhat difficult to read but that's half the fun right? :-) <span style="font-family: Courier New, Courier, monospace;">time_binary</span> is slightly mis-named as it can also display the time in hexadecimal and octal.<br />
<br />
But why stop there? I now wanted all sorts of Unicode indicators showing the level of things like sound, wifi signal strength, left/right speaker volumes, memory used, system load, etc. However, since UIs are very personal, I wanted to be able to write short custom indicators for each of the above but allow the choice of unicode indicators to be changed with minimal trouble as desired.<br />
<br />
<h2>
Enter byobu-ulevel</h2>
What I ended up doing is trawling through some of those myriad Unicode symbols looking for suitable groups and creating a small utility called <span style="font-family: Courier New, Courier, monospace;">byobu-ulevel</span>. This takes a numeric value and converts it into an appropriate "level" symbol. This all sounds a bit vague, so here's an example:<br />
<br />
<pre lang="shell">$ byobu-ulevel 100
█
</pre>
<div>
<br />
The above example has converted the value <span style="font-family: Courier New, Courier, monospace;">100</span> to its default <span style="font-family: Courier New, Courier, monospace;">100%</span> value using the default theme. And here's the default <span style="font-family: Courier New, Courier, monospace;">30%</span> value:<br />
<br /></div>
<pre lang="shell">$ byobu-ulevel 30
▂
</pre>
<div>
<br />
Starting to make sense? If your data source doesn't provide values from <span style="font-family: Courier New, Courier, monospace;">0%</span> to <span style="font-family: Courier New, Courier, monospace;">100%</span>, fear not. Let's say your data source ranges from <span style="font-family: Courier New, Courier, monospace;">-123</span> to <span style="font-family: Courier New, Courier, monospace;">+456</span> and your value to display is <span style="font-family: Courier New, Courier, monospace;">333.0</span>:<br />
<br /></div>
<pre lang="shell">$ byobu-ulevel -c 333 -m -123.45 -x +678.90
▄
</pre>
<div>
<br />
<br /></div>
This utility is generally only useful if your data has some specifiable bounded range. However, there is a "cheat mode" - if you are happy for values to overflow/underflow silently, you can specify "<span style="font-family: Courier New, Courier, monospace;">-p</span>":
<br />
<br />
<pre lang="shell">$ byobu-ulevel -c 999 -x 100
ERROR: value (999) > maximum (100)
$ byobu-ulevel -c 999 -x 100 -p
█
</pre>
<br />
<h2>
Template for a Byobu Unicode Custom Notification</h2>
We can create a Unicode custom notification:<br />
<br />
<pre lang="shell">$ mkdir -p ~/.byobu/bin
$ cd ~/.byobu/bin
$ cat >5_utest<<EOT
# XXX: rather than hard-coding the value, put your logic here
# XXX: to determine an actual value from some data source.
value=20
printf "U[%s]\n" $(byobu-ulevel $value)
EOT
$ chmod 755 5_utest
</pre>
<br />
In fact, as you'll see once you finish this article, the example notification above is really a template for most of your Unicode notifications.<br />
<br />
Once you've activated the custom notifications in byobu (F9 → Toggle status notifications → check the "<span style="font-family: Courier New, Courier, monospace;">custom</span>" option), and waited for a few seconds (5 in the example above), you'll see the following on your byobu status line:<br />
<br />
<pre lang="shell">U[▁]
</pre>
<br />
I've added a "<span style="font-family: Courier New, Courier, monospace;">U</span>" to denote my Unicode test indicator and delimited the actual indicator value glyph between square brackets, but of course you can use whatever ornamentation you wish.<br />
<br />
<h2>
Themes</h2>
<br />
So, we now have an indicator that, assuming you add in some appropriate logic to set "<span style="font-family: Courier New, Courier, monospace;">$value</span>" dynamically, will show a 1-character wide bar whose height varies depending on the magnitude of <span style="font-family: Courier New, Courier, monospace;">$value</span>.<br />
<br />
But what if you don't like that bar much? No problem because <span style="font-family: Courier New, Courier, monospace;">byobu-ulevel</span> comes with a set of built-in "themes" and since we haven't specified one, we get the default (<span style="font-family: Courier New, Courier, monospace;">vbars_8</span>). Let's list the available themes by asking <span style="font-family: Courier New, Courier, monospace;">byobu-ulevel</span> to display them:<br />
<br />
<pre lang="shell">$ byobu-ulevel -l
circles_2
diamonds_2
flags_2
hearts_2
squares_2
squares_small_2
stars_2
vdots_thick_4
vdots_thin_4
fractions_4
quadrants_4
shades_4
circles_5
dice_6
hbars_8
vbars_8
circle_number_10
solid_numbers_a_10
solid_numbers_b_10
</pre>
<br />
So now we know the theme names, let's take a look at them:
<br />
<br />
<pre lang="shell"></pre>
<pre lang="shell">$ for theme in $(byobu-ulevel -l); do byobu-ulevel -l -t "$theme"; done
Listing theme 'circles_2'
○ ●
Listing theme 'diamonds_2'
◇ ◆
Listing theme 'flags_2'
⚐ ⚑
Listing theme 'hearts_2'
♡ ♥
Listing theme 'squares_2'
◻ ◼
Listing theme 'squares_small_2'
◽ ◾
Listing theme 'stars_2'
☆ ★
Listing theme 'vdots_thick_4'
⣀ ⣤ ⣶ ⣿
Listing theme 'vdots_thin_4'
⢀ ⢠ ⢰ ⢸
Listing theme 'fractions_4'
¼ ½ ¾ ¹
Listing theme 'quadrants_4'
◔ ◑ ◕ ●
Listing theme 'shades_4'
░ ▒ ▓ █
Listing theme 'circles_5'
◦ ○ ◎ ◉ ●
Listing theme 'dice_6'
⚀ ⚁ ⚂ ⚃ ⚄ ⚅
Listing theme 'hbars_8'
▏ ▎ ▍ ▌ ▋ ▊ ▉ █
Listing theme 'vbars_8'
▁ ▂ ▃ ▄ ▅ ▆ ▇ █
Listing theme 'circle_number_10'
➀ ➁ ➂ ➃ ➄ ➅ ➆ ➇ ➈ ➉
Listing theme 'solid_numbers_a_10'
➊ ➋ ➌ ➍ ➎ ➏ ➐ ➑ ➒ ➓
Listing theme 'solid_numbers_b_10'
❶ ❷ ❸ ❹ ❺ ❻ ❼ ❽ ❾ ❿
</pre>
So how do you use these themes? Easy, just choose the one you like the look of and specify it using "<span style="font-family: Courier New, Courier, monospace;">-t $theme</span>". Let's try out the <span style="font-family: Courier New, Courier, monospace;">stars_2</span> theme:<br />
<br />
<pre lang="shell">$ byobu-ulevel -c 60 -t stars_2
★★★☆☆
</pre>
<br />
All themes ending with "<span style="font-family: Courier New, Courier, monospace;">_2</span>" contain two glyphs and are called "rating themes" (for obvious reasons I hope). The first glyph in a rating theme represents the "unused" part of the range and the second glyph represents the "used" part. So, here we see that <span style="font-family: Courier New, Courier, monospace;">60%</span> (equating to 3 solid stars) is "used" since the value is <span style="font-family: Courier New, Courier, monospace;">60</span> and the default range is <span style="font-family: Courier New, Courier, monospace;">0-100%</span>. Obviously, we can't display half glyphs so for this theme for example the value provided is "rounded to the nearest star" :-)<br />
<br />
By default rating themes are displayed left-to-right. However, you can change that using the "<span style="font-family: Courier New, Courier, monospace;">-r</span>" option:
<br />
<br />
<pre lang="shell">$ byobu-ulevel -c 60 -t stars_2 -r
☆☆★★★
</pre>
<br />
Or you can invert the theme such that the first glyph will represent the "used" part and the second the "unused" using the "<span style="font-family: Courier New, Courier, monospace;">-i</span>" option:
<br />
<br />
<pre lang="shell">$ byobu-ulevel -c 60 -t stars_2 -i
☆☆☆★★</pre>
<br />
But maybe 5 stars isn't enough for you? No problem, use the width option ("<span style="font-family: Courier New, Courier, monospace;">-w</span>"):<br />
<br />
<pre lang="shell">$ byobu-ulevel -c 60 -t stars_2 -w 10</pre>
<pre lang="shell">★★★★★★☆☆☆☆
</pre>
<br />
Themes with more than 2 glyph values are different to rating themes in that rather than using a combination of the "used" and "unused" glyphs to show the value, only a single glyph is selected. For example, here's our 60% again using the <span style="font-family: Courier New, Courier, monospace;">circle_number_10</span> theme:<br />
<br />
<pre lang="shell">$ byobu-ulevel -c 60 -t circle_number_10
➅
</pre>
<br />
Since I'm bound to have missed out your favourite Unicode glyphs, you can of course make your own theme trivially too. Let's create a rating theme:<br />
<br />
<pre lang="shell">$ byobu-ulevel -c 60 -u "· ☢"
☢☢☢··
</pre>
<br />
And now a normal theme with, in this example, 10 values:<br />
<br />
<pre lang="shell">$ byobu-ulevel -c 60 -u "a b c d e f g h i j"
f
</pre>
<br />
You could even use <a href="http://ifdeflinux.blogspot.co.uk/2012/09/out-output-utility.html">utfout</a> to construct your theme:<br />
<br />
<pre lang="shell">$ byobu-ulevel -c 45 -u "$(utfout -a ' ' '\{α..ω}')"
λ
</pre>
<pre lang="shell"></pre>
<h2>
<span style="font-family: 'Times New Roman'; white-space: normal;">Accessibility</span></h2>
<pre lang="shell"><span style="font-family: Times New Roman; white-space: normal;">But what about accessibility? What about users who cannot display some of these characters? </span><span style="font-family: Courier New, Courier, monospace; white-space: normal;">byobu-ulevel</span><span style="font-family: Times New Roman; white-space: normal;"> provides for these folk too: either specify "</span><span style="font-family: Courier New, Courier, monospace; white-space: normal;">-a</span><span style="font-family: Times New Roman; white-space: normal;">" or set the </span><span style="font-family: Courier New, Courier, monospace; white-space: normal;">BYOBU_A11Y</span><span style="font-family: Times New Roman; white-space: normal;"> environment variable to any value, and </span><span style="font-family: Courier New, Courier, monospace; white-space: normal;">byobu-ulevel</span><span style="font-family: Times New Roman; white-space: normal;"> will display purely numeric (floating point) values rather than Unicode bling. You can specify the number of decimal places too with the "</span><span style="font-family: Courier New, Courier, monospace; white-space: normal;">-e</span><span style="font-family: Times New Roman; white-space: normal;">" option.</span></pre>
<pre lang="shell"><span style="font-family: Times New Roman; white-space: normal;">
</span></pre>
<h2>
<span style="font-family: Times New Roman; white-space: normal;">Multiple Values</span></h2>
<pre lang="shell"><span style="font-family: Times New Roman; white-space: normal;">
</span></pre>
<pre lang="shell"><span style="font-family: Times New Roman;"><span style="white-space: normal;">
</span></span></pre>
<pre lang="shell"><span style="white-space: normal;"><span style="font-family: Times New Roman;">Great, but byobu-ulevel only handles a </span><i style="font-family: 'Times New Roman';">single</i><span style="font-family: Times New Roman;"> value - what if you want to display multiple values? For example a range of </span><span style="font-family: Times New Roman;">values over time? No problem because along with </span><span style="font-family: Courier New, Courier, monospace;">byobu-ulevel</span><span style="font-family: Times New Roman;"> we have </span><span style="font-family: Courier New, Courier, monospace;">byobu-ugraph</span><span style="font-family: Times New Roman;"> which takes a series of values and uses </span><span style="font-family: Courier New, Courier, monospace;">byobu-ulevel</span><span style="font-family: Times New Roman;"> to display them. The canonical example being the load average display. Let's create a rolling load average showing 5 data points by creating </span><span style="font-family: Courier New, Courier, monospace;">~/.byobu/bin/5_uload </span><span style="font-family: Times New Roman;">and making it executable:</span></span></pre>
<pre lang="shell"><span style="white-space: normal;"><span style="font-family: Times New Roman;">
</span></span></pre>
<pre lang="shell"><span style="white-space: normal;"><span style="font-family: Times New Roman;">
</span></span></pre>
<pre lang="shell">file=/tmp/load.dat
awk '{print $1}' /proc/loadavg >> $file
printf "L[%s]\n" $(byobu-ugraph -p 5 -x 3 -f $file)
</pre>
<br />
Leave it to run for a few iterations (to generate the 5 data points we've specified we want and we get something like this:<br />
<br />
<pre lang="shell">L[▁▄▇▇▇]
</pre>
<pre lang="shell"><span style="white-space: normal;"><span style="font-family: Times New Roman;">
</span></span></pre>
<pre lang="shell"><span style="white-space: normal;"><span style="font-family: Times New Roman;">
</span></span></pre>
<pre lang="shell"><span style="white-space: normal;"><span style="font-family: Times New Roman;">
</span></span></pre>
<pre lang="shell"><span style="font-family: Times New Roman;"><span style="white-space: normal;">
</span></span></pre>
<pre lang="shell"></pre>
<pre lang="shell"></pre>
<pre lang="shell"><span style="font-family: Times New Roman;"><span style="white-space: normal;">
</span></span></pre>
<pre lang="shell"><span style="font-family: Times New Roman;"><span style="white-space: normal;">
</span></span></pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">Like </span><span style="font-family: Courier New, Courier, monospace; white-space: normal;">byobu-ulevel</span><span style="font-family: 'Times New Roman'; white-space: normal;">, </span><span style="font-family: Courier New, Courier, monospace; white-space: normal;">byobu-ugraph</span><span style="font-family: 'Times New Roman'; white-space: normal;"> assumes all data values range from </span><span style="font-family: Courier New, Courier, monospace; white-space: normal;">0</span><span style="font-family: 'Times New Roman'; white-space: normal;"> to </span><span style="font-family: Courier New, Courier, monospace; white-space: normal;">100</span><span style="font-family: 'Times New Roman'; white-space: normal;"> so assuming your system load is somewhat more manageable than that, you'll want to specify a "reasonable" maximum value as I have with </span><span style="font-family: 'Times New Roman'; white-space: normal;">"</span><span style="font-family: Courier New, Courier, monospace; white-space: normal;">-x 3</span><span style="font-family: 'Times New Roman'; white-space: normal;">" to allow byobu-ugraph to </span><span style="font-family: 'Times New Roman'; white-space: normal;">scale the values appropriately.</span></pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></pre>
<span style="font-family: 'Times New Roman'; white-space: normal;">Again, you can specify a theme to pass through to </span><span style="white-space: normal;"><span style="font-family: Courier New, Courier, monospace;">byobu-ulevel</span></span>.
Here's the same data using the <span style="font-family: Courier New, Courier, monospace;">solid_numbers_b_10</span> theme:
<br />
<br />
<pre lang="shell">L[❷❺❾❾❾]
</pre>
<br />
... And the same data again this time using the <span style="font-family: Courier New, Courier, monospace;">vdots_thick_4</span> theme:
<br />
<br />
<pre lang="shell">L[⣀⣤⣶⣶⣶]</pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;"> </span></pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">Can console bliss get any better than this? Of course it can - we haven't introduced colours yet!</span></pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">This part is still rather a work-in-progress as my current unreleased colour utility is rather basic. I'm hoping to expand it and potentially merge it into </span><span style="font-family: Courier New, Courier, monospace; white-space: normal;">byobu-ulevel</span><span style="font-family: 'Times New Roman'; white-space: normal;"> "Real Soon Now"</span><span style="font-size: 125%;">™</span><span style="font-family: 'Times New Roman'; white-space: normal;">. However, my latest versions of </span><span style="white-space: normal;"><span style="font-family: Courier New, Courier, monospace;">byobu-ulevel </span></span><span style="font-family: 'Times New Roman'; white-space: normal;">and </span><span style="white-space: normal;"><span style="font-family: Courier New, Courier, monospace;">byobu-ugraph</span></span><span style="font-family: 'Times New Roman'; white-space: normal;"> now accept a </span><span style="font-family: 'Times New Roman'; white-space: normal;">"</span><span style="white-space: normal;"><span style="font-family: Courier New, Courier, monospace;">-y</span></span><span style="font-family: 'Times New Roman'; white-space: normal;">" option to enable colour. This allow us to</span><span style="font-family: 'Times New Roman'; white-space: normal;"> <i>bling up</i> </span><span style="font-family: 'Times New Roman'; white-space: normal;">that original load example rather trivially:</span></pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></pre>
<pre lang="shell">file=/tmp/load.dat
awk '{print $1}' /proc/loadavg >> $file
printf "L[%s]\n" $(byobu-ugraph -p 5 -x 3 -f $file -y)
</pre>
<br />
All I've done is add that "<span style="font-family: Courier New, Courier, monospace;">-y</span>" to produce:
<br />
<br />
<pre lang="shell">L[<span style="color: lime;">▁</span><span style="color: yellow;">▄</span><span style="color: red;">▇▇▇</span>]
</pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></pre>
<pre lang="shell"></pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">The colour script currently </span><span style="font-family: 'Times New Roman'; white-space: normal;">takes current, minimum and maximum values but also allows thresholds to be specified along with colours for each threshold. By default, two thresholds are specified: "low" (</span><span style="white-space: normal;"><span style="font-family: Courier New, Courier, monospace;">0-15%</span></span><span style="font-family: 'Times New Roman'; white-space: normal;">) and "high" (</span><span style="white-space: normal;"><span style="font-family: Courier New, Courier, monospace;">85-100%</span></span><span style="font-family: 'Times New Roman'; white-space: normal;">). If the value is within the low threshold, it will be coloured red, if in the high threshold it gets coloured green, and yellow if in the middle. </span><span style="font-family: 'Times New Roman'; white-space: normal;">The threshold colours can also be inverted so that for example an indicator showing 95% disk usage will default to displaying in red when inverted (as that's a bad scenario), whilst an indicator showing 95% free memory will show in green when not inverted (as that's good :-)</span></pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></pre>
<h2>
<span style="font-family: 'Times New Roman'; white-space: normal;">Obligatory Screenshot</span></h2>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">Here's an example of the sorts of indicators you can create with just a few lines of shell script for your custom indicators:</span></pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></pre>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-oKSQ4PNHCXI/UWx44sBsvoI/AAAAAAAAAYA/GfZ2tjXeXVw/s1600/byobu-unicode-large.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="199" src="http://4.bp.blogspot.com/-oKSQ4PNHCXI/UWx44sBsvoI/AAAAAAAAAYA/GfZ2tjXeXVw/s320/byobu-unicode-large.png" width="320" /></a></div>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></pre>
<div class="separator" style="clear: both; text-align: center;">
</div>
<pre lang="shell"></pre>
<pre lang="shell"><span style="font-family: Times New Roman;"><span style="white-space: normal;">As you can see, it's pretty sparse - just the way I like it.</span></span></pre>
<pre lang="shell"><span style="font-family: Times New Roman;"><span style="white-space: normal;">
</span></span></pre>
<pre lang="shell"><span style="white-space: normal;"><span style="font-family: Times New Roman;">The indicators, from left to right (ignoring the "</span><span style="font-family: Courier New, Courier, monospace;">0:*</span><span style="font-family: Times New Roman;">" byobu window number) are:</span></span></pre>
<pre lang="shell"><span style="font-family: Times New Roman;"><span style="white-space: normal;">
</span></span></pre>
<pre lang="shell"><ul>
<li><span style="white-space: normal;"><span style="font-family: Times New Roman;">Time (23:01) in binary ("</span><span style="font-family: Courier New, Courier, monospace;">T[...]</span><span style="font-family: Times New Roman;">")</span></span></li>
<li><span style="white-space: normal;"><span style="font-family: Times New Roman;">Free disk ("</span><span style="font-family: Courier New, Courier, monospace;">D[...]</span><span style="font-family: Times New Roman;">")</span></span></li>
<li><span style="white-space: normal;"><span style="font-family: Times New Roman;">Some sort of rating (you decide :-) using hearts ("</span><span style="font-family: Courier New, Courier, monospace;">R[...]</span><span style="font-family: Times New Roman;">")</span></span></li>
<li><span style="font-family: Times New Roman; white-space: normal;">Free memory ("</span><span style="font-family: Courier New, Courier, monospace; white-space: normal;">M[...]</span><span style="font-family: Times New Roman; white-space: normal;">")</span></li>
<li><span style="white-space: normal;"><span style="font-family: Times New Roman;">Another rating this time using stars ("</span><span style="font-family: Courier New, Courier, monospace;">R[...]</span><span style="font-family: Times New Roman;">")</span></span></li>
<li><span style="white-space: normal;"><span style="font-family: Times New Roman;">Load average with 5 values ("</span><span style="font-family: Courier New, Courier, monospace;">L[...]</span><span style="font-family: Times New Roman;">")</span></span></li>
<li><span style="white-space: normal;"><span style="font-family: Times New Roman;">Wifi signal strengh ("</span><span style="font-family: Courier New, Courier, monospace;">W[...]</span><span style="font-family: Times New Roman;">")</span></span></li>
</ul>
</pre>
<h2>
Tips and Observations</h2>
<div>
<ul>
<li>Sometimes, you may find that an indicator doesn't appear. This appears to be an issue with either tmux or the terminal emulator you are using (<span style="font-family: Courier New, Courier, monospace;">gnome-terminal</span>, <span style="font-family: Courier New, Courier, monospace;">rxvt</span>, <span style="font-family: Courier New, Courier, monospace;">urxvt</span>, <span style="font-family: Courier New, Courier, monospace;">xterm</span>, etc). The solution seems to be to exit byobu, close the tab/window and restart byobu.</li>
<li>When writing custom indicators, ensure that the output ends with a newline character ('<span style="font-family: Courier New, Courier, monospace;">\n<span style="font-family: 'Times New Roman';">'</span>)</span> - without it, byobu won't display the indicator.</li>
<li>Performance can be an issue if you specify a large (10+ say) number of values to <span style="font-family: Courier New, Courier, monospace;">byobu-ugraph </span>or if you have more than a few <span style="font-family: Courier New, Courier, monospace;">byobu-ulevel</span> and/or <span style="font-family: Courier New, Courier, monospace;">byobu-ugraph</span> indicators on-screen at once.</li>
<li><span style="font-family: Courier New, Courier, monospace;">byobu-ulevel</span> and <span style="font-family: Courier New, Courier, monospace;">byobu-ugraph</span> can run in a terminal as long as you don't enable colour mode (which outputs not colour escape sequences but tmux colour commands).</li>
<li>For extra fun, you could have your indicator script call byobu-ulevel specifying a random theme. Here's an indicator which will use a random theme to display the value (80) each time it is called:</li>
<pre lang="shell"># change this to by the value you want to display
num=80
count=$(byobu-ulevel -l | wc -l)
selected=$(((RANDOM % count)+1))
theme=$(byobu-ulevel -l | sed -n ${selected}p)
printf "X[%s]\n" $(byobu-ulevel -c $num -t $theme)
</pre>
</ul>
<div>
<br /></div>
</div>
<div>
<h2>
</h2>
<h2>
<span style="font-family: 'Times New Roman'; font-size: large;">Planned Future Work</span></h2>
<pre lang="shell"><ul style="font-weight: normal;">
<li><span style="white-space: normal;"><span style="font-size: small;"><span style="font-family: Times New Roman;">Finish the colour script and potentially merge with </span><span style="font-family: Courier New, Courier, monospace;">byobu-ulevel</span><span style="font-family: Times New Roman;">. New features that could be added to the colour script:</span></span></span></li>
<ul>
<li><span style="font-family: Times New Roman; font-size: small;"><span style="white-space: normal;">ability to specify arbitrary thresholds with arbitrary colours and attributes (bold, etc).</span></span></li>
<li><span style="font-family: 'Times New Roman'; white-space: normal;">ability to invert the colour applied to a value from a colour theme.</span></li>
</ul>
<li><span style="font-family: 'Times New Roman'; white-space: normal;">Improve performance (probably by rewriting in C).</span></li>
<li><span style="font-family: 'Times New Roman'; white-space: normal;">Improve accessibility support.</span></li>
<li><span style="font-family: Times New Roman; white-space: normal;">Finish another utility I've half-written that shows sound levels (with individual left and right speaker channels all in a <i>single</i></span><span style="font-family: Times New Roman; white-space: normal;"> on-screen character!)</span></li>
</ul>
<h2>
<span style="font-family: Times New Roman;"><span style="white-space: normal;">Contribute</span></span></h2>
<div style="font-weight: normal;">
<span style="font-family: Times New Roman;"><span style="white-space: normal;">There are lots of other Unicode possibilities. Just browse through the <a href="http://en.wikipedia.org/wiki/List_of_Unicode_characters">wikipedia Unicode page</a> for inspiration.</span></span><br />
</div>
<div>
<span style="font-family: Times New Roman; font-weight: normal;"><span style="white-space: normal;">For example, I'd love it for example if someone would write a Unicode weather app that downloaded the current weather for my location and then displayed it using suitable glyphs like: </span></span><span style="font-family: Times New Roman;"><span style="white-space: normal;">⁂ ※ ⁑ ⁕ ⁖ ⁜ ⁛ ☀ ☁ ☂ ☔ </span></span><span style="font-family: 'Times New Roman'; white-space: normal;">☃.</span></div>
<div>
<span style="font-family: Times New Roman;"><span style="white-space: normal;">
</span></span></div>
<div>
<span style="font-family: Times New Roman;"><span style="white-space: normal;">
</span></span></div>
<span style="font-family: 'Times New Roman'; white-space: normal;">Find an excuse to get involved!</span></pre>
<h2>
Conclusion</h2>
<br /></div>
<pre lang="shell"><span style="font-family: Times New Roman;"><span style="white-space: normal;">With minimal effort, you can now add custom Unicode indicators to your byobu session saving you precious onscreen space whilst also offering a dash of visual spice.</span></span></pre>
<pre lang="shell"><span style="font-family: Times New Roman;"><span style="white-space: normal;">
</span></span></pre>
<pre lang="shell"><span style="font-family: Times New Roman;"><span style="white-space: normal;">
</span></span></pre>
<pre lang="shell"><span style="font-family: 'Times New Roman'; white-space: normal;">
</span></pre>
James Hunthttp://www.blogger.com/profile/03416491632578860019noreply@blogger.com175tag:blogger.com,1999:blog-7349425905862300662.post-41719261914096648422013-04-15T19:16:00.002+01:002013-04-15T19:16:53.663+01:00Stateful re-exec for Upstart Sessions? You bet!As of <a href="http://ifdeflinux.blogspot.co.uk/2012/11/upstart-16-released.html">Upstart 1.6</a>, restarting PID 1 is supported without losing state. So as soon as you get an update to either Upstart itself or one of its dependent libraries in Ubuntu, you'll be running the new version automatically, without the need for a reboot.<br />
<br />
But what about <a href="http://ifdeflinux.blogspot.co.uk/2013/04/upstart-user-sessions-in-ubuntu-raring.html">User Sessions</a>? Are they "second-class citizens"? Of course not! <i>When PID 1 restarts, all the running Session Inits will automatically restart too</i>. You won't <i>notice</i> this of course as your desktop session will continue to run and be fully functional but you too will be running the latest version of Upstart.<br />
<br />
How does this work? When an instance of Upstart has restarted, it signals this fact over D-Bus. From the user perspective, the <span style="font-family: Courier New, Courier, monospace;">upstart-event-bridge</span>, listening to the system instance of Upstart running as PID 1, will generate a "<span style="font-family: Courier New, Courier, monospace;">:sys:restarted</span>" event when it detects the restart. This in turn will cause the session job <span style="font-family: Courier New, Courier, monospace;">/usr/share/upstart/sessions/re-exec.conf </span>to run. And that job requests the Session Init restarts. Since all Session Inits will have the <span style="font-family: Courier New, Courier, monospace;">re-exec</span> job registered, <i>all</i> Session Init instances will restart as soon as PID 1 does.<br />
<br />
To convince yourself, let's create a <i>session</i> job that will start when your Session Init restarts. We'll do this by having the job start when the <span style="font-family: Courier New, Courier, monospace;">re-exec</span> job has finished:<br />
<br />
<pre lang="shell">$ cat >$HOME/.config/upstart/restart.conf<<EOT
start on stopped re-exec
exec xclock
EOT
</pre>
<br />
Now, let's force Upstart running as the <i>system</i> init daemon to restart:
<br />
<br />
<pre lang="shell">$ sudo telinit u
</pre>
<br />
You should now see <span style="font-family: Courier New, Courier, monospace;">xclock</span> running!<br />
<br />
Note that the job could have specified any of the following <span style="font-family: Courier New, Courier, monospace;">start on</span> conditions...<br />
<br />
<br />
<ul>
<li><span style="font-family: Courier New, Courier, monospace;">start on starting re-exec</span></li>
<li><span style="font-family: Courier New, Courier, monospace;">start on started re-exec</span></li>
<li><span style="font-family: Courier New, Courier, monospace;">start on stopping re-exec</span></li>
<li><span style="font-family: Courier New, Courier, monospace;">start on stopped re-exec</span></li>
<li><span style="font-family: Courier New, Courier, monospace;">start on :sys:restarted</span></li>
</ul>
<div>
... however, I chose "<span style="font-family: Courier New, Courier, monospace;">start on stopped re-exec</span>" because I want <span style="font-family: Courier New, Courier, monospace;">xclock</span> to run just <i>after</i> the <span style="font-family: Courier New, Courier, monospace;">re-exec</span> job has finished.</div>
<div>
<br /></div>
<br />
<br />James Hunthttp://www.blogger.com/profile/03416491632578860019noreply@blogger.com0tag:blogger.com,1999:blog-7349425905862300662.post-15722566540129473732013-04-12T12:35:00.000+01:002013-04-12T12:35:12.877+01:00Upstart User Sessions in Ubuntu Raring<h2>
Overview</h2>
Ubuntu Raring now includes <a href="http://ifdeflinux.blogspot.co.uk/2013/03/upstart-18-released.html">Upstart 1.8</a>. <a href="http://ifdeflinux.blogspot.co.uk/2013/03/upstart-17-released.html">Upstart 1.7 </a>and 1.8 combined mark a major milestone since they bring the proven power of Upstart to the user as never before: not only is Upstart now managing the system, but it is also capable of managing the default Ubuntu user's desktop sessions too.<br />
<br />
<h2>
Why?</h2>
<h3>
<u>Question:</u> Why reuse a system facility at the user level in this way?</h3>
The modern Linux desktop environment is a very dynamic one: users start and stop applications, switch workspaces, search the dash, adjust personal settings in the panel, connect to different networks, hot-plug USB devices and so on. All of these activities can be represented by "events".<br />
<br />
Stepping back a second, recall that Upstart was written <i>from the outset</i> to take advantage of <a href="http://upstart.ubuntu.com/cookbook/#upstart-s-design-why-it-is-revolutionary">the dynamic nature of a modern Linux system</a>. Long gone are the days when a system booted serially. A modern Linux system abounds with "events" from all sorts of different sources:<br />
<div>
<ul>
<li>The user plugs or unplugs a peripheral device.</li>
<li>A process changes state.</li>
<li>A file gets created.</li>
<li>A file is deleted.</li>
<li>A D-Bus signal is emitted.</li>
<li><i>et cetera.</i></li>
</ul>
</div>
In recognition of this, Upstarts core concept <b><i><u>is</u></i></b> that of <a href="http://upstart.ubuntu.com/cookbook/#event">an event</a>. At its heart, Upstart is an event-based process supervisor. So what better than to use a proven facility such as Upstart at the session level too? Upstarts design fits perfectly with the requirements for an init daemon but <i>also</i> fits the requirements for a session supervisor!<br />
<h3>
Question: Yes but isn't having Upstart manage users sessions overkill?</h3>
Not really - Upstart is not a big application: it was written to be small, fast, safe and reliable, all of which are highly desirable attributes for controlling sessions.<br />
<br />
Aside: Upstart has in fact been able to run as a non-privileged user for a couple of years now (since version 1.3) although the facility was added initially for testing and then later for the <span style="font-family: Courier New, Courier, monospace;">init-checkconf(8)</span> utility.<br />
<br />
This cycle, the Ubuntu developers have gone to great lengths to squash down the default image making the overall system smaller, leaner and faster. By allowing Upstart to manage user sessions, even greater gains can be achieved going forwards since although we've now added a small number of extra (albeit small) user processes, these allow us to convert some of the historically "long-running" user applications to "on-demand" ones. Like the corresponding Upstart system jobs, by default these won't run at all: Upstart will start them <i>if and when they need to run</i>.<br />
<br />
Initial targets are apps like <span style="font-family: Courier New, Courier, monospace;">update-notifier </span>and <span style="font-family: Courier New, Courier, monospace;">update-manager</span>. However, for the "S" cycle we'll be looking even more closely at what is running by default to see if we can start more of those processes on demand.<br />
<br />
Some of the changes recently introduced to Upstart also mean that additional <i>system</i> services can also benefit further from on-demand startup. For example, the plan is for the <a href="https://wiki.ubuntu.com/ErrorTracker">whoopsie</a> daemon to only start on-demand.<br />
<br />
This will be an ongoing project, but the foundations have been laid and the concept proved.<br />
<br />
<h2>
Enabling</h2>
The default Ubuntu Desktop <b>(*)</b> in Raring is now fully capable of being managed by Upstart. However, since all the changes landed relatively late in the cycle, the decision was taken not to enable this feature by default.<br />
<br />
<b>(*)</b> - we'd like to offer this facility to all the others desktop environments so please contact us if you wish to get involved in making the remaining Xsession scripts Upstart-aware.<br />
<br />
However, the good news is two-fold:<br />
<ul>
<li>Upstart User Sessions for the default desktop will be enabled by default for the "S" cycle.</li>
<li>Enabling Upstart User Sessions for the default Ubuntu Raring desktop is extremely easy!</li>
</ul>
If you wish to try out User Sessions (and I'd like to encourage as many folks as possible to) and you use the default Ubuntu desktop, you just need to change a single file, then logout and back in again:<br />
<br />
To enable, simply uncomment "<span style="font-family: Courier New, Courier, monospace;">ubuntu</span>" in file <span style="font-family: Courier New, Courier, monospace;">/etc/upstart-xsessions</span><span style="font-family: inherit;"> </span>like this:<br />
<br />
<pre code="shell">sudo sed -i 's/^#ubuntu/ubuntu/' /etc/upstart-xsessions
</pre>
<br />
To disable, simply comment-out "<span style="font-family: Courier New, Courier, monospace;">ubuntu</span>" again by running the following:<br />
<br />
<pre code="shell">sudo sed -i 's/^ubuntu/#ubuntu/' /etc/upstart-xsessions
</pre>
<h2>
Show me my Session</h2>
<div>
When you've enabled sessions and re-logged in, everything will <i>look</i> the same but you'll be running within an Upstart environment. You can see the session you're running in by looking at the value of the <span style="font-family: Courier New, Courier, monospace;">$UPSTART_SESSION</span> environment variable:<br />
<br /></div>
<pre code="shell">$ echo $UPSTART_SESSION
unix:abstract=/com/ubuntu/upstart-session/1000/3066
</pre>
<pre code="shell"></pre>
<br />
To list all running sessions for your user:
<br />
<br />
<pre code="shell"></pre>
<pre code="shell">$ initctl list-sessions
3066 unix:abstract=/com/ubuntu/upstart-session/1000/3066
</pre>
<br />
The format of the <span style="font-family: Courier New, Courier, monospace;">list-sessions</span> output in case you haven't guessed it is:<br />
<br />
<br />
<pre code="shell">$pid $UPSTART_SESSION
</pre>
<div>
<br /></div>
<br />
Proof that we are running Upstart:<br />
<br />
<pre code="shell"></pre>
<pre code="shell">$ ps -fp 3066
UID PID PPID C STIME TTY TIME CMD
james 3066 2884 0 07:58 ? 00:00:00 init --user
</pre>
<h2>
Have I got any jobs running?</h2>
Yes!<br />
<br />
<pre code="shell">$ initctl list
xsession-init stop/waiting
dbus start/running, process 3304
firefox stop/waiting
procenv stop/waiting
term start/running, process 3289
gnome-session start/running, process 3343
logrotate stop/waiting
thunderbird start/running, process 3339
mumble stop/waiting
gnome-settings-daemon start/running, process 3309
emacs start/running, process 4324
re-exec stop/waiting
upstart-event-bridge start/running, process 3305
</pre>
<br />
What's interesting is that some of those jobs are "built-ins" from <span style="font-family: Courier New, Courier, monospace;">/usr/share/upstart/sessions/</span> whilst some are jobs I've created myself (<span style="font-family: Courier New, Courier, monospace;">thunderbird</span>, <span style="font-family: Courier New, Courier, monospace;">emacs</span>, <span style="font-family: Courier New, Courier, monospace;">term</span>, <span style="font-family: Courier New, Courier, monospace;">procenv</span> and <span style="font-family: Courier New, Courier, monospace;">mumble</span>).
<br />
<br />
Note that if you're already familiar with Upstart, there is a subtle behavioural change here - since you are running within an Upstart session, "<span style="font-family: Courier New, Courier, monospace;">initctl list</span>" shows jobs <i>in that session</i>. To see <i>system jobs</i>, you can run either of the following:<br />
<br />
<pre code="shell">$ sudo initctl list
$ initctl --system list
</pre>
<br />
<h2>
Writing your first user job</h2>
Let's create a job that pops up <span style="font-family: Courier New, Courier, monospace;">xclock</span> when it's started. Upstart 1.7 can read configuration files from multiple locations, but the main location for user jobs should be <span style="font-family: Courier New, Courier, monospace;">$XDG_CONFIG_HOME/upstart/</span>. For most folk, this equates to <span style="font-family: Courier New, Courier, monospace;">$HOME/.config/upstart/</span>:<br />
<br />
<pre code="shell">$ conf_dir=${XDG_CONFIG_HOME:-$HOME/.config}/upstart
$ mkdir -p $conf_dir
</pre>
<br />
(Note that if <span style="font-family: Courier New, Courier, monospace;">$conf_dir</span> did not exist initially, you will need to logout and back in for jobs in that directory to take effect - a one-off task)<br />
<br />
Now, create file <span style="font-family: Courier New, Courier, monospace;">$conf_dir/xclock.conf </span>containing:<br />
<br />
<pre code="text">start on desktop-start
stop on desktop-end</pre>
<pre code="text">exec xclock -update 1
</pre>
<br />
That's it. But to be sure we have specified the syntax correctly, let's check that Upstart is happy with it first:<br />
<br />
<pre code="shell">$ init-checkconf $conf_dir/xclock.conf
File /home/james/.config/upstart/xclock.conf: syntax ok
</pre>
<pre code="shell"></pre>
<br />
I'd recommend always using <span style="font-family: Courier New, Courier, monospace;">init-checkconf(8)</span> to sanity-check your jobs. So, Upstart is happy with this job. Let's start it:<br />
<br />
<pre code="shell">$ start xclock
</pre>
<br />
And you should now see the wonderfully retro <span style="font-family: Courier New, Courier, monospace;">xclock</span> running.
To stop the job simply run:
<br />
<br />
<pre code="shell">$ stop xclock
</pre>
<br />
Note that since we specified a "<a href="http://upstart.ubuntu.com/cookbook/#start-on">start on</a>" and "<a href="http://upstart.ubuntu.com/cookbook/#stop-on">stop on</a>" stanza in the <span style="font-family: Courier New, Courier, monospace;">xclock.conf</span> job configuration file, this job will now run every time you login and stop just before you logout. If you don't want that to happen, you have a few choices:<br />
<ul>
<li>Delete <span style="font-family: Courier New, Courier, monospace;">xclock.conf</span> - the job is then gone.</li>
<li>Add "<span style="font-family: Courier New, Courier, monospace;">manual</span>" anywhere <b><i><u>after</u></i></b> the <span style="font-family: Courier New, Courier, monospace;">start on</span> line in <span style="font-family: Courier New, Courier, monospace;">xclock.conf</span> such that the job will not be auto-started.</li>
<li>Create a new file called <span style="font-family: Courier New, Courier, monospace;">xclock.override</span> in the same directory that simply contains "<span style="font-family: Courier New, Courier, monospace;">manual</span>". Again, this will stop the file from being auto-started but allows you to leave the existing <span style="font-family: Courier New, Courier, monospace;">.conf</span> file as-is.</li>
<li>Delete (or simply comment out) the <span style="font-family: Courier New, Courier, monospace;">start on</span> condition.</li>
</ul>
<div>
<br /></div>
<div>
Let's create another job to start firefox which we shall call somewhat unimaginatively <span style="font-family: Courier New, Courier, monospace;">firefox.conf</span>:<br />
<br /></div>
<pre code="text">start on desktop-start
stop on desktop-end</pre>
<pre code="text">exec firefox
</pre>
<br />
Now, that is more useful - firefox will now start when we login and stop just before we logout!<br />
<br />
But what if we want xclock to start <i>after</i> firefox starts and stop just before firefox stops? Simple, just change the <span style="font-family: Courier New, Courier, monospace;">start on</span> and <span style="font-family: Courier New, Courier, monospace;">stop on</span> stanzas in <span style="font-family: Courier New, Courier, monospace;">xclock.conf</span>:<br />
<br />
<pre code="text">start on started firefox
stop on stopping firefox</pre>
<pre code="text">exec xclock -update 1
</pre>
<br />
<div>
The job events (events that are emitted by jobs in your "<span style="font-family: Courier New, Courier, monospace;">initctl list</span>" output) are called <span style="font-family: Courier New, Courier, monospace;">starting</span>, <span style="font-family: Courier New, Courier, monospace;">started</span>, <span style="font-family: Courier New, Courier, monospace;">stopping</span> and <span style="font-family: Courier New, Courier, monospace;">stopped</span> but there are <a href="http://upstart.ubuntu.com/cookbook/#ubuntu-well-known-events-ubuntu-specific">many more interesting ones</a>.</div>
<br />
<h2>
Application Output </h2>
Error output from applications started from within a session (that is applications started from the Unity dash or launcher) still gets redirected to the usual <span style="font-family: "Courier New",Courier,monospace;">$HOME/.xsession-errors</span> file. However, <i>all</i> output from Upstart session <i>jobs</i> is redirected automatically to <span style="font-family: "Courier New",Courier,monospace;">$XDG_CACHE_HOME/upstart/$job.log </span>,which equates to <span style="font-family: "Courier New",Courier,monospace;">$HOME/.cache/upstart/$job.log</span> by default.<br />
<br />
Since <span style="font-family: Courier New, Courier, monospace;">gnome-session</span> is now running as an Upstart job, it too gets it own log file. By each job having its own log file, you can now identify where messages are coming from (not always obvious when perusing <span style="font-family: 'Courier New', Courier, monospace;">$HOME/.xsession-errors</span>).<br />
<br />
We have included a <span style="font-family: Courier New, Courier, monospace;">logrotate</span> job that automatically keeps the Upstart logs conveniently compressed and rotated. In fact, if you discover whilst testing a job that it is starting to use too much space, or if you simply want to shrink the amount of space your logs are taking, you can run this job whenever you wish like this:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace;">$ start logrotate</span><br />
<br />
The <span style="font-family: Courier New, Courier, monospace;">logrotate</span> job (<span style="font-family: "Courier New",Courier,monospace;">/usr/share/upstart/sessions/logrotate.conf</span>) is rather interesting in that it changes its behavior based on whether it was started by the Session Init, or was invoked manually by a user. Study of this job could be instructive for those wishing to explore some of the possibilities provided by having Upstart manage your session.<br />
<br />
<h2>
upstart-monitor</h2>
To use the <a href="http://ifdeflinux.blogspot.co.uk/2013/03/a-basic-upstart-events-gui-and-cli-tool.html">GUI</a>, you'll first need to install it (as it is not part of the core upstart package):<br />
<br />
<pre code="shell">$ sudo apt-get install upstart-monitor
$ upstart-monitor
</pre>
<br />
If you have already enabled User Sessions, it will connect to your session. Otherwise, the default is to connect to the system Upstart (running as PID 1).<br />
<br />
<h2>
Auto-start an application on plugging a USB device</h2>
This is my favourite example of the power of user jobs. Let's create a job that auto-starts a VoIP client when you plug in your USB headset.<br />
<br />
The skeleton of the job is simple. So, I'll create <span style="font-family: Courier New, Courier, monospace;">$HOME/.config/upstart/mumble.conf </span>containing:<br />
<br />
<pre code="text"># start VoIP client of choice
exec mumble
</pre>
<br />
It's a start, but it's not terribly impressive is it? However, we can now "<span style="font-family: Courier New, Courier, monospace;">start mumble</span>" and "<span style="font-family: Courier New, Courier, monospace;">stop mumble</span>" and Upstart will DTRT.<br />
<br />
But what about the '<a href="http://upstart.ubuntu.com/cookbook/#start-on">start on</a>' condition? We need a way to tell Upstart when to start <span style="font-family: Courier New, Courier, monospace;">mumble</span>. The simplest way to ascertain this for your particular USB headset is to fire up the <span style="font-family: Courier New, Courier, monospace;">upstart-monitor</span>, plug in your headset and see what happens:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-E2b0HkjG4wI/UWfmMi1aDAI/AAAAAAAAAXQ/rjjhPLEWeEg/s1600/upstart-monitor-usb-headset.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="153" src="http://3.bp.blogspot.com/-E2b0HkjG4wI/UWfmMi1aDAI/AAAAAAAAAXQ/rjjhPLEWeEg/s320/upstart-monitor-usb-headset.png" width="320" /></a></div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
As you can see, I get quite a few (15!) events for my USB headset, partly because it is providing multiple sound devices (headphones and a microphone). But which event do I need? Luckily, the answer is simple for sounds devices: the <span style="font-family: Courier New, Courier, monospace;">sound-device-changed</span> event. So, let's copy it...<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-8ydkSF5XmYk/UWfmSUU_-ZI/AAAAAAAAAXY/71C0iwCPRe8/s1600/upstart-monitor-usb-headset-copy.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="152" src="http://1.bp.blogspot.com/-8ydkSF5XmYk/UWfmSUU_-ZI/AAAAAAAAAXY/71C0iwCPRe8/s320/upstart-monitor-usb-headset-copy.png" width="320" /></a></div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<br />
... and paste it here:<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">:sys:sound-device-changed KERNEL='card2' DEVPATH='/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2/2-1.2:1.0/sound/card2' SUBSYSTEM='sound' ACTION='change' ID_BUS='usb' ID_FOR_SEAT='sound-pci-0000_00_1d_0-usb-0_1_2_1_0' ID_ID='usb-Logitech_Logitech_USB_Headset-00-Headset' ID_MODEL='Logitech_USB_Headset' ID_MODEL_ENC='Logitech\x20USB\x20Headset' ID_MODEL_FROM_DATABASE='ClearChat Pro USB' ID_MODEL_ID='0a0b' ID_PATH='pci-0000:00:1d.0-usb-0:1.2:1.0' ID_PATH_TAG='pci-0000_00_1d_0-usb-0_1_2_1_0' ID_REVISION='1013' ID_SERIAL='Logitech_Logitech_USB_Headset' ID_TYPE='audio' ID_USB_DRIVER='snd-usb-audio' ID_USB_INTERFACES=':010100:010200:030000:' ID_USB_INTERFACE_NUM='00' ID_VENDOR='Logitech' ID_VENDOR_ENC='Logitech' ID_VENDOR_FROM_DATABASE='Logitech, Inc.' ID_VENDOR_ID='046d' SEQNUM='2803' SOUND_FORM_FACTOR='headset' SOUND_INITIALIZED='1' TAGS=':seat:' UDEV_LOG='3' USEC_INITIALIZED='795038608'</span><br />
<br />
<pre code="text"></pre>
As you can see, that's a lot of information! The most important part of this event and the reason we chose the <span style="font-family: Courier New, Courier, monospace;">sound-device-changed</span> event in the first place is highlighted below:<br />
<br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">:sys:sound-device-changed KERNEL='card2' DEVPATH='/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2/2-1.2:1.0/sound/card2' SUBSYSTEM='sound' ACTION='change' ID_BUS='usb' ID_FOR_SEAT='sound-pci-0000_00_1d_0-usb-0_1_2_1_0' ID_ID='usb-Logitech_Logitech_USB_Headset-00-Headset' ID_MODEL='Logitech_USB_Headset' ID_MODEL_ENC='Logitech\x20USB\x20Headset' ID_MODEL_FROM_DATABASE='ClearChat Pro USB' ID_MODEL_ID='0a0b' ID_PATH='pci-0000:00:1d.0-usb-0:1.2:1.0' ID_PATH_TAG='pci-0000_00_1d_0-usb-0_1_2_1_0' ID_REVISION='1013' ID_SERIAL='Logitech_Logitech_USB_Headset' ID_TYPE='audio' ID_USB_DRIVER='snd-usb-audio' ID_USB_INTERFACES=':010100:010200:030000:' ID_USB_INTERFACE_NUM='00' ID_VENDOR='Logitech' ID_VENDOR_ENC='Logitech' ID_VENDOR_FROM_DATABASE='Logitech, Inc.' ID_VENDOR_ID='046d' SEQNUM='2803' SOUND_FORM_FACTOR='headset' <b>SOUND_INITIALIZED='1'</b> TAGS=':seat:' UDEV_LOG='3' USEC_INITIALIZED='795038608'</span><br />
<br />
The event containing <b style="font-family: 'Courier New', Courier, monospace; font-size: small;">SOUND_INITIALIZED='1' </b>denotes that the "overall" headset device comprising multiple sound devices is "ready to use". So we have the correct type of event and we know that that event is only emitted when the device really is ready but we still need to identify the device uniquely. That might sound like an arduous task, but it really isn't so panic not! Since this is my workstation and since I know I've only got a single USB headset, all I have to do is select some unique value (or combination of values) from the variables in the above data.<br />
<br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">:sys:sound-device-changed KERNEL='card2' DEVPATH='/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.2/2-1.2:1.0/sound/card2' SUBSYSTEM='sound' ACTION='change' ID_BUS='usb' ID_FOR_SEAT='sound-pci-0000_00_1d_0-usb-0_1_2_1_0' ID_ID='usb-Logitech_Logitech_USB_Headset-00-Headset' <b>ID_MODEL='Logitech_USB_Headset'</b> ID_MODEL_ENC='Logitech\x20USB\x20Headset' ID_MODEL_FROM_DATABASE='ClearChat Pro USB' ID_MODEL_ID='0a0b' ID_PATH='pci-0000:00:1d.0-usb-0:1.2:1.0' ID_PATH_TAG='pci-0000_00_1d_0-usb-0_1_2_1_0' ID_REVISION='1013' ID_SERIAL='Logitech_Logitech_USB_Headset' ID_TYPE='audio' ID_USB_DRIVER='snd-usb-audio' ID_USB_INTERFACES=':010100:010200:030000:' ID_USB_INTERFACE_NUM='00' ID_VENDOR='Logitech' ID_VENDOR_ENC='Logitech' ID_VENDOR_FROM_DATABASE='Logitech, Inc.' ID_VENDOR_ID='046d' SEQNUM='2803' SOUND_FORM_FACTOR='headset' </span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">SOUND_INITIALIZED='1'</span><span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"> TAGS=':seat:' UDEV_LOG='3' USEC_INITIALIZED='795038608'</span><br />
<pre code="text"></pre>
<br />
There are some variables you do <b><i>not</i></b> want to select on:<br />
<br />
<ul>
<li><span style="font-family: Courier New, Courier, monospace;">KERNEL</span> and <span style="font-family: Courier New, Courier, monospace;">DEVPATH</span>: these refer to the physical location of the device so don't select these as that location will differ depending on which USB port you plug your device into.</li>
<li><span style="font-family: Courier New, Courier, monospace;">DEVNAME</span> and <span style="font-family: Courier New, Courier, monospace;">DEVNUM</span> are incremented every time a new device is plugged in (even if it's the same device you plugged in before).</li>
<li><span style="font-family: Courier New, Courier, monospace;">UDEV_LOG</span>, <span style="font-family: Courier New, Courier, monospace;">USEC_INITIALIZED</span>, <span style="font-family: Courier New, Courier, monospace;">SEQNUM</span>.</li>
</ul>
<br />
What you are looking for ideally is one more more variables that are:<br />
<br />
<ul>
<li>Describe your device clearly.</li>
<li>Will always have the same value whenever and wherever you plug the device.</li>
</ul>
<br />
Your mileage may vary (for example if you have multiple different USB headsets). However, for me, <span style="font-family: Courier New, Courier, monospace;">ID_MODEL</span> is sufficient and conveniently human-readable ;-)<br />
<br />
Back to our <span style="font-family: 'Courier New', Courier, monospace;">$HOME/.config/upstart/mumble.conf</span> session job. I can now specify the <span style="font-family: Courier New, Courier, monospace;">start on</span> condition:<br />
<br />
<pre code="text">start on :sys:sound-device-changed ID_MODEL="Logitech_USB_Headset" SOUND_INITIALIZED="1"
stop on desktop-end
# start VoIP client of choice
exec mumble
</pre>
<div>
</div>
<br />
I've added a <a href="http://upstart.ubuntu.com/cookbook/#stop-on">stop on</a> condition too such that Upstart will stop <span style="font-family: Courier New, Courier, monospace;">mumble</span> just before the desktop session ends (in case I forget to stop it myself). And that's it. To try it out, simply plug in your USB headset!<br />
<br />
An approximation of what is happening when the USB device is plugged in is:<br />
<ul>
<li>The kernel detects the device and generates a uevent.</li>
<li>The <span style="font-family: Courier New, Courier, monospace;">upstart-udev-bridge</span>, running at the <i><u>system level</u></i>, is monitoring the kernels netlink socket via libudev and therefore sees the event and emits a corresponding Upstart <u>system event</u>.</li>
<li>The users <span style="font-family: Courier New, Courier, monospace;">upstart-event-bridge</span> sees the system event and proxies it down to the users Upstart session by emitting an Upstart <u>user event</u> tagged with a "<span style="font-family: Courier New, Courier, monospace;">:sys:</span>" prefix.</li>
<li>The users Session Init compares the <span style="font-family: Courier New, Courier, monospace;">start on</span> condition in <span style="font-family: Courier New, Courier, monospace;">mumble.conf</span> with the Upstart event emitted by the <span style="font-family: Courier New, Courier, monospace;">upstart-event-bridge</span>, finds that it matches and so starts the <span style="font-family: Courier New, Courier, monospace;">mumble</span> job.</li>
</ul>
<br />
This example should give you a taste of the power you now have at your fingertips to harness system-level events in your own jobs :-)<br />
<br />
A summary of well-known user-level and system-level upstart jobs is available in the <span style="font-family: Courier New, Courier, monospace;">upstart-events(7) </span>manual page.<br />
<br />
<h4>
Restrictions</h4>
The above will not work if the device is <i>already plugged</i> at boot time. The problem is that although the <span style="font-family: Courier New, Courier, monospace;">upstart-udev-bridge</span> will emit the correct set of events for your device at boot time, your session won't have started at that point in time so there will be no <span style="font-family: Courier New, Courier, monospace;">upstart-event-bridge</span> to listen for them. So by the time your session starts, those events will have gone.<br />
<h2>
The File Bridge</h2>
Wouldn't it be great if you could create a job that would only start when a particular file got created? Well, now you can....<br />
<br />
Highly contrived, but let's start <span style="font-family: Courier New, Courier, monospace;">xclock</span> when the X session errors file gets modified:<br />
<br />
<pre code="text">start on file FILE=~/.xsession-errors EVENT=modify
exec xclock
</pre>
<br />
How about watching a directory for any activity?<br />
<br />
<pre code="text">start on file FILE=~/var/mydir/
exec xclock
</pre>
<br />
Now, xclock will get started when files get created, modified or deleted in the directory <span style="font-family: Courier New, Courier, monospace;">$HOME/var/mydir/.</span><br />
<br />
Or maybe a file glob might be useful to you?
<br />
<br />
<pre code="text">start on file FILE=~/.cache/myapp/*.crash
exec handle-my-buggy-app
</pre>
<br />
Here, the "<span style="font-family: Courier New, Courier, monospace;">handle-my-buggy-app</span>" application will be started to do something useful when your app creates "<span style="font-family: Courier New, Courier, monospace;">.crash</span>" files.<br />
<h2>
Understanding how jobs interact</h2>
<div>
Upstart provides a tool called <span style="font-family: Courier New, Courier, monospace;">initctl2dot</span> which generates <a href="http://www.graphviz.org/">graphviz</a> visualisations of jobs. Let's run it:<br />
<br /></div>
<pre code="shell">$ initctl2dot -o - | dot -Tpng -o upstart.png
</pre>
<br />
Here's the result on my system:
<br />
<div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-3w1n144D9Hw/UVKuZKx1taI/AAAAAAAAAXA/ft4Rlldpb-k/s1600/initctl2dot-for-user-session.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="260" src="http://2.bp.blogspot.com/-3w1n144D9Hw/UVKuZKx1taI/AAAAAAAAAXA/ft4Rlldpb-k/s320/initctl2dot-for-user-session.png" width="320" /></a></div>
<br /></div>
<div>
<br />
<ul>
<li>The rectangular nodes represent jobs.</li>
<li>The diamonds denote events.</li>
<li>Blue lines represent <span style="font-family: Courier New, Courier, monospace;">start on</span> conditions.</li>
<li>Red lines represent <span style="font-family: Courier New, Courier, monospace;">stop on</span> conditions.</li>
<li>Green lines show the events that jobs may emit.</li>
</ul>
<div>
There's quite a lot happening in that diagram partly as we've added some useful "well-known" events that you can use for your jobs. In particular, note the <span style="font-family: Courier New, Courier, monospace;">desktop-start</span> and <span style="font-family: Courier New, Courier, monospace;">desktop-end</span> events. These are not the first events emitted, but if your job specifies the following, it is guaranteed to run in a fully-functional desktop environment:<br />
<br /></div>
<div>
<pre code="text">start on desktop-start
stop on desktop-end
</pre>
<br /></div>
<div>
If you want your job to start as early as possible and end as late as possible, but don't care about whether the desktop is usable when the job starts, you can specify:<br />
<br />
<pre code="text">start on startup
stop on session-end
</pre>
<br />
Note: The <span style="font-family: Courier New, Courier, monospace;">desktop-start</span> and <span style="font-family: Courier New, Courier, monospace;">desktop-end</span> events are emitted by the job that actually starts the users graphical desktop. Currently, only the default Ubuntu desktop is emitting these events, but all desktop environments that support for Session Inits need to emit these events.<br />
<h2>
The Future</h2>
There are already a lot of <a href="http://upstart.ubuntu.com/cookbook/#ubuntu-well-known-events-ubuntu-specific">"well-known" events</a> your jobs can make use of, but we plan to introduce even more to enrich the available palette next cycle. Ideas for event sources include:<br />
<br />
<br />
<ul>
<li>Suspend/Resume events</li>
<li>Improved sound device events (for example, start a job when headphones plugged)</li>
<li>network events (for example, start/stop jobs when you connect to an open wifi network)</li>
<li>D-Bus signals (<span style="font-family: Courier New, Courier, monospace;">upstart-dbus-bridge</span>)</li>
<li>dconf/gsettings changes (<span style="font-family: Courier New, Courier, monospace;">upstart-dconf-bridge</span>)</li>
<li>cron-like temporal events (<span style="font-family: Courier New, Courier, monospace;">upstart-time-bridge</span>)</li>
</ul>
</div>
</div>
<h2>
Further Information</h2>
<ul>
<li><a href="http://upstart.ubuntu.com/cookbook/">The Upstart Cookbook</a></li>
<li>#upstart on freenode</li>
<li>The <a href="https://lists.ubuntu.com/mailman/listinfo/upstart-devel">upstart-devel</a> mailing list.</li>
<li>The all-important man pages:</li>
<ul>
<li><span style="font-family: Courier New, Courier, monospace;">init(5)</span></li>
<li><span style="font-family: Courier New, Courier, monospace;">init(8)</span></li>
<li><span style="font-family: Courier New, Courier, monospace;">upstart-monitor(8)</span></li>
<li><span style="font-family: Courier New, Courier, monospace;">upstart-events(7) </span></li>
</ul>
</ul>
<br />
<br />James Hunthttp://www.blogger.com/profile/03416491632578860019noreply@blogger.com25tag:blogger.com,1999:blog-7349425905862300662.post-75461630993812263602013-04-08T12:13:00.002+01:002013-04-08T12:13:35.117+01:00Coverity static analysis for C, C++ and Java code<br />
It's a well known principle of software engineering that the earlier bugs can be caught, the lower the overall cost. As such, testing needs to happen at <i>every</i> level. Once your project is at the coding stage, the earliest form of testing is <i><u><b>on the code itself</b></u></i>, not on the binaries the compiler produces.<br />
<br />
We run a variety of tools over critical codebases such as Upstart and <a href="https://launchpad.net/whoopsie">Whoopsie</a> regularly to identify issues well before they "escape into the wild". These tools include <a href="http://scan.coverity.com/">Coverity Scan</a> (see the <a href="http://scan.coverity.com/all-projects.html">list of projects already using it</a>).<br />
<br />
If you really care about your code and you are involved with a C, C++ or Java project, I'd strongly encourage you to take a look at this awesome tool. If you <i>aren't</i> directly involved in such projects, try contacting those running them and suggesting they use Coverity.<br />
<br />
The Coverity Scan service is entirely free for OSS projects. You will need to register to obtain an account and then download the client analysis tool. Once setup, a particularly attractive feature is the ability to auto-upload the analysis data generated for your project using <a href="http://www.catb.org/~esr/coverity-submit/">ESR's coverity-submit tool</a>. This could for example be hooked into your upload or release process to ensure no code quality regressions. After you have uploaded the analysis data, you can browse through the results of the scan using the web interface in a variety of ways, including a view that shows the errors "inline" with markers added around the code Coverity has identified as problematic.<br />
<br />
For those who have either never used static analysis tools, or have simply never used Coverity, don't fall into the trap of thinking that <span style="font-family: Courier New, Courier, monospace;">gcc -pedantic -Wall</span> or even LLVM's <span style="font-family: Courier New, Courier, monospace;">scan-build</span> should be "good enough for anyone" - it simply is not. Consider too Steckel's Rule to Success,<br />
<br />
<div style="text-align: center;">
<i>"Good enough is never good enough"</i></div>
<br />
Coverity performs <i>very</i> deep analysis and its results may well surprise you... but rather that than unexpected surprises for your users.<br />
<br />
Apologies if this post sounds like a bit of a sales pitch. It really isn't though: the Coverity service is free and what they are offering really is too good to ignore.<br />
<div>
<br /></div>
<div>
Note: I have no affiliation with Coverity - I'm just extremely impressed with their Scan tool! :-)</div>
James Hunthttp://www.blogger.com/profile/03416491632578860019noreply@blogger.com13tag:blogger.com,1999:blog-7349425905862300662.post-10126146620932617202013-03-22T15:16:00.003+00:002013-03-22T15:16:33.148+00:00Upstart 1.8 releasedHot on the heels of <a href="http://ifdeflinux.blogspot.co.uk/2013/03/upstart-17-released.html">Upstart 1.7</a> comes Upstart 1.8 which includes two interesting new features:<br />
<br />
<h2>
The File Bridge</h2>
Upstart now provides the <span style="font-family: Courier New, Courier, monospace;">upstart-file-bridge</span>, a bridge that allows jobs to react to file events.<br />
<br />
Here are a few examples:<br />
<br />
Start a job when file is created, modified or deleted:
<br />
<pre code="text">start on file FILE=/run/app.pid
</pre>
<pre code="text">
</pre>
Start job when file is created (only):
<br />
<pre code="text">start on file FILE=/run/app.pid EVENT=created
</pre>
<pre code="text">
</pre>
Start job when any files within a directory are created, modified or deleted:
<br />
<pre code="text">start on file FILE=/var/log/
</pre>
<pre code="text">
</pre>
Start job when files that match a glob pattern are created in the indicated directory:
<br />
<pre code="text">start on file FILE=/var/crash/*.crash EVENT=created
</pre>
<br />
Even better, this bridge is available to both system jobs and users session jobs.<br />
<br />
See <span style="font-family: Courier New, Courier, monospace;">upstart-file-bridge(8)</span> and <span style="font-family: Courier New, Courier, monospace;">file-event(7)</span> for further details.<br />
<br />
<h2>
The GUI</h2>
<br />
The <span style="font-family: Courier New, Courier, monospace;">upstart-monitor</span> tool covered <a href="http://ifdeflinux.blogspot.co.uk/2013/03/a-basic-upstart-events-gui-and-cli-tool.html">in a previous post</a> has also been added to the release. This allows you to see what events Upstart is emitting and how jobs are changing state both at the system and user session levels.<br />
<br />
You can download Upstart 1.8 from:<br />
<br />
<ul>
<li><a href="https://launchpad.net/upstart">https://launchpad.net/upstart</a></li>
</ul>
<br />
Upstart 1.8 should be landing in Ubuntu Raring in the next few days. Thanks to all involved!<br />
<h2>
Contributions</h2>
<div>
If you are interested in contributing to Upstart, we'd love to hear from you. Now is a <i>great</i> time to get involved, since with the advent of the <span style="font-family: Courier New, Courier, monospace;">upstart-monitor</span> the fun expands to include GUI hackers too!! ;-)</div>
<br />
<br />
<div>
<br /></div>
James Hunthttp://www.blogger.com/profile/03416491632578860019noreply@blogger.com2tag:blogger.com,1999:blog-7349425905862300662.post-67178950787741198482013-03-13T10:09:00.000+00:002013-03-13T10:09:29.946+00:00Upstart Cookbook updated for Upstart 1.7The <a href="http://upstart.ubuntu.com/cookbook/">Upstart Cookbook</a> has now been updated to take into account all the goodness that landed in Upstart 1.7. There are quite a few changes so if you'd like to review the diff, <a href="http://bazaar.launchpad.net/~upstart-documenters/upstart-cookbook/trunk/revision/176#upstart_cookbook.rst">it's here</a>.<br />
<br />James Hunthttp://www.blogger.com/profile/03416491632578860019noreply@blogger.com0tag:blogger.com,1999:blog-7349425905862300662.post-6410469718888895132013-03-04T12:56:00.002+00:002013-03-04T12:56:41.414+00:00A basic Upstart Events GUI (and cli!:-) toolIt didn't quite make it into the release, but we now have a very basic Upstart GUI that should appear in Ubuntu along with the Upstart 1.7 release soon.<br />
<br />
<h2>
What is it?</h2>
<span style="font-family: Courier New, Courier, monospace;">upstart-monitor</span> is a simple application that shows Upstart events as they are emitted. It can be used to view both system-level events and also session-level events when Upstart is running as a Session Init.<br />
<br />
It requires Upstart 1.7.<br />
<br />
<h2>
What can I use it for?</h2>
The tool allows you to see events as they occur which is an aid to understanding. It also helps with writing <a href="http://upstart.ubuntu.com/cookbook/#job-configuration-file">new jobs</a> since, if you are not sure which event to use, provoke the scenario you want, then just copy the appropriate event that appears in upstart-monitor to your <span style="font-family: Courier New, Courier, monospace;">.conf </span>file (more details on this in a future post...).<br />
<h2>
How do I use it?</h2>
Just start it to see events being emitted:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-t3NZ7vxPK5Y/UTSXGgTWFbI/AAAAAAAAAWY/xhqYdgbPjkE/s1600/upstart-monitor-gui.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="181" src="http://2.bp.blogspot.com/-t3NZ7vxPK5Y/UTSXGgTWFbI/AAAAAAAAAWY/xhqYdgbPjkE/s320/upstart-monitor-gui.png" width="320" /></a></div>
<br />
<br />
<br />
If you'd prefer a command-line version, run it with the '<span style="font-family: Courier New, Courier, monospace;">-n</span>' option:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-452-qPHgX6g/UTSXVxjpwKI/AAAAAAAAAWg/3qcFKlpHpIY/s1600/upstart-monitor-cli.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="98" src="http://1.bp.blogspot.com/-452-qPHgX6g/UTSXVxjpwKI/AAAAAAAAAWg/3qcFKlpHpIY/s320/upstart-monitor-cli.png" width="320" /></a></div>
<br />
<br />
<h2>
Where can I get it?</h2>
If you can't wait for it to land in Ubuntu, you can grab it from here:<br />
<br />
<br />
<ul>
<li><a href="http://people.canonical.com/~jhunt/upstart/gui/upstart-monitor.py">http://people.canonical.com/~jhunt/upstart/gui/upstart-monitor.py</a></li>
</ul>
<div>
<br /></div>
James Hunthttp://www.blogger.com/profile/03416491632578860019noreply@blogger.com5tag:blogger.com,1999:blog-7349425905862300662.post-13907603994511721112013-03-04T12:34:00.001+00:002013-03-04T12:34:10.491+00:00Upstart 1.7 releasedWe've just released Upstart 1.7 which expands Upstarts abilities a lot. I'll follow up with a further post explaining what these changes mean for users as this release is soon to appear in Ubuntu.<br />
<br />
<h2>
Summary of changes</h2>
<br />
<br />
<ul>
<li>New initctl commands: <span style="font-family: Courier New, Courier, monospace;">set-env</span>, <span style="font-family: Courier New, Courier, monospace;">unset-env</span>, <span style="font-family: Courier New, Courier, monospace;">get-env</span>, <span style="font-family: Courier New, Courier, monospace;">list-env</span>, <span style="font-family: Courier New, Courier, monospace;">reset-env</span>, <span style="font-family: Courier New, Courier, monospace;">list-sessions</span> (all except last with corresponding D-Bus methods).</li>
<li>New D-Bus-only signals <span style="font-family: Courier New, Courier, monospace;">EventEmitted</span>, <span style="font-family: Courier New, Courier, monospace;">Restarted</span>, and <span style="font-family: Courier New, Courier, monospace;">EndSession</span> method.</li>
<li>Ability to run with <span style="font-family: Courier New, Courier, monospace;">PID</span> >1 to allow Upstart to manage a user session.<br />Running Upstart as a 'Session Init' in this way provides features<br />above and beyond those provided by the original User Jobs such that<br />the User Job facility has been removed entirely: to migrate from<br />a system using User Jobs, simply ensure the user session is started with<br />'<span style="font-family: Courier New, Courier, monospace;">init --user</span>'.</li>
<li>New upstart-event-bridge bridge which proxies system-level events down to Session Inits, allowing users jobs to react to udev events.</li>
<li>Ability to read job configuration and override files from multiple freedesktop-compliant locations (Session Init only).</li>
<li>Ability to shutdown both via a system shutdown request and via a user logout request (Session Init only).</li>
<li>Additionally, there are a few bug fixes and 94 new tests.</li>
</ul>
<br />
Thanks to all the contributors, reviewers, testers and users!<br />
<br />
<h2>
Download</h2>
<br />
Get it here <a href="https://launchpad.net/upstart">https://launchpad.net/upstart</a><br />
<br />
<br />
James Hunthttp://www.blogger.com/profile/03416491632578860019noreply@blogger.com1