ANNOUNCE: numtype 1.0 — Type-level (low cardinality) integers

Since its inception my dimensional library has been built around a unary type-level representation of integers (NumTypes) defined in the Numeric.NumType module. This module has proven itself useful outside the context of dimensional and after dragging my feet for a long time I’ve finally gotten around packaging it up in its own library: numtype.

The Numeric.NumType module is completely self-contained (only imports Prelude) and is heavily commented in a narrative manner inspired by Oleg Kiselyov’s expositions. I believe it provides a good case study for type-level programming with multi-parameter type classes (MPTCs) and functional dependencies.

Addition, subtraction, division, and multiplication of NumTypes is supported. NumTypes have no value-level representation but can be converted to any Num instance with the toNum function.

The numtype library has two significant short-comings:

• Minimal haddocks — as with my dimensional library the literate Haskell source code is the documentation. The flip-side is that the code is very well-commented.

• Due to the unary implementation the practical size of the NumTypes is severely limited, making them unsuitable for large-cardinality applications. If you will be working with integers beyond (–20, 20) this package probably isn’t for you.

(If the second bullet is a show-stopper Edward Kmett’s type-int library may be a better choice. Peter Gavin’s tfp library also provides type-level integers but uses type families instead of MPTCs and fundeps. I cannot vouch for either of these libraries as I haven’t used them.)

Numtype version 1.0 can be downloaded from Hackage or the dimensional project page. I’ve also updated dimensional to version 0.8 with the Numeric.NumType module removed and Julian year and century units added. Enjoy!

Benchmarking Amazon EC2 with GHC

My personal computers are pretty old and/or slow. I have an old PowerBook G4 and a newish EEE PC. The PowerBook was top of its class when I got it with most options maxed out. Alas, that was five years ago. The EEE PC is by definition not a top performer, nor does it try to be. I find that when the two machines perform similarly in day-to-day tasks, at least when the EEE PC is in “Super performance” mode.

Truth is that for most of the mundane stuff I do these two machines perform acceptably. I’m obviously not going to be watching any 1080p movies on them, or enjoying the latest games (from a productivity standpoint I’m not sure these are such bad things), but most everything else works fine.

Where I feel the performance does hurt me is when compiling with GHC (or any compiler for that matter, it’s just that GHC is the one I use most). Often I spend way to much of my precious little private developer time waiting for the compiler to finish. This is in stark contrast to the situation at work where I run GHC on a pretty zippy Dell PowerEdge Blade Server.

In the near future I expect to be spending more time developing at home and want to be able to do so more efficiently, preferably on par with the situation at work. I could obviously buy shiny new hardware but being a miser (in case you couldn’t already tell based on my hardware) I’m looking for alternatives that would allow me to avoid or defer a hefty up-front investment. One such alternative I’m considering is to rent compute capacity in the Amazon Elastic Compute Cloud (EC2).

EC2 compute capacity is sold in the form of instances at an hourly rate ranging from $0.10 to $0.80 depending on capacity/performance plus some small-change for bandwidth and persistent storage. While there are a couple of hurdles to overcome in order to leverage EC2 as a development workstation the first thing I want to do is to make sure it is a goal worthy of pursuing in the first place, i.e. will I get the desired performance gains at a reasonable price?

To at least begin to answer this question I’ve done some informal benchmarking of the aforementioned systems, excluding the pricier EC2 instances. All the regular benchmarking caveats apply and to reinforce the unscientificity of it all I’m not going to bother providing complete specs for the systems. Here are the fundamentals:

• Apple PowerBook G4: 1.5 GHz PowerPC G4 processor, 2 GB RAM, 5400 rpm HD.
• Asus EEE PC 900HA: 1.6 GHz Intel Atom processor, 1 GB RAM, 4200 rpm HD.
• Dell PowerEdge 1855 Blade Server: Two single-core 3.2 GHz Xeon processors, 2 GB RAM.
• Amazon EC2 Small Instance: 1 EC2 Compute Unit (1 virtual core), 1.7 GB RAM, $0.10 per hour.
• Amazon EC2 High-CPU Medium Instance: 5 EC2 Compute Units (2 virtual cores), 1.7 GB RAM, $0.20 per hour.

According to Amazon “one EC2 Compute Unit provides the equivalent CPU capacity of a 1.0–1.2 GHz 2007 Opteron or 2007 Xeon processor.”

I figure the results are probably more interesting that the details of the tests so here they are, the systems are ordered by increasing performance which happened to be consistent across the tests:

Benchmark results, times in seconds, shorter times are better.

	astro-tables build	highlighting-kate build	fad test suite
PowerBook G4	292	(848)	28
EEE PC	291	643	18
EC2 Small	171	519	15
Dell PowerEdge	75	260	6
EC2 Medium	55	172	4

The EEE PC was in “Super Performance” mode during the tests and the PowerBook was at its highest CPU speed. All times are the “real” time as measured by the Unix time command and lower numbers are naturally better.

As can be seen an Amazon EC2 Small instance is only marginally faster than the EEE PC. An EC2 High-CPU Medium instance on the other hand is significantly faster than the zippy Dell PowerEdge Blade server. Is either one a good deal? Good question, I think a case could be made either way depending on your priorities but I’m not going to tackle that today.

If you care about the details of the tests read on, if not please move on to your next blog of choice!

astro-tables build

This package currently consists of a single automatically generated 4000-line monster of a module¹. The code is pretty straight-forward but the module takes ages to compile, almost certainly due to me giving the type checker an unnecessarily hard time. I have a trivial rewrite on my todo-list which I expect will shorten the compilation time dramatically but the current form comes in kind of handy for the purposes of this benchmark. The Git repo is git://github.com/bjornbm/astro-tables.git and the commit used in the benchmarking was e63b8978833878526870b2101697197ff64af593. I made sure the dependencies were already installed and ran time cabal install.

highlighting-kate build

From recent memory I knew that John MacFarlane’s highlighting-kate package has a hefty number of modules (the majority of which are also automatically generated) that take a fair amount of time to compile. I downloaded version 0.2.4 from hackage, made sure all dependencies were already installed, and ran time cabal install --flags=executable.

I ran into one snag with this test: the build wouldn’t complete with GHC 6.10.3 on the PowerBook G4 due to some problem with pcre-light (which tends to give me headaches on pretty much every platform). This particular headache² I was unable to resolve and had to run the test using GHC 6.10.1 on the PowerBook G4.

fad test suite

Finally I did a runtime performance (as opposed to compilation) benchmark: running the test suite of the fad library. The Git repo is git://github.com/bjornbm/fad.git and the commit used was cd2965a6741291570930e4bf6e9f8f9ab64ccadd. I ran ghc --make Test and then time ./Test.

---

1. An implementation of the 678 lunisolar terms and 687 planetary terms of the IAU 2000A Precession-Nutation Model. ↩

2. gcc: Internal error: Virtual timer expired (program cc1) ↩

May 2009 HCAR Submissions

Last weekend was the submission deadline for contributions to the May 2009 edition of the Haskell Communities and Activities Report (HCAR). I absolutely love the HCAR; it’s a terrific and comprehensive source of information on all the amazing stuff people are doing with Haskell. I’m very grateful to the editor Janis Voightländer (and Andres Löh before him) for compiling the report, in my opinion he does a great service to the Haskell community. I can’t wait to get my hands on the upcoming edition!

Below are my submissions to the HCAR (free blogging material, yay!) with some additional notes.

dimensional: Statically checked physical dimensions

Report by: Björn Buckwalter
Status: active, mostly stable

Dimensional is a library providing data types for performing arithmetics with physical quantities and units. Information about the physical dimensions of the quantities/units is embedded in their types, and the validity of operations is verified by the type checker at compile time. The boxing and unboxing of numerical values as quantities is done by multiplication and division with units. The library is designed to, as far as is practical, enforce/encourage best practices of unit usage.

The core of dimensional is stable with additional units being added on an as-needed basis. In addition to the SI system of units, dimensional has experimental support for user-defined dimensions and a proof-of-concept implementation of the CGS system of units. I am also experimenting with forward automatic differentiation and rudimentary linear algebra.

The current release is compatible with GHC 6.6.x and above and can be downloaded from Hackage or the project web site. The primary documentation is the literate Haskell source code, but the wiki on the project web site has a few usage examples to help with getting started.

Further Reading

http://dimensional.googlecode.com

Dimensional was largely the project that enticed me to learn Haskell, and is the one that I am most proud of to date. It relies on some pretty tricky type-level hackery, stuff I would never have been able to pull off without the help of many papers ranging from Oleg Kiselyov’s amazing type-hacks to Hudak et al’s A History of Haskell.

Its primary shortcoming at present is the lack of Haddock documentation. Dimensional is written in a literate style and I think anyone reading the source will find the documentation pretty comprehensive and easy to follow (barring the fact that type hacking is pretty tricky business). Alas, with the last few years’ infrastructure improvements and increased (de facto) standardization in the community not having proper haddocks is a wart.

fad: Forward Automatic Differentiation

Report by: Björn Buckwalter
Participants: Barak A. Pearlmutter, Jeffrey Mark Siskind
Status: active

Fad is an attempt to make as comprehensive and usable a forward automatic differentiation (AD) library as is possible in Haskell. Fad (a) attempts to be correct, by making it difficult to accidentally get a numerically incorrect derivative; (b) item provides not only first-derivatives, but also a lazy tower of higher-order derivatives; (c) allows nested use of derivative operators while using the type system to reject incorrect nesting (perturbation confusion); (d) attempts to be complete, in the sense of allowing calculation of derivatives of functions defined using a large variety of Haskell constructs; and (e) tries to be efficient, in the sense of both the defining properties of forward automatic differentiation and in keeping the constant factor overhead as low as possible.

Version 1.0 of fad was uploaded to Hackage on April 3. Recent changes can be found via git clone git://github.com/bjornbm/fad.git

Further Reading

http://github.com/bjornbm/fad
http://flygdynamikern.blogspot.com/2009/04/announce-fad–10-forward-automatic.html

The majority of work on fad (including most of the above report) is actually being done by Barak Pearlmutter, principial of the Hamilton Institute’s Brain and Computation Lab. As you can understand his brain is in a different league than mine when it comes to anything related to computation. My contribution at this stage is mostly reviewing his patches (often struggling to understand them) and providing guidance and support on Haskell infrastructure and conventions.

leapseconds-announced

Report by: Björn Buckwalter
Status: stable, maintained

The leapseconds-announced library provides an easy to use static LeapSecondTable with the leap seconds announced at library release time. It is intended as a quick-and-dirty leap second solution for one-off analyses concerned only with the past and present (i.e. up until the next as of yet unannounced leap second), or for applications which can afford to be recompiled against an updated library as often as every six months.

Version 2009 of leapseconds-announced contains all leap seconds up to 2009–01–01. A new version will be uploaded if/when the IERS announces a new leap second.

Further Reading

http://hackage.haskell.org/cgi-bin/hackage-scripts/package/leapseconds-announced
http://github.com/bjornbm/leapseconds-announced

Not much to add about leapseconds-announced other than that it fills a need I have. I elaborated a little on what it is (and what it doesn’t try to be) in the announcement thread.

That’s it! I hope to have another project or two to report on in the November edition…

ANNOUNCE: fad 1.0 — Forward Automatic Differentiation for Haskell

I’m pleased to announce the initial release of the Haskell fad library, developed by Barak A. Pearlmutter and Jeffrey Mark Siskind. Fad provides Forward Automatic Differentiation (AD) for functions polymorphic over instances of Num. There have been many Haskell implementations of forward AD, with varying levels of completeness, published in papers and blog posts, but alarmingly few of these have made it into hackage — to date Conal Elliott’s vector-spaces package is the only one I am aware of.

Fad is an attempt to make as comprehensive and usable a forward AD package as is possible in Haskell. However, correctness is given priority over ease of use, and this is in my opinion the defining quality of fad. Specifically, Fad leverages Haskell’s expressive type system to tackle the problem of perturbation confusion, brought to light in Pearlmutter and Siskind’s 2005 paper Perturbation Confusion and Referential Transparency. Fad prevents perturbation confusion by employing type-level “branding” as proposed by myself in a 2007 post to haskell-cafe. To the best of our knowledge all other forward AD implementations in Haskell are susceptible to perturbation confusion.

As this library has been in the works for quite some time it is worth noting that it hasn’t benefited from Conal’s ground-breaking work in the area. Once we wrap our heads around his beautiful constructs perhaps we’ll be able to borrow some tricks from him.

As mentioned already, fad was developed primarily by Barak A. Pearlmutter and Jeffrey Mark Siskind. My own contribution has been providing Haskell infrastructure support and wrapping up loose ends in order to get the library into a releasable state. Many thanks to Barak and Jeffrey for permitting me to release fad under the BSD license.

Fad resides on GitHub and hackage and is only a cabal install fad away! What follows is Fad’s README, refer to the haddocks for detailed documentation.

   Copyright  : 2008-2009, Barak A. Pearlmutter and Jeffrey Mark Siskind
   License    : BSD3

   Maintainer : bjorn.buckwalter@gmail.com
   Stability  : experimental
   Portability: GHC only?

Forward Automatic Differentiation via overloading to perform
nonstandard interpretation that replaces original numeric type with
corresponding generalized dual number type.

Each invocation of the differentiation function introduces a
distinct perturbation, which requires a distinct dual number type.
In order to prevent these from being confused, tagging, called
branding in the Haskell community, is used.  This seems to prevent
perturbation confusion, although it would be nice to have an actual
proof of this.  The technique does require adding invocations of
lift at appropriate places when nesting is present.

For more information on perturbation confusion and the solution
employed in this library see:
<http://www.bcl.hamilton.ie/~barak/papers/ifl2005.pdf>
<http://thread.gmane.org/gmane.comp.lang.haskell.cafe/22308/>


Installation
============
To install:
    cabal install

Or:
    runhaskell Setup.lhs configure
    runhaskell Setup.lhs build
    runhaskell Setup.lhs install


Examples
========
Define an example function 'f':

> import Numeric.FAD
> f x = 6 - 5 * x + x ^ 2  -- Our example function

Basic usage of the differentiation operator:

> y   = f 2              -- f(2) = 0
> y'  = diff f 2         -- First derivative f'(2) = -1
> y'' = diff (diff f) 2  -- Second derivative f''(2) = 2

List of derivatives:

> ys = take 3 $ diffs f 2  -- [0, -1, 2]

Example optimization method; find a zero using Newton's method:

> y_newton1 = zeroNewton f 0   -- converges to first zero at 2.0.
> y_newton2 = zeroNewton f 10  -- converges to second zero at 3.0.


Credits
=======
Authors: Copyright 2008,
Barak A. Pearlmutter <barak@cs.nuim.ie> &
Jeffrey Mark Siskind <qobi@purdue.edu>

Work started as stripped-down version of higher-order tower code
published by Jerzy Karczmarczuk <jerzy.karczmarczuk@info.unicaen.fr>
which used a non-standard standard prelude.

Initial perturbation-confusing code is a modified version of
<http://cdsmith.wordpress.com/2007/11/29/some-playing-with-derivatives/>

Tag trick, called "branding" in the Haskell community, from
Bjorn Buckwalter <bjorn.buckwalter@gmail.com>
<http://thread.gmane.org/gmane.comp.lang.haskell.cafe/22308/>

Space News: U.S. Air Force To Widen Access To Detailed Space Surveillance Data

Revisiting my posts on the Iridium 33/Cosmos 2251 collision and data…

The front page article of last week’s Space News (Volume 20, Issue 10, March 9th, 2009) reported that “U.S. Air Force To Widen Access To Detailed Space Surveillance Data” (article quoted in full here). The short story is that:

The U.S. Air Force has agreed to provide wider access to its high-accuracy catalog showing the whereabouts of orbital debris and operational satellites as part of an effort to enable commercial and non-U.S. government satellite operators to better avoid in-orbit collisions, according to U.S. Air Force officials.

The “high-accuracy catalog” would be what I referred to as Special Perturbations (SP) data. The article reiterates the limitations of the Two-Line Elements currently available to the public:

…U.S. Air Force Space Surveillance Network data is published, but only in a form that satellite operators have long said is not useful for space traffic management. This data, called Two-Line Elements (TLEs), has too great a margin of error to permit operators to act.

And says that:

The U.S. Air Force statement suggests that it will furnish more information to the public to enable operators to make a more highly informed decision.

Apparently a policy if forthcoming. The details of the policy are unknown but it should be announced before June.

All this is of course good news and cause for cautious optimism. Why do I say cautious? Here are two quotations from a March 5 roundtable discussion titled Challenges for Space Policy in 2009:

We’re horrible at implementing policy, absolutely horrible.

The problem is not that we have insufficient space policy or a bad space policy, but as we now understand, policy is not self-actualizing. We need to have some other mechanisms that take very sound policy and turn them into action and results.

Let’s hope those mechanisms are in place by June.

While on the topic of the Iridium 33/Cosmos 2251 collision: via Jeff Foust’s spacepolitics.com (good blog, love the tagline) I read that “some have tried to portray last month’s Iridium-Cosmos satellite collision as either a deliberate act by the US or a deliberate act by Russia.” I call bullshit on both of these. The Washington Times editorial is particularly embarrassing in its tin-foil-hattery.

Blogging with Pandoc, literate Haskell, and a bug

I use John MacFarlane’s pandoc for converting my blog entries into HTML. My workflow is roughly:

1. Write my posts vim using (pandoc’s enhanced) Markdown format.
2. Store my posts in a private Git repository on GitHub.
3. Convert posts to pastable HTML using pandoc and paste into Blogger.

pandoc is a great tool. It is a better Markdown than Markdown and also supports a bunch of other markup formats. The feature set is really extensive with plenty of useful options and sensible defaults. Kudos to John and the other contributors.

My previous post on Network.Curl was the first post where I included significant amounts of source code and was thus my first opportunity to exercise pandoc’s syntax high-lighting features. I was fortunate enough that version 1.2 of pandoc was released just days prior to my post with support for literate Haskell. The literate Haskell support meant that I could stop worrying about how to avoid my Haskell code being interpreted as Markdown block quotations (and if I need a block quotation I can always use <blockquote> tags).

By default pandoc will install without support for syntax high-lighting. To enable syntax high-lighting you must supply the highlighting flag when building, e.g. if you are using cabal:

cabal install pandoc --flags=highlighting

This triggers installation of John’s highlighting-kate library which is used for the syntax high-lighting. A snag you might run into here is the dependence on pcre-light which in turn relies on a PCRE dynamic library being available. In my case (on Mac OS 10.4) I had to install PCRE using MacPorts and add /opt/local/lib to $LD_LIBRARY_PATH.

When editing my posts I want to review occasionally and run pandoc with the --standalone option to generate a complete HTML document I can view in my browser. However, when pasting into Blogger I only want a fragment and omit said option. The problem with this is that the CSS style definitions used for the syntax high-lighting isn’t included in the fragment. My solution to this problem was to copy the <style> element from the --standalone output and paste into the <head> of my Blogger template (immediately prior to </head>).

That’s pretty much it. The complete command I use to generate the fragments I paste into Blogger is:

pandoc --smart --to=html+lhs input.lhs

(I pipe the output into e.g. pbcopy or xclip to get it straight into the clipboard.)

The +lhs portion of the --to option tells pandoc to keep the bird-tracks (those ’>’s at the beginning of a line of code). The beauty of this is that readers can copy and paste the entire (as in “Select All”) blog post into a .lhs file and it is valid literate Haskell that e.g. ghc will happily compile. (Well, at least when copied from Firefox. Safari has the nasty habit of omitting the empty line following a code block which doesn’t make for valid literate Haskell… I’ll be investigating work-arounds.)

When writing my previous post I did notice one discrepancy in the syntax high-lighting. The bird-track on Any line following a end-of-line comment (--) wouldn’t be correctly high-lighted. Example:

> -- | Additional options to simulate submitting the login form.
> loginOptions user pass = 
>   CurlPostFields [ "login=" ++ user, "password=" ++ pass ] : method_POST

Depending on your screens calibration it might not stick out like a sore thumb but the middle bird-track there is black while the other two are blueish. This was due to a bug in the syntax high-lighting definition file for literate Haskell (literate-haskell.xml) in the highlighting-kate package currently on Hackage (version 0.2.3). This is no fault of John’s as that file came straight from the KDE subversion repository. A couple of hours ago I submitted a patch to John as well as the KDE developers (who have already applied it to the their repo).

If you can’t wait for John to apply the patch to the highlighting-kate repo you can download it from darcswatch (it’s the one labeled “20090314201157”), apply it to your local copy of John’s repo, cabal install, and finally cabal install pandoc --flags=highlighting --reinstall.

With the patch applied the bird-tracks will render properly:

> -- | Additional options to simulate submitting the login form.
> loginOptions user pass = 
>   CurlPostFields [ "login=" ++ user, "password=" ++ pass ] : method_POST

Update 2009–03–16: John uploaded highlighting-kate 0.2.4 (with patches applied) to Hackage today. Reinstall with cabal (don’t forget to cabal update first).

Extended sessions with the Haskell Curl bindings

I recently needed to automate retrieving protected data from a secure web site. I had to:

1. Log into the website with a POST request.
2. Download the protected data with a GET request.

All this had to be done using SSL, and I suspected I’d need to handle cookies too.

I had read that libcurl had support for sessions and cookies spanning multiple requests, and knew that it could handle SSL. I was aware there is a Haskell binding to libcurl (aptly named “curl” but hereafter referred to as Network.Curl to avoid confusion) on Hackage so I had my tools cut out for me. While I had used the command line curl utility quite a bit I had never programmed against libcurl before and had some learning to do.

It wasn’t entirely clear to me from the haddocks how to use Network.Curl. This may not be a problem if you are familiar with libcurl already (I couldn’t tell) but for me it was quite a hurdle. Googling on the topic I found some blogged examples that got me started but I was unable to find an example demonstrating a multi-request session. However, with the basics from the blogs I was able to return to Network.Curl and figure things out by inspecting its source code. I’ll share an example here for the benefit of others who find themselves in the same situation. I’m using version 1.3.4 of Network.Curl.

As a contrived example let’s assume we want to write a small program that, given a user name and password, fetches the user’s API token from GitHub. Here is the code (literate Haskell, just copy and paste into a .lhs file):

> import Network.Curl
> import System (getArgs)
> import Text.Regex.Posix

> -- | Standard options used for all requests. Uncomment the @CurlVerbose@
> -- option for lots of info on STDOUT.
> opts = [ CurlCookieJar "cookies" {- , CurlVerbose True -} ]

> -- | Additional options to simulate submitting the login form.
> loginOptions user pass = 
>   CurlPostFields [ "login=" ++ user, "password=" ++ pass ] : method_POST

> main = withCurlDo $ do
>   -- Get username and password from command line arguments (will cause
>   -- pattern match failure if incorrect number of args provided).
>   [user, pass] <- getArgs

>   -- Initialize curl instance.
>   curl <- initialize
>   setopts curl opts

>   -- POST request to login.
>   r <- do_curl_ curl "https://github.com/session" (loginOptions user pass)
>     :: IO CurlResponse
>   if respCurlCode r /= CurlOK || respStatus r /= 302
>     then error $ "Failed to log in: "
>                ++ show (respCurlCode r) ++ " -- " ++ respStatusLine r
>     else do
>       -- GET request to fetch account page.
>       r <- do_curl_ curl ("https://github.com/account") method_GET
>         :: IO CurlResponse
>       if respCurlCode r /= CurlOK || respStatus r /= 200
>         then error $ "Failed to retrieve account page: "
>                    ++ show (respCurlCode r) ++ " -- " ++ respStatusLine r
>         else putStrLn $ extractToken $ respBody r

The first thing to note is that we use do_curl_ rather than e.g. curlPost and curlGet. The latter two don’t actually give you access to the response body but instead prints it on stdout! The general process is:

1. Initialize a curl instance.
2. Set options.
3. Call do_curl with the URL and request-specific options.
4. Inspect CurlResponse
5. Repeat from 3 until done.

Note that the all used of libcurl should be wrapped by withCurlDo. In the example I wrapped the entire body of main. Also note that the type of do_curl_ must be specified explicitly unless it can be inferred by later use. The CurlResponse type specified above uses vanilla Strings for everything.

For the POST request I added some CurlPostFields to the method_POST options predefined in the Network.Curl. For the GET request’s options since the predefined method_GET is sufficient.

A GitHub-specific peculiarity here is the error checking after the POST request. GitHub returns a 302 (“Moved Temporarily”) on successful login and a 200 (“OK”) when the credentials are bad. Stuff like this needs to be figured out on a site-by-site basis.

For completeness here is the function that extracts the token from the response body using a regular expression:

> -- | Extracts the token from GitHub account HTML page.
> extractToken body = head' "GitHub token not found" xs
>   where
>     head' msg l = if null l then error msg else head l
>     (_,_,_,xs)  = body =~ "github\\.token (.+)"
>                :: (String, String, String,[String])

If you Load this code in ghci and type :main username password the Octocat will deliver your token.

Programmer Man

While I haven’t tried pair programming I’m intrigued by the idea and would like to give it a shot some time. Rod Hilton recently blogged about his experiences with pair programming and despite vim (+command line) being my primary development environment I have to concede that it makes an interesting as well as entertaining read. As William Pietri noted in the comments the “insights on Programmer Man are both novel and keen” and well worth pondering even if you have no interest in pair programming per se.

Iridium 33, Космос (Cosmos) 2251, and Data

So, finally: data (previous posts here and here).

I may not have said it outright, but I certainly implied that the primary form of orbital data available to operators for collision prediction purposes is Two-Line Elements (TLEs). Specifically the TLEs published by the US Air Force. There are of course other types of data, although they are either not publicly available or not comprehensive.

Here’s a rundown…

Operator data

By “operator data” I mean whatever orbital data a satellite owner/operator uses for orbit control and/or prediction for its own spacecraft. This data is typically based on radiometric or GPS measurements and is much more accurate than TLE. The obvious shortcoming of operator data is that no such data exists for inactive spacecraft and debris orbiting Earth. Furthermore, satellite operators normally only have access to this information for their own spacecraft. Accurate knowledge of your own spacecraft’s orbit doesn’t really help much when you don’t have accurate information about all the other objects it might collide with.

In the geostationary realm, where the ratio of active spacecraft to debris is still relatively favorable, operators have recently started an effort to share data between themselves for purposes of collision avoidance. This is a unique cooperation among the otherwise very competitive telecommunication satellite owners (of which my employer, Intelsat, is one) and an important first step to safer satellite operations in the geostationary belt.

In lower orbits, such as those of Iridium 33 and Cosmos 2251, the ratio of operated spacecraft to debris is so small that sharing operator data would only marginally improve safety.

Two-Line Elements

The US Air Force publishes a catalog of orbital data in the form of TLEs for the majority of Earth-orbiting objects through its space-track.org web site.

TLEs are based on measurements made by the US Space Surveillance Network (SSN). The measurements are fitted to a relatively simple analytical model (SGP4) by an undisclosed process. The accuracy of any given TLE is generally unknown, perhaps even to the Air Force. Factors affecting the orbit quality are:

• The amount and age of measurements used to generate the TLE,
• The applicability of the SGP4 model to the particular orbit and object properties,
• The presence of artificial disturbances (i.e. maneuvers) during or after the time when measurements were collected.

In general the accuracy of TLEs is thought to be on the order of kilometers. However, the TLE catalog also has other problems, notably that classified US spacecraft (and related debris) are purposefully omitted from the catalog and that certain classes of non-classified objects are omitted for technical reasons (e.g. inability to track them).

Special Perturbations (SP) data

SP data is the moniker for orbital data from a classified catalog maintained by the US Air Force. It is essentially equivalent to the TLE catalog but with higher accuracy data based on a more accurate orbital model. It also includes the classified US spacecraft. The accuracy of SP data isn’t publicly known but it is thought to be significantly better than that of the TLEs.

The SP data catalog is used routinely for collision prediction for the Air Force’s (and other DoD organizations’) satellites as well as NASA missions (notably human spaceflight). However, by virtue of being classified the SP data isn’t generally available to other satellite operators. There is a process through which satellite operators can request support from the Air Force to confirm isolated close approaches. This process currently requires the operator to first identify the risk of a close approach by another means (e.g. using TLEs) and then go through a few layers of bureaucracy before the Air Force will investigate the close approach. It works somewhat in the not-too-densely-populated geostationary realm but I cannot see that it would scale to the level of support e.g. Iridium would need.

One can only hope that the collision between Iridium 33 and Cosmos 2251 prompts the Air Force to increase the level of support it provides to satellite operators and streamline the processes involved.

DISCOS

The European Space Agency maintains the DISCOS database (Database and Information System Characterising Objects in Space) which is based primarily on data from the US Space Surveillance Network. I don’t know any details about that data, perhaps it is simply the TLEs that the US Air Force publishes?

Late last year ESA initiated a program to develop its own space tracking capabilities.

MIT Lincoln Laboratory

MIT’s Lincoln Laboratory offers a close approach prediction service to geostationary satellite operators. To my knowledge the data used in the predictions is collected by the laboratory’s own sensors and is comparable to SP data. It’s a good service but the cost for operators is pretty steep (though certainly nowhere near the cost of losing a satellite in a collision) which prevents its use from being ubiquitous.

International Scientific Optical Network

A relatively new player in this field is the International Scientific Optical Network (ISON) coordinated by the Keldysh Institute of Applied Mathematics in Moscow. ISON is a collaboration between 18 scientific institutions in 9 states with the objective of establishing and maintaining a comprehensive database of objects in high altitude orbits (including the geostationary belt). ISON has so far identified several hundred objects not included in the US Air Force’s TLE catalog (roughly half of which are classified US objects).

The quality of ISON’s orbital data seems to be significantly better than TLEs, but at present the networks focus is on tracking object not in the TLE catalog rather than providing comprehensive orbital data for collision prediction purposes. However, this is a player worth keeping an eye on if you are operating geostationary spacecraft.

There’s more to it…

As is obvious to anyone working with the problem of satellite close approach prediction and avoidance I done a lot of hand waving and sweeping generalizations in these posts. My intent was never to go into pain-staking detail or to expose every facet of the problems satellite operators face. I am certainly not aware of them all myself despite the problem being part of my everyday work (just today we decided to reschedule a maneuver on one of our satellites in order to maintain safe separation distance during an upcoming close approach with a spent rocket body).

Iridium 33, Космос (Cosmos) 2251, and Odin

In my previous post I discussed the difficulties satellite operators like Iridium face in attempting to predict collisions. I’ll talk more about orbital data in the future but in this post I want to revisit the question of whether it makes sense to point fingers at the Russians for having an “out of control” spacecraft (Cosmos 2251) on orbit? Well, one could turn the tables and argue that the finger should be pointed at Iridium, since their spacecraft (Iridium 33) had the capability to maneuver and also was an asset worth protecting. But, as I illustrated previously, it is extremely difficult for a satellite operator in this particular orbital regime (Low Earth Orbits) to predict collisions with high enough confidence to warrant action.

In fact, many satellites in Low Earth Orbits don’t even have a propulsion system, which means that even given perfect information regarding a future collision they would be unable to move out of the way. An example of such a satellite is the Swedish space observatory Odin which orbits Earth in a circular polar orbit roughly 600 km above sea level, or 180 km below the altitude at which Iridium 33 and Cosmos 2251 collided. Odin is built and operated by the Swedish Space Corporation, my former employer, and is particularly dear to me as I participated in its launch campaign at Svobodny Cosmodrome, Russia, in 2001.

About a year ago Dr T.S. Kelso told me that one day when he had been checking on the close approach predictions from his SOCRATES service he saw a miss distance significantly smaller than he had ever seen before. I don’t remember the exact number but I believe he said it was less than a meter, which is to say it is not a miss at all if the combined dimensions of the involved spacecraft are greater than a meter. One of the objects involved in the prediction was a piece of debris while the other was the still operational Odin. Even with the caveats regarding the inaccuracies of Two-Line Elements T.S. felt compelled to contact Odin’s, to him unknown, operator and warn them of the potential danger. After a fair bit of detective work he eventually got hold of a phone number to the Odin control center at Esrange in northern Sweden. He placed the call and explained the predicament only to find out that Odin did not have maneuvering capability and could do nothing to ameliorate the situation.

So how did it go? Well, since you haven’t heard of this before I don’t have to tell you there was no collision and Odin is still out there doing good science. But the example still serves to illustrate the inability to mitigate a predicted close approach, which would hold true even if the data behind the prediction was better. In light of the fact that many satellites don’t have a propulsion system by design (and consequently have no interest in, or can not benefit from, collision predictions) it appears unreasonable to place blame on Iridium for omission of action, or on the Russians for inability to act. These are just par for the course.

(Now, if there was such a thing as perfect information regarding collision prediction perhaps it would be worthwhile including a minimal propulsion system solely for the purpose of collision avoidance, even on spacecraft like Odin where a propulsion system is otherwise considered unnecessary. Alas, it is not so.)

Iridium 33 and Космос (Cosmos) 2251

The big news in my industry right now is the collision between the satellites Iridium 33 and Cosmos 2251. While the Cosmos spacecraft was reportedly non-operational since 1995 Iridium 33 was operational and in service.

In some of the press the choice of wording (e.g. “out of control” and “slammed into the Iridium craft”) has been such as to unfairly suggest that Cosmos 2251 was at fault. For example, from BBC News:

“A collision occurred between an Iridium 33 satellite and a Russian Cosmos 2251 military satellite,” Major General Alexander Yakushin said.

The satellite was launched in 1993 and ceased to function two years later, he said according to the AFP news agency.

Russia has not commented on claims the satellite was out of control.

The Russian Major General said that Cosmos 2251 ceased to function in (or around) 2005, so by definition it is “out of control,” which is to say it was orbiting the Earth as dictated by fairly well understood laws of physics. However, the term “out of control” is used by BBC (and others) in a way that implies that Cosmos 2251 is flying around all over the place in an unpredictable manner endangering other spacecraft left and right.

The truth is, it doesn’t really matter, this could just as well have happened between any two operational spacecraft. The key here is that in general, no one is looking. And even if you are looking, it is difficult to know what to do.

I first heard of the event about twelve hours after the fact from Dr T.S. Kelso at the Center for Space Standards & Innovation. Dr Kelso operates the venerable CelesTrak website and one of the services his site provides is SOCRATES (Satellite Orbital Conjunction Reports Assessing Threatening Encounters in Space). SOCRATES compares Two-Line Elements (parameters describing satellite orbits, hereafter referred to as TLEs) from the US Air Force in order to predict close approaches between objects in orbit around Earth. SOCRATES had predicted a close approach between Iridium 33 and Cosmos 2251 of 584 m at the time of collision. This highlights a significant problem: the data generally available to predict close approaches isn’t very accurate. The actual accuracies of any given TLEs are not well known, perhaps not even to the Air Force. In this case the data was obviously off by about 584 m (the satellites themselves may be up to tens of meters large).

So, should Iridium and other satellite operators look out for, and maneuver their satellites to avoid, anything that comes within a given distance of their own spacecraft? Apparently a 584 m predicted miss distance is not safe so for the sake of argument let’s assume that we use that as a safety threshold. Searching for “IRIDIUM” in SOCRATES right now returns 16 close approaches with distances less than 584 m over seven days:

Time of Closest Approach (UTC)	Object 1	Object 2	Min Range (km)
2009 Feb 13 14:38:21	IRIDIUM 16	COSMOS 1275 DEB	0.137
2009 Feb 16 05:47:41	IRIDIUM 97	CBERS 1 (ZY 1A)	0.184
2009 Feb 13 03:22:09	IRIDIUM 16	FENGYUN 1C DEB	0.213
2009 Feb 15 23:59:47	IRIDIUM 45	SL–8 DEB	0.251
2009 Feb 12 19:38:39	IRIDIUM 24	COSMOS 1275 DEB	0.274
2009 Feb 12 07:19:35	IRIDIUM 26	NOAA 6	0.315
2009 Feb 12 14:18:32	IRIDIUM 36	CZ–4 DEB	0.348
2009 Feb 18 18:08:06	IRIDIUM 59	COSMOS 990	0.363
2009 Feb 16 00:12:50	IRIDIUM 54	THORAD AGENA D DEB	0.373
2009 Feb 17 07:48:07	IRIDIUM 38	DELTA 1 DEB	0.417
2009 Feb 17 06:36:02	IRIDIUM 50	ARIANE 40 R/B	0.483
2009 Feb 12 17:16:04	IRIDIUM 21	SL–8 DEB	0.490
2009 Feb 18 01:35:15	IRIDIUM 54	OPS 5058 (DMSP 5B F2)	0.491
2009 Feb 15 03:23:34	IRIDIUM 11	SL–18 DEB	0.495
2009 Feb 12 12:41:48	IRIDIUM 81	COSMOS 1190	0.555
2009 Feb 16 11:51:54	IRIDIUM 4	TITAN 3C TRANSTAGE DEB	0.578

Note that none of the above include debris originating from the Iridium 33/Cosmos 2251 collision - most likely because the Air Force hasn’t begun to publish that data yet. And we’re only looking at close approaches involving the 92 Iridium spacecraft, while there are over 10000 known objects out there. Check out the SOCRATES Top 10 listings at any given time for even closer approaches. The point is that services like SOCRATES predict lots of close approaches where the miss distance is significantly below the known to be unsafe 584 m, but actual collisions are still extremely unlikely and (to date) extremely rare.

Now, from the point of the satellite operator it should be understood that maneuvering a satellite to avoid a conjunction generally carries a quantifiable monetary cost with it. If the available information is so inaccurate you don’t even know for sure if you’ll be improving the situation by maneuvering (which is often the case when the information is TLEs) chances are you’ll do nothing and hope for the best. Indeed, there is a good chance you won’t even bother looking.

What is needed to make it operationally feasible to react to predicted close approaches is better data. Operators like Iridium have to know that if the data says the miss distance is, say, 100 m they are safe, and that for the infrequent cases where the miss distance is less the risk is so high that it is worth doing the analysis and spending the fuel necessary for an avoidance maneuver.

I have more to say on this topic but it’s getting late. In the mean time, check out some visualizations courtesy of Analytical Graphics, Inc.

Arch Linux in VirtualBox on EEE PC

For background refer to my previous post.

Now I’m going to go into some detail of the Arch Linux (hereafter referred to as Arch) installation and configuration process, or at least how it played out for me. I won’t be addressing those parts where the Beginners Guide was crystal clear and everything went as expected. I will be addressing the parts where I got confused or alarmed or where things just weren’t working as I would have expected.

VirtualBox

I should start off by mentioning the rather significant detail that I decided to install Arch as a guest OS running in VirtualBox under Windows XP. When I started out this seemed like a good idea; I wouldn’t have to worry about repartitioning the drive I currently had Windows XP on, I would be able to familiarize myself with Arch and how well it functions on the EEE PC before committing to it, it would be fun to try out VirtualBox, and if everything was hunky dory I hoped I could just take my VirtualBox disk image and clone it to a physical partition and be done. At that point I’d turn the tables and run Arch as my main OS while booting Windows XP as a VirtualBox guest OS on and as-needed basis.

Turns out I was naive. As I worked my way through the installation and configuration I realized that I’m failing to test how well Arch runs on the EEE PC. The stuff I would expect to be most problematic or difficult to configure doesn’t really come into play when running as guest OS. Video (including “compressed” 1024x768), sound, network cards, wifi, webcam, multi-touch trackpad, power management (including “Super Hybrid Engine”); all these are either not applicable or free-riding on the Windows drivers. I still have no idea how well all these things would work without Windows XP behind the scenes. In addition my Arch installation is configured for the mock hardware interfaces that VirtualBox exposes to it; for all I know it probably won’t run at all outside of VirtualBox!

Even had I realized the above I suspect I would still have chosen to install in VirtualBox first to try things out. Or perhaps I would have opted to try out another distro with a Live CD (or, more specifically, live thumb drive). Anyway, back to the Arch installation process…

Recovery

It turned out the mistake that I was referring to in my previous post wasn’t as fatal as I feared. I was in the Configure the System section of the Beginners Guide and believe I had made my way through initramfs and was about to select /etc/rc.conf in the menu when I inadvertently hit the ESC key. This exited the menu and a bunch of processing commenced, entirely prematurely. I had just skipped the steps where the guide said (with emphasis as it appears in the guide):

Closely following and understanding these steps is of key importance to ensure a properly configured system.

and, a few paragraphs further down:

Note: It is very important at this point to edit, or at the very least, to verify by opening, every configuration file. The installer script relies on your input to create these files on your installation. A common error is to skip over these critical steps of configuration.

I was fully expecting to have to wipe everything out and restart the installation procedure from scratch. However, once the configuration was done I was returned to the main menu and, lo and behold, I could back up one step and chose to reconfigure. This I did, following the guide closely, and as far as I can tell all is well. Sometimes you get lucky.

General installation process

I decided to install off the most recent core ISO (archlinux–2008.06-core-i686.iso). The relative merits of core vs FTP were unclear at the time but I believe if you use the FTP ISO you’ll end up with an up to date system from the start, while I ended up with eight month old packages.

I’m not particularly fond of the fuzzy compressed mode or scrolling the screen so I chose vga=771 to get 800x600 resolution instead of vga=773 and 1024x768. I imagine that without VirtualBox the scrolling and compressed mode wouldn’t have been available at all in which case vga=773 would be a very poor choice.

The first point of confusion arose when partitioning the hard drive. In the cfdisk interface a partition could be flagged as “bootable,” which sounds like it would be a desirable property for the partition with /boot filesystem. However, there is no mention of the flag in the Beginners Guide and in the example no flags are set on the partitions. I stayed true to the guide and didn’t set the flag and everything has worked fine. So while there isn’t an error per se in the guide it would be nice if it explicitly mentioned that the flag doesn’t need to be set. It certainly would have saved me some angst.

For the record I didn’t want to deal with deciding what is an appropriate size for all the file systems so I just went with one for / and a separate swap partition. It’s probably ridiculously easy to resize partitions after the fact but I wasn’t in a mood to investigate at the time.

At first I disabled crond in /etc/rc.conf but later realized I was missing out on good stuff like updatedb (necessary for locate to work) so I re-enabled it.

Setting up sound with alsa had me scratching my head for a while until I realized I had to enable sound for the guest in the VirtualBox settings.

I skipped part IV of the guide since I had already decided to use xmonad as my window manager. This was readily accomplished by pacman -S xmonad and I continued with other packages: vim, texlive, firefox…

The first “discrepancy” I noticed occurred when pacman was “updating kernel-headers” after installing cairo. It reported the rather disconcerting message:

error: scriptlet failed to execute correctly

I don’t mean to imply that the message itself is terribly scary, scriptlet even has a cutesy ring to it, but I’d imagine its not good to have stuff going wrong when messing with your kernel. Anyway, I have no idea what that means or what the scriptlet that failed was supposed to do. The error didn’t show up in /var/log/pacman.log so I presumed it must have come from a build script external to pacman or something along those lines. The next time I saw a scriptlet fail was when updating shadow:

Fixing gshadow file...
add group 'uucp' in /etc/gshadow ?grpck: the files have been updated
error: scriptlet failed to execute correctly

Again not sure what that meant, nor what to do about it. Hoping it was nothing.

VirtualBox Guest Additions: Take One

At some point I realized that to get decent video drivers supporting the native 1024x600 resolution of the EEE PC I had to install the VirtualBox Guest Additions. I had the option of installing from the ISO that came with VirtualBox, but there were no instructions specifically for Arch. I saw mention in some forum of a virtualbox-ose-additions community package. Doing a pacman -Si virtualbox-ose-additions revealed that it was version 2.0.4–1 while my installation of VirtualBox is 2.1.2. The build date was 2008–11–12 so I decided to try my luck hoping not much would have changed between the two versions, especially given that the ISO is an auxiliary part of VirtualBox. Well, I was stopped short by pacman:

error: failed to prepare transaction (could not satisfy dependencies)
:: virtualbox-ose-additions-modules: requires kernel26<2.6.28>

I checked my kernel26 and it was version 2.6.25.6–1 which certainly seemed to comply with the stated requirement. However, a pacman -Si virtualbox-ose-additions-modules revealed the following requirement:

Depends on     : kernel26>=2.6.27  kernel26<2.6.28>

OK, my kernel26 didn’t fulfill the requirements, although the requirement reported wasn’t the one that was violated.

By now I was thinking maybe I should upgrade my kernel26. In fact, I was mildly surprised that pacman didn’t go ahead and do that for me. I checked with pacman and saw that the latest version of kernel26 is 2.6.28.2–1. It dawned upon me that the reason pacman reported the requirement kernel26<2.6.28 was because that was the one that would have been violated had the kernel26 been upgraded to the latest version. Makes sense I guess but would have been nice if the error message was more explicit about it.

I decided to hit the AUR site to see if it could shed any additional light on these packages and indeed, there I saw that virtualbox-ose-additions had been flagged as out of date! I can’t help but wish that a pacman would reveal this rather important piece of information; pacman -Si loses a lot of its appeal if I have to double-check each package on the web before installing. (One could argue that pacman implicitly “knows” this particular package is out of date by virtue of its dependencies being unsatisfiable, but I believe this is a special case; other packages may have been flagged e.g. for security reasons even though there dependencies are satisfiable.)

Recommendation: Always check package status in AUR before installing, especially before deciding to “try your luck.”

(I later realized that I was using the PUEL VirtualBox and not VirtualBox OSE so the packages in question may never have been applicable in the first place.)

Upgrading all packages

Given that the VirtualBox Guest Additions seem to be particular about the kernel version I decided that it would be a good idea to update the kernel and all my other packages before proceeding. A pacman -Su identified 76 packages needing upgrades and started downloading (these are presumable the packages that were installed from the eight months old core ISO). Upon completion of the downloads pacman complained:

error: failed to commit transaction (conflicting files)

and listed a mass of files in /usr/lib/klibc/include/asm. A quick google revealed an announcement and fix for this: rm /usr/lib/klibc/include/asm. Having removed the symlink all 76 packages upgraded just fine.

VirtualBox Guest Additions: Take Two

Back to the VirtualBox Guest Additions; it seemed my only option now was to install them from the ISO. I could either use the ISO provided with my Windows version of VirtualBox or I could install the virtualbox-additions package which to my understanding would just download the same ISO for me; I’d still have to mount it and install the actual software myself. I decided to go with the ISO that came with my VirtualBox.

I mounted the ISO in the VirtualBox Media Manager but when attempting to mount /media/cdrom in Arch I was told that the cdrom was missing from /etc/fstab. This was perfectly true; the EEE PC has no optical drive so during Arch configuration I had left the cdrom commented out. At the time I hadn’t fully internalized the distinction between installing straight to the EEE PC vs installing on VirtualBox, and it hadn’t occurred to me that VirtualBox provided emulated optical drives. I uncommented the cdrom in /etc/fstab and it mounted without further drama.

The installation instructions for the VirtualBox Guest Additions were conflicting and confusing between the wiki, forums, and VirtualBox manual. The VirtualBox manual recommended installing dkms first, but the dkms package in AUR was flagged as out of date. I tried my luck with the instructions on the ArchWiki’s VirtualBox page instead; verify gcc and make are installed and sh /media/cdrom/VBoxLinuxAdditions-x86.run. The wiki warned of “errors about init scripts and run levels and what not” but to my great surprise everything built and installed with nary a complaint.

I rebooted, and then shutdown. After the shutdown Arch wouldn’t start; it would switch resolution to 800x600 but get stuck there with a black screen. I rebooted in fallback mode, removed vga=771 from /boot/grub/menu.lst and retried. Now that the boot process stayed in 640x480 I could see what was going wrong, although I little idea what it meant:

Code:  Bad EIP value.
EIP: [<00000000>] 0x0 SS:ESP 0068:c042fdf4
Kernel panic - not syncing: Fatal exception in interrupt

I guess someone doesn’t like zeroes. Restarted in fallback mode: same error this time! Normal mode: same error. Rebooted the EEE PC completely (this had helped previously with another VirtualBox problem): same error in both modes. I noted that there seemed to be some variation in when in the boot process the kernel panic occurred, although it was always very early. I had recently unplugged the EEE PC at which time it switched to its “Auto Power-Saving” mode with lower (and probably variable) clock speed. I speculated that maybe this mode was causing some random timing error (“not synching,” remember?) in the early bootup process. I changed to the “Super Performance” mode, rebooted Arch, and lo and behold, it worked! Once the boot was complete I switched back to the Auto Power-Saving mode.

I’ve had these kernel panics several times since. Also in Super Performance mode and with AC connected. They’re more frequent when power is disconnected and CPU throttled and I’ve rarely had two in a row with AC connected. I don’t think I had any of these before installing the VirtualBox Guest Additions.

Recommendation: Do not change the resolution of your fallback installation.

Recommendation: If you get kernel panics during bootup disable CPU throttling and if possible connect AC.

VirtualBox Guest Additions: Configuration

I added rc.vboxadd to /etc/rc.conf. During bootup the message

Starting VirtualBox host to guest time synchronization

is printed to the console which I presume means that rc.vboxadd-timesync is started by rc.vboxadd without having to be added individually to the daemons in /etc/rc.conf.

When shutting down Arch I noticed the error message:

/etc/rc.d/functions: line 167: /etc/rc.d/vboxadd-timesync: No such file or directory

The correct file name should be /etc/rc.d/rc.vboxadd-timesync. I tried to figure out what was the problem but wasn’t familiar enough with rc.d to be able to figure it out. In fact, I couldn’t even locate where the stop_daemon function where the error occured was called from.

Update 2009–02–20: Alan Savage figured this one out and submitted a patch for it.

Also, running /etc/rc.d/rc.vboxadd-timesync start (or stop) from the command line resulted in:

/etc/rc.d/rc.vboxadd-timesync: line 226: begin: command not found

The begin() function was indeed missing from the code block that defined functions for Arch so I copied the one from the Gentoo block and inserted it at line 169.

Update 2009–02–14: A better place to copy begin() from is the Arch-specific definition in rc.vboxadd. I’ve submitted a patch for this to the vbox-dev mailing list.

Configuring Xorg

Here are the contents of the xorg.conf file that VirtualBox Guest Additions installed:

# Default xorg.conf for Xorg 1.5+ without PCI_TXT_IDS_PATH enabled.
#
# This file was created by VirtualBox Additions installer as it
# was unable to find any existing configuration file for X.

Section "Device"
        Identifier      "VirtualBox Video Card"
        Driver          "vboxvideo"
EndSection

My impression was that Xorg was supposed to use hal to discover the video modes that the “VirtualBox Video Card” supported, and that the X screen resolution would seamlessly follow the VirtualBox window size. This didn’t work for me and I was stuck with 640x480 with no other modes available according to xrandr. I had to add the following to /etc/xorg.conf to get additional resolutions:

Section "Screen"
        Identifier       "Default Screen"
        Device           "VirtualBox Video Card"
        SubSection "Display"
                Modes   "1024x600" "800x600" "640x480" "1024x768"
        EndSubSection
EndSection

(As an aside I also tried generating an xorg.conf using hwd but when it didn’t use the vboxvideo driver (opting instead for vesa) I didn’t pursue that avenue further.)

The only other problem I’ve had with Xorg is that switching modes with Ctrl+Alt+Keyopad-Plus/Minus was unreliable at best at rendered the session unusable at worst. I disabled these keys with the DontZoom option as described in the xorg.conf manpage:

Section "ServerFlags"
        Option "DontZoom" "true"  # Disable mode-switching keys.
EndSection

Instead I use xrandr to switch modes, e.g:

xrandr --output VBOX1 --mode 1024x768

I have set up aliases for this, e.g.:

alias 1024x768='xrandr --output VBOX1 --mode 1024x768'

It’s a wrap!

That pretty much sums it up. Most everything seems to be in working order now and it’s trivial to install additional software with pacman as needed. The one thing I know I’ll want to figure out sooner or later is how to how to “hotswap” keyboard layouts but I’m in no hurry.

Arch Linux on EEE PC: Prelude

I’m a long-time (as in since the Motorola 68030 and System 7 days) Mac user and currently own a PowerBook G4 going on it’s fifth year. However, I’ve been following the emergence of netbooks heralded by the EEE PC with a fair amount of interest and in November finally bought myself an EEE PC 900HA (which I may do a mini-review of some day).

I wasn’t thrilled by the prospect of having Windows XP on it but I do live with Windows everyday at work, and given that I spend most of my time in a web browser or terminal window I figured it wouldn’t matter too much. I also told myself I’d install Linux on it fairly immediately. Well, now it’s February. I’ve been putting off the Linux install, struggling along with Cygwin instead. I’d have to say the upside of this was that I’ve had two months to try out Google Chrome. My impression is that Google got a lot of things right and managed to streamline the browsing experience quite a bit; now I’m not at all so content with Firefox and Safari…

Anyway, back on topic, today I decided to take some time to get Linux on the EEE PC. I’ve been deliberating for some time which distro to use; on the one hand there’s Ubuntu which I imagine is friendly as can be and probably a great distro all over, on the other hand I’m eager to try using xmonad as my window manager and drawn towards choosing a similarly bare bones distro to run it on. I’ve finally settled on the latter with Arch Linux; the main selling points, in my perception, being:

• Excellent Haskell support.
• The rolling release system.
• The package system. I’ve had a good experience with MacPorts, and Arch Linux’s pacman/ABS appears to be similar.
• The bare-bonesiness. If nothing else I hope to learn quite a bit from installing and configuring Arch Linux. I also imagine that by limiting the installation to what I use and need there’ll be less noise and more signal when I inevitably have to trouble-shoot something.

I’m working my way through the installation process as I type, and it is indeed a rather arduous process. The Beginners Guide to installing and configuring Arch Linux prints to a hefty 53 pages, not counting the 18 page appendix.

Ouch! It would appear I just screwed up by hitting the ESC key at an inopportune moment in the installer (my motor memory must have thought I was in vi). This appears to have bypassed steps which the guide prefaces by repeated italicized notes regarding their importance. Oh well, I guess that’s as good an excuse as any to cut off this post early and go to bed.

At a later date I’ll comment in more detail on the installation experience; what confused me, what worked, and what didn’t.