Mike’s Journal

… is growing slowly, but most people still aren’t making the connection between high oil prices and the fundamental shift in production rates that is peak oil. This image shows Google searches for “oil price” (red) against “peak oil” (blue):

Yay, it finally happened. Google released protocol buffers as an open source library.

Protocol buffers are one of those ideas that has a million different implementations but very few good implementations. As a result everybody invents their own and we lose interopability and time for no good reason. Protocol buffers more or less hit the sweet spot between features and simplicity - I certainly wouldn’t want to design a file format or network protocol without them these days, and now I won’t have to, even if I’m doing work outside of Google.

Here’s a quick intro to what they are.

Protocol buffers are binary XML

Well, sort of. XML is more complicated than protocol buffers are - protobufs have no concept of namespaces, attributes vs subelements, character escaping, DTDs or any of the other things that make XML complicated. But the essential idea is the same - it’s a way of representing trees of structured data in such a way that they can be extended in a backwards compatible manner.

The key features of protocol buffers are:

• A very efficient yet simple binary encoding. A minimal protobuf takes only a few bytes due to clever use of variable-length integers.
• A simple specification syntax that lets you define a schema far more easily than an XML Schema would be.
• A compiler that produces objects representing your structures in either C++, Python or Java. These objects present a much cleaner and simpler interface than the XML DOM - it’s quite feasible to represent all your programs internal state this way, whereas it’d be painful to replace a native object heirarchy with XML.
• A set of tools that let you quickly serialize and deserialize them.
• A lightweight (incomplete) set of interfaces that let you hook protobufs up to an RPC system.

Usage in RPC

Protocol buffers are so named because they were developed as a wire protocol for server communication within Google. Over time this developed into a full high performance RPC system that supports many advanced features. In particular it’s very easy to debug and troubleshoot, a feature I find invaluable day in, day out. At heart though, this RPC system is still based on protocol buffers - the RPC protocols are defined using them.

The key feature here is that it’s very easy to extend protocols based on protobufs over time. Alternatives like CORBA, DCOM or Ice don’t have a particularly elegant approach to this - if you want to introduce a new parameter to an RPC for instance, you need to introduce a whole new interface, and then translate the call through. In a protobuf based system, you just mark the new field as optional and then use the has_foo() function before accessing it in your new server. When the client is ready to use the new parameter, one set_foo() call is all it takes.

The next key feature is that it’s very efficient on the wire. Protocols like SOAP or XML-RPC are really not designed for efficiency at all. In many cases, this won’t be a big deal, but for Google it is because we push everything so hard.

Protocol buffers as a file format

Because they efficiently serialize to binary, can be used from at least 3 key languages (and more in future) and can be extended over time, protocol buffers are a perfect fit for many file formats. Most open source programs these days have based their file formats on XML. OpenOffice, AbiWord, Inkscape and more all use XML markup languages to save their data. Because XML tends to be very large, they often zip it, resulting in a very slow and complicated piece of code to load or save these files. This matters - a big part of the complexity of the Word/Excel formats is due to “quicksave”, a feature that users love as it lets auto-save be less intrusive and more frequent, but which complicates the codebase considerably.

Protocol buffers have a more or less optimal binary representation and can be deserialized into in-memory objects extremely fast. Nothing stops you gzipping part or all of them if you want to eliminate some redundancy, but it’ll be redundancy on the application level that is eliminated, not on the format level.

Go check it out

Seriously, it’ll only take a few minutes to read the docs, and you’ll add a valuable tool to your arsenal. Whilst the problem that protocol buffers solve isn’t new, this is one of the best implementations I’ve seen so far. I hope the open source community embraces this system as a way to make easier to use, more efficient file formats and network protocols for its applications.

I moved my blog to mikehearn.wordpress.com - I got tired of having to maintain it myself. If you notice any problems let me know.

Last time, I talked about the basics of Singularity: SIPs, manifests, connections, and software isolation. Let’s have a look at how this applies to driver development.

Drivers are a Big Deal for us, because:

• About 85% of all Windows crashes are caused by bad drivers
• An analysis of Linux showed that driver code is 7 times more likely to contain bugs than the rest of the kernel
• When drivers crash, they take the whole system with them, thus losing the users work and - by extension - users trust.
• When drivers contain bugs, they can sometimes be used for privilege escalation. It only takes one privilege escalation for the bad guys to win.

So getting drivers right would be a huge win, both for reliability, and for building a foundation on which we can get security right too.

drivers in singularity

Because Singularity is a type system based micro-kernel, a driver is simply a set of Sing# classes (Sing# is C# with extra bits), shipped as verifiable MSIL bytecode images. It runs as a SIP, like any other program does. Obviously, using C# instantly solves a large class of common bugs - pretty much any exploit with “overflow” in its name is solved by using these languages.

You can’t represent the specialized instructions needed to control hardware in safe MSIL, so we need a workaround. This comes in the form of an unsafe DLL which provides objects abstracting the hardware control instructions. This DLL (Singularity.DriverRuntime) is a part of what Microsoft Research call the “trusted computing base”, but that makes it sound like it’s to do with Palladium and DRM, which it isn’t. It’s easier to think of this DLL just as logically part of the kernel, even though it doesn’t run in the context of the main kernel program. These classes have names like IoPortRange, IoIrqRange and so on, and provide a simple OO interface to programming hardware.

Singularity won’t let you construct objects of these types yourself, you have to use a set of special static method calls. And you can’t invoke them directly either. It seems we’re stuck - how do we get a reference to such an object, if we can’t construct them ourselves? And what’s the point of being so awkward?

The answer is that Singularity requires you to annotate your code with metadata, describing what hardware your driver is for and what resources it needs. This is done in the usual .NET fashion, with attributes that decorate your classes and fields. At install time, this metadata is used to build an XML manifest, describing what resources your code needs, and is input to a compile time transform, which fills out an empty method you provide. This transform is itself a part of the kernel, and is applied when you install the driver - it is what adds the protected call to the hardware abstraction DLL.

In this way, Singularity ensures that it has accurate declarative metadata on what exactly the driver needs. Because you can’t get hold of the relevant objects and thus program the hardware unless you define accurate metadata (and this is enforced by the system), the metadata is guaranteed to be accurate. That, in turn, makes it easy to solve problems like avoiding driver conflicts and figuring out the order in which drivers should be loaded at bootup.

the hole in the plan

This would all be fine and dandy, if it weren’t for The Catch (why is there always a catch?). The catch is this - any driver that can access a DMA capable device can subvert the type system by overwriting arbitrary areas of physical memory.

Direct Memory Access is an optimisation that lets hardware devices directly write data into RAM, bypassing the CPU. It’s very useful and speeds things up a lot, especially when moving lots of data around like with graphics cards, hard disks, network cards and so on. Unfortunately, DMA not only bypasses the CPU but also the MMU, thus a driver that can control a DMA capable piece of hardware can control the contents of memory. In a regular operating system that doesn’t matter, because such drivers are running in kernel mode anyway and are fully trusted. But for Singularity it’s a problem.

Fortunately for us, DRM comes to the rescue! Yes, I’m serious. Newer CPU architectures being developed by Intel and AMD feature something called, appropriately enough, an IOMMU. This does for hardware devices what the regular MMU does for the CPU - regulates memory access by mapping DMA read/writes through a set of page tables. By configuring the IOMMU, you can stop hardware from fiddling with memory. This feature was originally intended to stop people breaking into kernel space by plugging in physical pieces of hardware, which apart from the obvious downsides for DRM manufacturers can also be used to do things like unlock screens. And of course it protects computers from buggy hardware, which is not such an uncommon thing.

It’s also mighty handy for us, because it closes the hole that would mean DMA-aware drivers could break out of their type-verified jail.

how it helps security

Before you can make something secure, you need a threat model. What exactly do we want to stop?

I’m going to skip over that here and mumble something about stopping malware and viruses. It’s vague and not that useful, but it’ll help illustrate these examples.

Back in 2001 the idea of Microsoft restricting what software you could put on your system seemed grotesque. They were just coming to the tail end of the the browser wars, broadband was just beginning its rollout and the general problem of botnets had not yet reared its ugly head. The potential for abuse seemed way too high, the benefits way too low. Despite that, drivers were known to be such huge reliability problems that a signing program was instigated …. Microsoft would provide you with a unit test suite, and if it passed, that suite would sign the driver for you. The test suite checked for many common problems and didn’t require anybody to hand over their code to Microsoft. Seemed like a good idea, right?

Well, of course it was controversial. Worse, it didn’t necessarily work. Rather than pass the tests, some manufacturers cheated. Rather than risk a schedule slip, or pay the $250 verification cost, some moved the mouse and clicked OK on the unsigned driver warning for you. D’oh.

If we’re willing to be a bit despotic and require signing that cannot be disabled for some things, can we do better? Yes, but it would suck to require every driver to be signed. We can avoid this in the most common cases.

For instance, thanks to the declarative metadata, we can now statically inspect a driver and say, well, gee, this driver will only be activated for devices of class “sound card”, and it doesn’t use DMA, and it doesn’t read or write any files (we know this because we can statically verify that it can’t get any connections to the filesystem server). What can this driver do, except mess up the sound card? Not a whole lot, actually. It could still hose the box by misprogramming the hardware, but such bugs are rare. But it can’t blue-screen us, because it’s not running in the actual kernel. If it crashes, the worst that happens is audio is interrupted until the driver restarts and programs re-establish their connections (assuming we write software to handle driver crashes of course - more on that later).

Generally, we can define some set of privileges (in Singularity this is equivalent to defining some set of connections a SIP can access) which might be dangerous, and only require signing for those drivers. What might trigger such a requirement? Accessing system files or configuration is an obvious case. Being a network driver is another case (how to DoS a target without falling foul of the firewall? Become a network driver). Fortunately, i
t’s rare that drivers need access to anything other than the hardware and their own configuration data, and network drivers are typically uninteresting enough that the default drivers suffice for most people. Thus the majority of drivers can be safely written, distributed and installed without Microsoft ever needing to be involved whilst preserving the security of the system.

Last time I talked about Singularity I rambled about micro-kernels for a while. You probably knew that stuff backwards anyway. Onwards!

**How does it work?**

Singularity pulls off the impressive feat of being a pure microkernel design that nonetheless is ~30% faster than the traditional approach, and about 10% faster than the monolithic approach for a file-io heavy benchmark. How does it do that?

The trick is very simple - they just throw away hardware memory protection entirely. In Singularity, everything runs in kernel mode, and everything runs in the same address space. The MMU, in other words, doesn’t do anything. There are no “processes” in the traditional sense.

Obviously, if that was the _only_ thing they do it wouldn’t be very interesting. 85% of Windows blue-screens are caused by drivers, not kernel bugs (I suspect the rest are caused by hardware failure). Innumerable privilege escalation vulnerabilities can be caused by bad drivers. Every privilege escalation is a gift to the bad guys. Protection against software bugs is what makes micro-kernels useful.

Programs in Singularity _are_ isolated from each other, but the isolation is done entirely in software, using type theory instead of silicon. They can do this, because programs in Singularity are all implemented in a C# derivative called Sing# (although in theory any .NET language could be used, Singularity uses a few features that they added to C#, so other languages would need the same minor extensions).

You already know that in most modern languages like Java and C# you can’t access memory directly. Most obviously, there’s no way to write the following C in Java:

*((char *)0×1234) = ‘X’; // overwrite the byte at location 1234

It’s not just that there’s no syntax for it. It’s that the Java/C# compilers produce a form of assembly language that can be checked by a bytecode verifier to mathematically prove that the program does not do this. Once proven, we can translate the JVM/.NET opcodes into actual code that the CPU can run, confident that if we don’t give the code a reference to an object, it can’t read or write to it. That’s not only good for reliability - we can build a security system on top of that!

**exceptions to the above**

OK, so that was the theory. In practice it wasn’t so easy. Just like theoretically “sandboxed” programs written in C can escape the sandbox by exploiting kernel bugs, the same is true of Java and .NET - by exploiting bugs in the VM or native class libraries (written in C++) the security system can be compromised.

Worse, in some cases, features like reflection can be used to access objects that people did not expect.

Both these problems have caused breaches in JVM applet security in the past. Given how huge the modern JREs are, that isn’t surprising.

**the solution**

Singularities solution is again straightforward - almost all of the system is itself written in C#, with only a small amount written in C++ and assembly language. This immediately makes the core “kernel” robust against the most common types of attacks. Even then, Microsoft are working on research that will let them prove the safety of the tiny remaining pieces of unsafe code.

The building block in Singularity is called a SIP, for “software isolated process”. Singularity retains the idea of a process being something that owns resources, has its own memory space and is independently scheduled, but it’s all enforced with software and mathematics.

A SIP has its own heap, its own garbage collector (you can choose one from several the OS offers, reflecting the fact that no one GC fits all situations), and its own memory pages. So it’s more isolated than some similar approaches (like KaffeOS), in which all code is loaded into a single runtime, and objects can be exchanged between different programs. This has the disadvantage that you can’t easily exchange objects between two SIPs. It has the pragmatic advantage that SIPs can be quite different from each other - not only using different garbage collectors, but also different language runtimes and in-memory layouts. And it means they can be deallocated quickly, without doing a full-heap GC, which could potentially be extremely slow.

Given that we’re no longer using the CPUs hardware modes, it’s no longer clear how we should define what the “kernel” really is. In Singularity, the “kernel” is the software component that connects SIPs together, does memory management, loads new SIPs and handles some other tasks. Logically, it also includes the trusted/unsafe parts of the system written in C++ or assembly. Some of these are actually attached to a SIP, like the garbage collectors, but because they are just trusted to be correct they can be thought of as a part of the kernel.

SIPs call into the kernel simply by doing a regular function call. You need a way to mark the stack so the kernels own garbage collector doesn’t interefere with the SIPs garbage collector, but that’s easy and fast. Thus all the syscall overhead is avoided.

SIPs communicate via “channels”. These are message based pipes, sort of like UNIX sockets, except strongly typed and faster. A channel is actually a mathematical abstraction - sending a message via such a pipe does not involve any copying or fancy hardware tricks. It’s simply updating a few pointers in memory. Because of this, sending even very large amounts of data between SIPs (say between the network driver and a web server) is fast.

**enforcing security**

It might seem odd that invoking the kernel is just a regular function call. Surely that’s not possible? What stops a SIP from simply invoking the instructions to control the hard disk itself?

The answer is that a SIP is shipped (say, on CD) as a set of MSIL bytecode files. These files are not compiled at runtime as in a regular Java or .NET system. Instead, when the software is installed, it is compiled ahead of time into native code after being statically checked using a variety of analyses. Software installation is a privileged operation in Singularity - it’s handled by the kernel itself. Only software installed by the kernel will be allowed to run.

Because you can’t represent CPU specific instructions like “write to this IO port” or “trigger this interrupt” in safe MSIL, the only way for a piece of code to do that is to be linked with some trusted native library that will do it for you. Because the kernel is in charge of software installation, it can verify that only certain software is linked with such libraries, and even then, only in certain ways. Thus, the kernel can control access to hardware resources without relying on hardware checks - by controlling who has access to the CPU (and how) at install time.

**manifests**

In a regular OS, a process more or less corresponds to a program. A few programs have multiple processes, for instance, iTunes installs a program that navel gazes until you plug in an iPod. But generally speaking, one program uses up one process, and actually it’s quite common for one process to contain more than one program - for instance, a web browser that hosts plugins.

Because SIPs are so cheap, it’s reasonable - encouraged, even - to split a single program into several co-operating SIPs. Thus we have a problem - how do we start a program? What even _is_ a program in such a setup?

The answer is that a program in Singularity is defined by a manifest. A manifest is (like in .NET) an XML file describing the SIPs that make up a program, and the connections between them. A manifest is mostly auto-generated from metadata annotations in the code. When you start a program, you actually invoke a manifest.

**drivers in singularity**

This has interesting implications for how drivers are managed. Unfortunately, I blew my word count again. It’ll have to wait for next time.

I want to write some stuff about Microsoft Singularity. It’s cool and everybody with an interest in computing should be talking about it. Here’s a summary for those who don’t want to read all the papers.

**what is it**

Singularity is an operating system research project. It’s a team of smart people who were told “what might an operating system look like, if it was designed from the ground up for dependability”.

People on pop forums like Slashdot and OSNews have been wishing for _years_ that Microsoft would throw away Windows and start from scratch, to address problems like reliability and malware. Usually their wish revolves around rebasing Windows onto some form of UNIX, but that’s a crap idea and wouldn’t actually achieve their wish at all. If you want to address problems that are caused by fundamental design decisions, you need to revisit them. This is what Singularity does.

Dependability is a pretty broad topic. At minimum it means not crashing, and it means being secure. But, although the Singularity researchers are exploring many topics, they don’t have a wide-open mandate … it’s not chartered to do GUI research for instance.

**how is it different?**

That’s what I want to talk about here.

Singularity is a high performance, single address space microkernel design, which uses static type verification to enforce reliability properties and flexible pattern based ACLs.

Oooh, there’s a lot of scary academic talk in that sentence. Let’s figure out what it means. This is going to be complicated, because Singularity is pretty different to textbook OS designs.

It’s _high performance_. Performance isn’t actually a goal of the project, but the researchers are smart enough to realise that if they don’t keep it in mind, their research will wander into the weeds and become completely uncommercialisable. There are hints that Microsoft are thinking of one day using this research in real products, so it’s important to be fast (more on that later).

It’s also a _microkernel_.

For now let’s just focus on the fact that it’s a microkernel. We can cover the other things in future blog posts. Skip to the bottom if you already know this stuff - I’ll assume here that you, my dear reader, aren’t entirely sure what a microkernel is or why they are supposed to be slow.

**a microwhat?**

Historically, there are two ways to design an operating system, which are actually the same way but using different cost:benefit analyses for certain decisions. These are whether to use a microkernel, or a monolithic kernel. Note that we’re talking low level designs here … this stuff is all independent of whether you use a task bar or a dock in your UI.

Recall that in a microkernel design, subsystems like the filesystem and network drivers run as more-or-less regular programs outside the kernel itself (which is distinguished from other programs bascially by running in a special CPU mode). The kernel proper only handles starting processes/threads, sending messages between them and a small number of misc things like CPU scheduling. True microkernels are hard to find these days. You’ve probably used them without realising it - for instance, QNX is an operating system designed for embedded applications like Cisco routers, and QNX is a pure microkernel design.

In a monolithic design, filesystems, drivers and even web servers are all loaded into the kernel itself and all run in privileged code. The kernel still provides message passing systems for user-mode processes to communicate, but they aren’t used anywhere near as much. Every mainstream server or desktop OS is monolithic - Windows, Linux and MacOS. Note that whilst Linux has always been monolithic, Windows NT started out as a microkernel, and MacOS X - being based on Mach - is theoretically one today. I don’t know anybody who believes that though.

It can be hard to say whether a particular system is truly a microkernel or a monolithic design, because it’s not a boolean yes/no thing - for instance, Linux runs its graphics subsystem in a separate process (the X server) whereas Windows _used_ to do that but doesn’t do it anymore. Nonetheless, everybody agrees that Linux is not a microkernel. A good smell test is whether the filesystems are running in kernel mode or not - graphics can be a grey area, but the filesystems are generally not.

Anyway, Singularity being a microkernel might seem strange, because historically the debate has in academia always been won by microkernels and in the market has always been won by monolithic kernels, largely for performance reasons. These arguments were going strong in the 80s and you can read the infamous Torvalds vs Tanenbaum debate on it here. So at first it might appear that Singularity is just another academic exploration of the theoretically clean thing to do, at the cost of real world usability. But it’s not so.

He won the debate

**why are microkernels slower than monolithic kernels?**

Microkernels are typically slower than monolithic kernels because there is a cost associated with transitioning between user mode, kernel mode, and back again. What’s more, there’s an additional cost for switching the CPU between two user mode processes: a context switch.

These costs are small but real, and when you do bazillions of them per second can come to completely dominate the CPU such that you’re not getting any actual work done. Measuring those costs is hard, although the Singularity team have managed it.

The reason they cost precious time is because the CPU has to do unusual work to make them happen, and because the majority of the CPUs time is spent _not_ doing that unusual work, it tends to not be well optimized (this has changed in recent generations of x86 chips, but the general point holds).

For example, when invoke a syscall to make the kernel do something, you use a special CPU instruction. That used to be “int $80″ on Linux but these days you can use the “sysenter” opcode on kernels and x86 CPUs that support it (nearly all do). Control then transitions to the kernel. This is pretty fast on modern computers, but it wasn’t always so - and in fact early versions of Windows actually abused an illegal instruction because they found triggering a CPU exception was a faster way to get into kernel mode than using an interrupt (the official way). Intel fixed that
Context switching is more expensive, firstly because it obviously involves a transition to kernel space, so you pay for the cost of doing that, but mostly because reconfiguring the page tables is slow.

Reconfiguring the page tables is slow partly because, again, it’s an unusual operation (it involves poking special registers on x86 chips), but mostly because it requires flushing the _translation lookaside buffers_. These buffers cache the result of the MMUs lookup. Even though MMUs are custom designed hardware and very fast, they’re still not free and yet the translations are needed every time code accesses memory, which is all the time, thus caching makes a lot of sense.

This also makes it hard to quantify exactly what a context switch costs you. We know it costs _something_ because of CPU design fundamentals, but the actual cost is spread out over the code in the new process to run. Immediately after a context switch then, your computer is running a little bit slower, and then picks up steam as the TLB fills up.

So we have two conflicting priorities here. On one hand, using virtual memory to separate address spaces can improve reliability by insulating programs from each other, which is good, but on the other hand, it costs us some hard-to-measure amount of performance, which is bad. Worse, although it’s true that CPUs have got faster over time, they got faster at running code and not at doing address space manipulations, so we can’t rely on Moores Law to bail us out this time.

message passing

Micro-kernels are based on the idea of sending messages between processes running in separate address spaces. Thus to read a file, first you have to send a message from your program to the filesystem server. This means formatting the message in your own memory space (fast), invoking the “send message” syscall (not quite as fast), the kernel then copies the message into its own address space (sorta slow), does a context switch to the filesystem server (slow), and then copies the message into the filesystem server memory space before leaving kernel mode.

Once the filesystem reads the data, you have to do the whole thing in reverse but this time copying the data back in a message …. because the cost of a message send goes up with its size, this is even slower than the initial request!

In contrast, in a monolithic design, you format your request (fast), do a read syscall (not quite as fast), wait whilst the filesystem gets your data, the kernel then copies your data into your memory space (or perhaps the hardware will do that if you’re using DMA) and returns to user mode …. wow, simpler and faster! The disadvantage is that if the filesystem code is buggy, you blue-screen and lose everything.

About 80% of Windows crashes are caused by crappy drivers. So, if you could stop drivers from exploding the system in the same way we can stop apps, we can eliminate 80% of the worlds blue screens! That’s pretty cool. It also means that drivers could have security enforced. Today if you install a 3rd party filesystem, who knows what you’re getting? Unless you review and compile the code yourself, you just have to trust whoever gave it to you. Even if they mean well, a bug in the new driver can open a local root exploit, compromising the entire security system
It’s probably not surprising then that academia preferred the slow-but-robust solution, and desktop OS vendors preferred the fast-but-unstable solution. It’s not that they didn’t try! Windows NT started out as a pure microkernel solution, but even with super-optimized IPC they eventually gave up and moved the whole shebang into kernel space, including the GUI, which they got a lot of stick for but made Windows feel snappier and more responsive thus making users happy.

the singularity approach

Singularity manages to have its cake, and eat it. It gets both the robustness benefits of a microkernel, and manages to get even better performance than a monolithic kernel. Neat!

But I wrote too much above. How it pulls off this trick will have to wait for next time.

cpu photo by nadya peek

Here’s a note for the interwebs. The GigaSet SE 551 adsl/cable router firmware is really quite nice, and gets a lot of things right. Sadly, the same cannot be said for the people who run their website.

Unfortunately, like all software, you may need to upgrade it from time to time. The firmware is shipped on their site as a Windows EXE, despite consisting only of a readme file and the BIN that you upload into the router. What is a humble Linux or Mac user to do?

Fortunately, the Windows EXE is only a Winzip self extractor. Here’s the magic incantation to get your grubby paws on it using Linux:

Probably, a similar trick will work on MacOS.

Looks like Saudi Arabia stopped another planned al-Qaeda attack. From the article:

The conservative Muslim kingdom also said it arrested 208 suspected Al-Qaeda militants over the past few months who were plotting assassinations and an attack on a logistical oil facility.

208 seems absurdly high, and given the KSAs rather dubious standards of justice, it’s unlikely there were actually 208 actual terrorists involved.

Nonetheless, the risk remains of a suicide attack on key pieces of infrastructure such as the Abqaiq stabilisation facility, or the refinery at Ras Tanura. I last wrote about this in April - to recap, approximately 6-7% of the worlds oil supply passes through Abqaiq, and any attack that managed to shut it down would cause an instant oil shock in the west.

That makes it a very juicy target for anybody who doesn’t like us. Back then, only a fairly ragtag bunch of terrorists were trying to damage it. Now we have another risk - if Iran is attacked it will fire missiles not at our ships (which they can easily destroy) but at Abqaiq or a part of the KSA pipeline network. A clever strategic strike for sure, because it doesn’t hurt soldiers …. it hurts the average US citizen, in the wallet.

The Saudi regime takes this threat seriously, and is building up a dedicated army to protect the petroleum infrastructure:

Saudi security officials say the force now has about 9,000 of its planned 35,000 troops. Still to come are a helicopter force crucial for pipeline defense and an air-defense system designed to thwart both suicide aircraft and missiles.

In other news, oil brings in the new year by passing $100/barrel. Cheers!

The oil is running out. I talked about this on my blog before. Oil has thousands of uses, but one dominates all the others - petroleum manufacture for our vehicles. Because of this, it’s commonly assumed that leaving behind oil means leaving behind petroleum too … and perhaps even cars. But does it?

New technology currently in the R&D labs suggests another idea - manufacturing our own petroleum in a speeded up, industrial scale version of the same process that nature uses.

Question is, how?

German researchers are working on replicating photosynthesis, with the goal of converting atmospheric CO2 into CO. CO and hydrogen are the main components of syngas which can be used to manufacture gasoline. They’ve managed it too, but their current reaction uses energy from benzene and has low yield. They need to replace that energy input with something else and increase the efficiency for it to be a viable building block for manufacturing gasoline.

Another approach is being pushed by VC firms in the Valley: LS9 is a company genetically engineering microbes to create petrol from biomass. It’s a bit like ethanol manufacture, except that ethanol isn’t backwards compatible with our existing infrastructure (pipelines and engines all need modifications). LS9 claim they have been able to build microbes that eat sugar and expel gasoline … a remarkable feat if true. But like all biofuels, it’ll be constrained by the size of the biomass input. Their website is pretty vague about what sort of inputs the process can take. Even if it can eat more or less anything, there’ll still be huge issues with scaling up such an approach - just moving the biomass to the refineries is as yet an open question, viability wise. They think they’ll be competitive by 2010.

Why it matters

The fact that both techniques can (in theory) produce petroleum is a huge deal. Other than the fact that using it shifts huge quantities of carbon from the ground into the air, petroleum is a really great fuel. It’s amazingly energy dense, easy to transport, and we have a lot of infrastructure built to handle it.

Of both, the German approach appeals to me more. Photosynthesis isn’t particularly efficient and the LS9 approach still relies on it. Although making petroleum instead of ethanol is a big leap forward, as it solves the sticky backwards compatibility issue, the other problems with biofuels (land use, etc) are still open.

Sucking CO2 straight from the air using mechanized, optimized and industrial-scale processes seems like a better way forward. Whilst the air doesn’t contain a whole lot of CO2 proportionally, and using plants lets you cover large areas of land with ease, the ability to build sun-powered carbon-neutral gasoline factories is a very tempting prospect. I need to run the numbers some time on how much air you’d need to process to create a gallon of gasoline, to see how feasible such a scheme might be.

Well, the time has finally arrived. The first OLPC units are rolling off the production line.

Previously on this blog I’ve talked about (enthusiastically!) the Sugar user interface. It’s certainly quite cool, and takes into account a lot of the lessons learned over the past twenty years in UI design. The hardware seems pretty interesting too - in particular, the way they’ve got the laptops doing bazillions of suspend/resume cycles with no glitches (Apple could learn from that). But, this doesn’t mean I’m as enthusiastic about the rest of the project.

I installed the XO QEmu image and had a play around with it. Note, the fonts will be super-small if you try this, due to the different resolution of the OLPC screen. The wiki recommends changing your X config, but I found if I do that I can no longer launch all the apps, as the “app bar” at the bottom just clips icons that don’t fit, because the whole UI scales up. A better way is to press Alt-0, su to root (password is blank) and then edit /usr/share/sugar/sugar-xo.gtkrc to up the font size to something better, like 17. Then kill X and fonts will be readable, without scaling the whole UI.

My concerns about the way OLPC is being run aren’t original but I’ll reiterate them here anyway:

• There is, as far as I can tell, no user testing at all. Negroponte is apparently convinced testing isn’t necessary because laptops in education have already been tried and - he says - had great results. Maybe so. But the OLPC is very different even to those projects. But when you’re asking governments to pony up huge sums of cash, you owe it to them to prove the benefits of what you’re selling beyond all reasonable doubt.
• In particular, the Sugar UI does not seem to have been tested on the target audience, which isn’t even well defined (other than being “children”). I’d originally expected multi-year trials before the mass manufacturing stage, but no - they’re ploughing right into it, convinced they got it right the first time. To some extent the UI is separate from the hardware, but what if they find they got it wrong? What if they find the “view source” button is confusing and people hit it accidentally? What if the Sugar UI, despite feeling right to geeks like me, actually is a dud? It’s a bit late to go back to the drawing board now.
• The apps being shipped with the latest builds don’t seem planned - seems they are just a mishmash of things thrown together based on who volunteered to write them. There doesn’t seem to be any coherent educational plan to them. For instance, an RSS feed reader is included, as is a Tetris clone, but there are no mathematical games or geography products.
• There seems to be an overemphasis on learning programming. That bothers me.

  Squeak EToys is included, complete with a “haha, i am a demon and you must escape this castle” tutorial game, and of course there is the famous view source button on the keyboard. The point of all this was never clear to me. Most people aren’t going to become computer programmers, and the ones that are will need direction and help from adults.

  I remember when I learned programming, I needed a lot of help from my father to understand even a very simple language like BASIC (I was about 6 or 7, I think). Later on I knew enough (and had the motivation) to learn from books and online. I decided to find out how feasible it might be for young children to actually hack on the activities they are given. I chose BlockParty, partly because games are always what appeal to kids and partly because it’s the first on the app bar.

  You can find the BlockParty source code in /usr/share/activities/BlockParty.activity/BlockParty.py or on the web here. In the very earliest days of the Sugar project, I jumped on IRC and asked why Python was being used for all the code. After all, XOs are very hardware constrained and Python isn’t known for being efficient. I was told Python was the language of choice because it’d allow children to understand and edit the source code of their laptop. Let’s see how that worked out in practice.

  First thing you notice is that BlockParty is completely uncommented. Seriously, the only comment in the entire file is the copyright notice at the top. If by some chance a child can read English they won’t find any help in the code itself.

  Second thing is that using Python doesn’t really make the code easier to understand. Maybe a bit easier than if it was written in C++, but not much. Code is code, and there are still plenty of lines like these to puzzle over:

  pix_data = “”"/* XPM */ static char * invisible_xpm[] = {”1 1 1 1 c None”};”"”
  bwpx,bhpx,score,bw,bh,cnt=0,0,0,11,20,0
  msg = “csound.SetChannel(’sfplay.%d.on’, 1)\n” % self.csid

  You get the picture. This is advanced code that interacts with sound servers, graphics libraries, widget toolkits and so on. To put it bluntly, I am not sold on sacrificing efficiency (and thus unit price) for some vague, unplanned, untested vision of small children learning programming from it. I think they should have stuck with C++.

My conclusion from all of this is that the XO seems like a small replica of the Linux community: a bunch of smart people hacking on good ideas, but without any overall focus or master plan, and an over-strong focus on the importance of programming. That’s not surprising - it is a small replica of the Linux community. But the point of OLPC is not to be free or open source, or a platform for UI research, or even to teach programming - actually, it’s to be an educational aid for the school-age curriculum.