Layman's Guide to Computing - Season 12

Issue 156: Translation

2022-01-22T08:00:00+08:00

Previously: Translating a set of instructions before executing it will always lead to a slowdown, although sometimes this may not be noticeable to users.

So, just-in-time (JIT) compilation is really cool and mostly works. Feed in enough instructions to fill a buffer, and execute them. Keep your fingers crossed and hope the buffer doesn’t empty. That’s kind of how our global supply chain works too.

But sometimes it doesn’t go smoothly. The program hits a code branch, new instructions have to be unpredictably injected. The emulation layer halts temporarily. The program stutters.

We can’t really avoid that, not without rewriting the program anyway. But we can at least decide when to carry out the translation.

Ahead-of-time translation

What if we translated whatever we could ahead of time, and stored the native instructions? Then, whenever we need that chunk, instead of translating the original program chunk, we just load the already-translated instructions?

This is called ahead-of-time (AOT) translation, and is what Apple Rosetta 2 does with MacOS programs compiled for Intel x86-64. While installing those applications, it also carries out translation into native ARM instructions that the M1 later uses for execution.

Android AOT translation

The Java virtual machine (VM), also called the Java Runtime (Issue 151)), is the interpreter that carries out the Java bytecode that a Java program comprises.

Android apps, though themselves Java programs, are run not by the Java Runtime, but by the Android Runtime¹.

Whenever your Android phone finishes installing an Android update, there is always a significant block of time that it takes up “optimising” your apps. What it is doing is actually AOT translation, of the app’s Java bytecode into ARM instructions.

Compilers vs interpreters

Back in Issue 54), I mentioned in a footnote that

Purists will argue with me that Python technically runs through an interpreter, not a compiler. At this point, the distinction between the two terms for layfolks is not critical, and I choose clarity over accuracy at this point until I can delve into more detail in a future issue.

Here is where I can draw the distinction more clearly.

A compiler compiles a programming language into another language; usually this means translating a programming language into machine instructions, or virtual machine (VM) bytecode (Issue 150)). But it is also not unheard of for compilers to translate one programming language into another, e.g. translating to Javascript for use in a webpage.

An interpreter interprets a programming language, and executes it to bring about the intended effect (creating/changing/deleting a file, producing a sound, displaying an image, …).

It may be helpful to think of a compiler as carrying out AOT translation, and an interpreter as carrying out JIT translation. But many systems will use a mix of both. The Python interpreter, for instance, actually translates Python code into intermediate Python bytecode, and then executes the bytecode. When you run the same Python script, it executes the bytecode if the Python code has not changed. If it has changed, the interpreter will recompile the bytecode first, then execute it.

Issue summary: To speed up execution and avoid translation overhead, some systems employ ahead-of-time translation, storing the translated instructions to be executed in future. But many systems employ a mix of just-in-time (JIT) and ahead-of-time (AOT) techniques.

What I’ll be covering next

Next issue: [LMG S13] Issue 157: NTP and time-syncing

In the season finale, I’ll wrap up with a mishmash of things that are not the Internet per-se, but very much a part of the Internet and our lives. First up: how do our phones always know the actual time?

Sometime in the future: What is:

XSS? [Issue 8]
OpenType? And what are fonts anyway? [Issue 42]

Recall from Issue 151) that one needs a distribution license to distribute the Java runtime environment (JRE); Google would have to pay hefty licensing fees to Oracle to bundle the JRE with each copy of Android. Instead, they decided to write their own compatible runtime: the Android Runtime. Is this legal? They already fought that battle in court (Google LLC v. Oracle America, Inc), and it seems the answer is “yes” (with a heap of caveats). ↩

Issue 155: Emulation performance

2022-01-15T08:00:00+08:00

Previously: Programs that were not compiled for the instruction set of the host OS have to go through an emulation layer program. This program translates the instructions of that program into compatible instructions that its own processor can execute.

The Apple M1 is an ARM processor that executes 64-bit ARM instructions. MacOS programs that were compiled for Intel 64-bit x86-64 processors go through the Apple Rosetta 2 emulation layer to run on the M1.

Yes, that’s what I said last issue. But if that were all the Apple Rosetta 2 emulation layer did, The M1 Macbook would not have gotten its rave reviews.

The act of translation

Think about it: the ARM and x86-64 instruction sets are very different! They have different lengths, different instruction names and bit values, different concepts of operation, … they’re very different. While the Intel Core and ARM processor architectures share some similarities, translating instructions from one set to another, though possible, is still no simple feat.

Keep in mind that in addition to translating the x86-64 instructions to ARM, the M1 also has to execute them—the program must go on. It should not be surprising at all that there is a performance hit compared with executing native ARM instructions.

If you were given a set of execution instructions in your native language, you would have little difficulty carrying out the required task. But if the execution instructions are now in alien language? Suppose you were given translation instructions, in your native language, for understanding the alien language. You would definitely be doing it slower than if the execution instructions were in your native language.

Just-in-time translation

These days, our processors are fast enough that if they are not too bogged down, they can actually read in instructions slightly ahead of time, translate them, and store the translated instructions for a short period of time before they are executed. This way, the operating system remains ever so slightly ahead of the program being translated, in a way reminiscent of our currently strained global supply chain.

Naturally, it doesn’t make sense to keep doing translation work. As much as possible, we want to minimise it! But how?

Issue summary: Translating a set of instructions before executing it will always lead to a slowdown, although sometimes this may not be noticeable to users.

What I’ll be covering next

Next issue: [LMG S12] Issue 156: Translation

In the last issue of season 12, we talk about why app installation or system updates can take so long.

Sometime in the future: What is:

XSS? [Issue 8]
OpenType? And what are fonts anyway? [Issue 42]

Issue 154: Emulation

2022-01-08T08:00:00+08:00

Previously: The cloud offers standard digital business services, accessible through a web interface and API, which any developer (with a credit card) can use. Developers don’t have to reinvent the wheel, so long as they know how to use web APIs.

Virtualisation, particularly system virtualization, is a real game-changer for those of us who like to have our apps all running in the same operating system, instead of switching operating systems all the time through dual-booting (or Apple Parallels).

But what is stopping us from allowing them to run near-natively in the desktop, their windows directly showing up in the taskbar, without the distracting abstraction of the virtual machine?

Introduction to Emulation

What you are seeking is a feature known as emulation, in which your operating system (OS), which we shall again call the host, emulates the instruction set (Issue 53)) that the application is compiled for. In other words, the host OS:

presents itself as the “correct” machine type to the application (“hello program, I listen to x86 instructions and respond to x86 instructions, so please treat me like an x86 processor”),
transparently interprets its machine code into its instruction set’s machine code (through a program called an emulation layer),
executes the interpreted version, producing its intended effects, and returning any intended output back to the application.

Depending on how different the two instruction sets are, the complexity of this task differs greatly. Not much point going into detail here in a layman’s newsletter, so instead I’ll briefly illustrate some instances of emulation in the wild.

The three main instruction sets discussed here are x86 (32-bit), x86-64 (64-bit), and ARM¹ (Issue 53)).

Windows-to-Windows emulation (WOW64)

Do you remember the great 32-to-64-bit schism of the late 2000s (Issue 55))? There was a period of time when people got confused whether a Windows program they had could run on a 32-bit x86 processor or a 64-bit x86-64 processor: programs compiled for the latter could not run on the former, but programs compiled for the former could run on the latter.

Some download sites still ask you to make this choice between downloading the 32-bit or 64-bit version, usually for users who for whatever reason have opted not to upgrade to 64-bit processors.

That was a lie. Programs compiled for Windows on x86 cannot run natively on x86-64, and vice-versa. x86 and x86-64, while looking similar, are different instruction sets. x86 instructions have to be translated into x86-64 instructions to run on a 64-bit processor.

What happened was Microsoft developed the WOW64 subsystem, an emulation layer that translated 32-bit x86 instructions into 64-bit x86-64 instructions. When users tried to run a 32-bit application, Windows plugged the instruction stream into WOW64, executing the interpreted instructions and allowing it to run near-natively².

And so the 32-to-64-bit transition took place more smoothly than it would otherwise have.

Windows ARM emulation for x86

In 2019, Microsoft released the Surface Pro X, its second³ ARM-powered laptop. That’s right, it’s Windows not running on an Intel chip. Microsoft does actually have a version of Windows, called Windows ARM, which runs on ARM chips. But what about all the programs you know and love?

WOW64 to the rescue again! This emulation layer also translates 64-bit x86-64 instructions to 64-bit ARM instructions, allowing them to run on Windows ARM (with a performance penalty due to the translation required).

Game console emulation

If you find any “reborn” retro gaming products floating around, these are guaranteed to be emulators in disguise: the original hardware that the consoles used are no longer in production. (If you don’t find any, you can also just google “console emulation” to find a whole collection of them.)

These emulators are usually hobby projects by skilled amateurs, who attempt to reverse-engineer the workings of the original hardware. They then write programs to emulate these processors on modern hardware, allowing you to “boot” a digital copy of the games that worked on those platforms.

Apple Rosetta

The Apple M1 is an ARM processor that executes 64-bit ARM instructions. MacOS programs compiled for Intel 64-bit x86-64 processors must go through the Apple Rosetta 2 emulation layer to run on the M1. This works like WOW64 but in the reverse direction: it takes in an x86-64 instruction stream, and produces interpreted instructions for Apple ARM processors.

Issue summary: Programs that were not compiled for the instruction set of the host OS have to go through an emulation layer program. This program translates the instructions of that program into compatible instructions that its own processor can execute.

Simple to describe, much more difficult to execute … 😩

What I’ll be covering next

Next issue: [LMG S12] Issue 155: Emulation performance

Is this how emulation actually works? I wish! Don’t want to go too deep, but I think it is instructive to briefly discuss performance issues, and why it leads to some things you might have observed.

Sometime in the future: What is:

XSS? [Issue 8]
OpenType? And what are fonts anyway? [Issue 42]

ARM actually has a 32-bit instruction set—AArch32 and a 64-bit instruction set—AArch64, which are incompatible. But since Apple switched to AArch64 starting from the iPhone 5S (2013), other mobile device manufacturers have followed suit, and AArch64 is now the main instruction set used on mobile. In this issue, I use ARM to refer to AArch64. ↩
You can still see this happening with some old programs; this is usually indicated in the title bar as compatibility mode. ↩
Their first was the ill-fated Surface RT, which these days is only whispered about. ↩

Issue 153: Using the cloud

2022-01-01T08:00:00+08:00

Previously: Actually making a web application requires you to set up lots of supporting software and carry out lots of steps to create a suitable app environment.

Last issue, I described the whole host of things that need to be done just to make a web application work on another server, different from where you did your programming.

How do people deploy web services so quickly if there is so much tedium involved?

Birth of the Cloud

You could set up the environment, containerise it, and deploy it through a container … that’s one way to solve for distribution.

But somebody smart realised that this is likely a common problem. 99%¹ of applications are going to need the same building blocks: one or more databases (Season 7)), remote storage servers for storing large files (Issue 106)), web servers (Issue 8)), and of course the application itself.

All these pieces must themselves run on a machine (physical or virtual). The hardware, the network resources, the type of database, the type of storage, … these are usually not key differentiators for the business. They do not derive their business value by managing it differently from other companies.

Amazon was the first to realise that whatever they were doing to scale Amazon’s business globally, their competitors and other businesses would eventually need as well.

The Cloud is born

The Cloud is a collection of services that can be plugged in to an application, in lieu of writing your own code. Instead of implementing your own storage server, you could use Google Cloud Storage, or Amazon S3, or Microsoft Azure Storage, etc. You access these and other services typically through a web API (Issue 4)), and have the option to manage them through a web interface.

Google Cloud Storage web interface

Besides virtual machines (Issue 147)), cloud offerings span multiple layers of abstraction.

Google Cloud main offerings

Here are Google’s mainstay offerings:

Compute Engine: virtual machines
App Engine: a pre-configured, setup-free virtual machine that runs code which you upload, through a web API or their web interface
Kubernetes Engine: a hypervisor (Issue 148)) that runs your containers
Cloud Storage: a storage service for files (see earlier screenshot)
Cloud SQL: a relational database service (Issue 87))
Cloud Bigtable: a NoSQL database service (Issue 90))

Then there are variants for running big data queries, using machine learning nodes, and rebranded services for running the backends of mobile apps, … the key common factor here is that using these services is much simpler than rolling and maintaining your own version!² And it lets you speed up development by not having to reinvent the wheel that cloud services have implemented for you.

The Cloud grows

As web applications got larger and larger, beyond the capacity of even a single high-end server to manage, they had to be redesigned so that they could run on multiple servers while maintaining data synchronicity. As businesses standardise on ways to do that, cloud providers add these tools as part of their offerings.

For example, sending/receiving messages between servers is a key engineering problem. Data packets sometimes get dropped en-route, or when a server gets overloaded. Sometimes they get held up at a server, time out, and then they get resent by the client (Issue 9)); the server receives two identical requests, and sends two identical responses, resulting in a duplicate response.

If you don’t want to write your own software for managing communication between servers, the cloud lets you write code for your machines to communicate easily, without having to crack your head thinking about how to make it happen.

Issue summary: The cloud offers standard digital business services, accessible through a web interface and API, which any developer (with a credit card) can use. Developers don’t have to reinvent the wheel, so long as they know how to use web APIs.

There is, of course, much more to developing an application than just gluing services together. For commercial applications, you still have compliance requirements, logging, monitoring, and other things to set up. But these are not new needs, and not really worth going into detail in a layman’s newsletter.

I hope the gist of what the cloud does is at least clearer!

What I’ll be covering next

Next issue: [LMG S12] Issue 154: Emulation

I am done with virtualisation and the cloud at this point. You’ve learned about hardware virtualisation (through drivers), system virtualisation (through system VMs), process virtualisation (through process VMs), and service virtualisation (through APIs) so far this season.

I’m going to use the last three issues to talk about a related and current thing: instruction translation and emulation. Let’s start with a question: How is the Apple M1, an ARM chip, able to run MacOS programs compiled for the Intel x86-64 chips? Aren’t they two very different instruction sets (Issue 53))?

Yes, yes they are. More next issue ;)

Sometime in the future: What is:

XSS? [Issue 8]
OpenType? And what are fonts anyway? [Issue 42]

Illustrative but not accurate estimation! ↩
Provided your use case is in the 99-percentile! If yours is an edge case, you might find that rolling your own version is better value for money in the long run. ↩

Issue 152: Getting started with programming

2021-12-25T08:00:00+08:00

Previously: The Java Runtime Environment (JRE) bundles the Java VM and supporting libraries. The JRE has to be installed on the user’s system for Java programs to work, unless the program bundles the supporting libraries. Solo programmers can start programming with OpenJDK for free with fewer features and less support, while commercial companies can license Oracle JDK for better support and features.

So you started taking up programming. Maybe you went to a class, where everything was set up for you and you didn’t have to worry about installing and configuring necessary software. Or you took an online course, where step-by-step instructions were provided and you mostly didn’t have to spend time scratching your head. That’s how it should be; you paid to learn programming, not to learn how to configure a software development environment.

Once you actually have to start writing code though …

Setting up a development environment

Let’s suppose you are writing code for a web application. You start by installing the software for the compiler/interpreter program, which either executes your code directly or compiles it into an executable binary (Issue 54)).

Now you begin writing your code. Along the way, you begin to install various libraries (Issue 17)) and frameworks (Issue 18)) that your code relies on. These are provided in things called packages, which are basically zipped files containing all the files and metadata for the library/framework. You install these packages using another program, called a package manager, through the command line terminal (Issue 15)).

You write more code. And one day … you’re done! The real pain has just begun!

Deploying code

Thus far, you have been programming on your own laptop. But your laptop can’t handle a full webserver load once people start using your app, so you wisely decided to lease a virtual machine (VM: see Issue 147)) from a cloud provider instead. You boot up the VM, it goes through its bootup process (Issue 112)), and finally completes. You are greeted with a familiar command line, the text cursor blinking cheerfully.

How are you going to get your code on that machine?

Maybe you set up a code repository (Issue 19)) on another server, and then download your code onto the VM with some commands.

Maybe you decide to turn your app into a package instead: you write software instructions (more code!) to tell the package manager (mentioned earlier) how to install the package, and how to configure everything. You add some files with metadata, a file manifest containing all the files used by the package, and then pack it up. You install yet more software (called build tools) to help you automate this part. Then you set up another file server, upload the package onto it from your laptop, download it from the VM, and install it on the VM using the same package manager software.

You test it, and after many hours of cursing, confused pacing and mumbling, and much hair-tearing, it finally works. Phew!

Expanding the app

Unfortunately, the tiny toy server that you used to test your web app doesn’t hold up to real-world network loads. You’ll need to put the app behind a Real™ web server; as more and more users use it, you may even need to deploy your app to multiple servers to handle the load, all managed by a load balancer. The balancer receives the web requests (Issue 9)), decides which of the multiple servers has the lowest load so far, and directs the request to it so it can serve a web response (Issue 8)).

Deploying more servers … does that mean you have to do the above all over again?!

Issue summary: Actually making a web application requires you to set up lots of supporting software and carry out lots of steps to create a suitable app environment.

What I’ll be covering next

Next issue: [LMG S12] Issue 153: Using the cloud

I hope this issue adequately describes the problem that the cloud attempted to resolve! When people talk about “using the cloud”, what is that actually like?

Next issue, I’ll give a peek. Short issue guaranteed.

Sometime in the future: What is:

XSS? [Issue 8]
OpenType? And what are fonts anyway? [Issue 42]

Issue 151: the Java VM

2021-12-18T08:00:00+08:00

Previously: System VMs provide a set of virtualised hardware that the OS interacts with. Process VMs provide a set of libraries that a program (written in that programming language) interacts with.

If the Java VM lets us write programs that work across multiple Oses, why don’t we write everything in Java then?

Actually a lot of enterprises do! But there are some tradeoffs to make this work.

What’s bundled

The code required to make a Java program run on every operating system is not simple; somehow all this complexity needs to make its way to the target computer, whether it uses it or not.

Ideally, we have one part: the program you want to distribute, bundled and delivered to the user. The other part, like an adapter (the Java VM), allows your program to work on the target computer.

Up to this point I have given the impression that the Java VM is all that is needed for the adapter to work. This is inaccurate; the Java VM interprets the program instructions (in an intermediate set of instructions called bytecode). It still needs a whole set of supporting libraries to enable interfacing with the OS.

The Java VM and these supporting libraries are installed in a software package called the Java Runtime Environment (JRE). If you hear the term “runtime environment”, this is what it refers to: the program plus the libraries it needs to run.

If you want to write Java programs, you’ll need more than the JRE though. you will need supporting tools especially for debugging, and these are provided in the Java Development Kit (JDK). This is not only true for Java, but for many other languages as well; if you are getting into programming, and asked to pick a download, you usually want the one that says “development kit”.

Tradeoffs in distribution

If you want to keep the software bundle size small, you’ll have to ask the user to install the JRE separately (about ~80MB downloaded, more when installed). If you don’t want your users to face that hassle, you’ll have to take on the work of bundling the required libraries into your program yourself. Are you going to bundle libraries for all targeted OSes, or just for one particular platform? The more you bundle the larger the size …

Nobody said multiplatform support is easy!

The legalese

Java is free to use—but only on a personal basis. Once you intend to distribute your program, and maybe even make money, the issue of licensing rears its head.

Java was originally created at Sun Microsystems in 1995, but was acquired by Oracle in 2010. They do enforce their licensing pretty strictly, so be prepared to pay for the convenience!

Java and open-source

If the only way to distribute programs with multiplatform support is to pay a licensing fee, Java would see much lower takeup, which would hurt long-term profits … so a big portion of the Java core is open-sourced as OpenJDK. This is free to use and extend, and many businesses have been successfully releasing software based on it. But it is going to be a lot more work carrying out testing and writing code for features which are not provided.

On top of OpenJDK, Oracle JDK—Oracle’s commercial release of the JDK—adds some proprietary code, plus lots of enterprise support and testing. In general you’re going to have a much easier time writing your code with Oracle JDK instead of OpenJDK—that is how Oracle makes money!

If you just want to get familiar with the language, or are working on a personal project, OpenJDK lets you do so for free, legally. If you are a corporation trying to get your engineers to write code that runs on multiple platforms so as to simplify your systems, Oracle JDK helps you to save time doing that, for a fee.

It’s the best of both worlds.

Issue summary: The Java Runtime Environment (JRE) bundles the Java VM and supporting libraries. The JRE has to be installed on the user’s system for Java programs to work, unless the program bundles the supporting libraries. Solo programmers can start programming with OpenJDK for free with fewer features and less support, while commercial companies can license Oracle JDK for better support and features.

What I’ll be covering next

Next issue: [LMG S12] Issue 152: Getting started with programming

Hold on, I thought—

Nah, Layman’s Guide to Computing has not suddenly switched to being a Guide to Programming! We are still on the topic of the cloud and the history of commercial computing. Where I last stopped, I was talking about how co-hosting started from virtual-hardware VM rental, and then added containerisation to its bag of tools. Then I segued into talking about process virtualisation and the Java VM (last issue & this issue).

A short note though: process virtualisation isn’t a co-hosting tool, but a programming tool. We use process virtualisation while writing programs, not while deploying programs to the cloud. And I do want to get back to talking about the cloud.

But before I do that, there’s one thing that the average layperson would not be familiar with, and which I need to talk about:

Why is it so difficult to do programming, even with experience? Yes, do programming, not merely learn it!

Sometime in the future: What is:

XSS? [Issue 8]
a good reason developers write code and give it away for free online? [Issue 21]
OpenType? And what are fonts anyway? [Issue 42]

Issue 150: System VMs vs Process VMs

2021-12-11T08:00:00+08:00

Previously: Containers are one layer of virtualisation above virtual machines: containerisation systems virtualise access to the operating system, presenting a virtual interface that provides software with the resources it needs, without being aware of software running in other containers on the same system.

Recap

If I need to configure an entire machine to install and configure my own operating system (OS), I can rent a virtual machine — this is system virtualisation.

If I just need to run a set of software on a particular OS but don’t want the hassle of managing the rest of the OS, I can containerise them for the OS — this is containerisation.

What if I want to write software that will work the same on multiple OSes? Is that possible?

Before 1995, no!

OS-level details are tricky

OSes mediate access to resources through libraries (Issue 17)). Instead of having to deal with storage disk sectors and blocks, you can just use library features to ask the OS to help you create a file and write data to it. The OS and filesystem (Issue 106)) take care of the details.

I don’t know how to explain this more clearly without showing some programming code; the OS controls a lot of things! Which is a problem when it comes to multi-OS support, because different OSes control different things differently.

The library features for lots of things — storage, searching through directories, networking, etc — are different between Windows, Linux, MacOS, …

Abstracting away the details

Can’t we program in a higher-level language, and use another program (the code interpreter) to break it down to different types of operations on different operating systems? E.g. python.exe on Windows will try to make my Python code work in Windows by using Windows libraries, python on Linux will do so using Linux libraries, and python on MacOS will do so using macOS libraries. Javascript, Ruby, Perl, VBScript, and other interpreted programming languages work this way too¹.

To a large extent, this is possible. But there remain some irreconcilable differences:

if you need to use Python to run a command-line program, such as a shell script (Issue 16)), those will depend on terminal availability: not all terminals have cross-OS support!
If you need to specify a file location, Windows and Linux do not use the same path separators (the character that separates folder names in a folder hierarchy); Windows uses \ to separate directory names, while Linux and MacOS uses /. This is often a source of bugs and headaches.
Important advanced features, such as multiprocessing (on multiple CPU cores simultaneously), are handled differently in different OSes in ways that may be incompatible within the same program.

Virtualising processes vs virtualising hardware

So it looks like we need something more. We need a programming language that can be interpreted into intermediate instructions (these are called bytecode), and we need a program for each OS that can carry out these intermediate instructions.

That program is also called a virtual machine (VM), but this is not hardware virtualisation. It is process virtualisation.

The OS interacts with virtual hardware without directly accessing the underlying hardware. Similarly, we want our code to interact with a virtual process instead of using OS libraries directly.

This is how the Java programming language works. It provides a (process) VM, called the Java VM, for each operating system. Your Java program interacts with the Java VM only; it mediates all access to the operating system.

Issue summary: System VMs provide a set of virtualised hardware that the OS interacts with. Process VMs provide a set of libraries that a program (written in that programming language) interacts with.

What I’ll be covering next

Next issue: [LMG S12] Issue 151: the Java VM

The idea sounds pretty cool … so why don’t we write more programs this way? And what is the Java VM anyway?

Light discussion on code-adjacent issues next issue!

Sometime in the future: What is:

XSS? [Issue 8]
a good reason developers write code and give it away for free online? [Issue 21]
OpenType? And what are fonts anyway? [Issue 42]

As opposed to compiled programming languages (Issue 54)), where code is compiled into CPU instructions for one hardware platform for one OS only. ↩

Issue 149: History of commercial computing - containerisation

2021-12-04T08:00:00+08:00

Previously: Renting out virtual hardware instead of physical hardware meant that instead of having to move hardware around and manage it, you could send the data for running an OS to the hosting company and have them be responsible for hardware operations.

Business concerns

Every business computer you have encountered likely runs an operating system (OS). And yet, what value does managing the OS have for the business? They have business software to run—point-of-sale systems, accounting systems, communication systems e.g. email—but the OS is no big concern for them, as long as it runs the software!

If I have point-of-sale software that only runs in Windows, and I’m paying for a company to provide it as a service, I don’t care if the software actually runs on Windows with direct hardware access, or if it is doing it through a virtual machine, so long as it works.

If I am going to help other companies run point-of-sale systems, one way to do it securely (so that data from different terminals/companies do not mix) would be to run a separate virtual machine (VM) for each system. I would do this through a hypervisor, a specialised thing (software/firmware/hardware) meant for running VMs.

Virtualising OSes

This is kind of wasteful; I am running multiple versions of the same OS to support as many copies of the point-of-sale system, even though they can all run on the same OS. I am just loathe to do so for security reasons.

What if we could apply virtualisation one layer up: instead of just virtualising hardware, we virtualise the OS?

To recap: when we virtualise the hardware, we provide virtual drivers that the OS can accept as valid hardware.

But programs don’t need real or virtual hardware; they need OS libraries which provide common resources and services: network, storage, compute, memory, windowing/display. If we can provide virtual libraries which respond like the actual OS libraries would, the programs would be able to run as normal.

The technology that enables OSes to let programs think they are running exclusively, protected from other programs, is called OS-level virtualisation, but more widely referred to as containerisation.

An OS (with the appropriate software & support) can run multiple containers, each container acting like a sandbox (Issue 92)) for the software within. In each container, software has access to OS features, but are unable to affect software outside of the container. Each container appears to have exclusive access to (a portion of) the system’s network, storage, compute, and memory resources.

Managing containers

The word containers may seem like a misnomer, for what are effectively software wrappers. But these do work almost like shipping containers: set up a container in an OS, install the required software, configure it, and now you have a container ready to meet business needs. You could send this container to any virtualisation service, they drop that container into their hosting system, and it runs like you expect.

Issue summary: Containers are one layer of virtualisation above virtual machines: containerisation systems virtualise access to the operating system, presenting a virtual interface that provides software with the resources it needs, without being aware of software running in other containers on the same system.

What I’ll be covering next

Next issue: [LMG S12] Issue 150: System vs Process VMs

Up to this point, we have been looking at two different kinds of virtualisation: system virtualisation (virtualising hardware), and containerisation (virtualising operating system environments).

Next issue, we examine a third kind: process virtualisation.

Sometime in the future: What is:

XSS? [Issue 8]
a good reason developers write code and give it away for free online? [Issue 21]
OpenType? And what are fonts anyway? [Issue 42]

Issue 148: History of commercial computing - cohosting

2021-11-27T08:00:00+08:00

Previously: Running a virtual machine is like running a physical machine, but within a window in your OS.

Co-located hosting

A not-so-long time ago, to run a website, you literally just ran a webserver on your desktop, connected it to the internet, and gave your IP address to other people. This is a pretty unreliable way to host a business website though. A big company would make business arrangements to procure a reliable internet connection, set up the infrastructure (power, cooling, mounting hardware) required to run multiple computers, and then manage their multiple systems with a full IT management team (hardware & software).

Not every company can afford this. Smaller companies would therefore co-locate their computers (called colo boxes) with bigger companies, enjoying service support and infrastructure for a monthly/yearly fee. Some companies decided to just provide these services as their full-time business, and the hosting business was born.

The difficulties of troubleshooting remotely

Running your computer on someone else’s premises is no joke. If something went wrong, there was no way to do troubleshooting remotely. You had to drive down and do the troubleshooting onsite, usually wasting at least half a day in the process. Unsurprisingly, this was a problem many companies were happy to abstract away by paying more money. Soon, hosting companies offered to rent your their computers, configured to standard specs, and provide basic onsite troubleshooting. You would email/send them the software you wanted to run on those computers, with instructions, and they would do it for you. A huge timesaver; most hardware issues are now out of mind for business owners.

You still had to worry about OS issues though. If you need to have multiple pieces of software set up and configured, this was something a hosting company could not do for you. And this was where virtual machines (VMs) came in handy. What if you could set up a virtual machine, with virtual CPU and virtual memory, install your OS on a virtual disk, install all your required software in that OS, and then send that virtual disk (as data) to a hosting company? They would then run that disk through their hypervisor, a piece of software that manages virtual machines.

The promise of virtualisation

This was the promise that virtualisation companies offered to businesses. You could manage your virtual machines remotely, choosing when to boot them up or shut them off, paying only for virtual hardware you requested, without affecting the virtual machines of other companies. Hosting companies could “collect rent” for multiple VMs running on a single computer.

After all, as long as you have an interface to manage it, and you are able to set up your software on it, does it really matter whether it is a physical or virtual machine?

Issue summary: Renting out virtual hardware instead of physical hardware meant that instead of having to move hardware around and manage it, you could send the data for running an OS to the hosting company and have them be responsible for hardware operations.

What I’ll be covering next

Next issue: [LMG S12] Issue 149: History of commercial computing - containerisation

Can we push this further? Could we get hosting companies to not only help us run the hardware, but the operating system as well? Yes, yes we can!

Sometime in the future: What is:

XSS? [Issue 8]
a good reason developers write code and give it away for free online? [Issue 21]
OpenType? And what are fonts anyway? [Issue 42]

Issue 147: Operating systems on virtual hardware

2021-11-20T08:00:00+08:00

Previously: Virtual hardware can be created in the form of drivers that respond to a program’s requests for hardware resources. If a bootup program enumerates hardware devices and receives a response, then as long as it continues to receive valid and correct responses, it can work with the virtual hardware to run an operating system.

So … what is it like to run an operating system (OS) on virtual hardware? I promised screenshots, but they probably won’t be as exciting as you expect—it looks quite normal!

Creating a virtual machine

I don’t want to purchase a VMware license, so I will be using an alternative virtual machine product instead: Oracle’s free Virtualbox. This is what it looks like, running on Arch Linux on my laptop:

Virtualbox main interface

Let’s create a new virtual machine. It asks which OS I intend to use, presumably so it can pick the right virtual hardware to use (MacOS virtual machines may be more picky? I’m not too sure here either). I don’t have a valid Windows license to use, so I’m just going to demo with Arch Linux instead.

Whichever OS you pick, you are going to have to find a way to install that OS into your virtual machine—whether you are using actual boot media (such as a DVD drive), or virtual media.

Creating a virtual machine

Cool, you actually get to configure how much of your computer’s memory the virtual machine will get to use.

Configuring memory size

Without a storage disk, a computer isn’t much. Virtualbox needs something it can use as a disk; the usual way is to create a file on your system that Virtualbox uses as a virtual disk. The space occupied by this virtual disk can be preallocated up front (so that your virtual machine doesn’t accidentally “run out of storage space” before the disk is full), or it can be dynamically allocated, only taking up as much space as is actually used.

I don’t have an existing virtual disk file, so I will ahead and create one here (not shown). If I have one from a previous installation, I can use it here instead.

Configuring hard disk

The machine is created; the main interface now looks like this. Note that all the “hardware” you see there is virtual!

Virtualbox main interface, with one virtual machine

If you are doing this on your own computer, at this point you might want to go into Settings and see what else you can toy around with: number of (virtual) CPUs, sharing some folders on your system with the virtual machine (they show up as shared network folders), adding more disks or even a virtual optical drive, etc.

Setting up boot media

I could try to boot it now, but I already know it won’t work; there is nothing in the hard disk to boot from. With a physical computer, at this point we will attempt to install the OS from a DVD drive or flash drive. You could allow your virtual machine to access the DVD drive or flash drive in order to do this, or you can do it virtually. Most operating systems (including Windows) provide virtual boot media for installation: an ISO file is a virtual optical disk.

I can download the ISO boot file for Arch Linux, but it seems they have gotten savvier lately and actually provide virtual boot media in the form of virtual hard disks! Let’s use that instead. I add it as a second virtual disk in Virtualbox:

Virtualbox hard disk configuration. packer-virtualbox.vmdk is the virtual boot media for Arch Linux

Once I start the virtual machine, it begins its boot sequence, and I interrupt it by pressing F12 to go to the boot menu (otherwise it will attempt to boot from the main virtual hard disk, and fail to detect any OS). Interesting to see here that it isn’t actually connected to a monitor: virtualbox presents a virtual display device, to which the virtual machine sends its video signal. Virtualbox captures these signals and displays them within the window instead. So you can flexibly configure the window size, and the virtual machine just thinks the display device it is connected to has been resized.

Virtualbox boot menu. Through the virtual (AHCI) disk controller, the virtual machine detects two (virtual) disks: the new disk I created, and the Arch Linux boot media that I loaded.

Running a virtual machine

I select the second disk to boot up Arch Linux for installation, and the login prompt appears:

The Arch Linux login prompt, in Virtualbox. Yes, this is how Arch Linux gets installed the first time.

I wont go any further at this point, because then I’d just be showing you how to set up Arch Linux. But I trust this is enough to give you an idea: running a virtual machine feels just like running a physical machine, but in a window!

Okay hold on, how do we shut this thing down? It is not recommended for a computer to be unplugged without a proper shutdown, so … how do we do that with a virtual machine?

Even the powerdown and reset buttons on a computer are actually hardware signals which the OS receives and uses to trigger a series of actions. We can send these signals virtually too, through Virtualbox’s Machine menu:

The Virtualbox Machine menu.

And with that, let’s shut it down.

Issue summary: Running a virtual machine is like running a physical machine, but within a window in your OS.

What I’ll be covering next

Next issue: [LMG S12] Issue 148: History of commercial computing - cohosting

Right, so this has been really cool and all, but not something a layperson would use on a daily basis usually. And the setup still seems … rather technical? So why does this deserve its own season?

Remember that this happened before the turn of the century, so the tech industry has had two decades to figure out how to make money out of this. And two decades is the equivalent of a whole lifetime in this industry. How did virtualisation change the landscape of commercial computing? Time to take another walk in recent history.

Sometime in the future: What is:

XSS? [Issue 8]
a good reason developers write code and give it away for free online? [Issue 21]
OpenType? And what are fonts anyway? [Issue 42]

Issue 146: Virtual hardware

2021-11-13T08:00:00+08:00

Previously: Programs do not usually deal with the gnarly details of hardware, but instead access it through an interface. They access storage devices through a filesystem, and access hardware through drivers.

How does one trick an operating system (OS) into coexisting with other operating systems on a single machine? By virtualising hardware into virtual drivers!

Virtual network hardware

Let’s take an example. Take a look at your network devices: there’s one for your LAN port, there’s one for your wifi card, and these days there may be one each for your Bluetooth chip and 4G/5G modem too.

If your workplace requires a VPN, you may have noticed that it adds a new network device. How is this possible if you don’t actually have any new network hardware?

Remember that in an operating system, access to hardware is mediated through drivers. A driver decides how to present its interfaces to the operating system. In essence, nothing prevents a driver from presenting multiple interfaces to the OS, provided the driver is able to receive requests and respond to them.

A VPN uses its own driver to present an additional interface to the OS, and that is how we end up with “virtual hardware”.

Virtual storage

We are used to thinking of storage as referring to a hard disk, or solid state disk. But technically anything that is capable of representing bits can be used as storage—with an appropriate driver.

Some operating systems/programs provide drivers for RAM disks—a storage disk that uses computer memory. These appear as a normal disk drive (in Windows) or mountpoint (in Linux). Managing files in a RAM disk is speedy, because computer memory is much faster than a storage device.

Virtual memory

In Issue 55), I explained how the operating system offers and controls access to computer memory, the pagefile (Issue 117)), as well as hardware devices through a single addressing interface: virtual memory.

When a program requests access to the printer and the OS responds with “here, you can send your request to memory address 0x35a4b2ff”, how is it to know if the data is going to a physical printer, or to a virtual one¹?

Virtual hardware

Take a look at your Device Manager in Control Panel. What do you see?

Device Manager in Windows 10
Note that what you are seeing are not the actual hardware (which the OS cannot possibly know). These are interfaces to the hardware.

A whole set of drivers and interfaces which the OS uses to carry out its work.

Many of these were initialised during bootup (Issue 112)), when the OS kernel (the core of the OS) enumerates the available hardware by sending out signals and seeing what hardware responds.

So a bunch of engineers at VMware thought: what if we … made drivers to present virtual hardware emulating the CPU, memory, storage devices, … and even the chipset? What if we then we booted the BIOS (the bootup program loaded on a computer’s mainboard; see Issue 112)), got the virtual hardware to respond when the BIOS enumerates hardware, and then basically simulated all the signals that hardware would actually send?

We end up with a virtual machine—one that you can actually install an OS on!

Issue summary: Virtual hardware can be created in the form of drivers that respond to a program’s requests for hardware resources. If a bootup program enumerates hardware devices and receives a response, then as long as it continues to receive valid and correct responses, it can work with the virtual hardware to run an operating system.

What I’ll be covering next

Next issue: [LMG S12] Issue 147: Operating systems on virtual hardware

So … what is it like to run an operating system on virtual hardware? Screenshots incoming!

Sometime in the future: What is:

XSS? [Issue 8]
a good reason developers write code and give it away for free online? [Issue 21]
OpenType? And what are fonts anyway? [Issue 42]

These virtual printers do, in fact, exist. It is why some OSes offer a “Print to PDF” printer device: the program effectively sends print commands to another program, which interprets the commands to produce a PDF file. This is possible because both printers and the PDF format share a common language: Postscript (see Issue 51)). ↩

Issue 145: What an app wants, what an app needs

2021-11-06T08:00:00+08:00

Previously: In 1999, VMware launched VMware Workstation, which allowed multiple operating systems to run off a single machine.

In Season 5 (Issue 53)), I went into some detail on how our programs work. The programming language they are written in gets compiled into CPU instructions, which get carried out by the CPU.

But the program does not handle everything on its own. In fact, it does not usually have direct access to hardware (unless requested from and provided by the operating system (OS)). All such access is abstracted and mediated through the operating system. The program requests and receives memory space, reads and writes files, and processes keyboard/mouse input—through the operating system.

How does the program know it is not living in a simulation? If the environment it operates in responds to its requests, the program continues running without a care. A keypress event is a keypress event, and the program can respond to it, whether it really came from a keyboard or not. A file that has data to read can be treated like a file, whether it is really stored on the hard disk or actually streamed from cloud storage.

Computing interfaces

This paradigm is really powerful, because it enables us to pipe data from place to place. It enables us to build interfaces: instead of writing a program that has to grapple with the gnarly details of files in binary format, I can write a program that deals with a file interface instead. The interface lets me read and write data, while the gnarly details are handled one layer down, by the filesystem.

And this is how we can open documents from a flash drive without even realising that the flash drive uses a different filesystem from our system disk.

The OS is just another special set of programs that mediate access to the hardware. Yet, it is easy to forget that even the OS’s many programs do not actually deal with the gnarly details of hardware, but with an interface to the hardware. This interface are the drivers (Issue 120)) we keep hearing about, and which our OS keeps telling us could not be found or are outdated.

As long as the “drivers” respond in the right way, the operating system continues to carry out its instructions as programmed.

How does one trick an operating system into coexisting with other operating systems on a single machine?

Issue summary: Programs do not usually deal with the gnarly details of hardware, but instead access it through an interface. They access storage devices through a filesystem, and access hardware through drivers.

What I’ll be covering next

Next issue: [LMG S12] Issue 146: Virtual hardware

Examples incoming!

Sometime in the future: What is:

XSS? [Issue 8]
a good reason developers write code and give it away for free online? [Issue 21]
OpenType? And what are fonts anyway? [Issue 42]

Issue 144: Programs-in-a-vat

2021-10-30T08:00:00+08:00

Previously: The Apple M1 is a souped-up iPhone processor, with unified memory.

I want to circle back to talking about processors again in this season, because there are a couple of pretty world-shaking ideas I haven’t fully fleshed out in Layman’s Guide yet.

One of them is—hmm where do I begin. As early as 1641, in Meditations on First Philosophy, Descartes proposes that “All that up to the present time I have accepted as most true and certain I have learned either from the senses or through the senses; but it is sometimes proved to me that these senses are deceptive, and it is wiser not to trust entirely to anything by which we have once been deceived.” In other words, Descartes isn’t always sure that he believes what he sees; his senses sometimes deceive him about the nature of reality.

More than three centuries later, in 1999, the Wachowski brothers translate this idea into a more modern form: what if the world as we know it is a simulation running on some other cosmic, otherworldly hardware? Is it possible to signal to our senses so convincingly that a simulacrum may be thought of as real?

Hold that thought, because this is the Layman’s Guide to Computing, not Philosophy (although there is plenty of that in Computing as well!). For millennials like me who came of age in the early noughties, The Matrix defined the zeitgeist for the next couple of decades, whatever one may think of its aesthetics. It is difficult to overcredit it with this.

VMware is founded

But allow me to put a little dent in that reputation: the year is 1998, in Palo Alto, California. A company named VMware had just been founded. One year later, in 1999, they demoed and launched VMware Workstation, their first product. In that demo, engineers demonstrated how VMware Workstation could allow users to run MS-DOS, Linux, FreeBSD, and multiple versions of Windows—that’s multiple operating systems (OS)—off a single machine.

The release of VMware Workstation probably happened too late to influence the direction of The Matrix, but no doubt there was talk in the air of virtualising OSes in the years leading up to it. To be clear, this is not dual-booting, in which a user can choose, through a boot menu, which OS to boot into (Issue 112). We are talking about multiple OSes, running simultaneously, off a single machine.

How does such a thing happen?

Issue summary: In 1999, VMware launched VMware Workstation, which allowed multiple operating systems to run off a single machine.

What I’ll be covering next

Next issue: [LMG S12] Issue 145: What an app wants, what an app needs

What enables an operating system to function, convinced it has full control of hardware? For that matter, how would it even know if the hardware … is really hardware?

… 👻

Sometime in the future: What is:

XSS? [Issue 8]
a good reason developers write code and give it away for free online? [Issue 21]
OpenType? And what are fonts anyway? [Issue 42]