ToaruOS 2.0

[klange]

ToaruOS has consumed the greater part of my life for the last eleven years. This release has been a long time coming: My first plans for a 64-bit, SMP-capable port of the OS date back to before the 1.0 release. The repository in which I built the new kernel was something I had already set up as far back as 2015. And now, finally, I think it is ready.

What is ToaruOS?

ToaruOS is a complete hobby operating system for x86-64 PCs. ToaruOS is made up of several components:

• Misaka, an SMP-enabled kernel for x86-64. Misaka is the main focus of this release, as it replaces the older "toaru32" kernel from 1.x.
• Yutani, a software-compositing windowing system. Yutani has been a core part of ToaruOS since very early in its development, and this release brings a number of improvements to the windowing protocol and compositor implementation.
• Kuroko, a bytecode-compiled interpreted programming language. Kuroko first appeared in ToaruOS 1.14 as a replacement for Python, and has improved greatly since it made its debut.
• Bim, a Vim-inspired code editor with syntax highlighting and plugins powered by Kuroko. All code written for ToaruOS in the last few years has been written in Bim.
• Graphical applications for Yutani, including a file browser, terminal emulator, widget-powered panel, and a package manager.
• A vast collection of terminal utilities, including a large subset of standard POSIX utilities.
• The ToaruOS libc, a cleanroom implementation of the C standard library. ToaruOS's libc has proven itself by supporting a considerable set of complex ports.
• A full suite of userspace libraries to provide support for PNG and JPEG images, TrueType fonts, and more.
• A dynamic linker (ld.so).
• A set of bootloaders for BIOS and EFI systems.

All software components of ToaruOS are original to the project.

What's New in ToaruOS 2.0?

There are two ways to look at 2.0: First, as a follow up to 1.14.1. Compared to the last "stable" release, the changes in 2.0 are extensive but iterative. Alternatively, 2.0 can be compared to 1.0. That release was built on a number of third-party elements, and 2.0 demonstrates how those elements have been successfully replaced.

Features of Misaka

• x86-64 architecture support and a far more portable base than its predecessor.
• Support for symmetric multiprocessing (SMP).
• VFS and process support for filesystem groups and supplementary group lists.
• Per-thread and per-core CPU usage tracking.
• Support for gzip-compressed ramdisks.
• Loadable driver modules, which are now loaded from the ramdisk.
• Process tracing with ptrace, providing visibility into system calls and signal delivery, single-step debugging, and cross-process memory access.
• A newly-rewritten network stack providing a BSD-style sockets API, support for multiple interfaces, a loopback interface, and listening sockets.

Changes Since 1.14.1

• New userspace utilities exposing functionality added in Misaka: top, strace, dbg, ping, cpuwidget.
• Several improvements have been made to the graphics library, including the addition of affine transformations.
• Several improvements have been made to the window compositing framework.
• A new TrueType text rasterizer has been added.
• A new marked-up text formatting library has been added.
• The BIOS loader has been improved and should support a wider range of hardware configurations (including hard disk boot), the EFI loader has been rewritten, and both now support editing the kernel command line.
• The panel has been redesigned, and widgets have been improved with loaded libraries, dynamic layout, and new popups.
• The Julia set viewer has been rewritten with new palettes and exploration features.
• A new calculator app has been added, powered by Kuroko.
• The libc now supports basic timezone configuration. Please set your system RTC to UTC for accurate time.
• The timezone offset and panel weather widget are automatically configured by location services, when available.
• Driver support has been added for the Ensoniq ES1371 chipset emulated by VMware Workstation.
• Many more things I lost track of.

Known Issues

• The new network stack remains experimental and the TCP implementation is missing functionality I hope to deliver in a future update.
• As in previous releases, ToaruOS's libc remains incomplete. Some functions may be provided as stubs, or may not accept all expected options.
• There are known issues with the ext2 implementation and, while possible, disk installations are not recommended.

What's Next?

Several features were deferred to future releases in order to avoid scope creep and get 2.0 finalized. My roadmap for the future looks something like this:

• 2.0.x: Various bug fixes and functionality that was almost ready for 2.0, such as TCP stack improvements.
• 2.1: AHCI, xHCI, USB HID devices, and other improved hardware support that was specifically deferred.
• 2.2: AArch64 port.

Running ToaruOS

About the Live CD

ToaruOS is distributed as a "live CD". The default configuration will boot into a user session for local. The password for this account is also local, and it is permitted to use the sudo command. When prompted for a password by either the graphical or command-line sudo utilities, enter local. The live CD is configured to provide an in-memory read-write filesystem and packages can be installed up to the limits of system memory, but no data will persist between boots. As in previous releases of ToaruOS, an experimental read-write ext2 implementation is available (insmod /mod/ext2.ko) and drivers for IDE hard disks are provided (insmod /mod/ata.ko). While these drivers have seen significant improvements since previous releases, their use on real systems is not recommended.

Virtual Machines

I generally recommend using a virtual machine such as QEMU, VirtualBox or VMware Workstation to run ToaruOS, as driver support for real hardware configurations is limited.

QEMU can be configured as follows:

qemu-system-x86_64 -cdrom image.iso -m 1G -device AC97 -enable-kvm

You may also wish to use multiple virtualized CPUs with -smp 2. For platforms other than Linux, substitute an appropriate command line flag for -enable-kvm to enable hardware virtualization.

For VirtualBox, be sure to provide 32MB of display memory and configure an Intel NIC.

ToaruOS 2.0 has also been tested in VMware Player.

Real Hardware

Users wishing to try ToaruOS 2.0 on real hardware are recommended to use GRUB. Booting ToaruOS 2.0 is notably simplified over 1.x, as modules no longer need to be loaded by the bootloader to achieve a functioning system. Extract the kernel and ramdisk files from the release CD and place them in a path accessible from your GRUB loader, then add a menu entry with the following commands:

multiboot2 /kernel root=/dev/ram0 vid=auto migrate
module2 /ramdisk.igz
set gfxpayload=keep # You may need to specify an appropriate mode here.

[klange]

Known Issues

• The Tutorial references Python as an installable package. No Python build is available for 2.0; I am working to add support for necessary pthread locking functions to port a newer version of Python than 3.6, and Kuroko is expected to fill the same role.
• Issues have been found in the network stack when using older versions of VirtualBox. Users who experience problems in VirtualBox 5 are advised to upgrade to 6. The specific cause of this is issue is not known.
• VirtualBox and VMware may not automatically adjust the display resolution at boot. Changing the VM window size after the GUI has loaded should correct this.
• QEMU may experience issues with simultaneous mouse and keyboard input (such as when playing Quake). This is a known bug in several versions of QEMU itself and was only fixed recently; a workaround is available in the boot menu.
• Systems with more than ~60GB of RAM will panic on boot due to an oversight in a safety check. If you experience this issue, you may wish to try the latest CI build.

[MrOnlineCoder]

ToaruOS has consumed the greater part of my life for the last eleven years

"klange is a man of focus, commitment, sheer will and pure dedication."

Good job and congrats!