Asahi Linux - Porting Linux to Apple Silicon

Asahi Linux - Porting Linux to Apple Silicon

Speakers: Sven

Conference: 39C3

YouTube: https://www.youtube.com/watch?v=3OAiOfCcYFM

Overview

This talk provides an in-depth look into the monumental effort of porting Linux to Apple Silicon Macs, a project known as Asahi Linux. Presented by Sven, a key contributor to the project, the talk details the innovative reverse engineering methodologies and the significant technical hurdles overcome to enable a fully functional Linux distribution on Apple's proprietary hardware. The talk not only celebrates the technical achievements of the Asahi team but also highlights Apple's surprising decision to intentionally allow custom operating systems to boot on their ARM-based Mac lineup, a policy that stands in stark contrast to their more locked-down mobile devices.

The significance of Asahi Linux cannot be overstated. Apple Silicon Macs, introduced starting in 2020, represent a significant shift in computing architecture, offering unparalleled performance and power efficiency. However, their custom ARM architecture and lack of public documentation for hardware components present a formidable challenge for alternative operating system development. Asahi Linux addresses this by meticulously reverse engineering Apple's hardware, driver by driver, to bring the open-source Linux ecosystem to these powerful machines.

Sven's presentation serves as a testament to the power of collaborative open-source development and sophisticated reverse engineering techniques. It is crucial for anyone interested in operating system porting, hardware reverse engineering, or the future of open-source software on cutting-edge hardware. The project's commitment to upstreaming its drivers into the mainline Linux kernel ensures that its contributions benefit the broader Linux community and pave the way for widespread Linux support on Apple Silicon for years to come.

Background

▶ Watch: Project introduction and team credits (0:00)

Apple's announcement in 2020 to transition its Mac lineup from Intel x86 processors to custom-designed ARM-based Apple Silicon chips marked a pivotal moment in the computing industry. While this move promised significant performance and efficiency gains, it also raised questions about the feasibility of running alternative operating systems like Linux, given Apple's historically closed ecosystem for its mobile devices.

The Asahi Linux project was born from this challenge. Initiated by Markin in December 2020 through a Patreon campaign, the project quickly garnered sufficient funding to begin full-time development. Within months, Markin achieved the first significant milestone: getting initial patches into the upstream Linux kernel tree, allowing a very basic shell to boot on Apple Silicon hardware. This early success demonstrated the potential, despite the unique customizations and divergences from standard ARM machines. By early 2022, Asahi Linux released its first alpha version, making a proper operating system available to end-users on these powerful machines.

A critical, often misunderstood aspect of this endeavor is how it's even possible. Unlike iPhones and iPads, where Apple tightly controls the boot chain and requires all code to be signed, Apple Silicon Macs intentionally allow users to boot custom, unsigned code. This is achieved through a mechanism Apple calls "one true recovery." To access this mode, a user must be physically present at the machine (long-pressing the power button during a hard reboot) and authenticate with the machine owner's password. Once authenticated, the system allows the execution of completely unsigned code. This process drops the custom code into EL2 (Exception Level 2), the highest privilege level on the ARM CPU, with no Apple firmware or management engine running on the main core. This intentional design choice, confirmed by the Apple engineer who designed the mechanism, means Asahi Linux operates without exploits or voiding warranties. Users can create a secondary partition and use Apple's boot picker to select their custom boot object, which in Asahi's case, launches Linux.

The primary hurdle, however, remains the complete lack of documentation for Apple's custom hardware. To port Linux, fundamental components like an interrupt controller and a timer are essential. While the timer is a standard ARM component, Apple's custom Apple Interrupt Controller (AIC) required extensive reverse engineering. Early attempts by Markin involved "poking" known memory-mapped registers to understand their function, but the slow reboot cycle associated with testing custom boot objects quickly highlighted the need for more efficient reverse engineering tools. This necessity drove the development of the innovative tooling that became central to the Asahi Linux project.

Key Findings

▶ Watch: Asahi Linux project initiation and early milestones (2:20)

The Asahi Linux project has yielded several pivotal findings that have not only enabled Linux on Apple Silicon but also advanced the field of hardware reverse engineering.

Foremost among these is the confirmation that Apple intentionally designed its Apple Silicon Macs to permit the booting of arbitrary, unsigned code. This crucial architectural decision, contrasting sharply with Apple's mobile device security model, provides the fundamental opening for projects like Asahi Linux. It means the project operates within Apple's intended framework, without relying on exploits or vulnerabilities, granting users unprecedented control over their hardware.

A second major finding is the development and application of a sophisticated dynamic reverse engineering toolchain. This includes M1N1, a lightweight, freestanding C application that acts as an early bootloader and provides a Python shell interface over UART for real-time hardware register manipulation. For more complex components, the team developed an MMIO Tracing Hypervisor that runs the macOS kernel (XNU) within a custom virtual machine, logging all memory-mapped I/O (MMIO) accesses to a host machine. This "hardware s-trace" methodology allows for an iterative, highly efficient understanding of undocumented hardware behavior, transforming what would typically be a static, laborious disassembly process into a dynamic, observational one.

Through these tools, the project has uncovered specific details about Apple's hardware implementations:

• The identification of Apple's Synopsys DesignWare USB controller (version 3), whose documentation was unexpectedly found within public datasheets from other SoC vendors like Intel and Rockchip, and whose Linux driver (with historical dual-licensing) provided a valuable reference.
• The discovery of a TI-based USB Power Delivery (PD) controller responsible for power negotiation and alternate mode switching on the Type-C ports.
• The unique and complex behavior of Apple's USB controllers, which mandate a full teardown and hard reset of the entire USB stack upon every device unplug event. This counter-intuitive design requires specific driver logic to ensure reliable USB operation.
• The successful implementation and ongoing upstreaming of drivers for core system components, including the system controller, USB3, parts of the audio stack, and the GPU user space, into the mainline Linux kernel. This commitment to upstreaming ensures long-term maintainability and broad distribution support.
• Initial breakthroughs in supporting the newer M3 chip, with a new contributor (integral pilot) achieving successful Linux boot-up with all CPU cores and storage functionality, although graphics acceleration remains a significant challenge.
• The identification of a new reverse engineering hurdle for M4 and M5 chips, where Apple has disabled certain virtualization instructions for "raw boot objects." While Linux can still boot, this change significantly hampers the effectiveness of the project's established MMIO tracing tools, necessitating new approaches for hardware analysis on these future generations.

These findings collectively demonstrate the Asahi Linux project's profound technical expertise and its role in pushing the boundaries of what's possible in open-source hardware enablement.

Technical Deep Dive

▶ Watch: Explaining how Apple Silicon allows custom OS (3:15)

The core of Asahi Linux's success lies in its sophisticated approach to reverse engineering undocumented hardware. Given Apple's proprietary nature, traditional methods of driver development are impossible. The team instead developed a unique, dynamic methodology to understand and control Apple Silicon's intricate components.

Reverse Engineering Methodology: From Poking to Tracing

Initially, for simpler hardware like the Apple Interrupt Controller (AIC) or I2C buses, the approach involved "blindly poking" memory-mapped registers. Markin, with his deep understanding of how such controllers typically function, would write small freestanding C applications to directly manipulate registers and observe the hardware's response. However, this iterative process was severely hampered by the need to constantly reboot the machine into "one true recovery" mode, install a new boot object, and test. This slow feedback loop was unsustainable for complex hardware.

To accelerate this, the team developed M1N1, a minimal, freestanding C application that acts as an early bootloader. M1N1 provides a small proxy over a hardware UART connection (accessible via specific USB pins), allowing a Python shell on a separate host machine to interact with the target Apple Silicon device. This setup enables real-time experimentation: developers can read and write hardware registers, allocate memory buffers, monitor MMIO ranges, and even trigger reboots—all from a Python script on their host machine. Sven highlighted how he prototyped an I2C driver entirely in Python using this method. This rapid prototyping allows developers to establish a working hardware model in their minds before writing a single line of Linux kernel code, ensuring that subsequent bugs are likely in the driver implementation rather than a misunderstanding of the hardware.

For highly complex components like the GPU or various co-processors, where guessing register functions is infeasible, the team developed an MMIO Tracing Hypervisor. This ingenious tool runs the macOS kernel (XNU) inside a specially crafted virtual machine, with M1N1 acting as the hypervisor. Initially, all hardware is mapped directly through, but for specific components of interest, the page tables are configured to trap every MMIO access. When XNU attempts to interact with the target hardware, the access is intercepted by M1N1, logged, and sent over USB to a host machine. This generates a detailed trace of how XNU drives the hardware. The tooling allows for iterative refinement: developers can define known registers in Python, correlate XNU's debug output with MMIO writes, and gradually build an understanding of the hardware's function. Sven likened this to building an "s-trace for hardware," a dynamic analysis technique that often negates the need for tedious static disassembly of XNU's C++ and IO Kit drivers.

USB Port Reverse Engineering: A Case Study

Sven's work on the USB ports exemplifies this methodology. The first step was identifying the USB controller. By dumping MMIO regions and comparing them to known controllers, he quickly determined Apple uses a Synopsys DesignWare controller, version 3. Crucially, official datasheets for this controller are often under NDA, but Sven found comprehensive documentation embedded within public datasheets from other SoC vendors like Intel and Rockchip, alongside existing Linux drivers for older versions.

A significant challenge emerged when implementing USB host mode: devices would only work once. After an unplug/replug, the device would not reappear. Using the MMIO tracing hypervisor, Sven observed XNU's behavior during a plug-unplug cycle. He discovered a highly unusual sequence: upon unplugging a USB device, XNU would assert a full reset of the USB controller, the port, and the bus, then gate clocks on everything, effectively powering down the entire system. When a device was plugged back in, XNU would reverse this process, bringing everything back up from a cold state. This meant Apple's controller was designed such that it could not be re-initialized after a single use without a complete hardware reset.

The solution involved implementing this teardown and hard reset logic within the Linux USB driver. Initially, Sven's implementation was "hacky," but through the upstreaming process, a Linux kernel maintainer provided a more robust and standard approach, underscoring the benefits of early upstreaming. With this, USB 1 and 2 functionality was fully achieved. The same observation-and-implementation cycle was then applied to USB 3, which involves more complex link training and higher speeds, eventually leading to its upstreaming into the kernel.

DisplayPort and Future Challenges

The quest for DisplayPort support, a highly requested feature, presented its own set of complexities. It requires configuring a separate display controller and communicating via two or three distinct serialization protocols, all coordinated with the already upstreamed USB3 PHY. Sven demonstrated working DisplayPort during his talk, a testament to the team's recent efforts, with the enabling code pushed for developers shortly before the presentation, aiming for general availability in early 2024.

Looking ahead to M3, M4, and M5 chips, the project anticipates generally minor hardware changes between generations, with the significant exception of the GPU, where Apple continues to innovate rapidly. A new contributor, "integral pilot," has already made strides on M3, achieving boot-up with all CPU cores and functional storage, though graphics acceleration remains a long-term goal.

However, M4 and M5 chips introduce a new hurdle: Apple has disabled specific virtualization instructions for "raw boot objects" (the mode used to boot Linux). While Linux can still boot, this change prevents the Asahi team from using their highly effective MMIO tracing hypervisor to observe XNU's hardware interactions. This means reverse engineering on these newer chips will revert to more challenging, less dynamic methods, requiring new strategies to overcome this limitation.

Demo / Proof of Concept

▶ Watch: Apple Silicon boot process and custom OS integration (5:00)

The talk itself served as a compelling demonstration of Asahi Linux's progress, particularly concerning DisplayPort functionality. Sven delivered his presentation directly from an Apple Silicon machine running Linux, showcasing a live, functional Linux environment. This setup directly addressed a frequently asked question within the Asahi community regarding external display support. He mentioned that the code enabling this was pushed just minutes before his talk, highlighting the cutting-edge nature of the demonstration and the rapid pace of development.

Beyond the live presentation, Sven also referenced other proofs of concept related to the M3 chip. He highlighted screenshots shared by a new contributor, integral pilot, showing an M3 Max machine successfully booting Linux with all its cores active and storage working. While graphics acceleration for M3 is still under development and considered a significant long-term task, the ability to boot and even run applications like Doom on the M3 chip already demonstrates substantial progress and the potential for a fully functional Linux experience on Apple's latest hardware generations. These demonstrations collectively underscore the tangible achievements of the Asahi Linux project and its continuous forward momentum.

Defensive Implications

▶ Watch: Major challenge: reverse engineering without documentation (6:15)

The Asahi Linux project, as detailed in this talk, primarily focuses on enabling user choice and control over hardware rather than exploiting vulnerabilities. Therefore, it does not carry typical "defensive implications" in the context of protecting systems from attacks. Instead, the talk highlights Apple's unique and intentional design decision to allow unsigned code execution on Apple Silicon Macs, which is a security feature for the user, ensuring they retain ownership and the ability to run alternative operating systems. This contrasts sharply with the locked-down boot chains found in iPhones and iPads.

From a broader security perspective, however, the detailed reverse engineering methodologies presented could offer insights. The techniques for MMIO tracing and dynamic hardware interaction using M1N1 are powerful tools. While Asahi Linux uses these for benign purposes (driver development), an adversary who gains initial access to a highly privileged execution environment (like EL2) on other ARM platforms could theoretically adapt similar techniques to understand and manipulate undocumented hardware for malicious ends. However, gaining such a privileged execution level without manufacturer intent, as is the case with Apple Silicon Macs, would typically require significant exploits.

The talk also implicitly demonstrates how vendor-specific hardware quirks, such as the unusual USB controller reset behavior on Apple Silicon, can create unique challenges for operating system development. While not a security vulnerability, such complexities could potentially introduce subtle bugs or instability if not handled meticulously, which is why the Asahi team's rigorous reverse engineering and upstreaming process is crucial for robust and secure driver implementations. In essence, the talk is a celebration of open hardware enablement, reinforcing the principle that users should have the freedom to run any software they choose on their purchased computing devices.

Key Takeaways

• Intentional Openness: Apple Silicon Macs are uniquely designed to intentionally allow users to boot unsigned, custom operating systems like Linux, dropping into the highest privilege level (EL2) without requiring exploits. This contrasts with Apple's mobile devices and empowers user choice.
• Dynamic Reverse Engineering: The Asahi Linux project developed groundbreaking dynamic reverse engineering tools, including M1N1 (a Python-controlled hardware interaction proxy over UART) and an MMIO Tracing Hypervisor (running XNU in a VM to log hardware accesses), enabling rapid understanding of undocumented Apple hardware.
• Unique USB Hardware Challenges: Apple Silicon's USB controllers exhibit unusual behavior, requiring a complete teardown and hard reset of the entire USB stack for every device unplug and subsequent replug event, a complexity meticulously reverse engineered and implemented in the Linux driver.
• Commitment to Upstreaming: Asahi Linux prioritizes upstreaming its developed drivers for components like the system controller, USB3, and parts of the GPU user space into the mainline Linux kernel, ensuring long-term sustainability and broad distribution support.
• Progress on Newer Chips: Initial Linux boot-up has been achieved on M3 chips, with all cores and storage functional, indicating continued progress across Apple Silicon generations, though graphics acceleration remains a significant challenge.
• Future Reverse Engineering Hurdles: For M4 and M5 chips, Apple has disabled certain virtualization instructions when booting raw objects, which impacts the effectiveness of Asahi's MMIO tracing tools, posing new challenges for hardware analysis on these future platforms.

About the Speaker(s)

Sven, the speaker for this insightful presentation, is a dedicated contributor to the Asahi Linux project. While his specific title and company affiliation were not detailed in the talk's metadata or transcript, his presentation clearly demonstrates his deep technical expertise and significant involvement in the project's success. Sven played a crucial role in the reverse engineering of the Apple Silicon USB ports, meticulously unraveling their complex behavior and contributing to the upstreaming of the USB 1/2/3 drivers to the Linux kernel. He also acknowledged the extensive collaborative effort behind Asahi Linux, crediting other key individuals such as Markin (project founder), Alyssa (graphics driver lead), Asylina (GPU kernel driver), and Yanu (downstream kernel maintenance, particularly for DisplayPort). Sven's work, along with that of the entire Asahi team, is instrumental in bringing a fully functional Linux experience to Apple's cutting-edge M-series hardware.