Introduction

When you open Terminal.app on your Mac, you’re not just launching an application—you’re opening a portal to a Unix system that traces its lineage back to the 1970s at Bell Labs. Beneath macOS’s polished graphical interface lies Darwin, a fully certified Unix operating system built on decades of BSD development and NeXT innovation.

This book is your comprehensive guide to understanding and leveraging that Unix foundation.

Who This Book Is For

This book serves several audiences:

Linux and BSD users coming to macOS will find detailed explanations of where macOS diverges from familiar Unix conventions. Why doesn’t sed -i work the same way? Where did /etc/init.d go? Why are there so many .DS_Store files everywhere? This book answers these questions and many more.

Mac users discovering the command line will find a thorough introduction to Unix concepts, contextualized for macOS. You’ll learn not just what commands do, but why macOS implements things the way it does, and how the terminal integrates with the graphical system you already know.

System administrators managing Mac fleets will find practical guidance on automation, configuration management, and security hardening. From understanding launchd to deploying configuration profiles, this book covers enterprise-relevant topics in depth.

Developers building for Apple platforms will understand the toolchain beneath Xcode, learn to compile Unix software on macOS, and navigate the complexities of code signing, notarization, and Apple Silicon.

What Makes macOS Unique

macOS occupies a unique position in the Unix landscape. It’s the only Unix-certified operating system with significant desktop market share. It combines:

• A BSD userland derived from FreeBSD
• A hybrid kernel (XNU) blending Mach microkernel concepts with BSD
• NeXT heritage in its service management and development tools
• Apple innovations in filesystem design, security architecture, and hardware integration

This combination creates both opportunities and challenges. You have access to a robust Unix toolkit, but many assumptions from Linux or traditional BSD don’t apply. GNU tools behave differently than their BSD counterparts. The filesystem hierarchy has macOS-specific locations. System services use a completely different model than systemd or traditional init.

How This Book Is Organized

The book progresses from foundations to practical applications:

Part I: Foundations & History explains where macOS came from and how its Unix implementation differs from others. Understanding this context helps you reason about why things work the way they do.

Part II: Filesystem & Storage covers APFS, HFS+, extended attributes, and macOS’s unique approach to file organization. You’ll learn to work with—not against—macOS’s filesystem quirks.

Part III: Shell & Terminal addresses the transition from bash to zsh, terminal configuration, and macOS-specific shell features. You’ll learn to create a productive command-line environment.

Part IV: Package Management provides a deep dive into Homebrew and alternatives. You’ll understand not just how to install software, but how these systems work and how to troubleshoot them.

Part V: System Services & Processes explains launchd, Apple’s replacement for init, systemd, and cron combined. You’ll learn to create and manage services the macOS way.

Part VI: Unix Commands & Tools documents the differences between BSD and GNU commands, introduces macOS-specific tools, and helps you write portable scripts.

Part VII: Development Environment covers Xcode Command Line Tools, the LLVM toolchain, library linking, and the complexities of building software on modern macOS.

Part VIII: System Administration addresses user management, permissions, System Integrity Protection, firewalls, and logging—everything you need to manage macOS systems.

Part IX: Networking explains network configuration, Bonjour, VPNs, and sharing services from the command line.

Part X: Security Model demystifies Gatekeeper, code signing, notarization, sandboxing, and the privacy controls that increasingly define the macOS experience.

Part XI: Interoperability helps you work across platforms—sharing files, running containers, writing portable scripts, and connecting to remote systems.

Part XII: Performance & Optimization teaches you to diagnose and resolve performance issues using both GUI and command-line tools.

Conventions Used in This Book

Command Examples

Terminal commands are shown in code blocks with a $ prefix indicating the shell prompt:

$ uname -a
Darwin MacBook-Pro.local 23.0.0 Darwin Kernel Version 23.0.0: Fri Sep 15 14:41:43 PDT 2023; root:xnu-10002.1.13~1/RELEASE_ARM64_T6000 arm64

Commands requiring sudo (administrator privileges) are shown with #:

# systemsetup -setremotelogin on

File Paths

File paths are shown in monospace: /Library/LaunchDaemons/com.example.daemon.plist

When a path begins with ~, it refers to your home directory:

• ~/.zshrc expands to /Users/yourusername/.zshrc

Admonitions

Throughout the book, you’ll find specially marked sections:

Note: Additional information or context that’s helpful but not critical.

Warning: Important caveats or potential pitfalls to avoid.

Tip: Practical advice for improved workflows.

Version Information

This book is written for macOS Sonoma (14.x) and later, running on both Intel and Apple Silicon Macs. Most content applies to earlier versions, but some features—particularly security and filesystem capabilities—are version-specific. When version matters, it’s noted explicitly.

Prerequisites

This book assumes:

• You have a Mac running a recent version of macOS
• You’re comfortable using applications and basic computer operations
• You have an interest in learning command-line tools

No prior Unix experience is required, though readers with Linux or BSD backgrounds will find much that’s familiar—and much that differs in important ways.

A Note on Terminology

Throughout this book, we use several terms that deserve clarification:

• macOS: Apple’s desktop operating system, formerly known as Mac OS X and OS X
• Darwin: The open-source Unix foundation underlying macOS (and iOS, tvOS, watchOS)
• Unix: The family of operating systems descended from or inspired by the original AT&T Unix
• BSD: Berkeley Software Distribution, a Unix variant that heavily influences macOS
• POSIX: Portable Operating System Interface, the standards that define Unix compatibility

When we say macOS is “Unix,” we mean it literally—macOS is UNIX 03 certified by The Open Group, the organization that owns the Unix trademark.

Let’s Begin

The command line on macOS is not a relic of the past—it’s a powerful interface that complements the graphical experience. Whether you’re automating tasks, managing servers, or building software, understanding macOS’s Unix foundations will make you more effective.

Open Terminal.app (you’ll find it in /Applications/Utilities/), and let’s explore what lies beneath the surface.

The Unix Heritage of macOS

macOS is not merely “Unix-like”—it is Unix. Apple’s desktop operating system carries official UNIX 03 certification from The Open Group, making it one of the few consumer operating systems that can legally use the Unix name. But how did a company known for the Macintosh end up shipping a certified Unix system?

The answer involves a journey through computing history, touching Bell Labs, Berkeley, Carnegie Mellon, NeXT, and finally Apple. Understanding this heritage is essential for anyone who wants to truly master macOS’s command-line environment.

The Path to Darwin

macOS’s Unix foundation is called Darwin. Released as open source since 2000, Darwin combines:

1. The XNU kernel: A hybrid design merging the Mach microkernel with BSD components
2. A BSD userland: Command-line tools and libraries derived primarily from FreeBSD
3. Apple frameworks: Extensions and adaptations for macOS-specific functionality

Darwin is not macOS—it lacks the Aqua graphical interface, the Cocoa frameworks, and the proprietary applications that define the Mac experience. But it provides the Unix substrate on which everything else runs.

What You’ll Learn in This Part

This section traces macOS’s Unix lineage and explains how it differs from other Unix-like systems:

Darwin: The Open Source Core introduces Darwin, examining its components, its open-source status, and its relationship to macOS.

From NeXTSTEP to macOS tells the story of how Steve Jobs’s post-Apple company created the technology that would eventually become macOS.

The XNU Kernel Architecture dives deep into macOS’s hybrid kernel, explaining how Mach and BSD components work together.

macOS vs Linux vs BSD: Key Differences provides a practical comparison for users coming from other Unix systems, highlighting where macOS does things differently and why.

POSIX Compliance and Standards examines macOS’s adherence to Unix standards and what this means for portability and compatibility.

Why History Matters

You might wonder why a practical guide spends time on history. The answer is simple: understanding why macOS works the way it does helps you predict how it will behave. When you understand that macOS’s service management comes from NeXT, that its command-line tools come from FreeBSD, and that its kernel blends two distinct traditions, you can reason about unfamiliar situations instead of just memorizing facts.

The next chapters provide that understanding.

Darwin: The Open Source Core

Darwin is the open-source operating system that forms the foundation of macOS, iOS, iPadOS, watchOS, tvOS, and visionOS. While Apple’s consumer products include proprietary components layered on top, Darwin itself has been available under various open-source licenses since its initial release in 2000.

What Is Darwin?

Darwin is a complete Unix operating system, comprising:

• XNU: The hybrid kernel combining Mach and BSD
• BSD subsystem: Process management, networking, filesystem interfaces
• IOKit: The driver framework for hardware interaction
• Userland tools: Command-line utilities derived from FreeBSD
• Core libraries: libSystem and related foundational libraries

When you type commands in Terminal.app, you’re interacting directly with Darwin. The graphical macOS experience—Aqua, Finder, the Dock—sits above Darwin but depends entirely on it.

Darwin’s Open Source Status

Darwin’s licensing is complex. Different components use different licenses:

The Apple Public Source License is OSI-approved but not GPL-compatible. This means Darwin code can’t be directly combined with GPL-licensed projects like Linux.

Accessing Darwin Source Code

Apple publishes Darwin source code at opensource.apple.com. You can browse and download:

# View available Darwin components
$ curl -s https://opensource.apple.com/tarballs/ | grep -o 'href="[^"]*"' | head -20

Key repositories include:

• xnu: The kernel
• Libc: The C standard library
• shell_cmds: Shell built-in commands
• file_cmds: File manipulation commands (ls, cp, mv, etc.)
• network_cmds: Networking utilities

Building Darwin

While Apple publishes source code, building a complete Darwin system is challenging. The community-driven PureDarwin project maintains bootable Darwin distributions, though these lack the polish and hardware support of macOS.

Darwin vs macOS

The relationship between Darwin and macOS is analogous to the relationship between the Linux kernel and a Linux distribution like Ubuntu:

However, there’s a crucial difference: Darwin without macOS is rarely used, while Linux without any particular distribution doesn’t exist as a consumer product.

Examining Your Darwin Installation

You can inspect Darwin’s presence on your system:

# Display Darwin version
$ uname -a
Darwin MacBook-Pro.local 23.4.0 Darwin Kernel Version 23.4.0: Fri Mar 15 00:12:49 PDT 2024; root:xnu-10063.101.17~1/RELEASE_ARM64_T6020 arm64

# View Darwin kernel version details
$ sysctl kern.ostype kern.osrelease kern.version
kern.ostype: Darwin
kern.osrelease: 23.4.0
kern.version: Darwin Kernel Version 23.4.0: Fri Mar 15 00:12:49 PDT 2024; root:xnu-10063.101.17~1/RELEASE_ARM64_T6020

The uname output reveals key information:

• Darwin: The OS type
• 23.4.0: The Darwin kernel version (not the macOS version)
• xnu-10063.101.17~1: The specific XNU build
• RELEASE_ARM64_T6020: Release build for ARM64, specifically the M2 Pro chip

Darwin Version vs macOS Version

Darwin versions don’t match macOS marketing versions. Here’s the mapping for recent releases:

The pattern: Darwin version = macOS major version + 9 (approximately, with some variation).

Darwin Components in Detail

The C Library (libSystem)

Unlike Linux (which uses glibc or musl) or FreeBSD (which has its own libc), macOS uses libSystem, an umbrella library that combines:

• Libc: Standard C library functions
• libinfo: Directory services
• libkvm: Kernel memory interface
• libm: Math library
• libpthread: POSIX threads

# Examine libSystem
$ otool -L /bin/ls | grep libSystem
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1336.61.1)

BSD Commands

Most command-line tools come from FreeBSD, with Apple modifications. You can verify this:

# Check version strings in common commands
$ what /bin/ls
/bin/ls:
    PROGRAM:ls  PROJECT:file_cmds-430.100.5

$ what /usr/bin/grep
/usr/bin/grep:
    PROGRAM:grep  PROJECT:text_cmds-154

The what command extracts version information embedded in binaries—a BSD convention.

Darwin’s Unique Position

Darwin occupies an interesting position in the Unix ecosystem:

1. Certified Unix: Darwin-based macOS is UNIX 03 certified
2. BSD heritage: Derived primarily from FreeBSD
3. Unique kernel: XNU is unlike both monolithic Linux and pure Mach microkernels
4. Apple stewardship: Developed primarily by Apple, with limited community input
5. Commercial backing: Unusual for an open-source Unix

This combination means Darwin benefits from significant engineering investment while maintaining Unix compatibility. However, Apple’s control means Darwin’s direction serves Apple’s product needs, not the broader open-source community.

Working with Darwin

For practical purposes, working with Darwin means working with macOS’s command line. Throughout this book, we’ll explore Darwin’s capabilities:

• Using BSD commands (and understanding how they differ from GNU)
• Managing processes with Darwin’s unique tools
• Understanding the kernel’s behavior and configuration
• Leveraging Darwin’s networking stack
• Working with the filesystem abstractions Darwin provides

Darwin is your gateway to macOS’s Unix power. Understanding it transforms macOS from a consumer operating system into a professional Unix workstation.

From NeXTSTEP to macOS

The story of macOS’s Unix foundation begins not at Apple, but at a company Steve Jobs founded after leaving Apple in 1985. NeXT, Inc. created an operating system so advanced that when Apple needed to modernize its aging Mac OS, it acquired the entire company to get it.

The Classic Mac OS Problem

By the mid-1990s, Apple faced a crisis. The original Macintosh operating system, while innovative for its time, lacked fundamental features that modern operating systems required:

• No protected memory: A crash in any application could bring down the entire system
• Cooperative multitasking: Applications had to voluntarily yield CPU time
• No modern networking stack: TCP/IP was bolted on awkwardly
• Limited scalability: The architecture couldn’t evolve to meet future needs

Apple attempted several internal projects to create a modern replacement—Copland, Taligent, Pink—but all failed. By 1996, Apple was searching externally for a solution.

NeXTSTEP: Unix Made Elegant

While Apple struggled, NeXT had been building something remarkable. NeXTSTEP, first released in 1989, was a Unix-based operating system with several distinctive characteristics:

Mach Microkernel Foundation

NeXTSTEP used the Mach microkernel developed at Carnegie Mellon University. Mach provided:

• Protected memory between processes
• Preemptive multitasking
• Modern IPC (inter-process communication)
• Support for multiple processor architectures

BSD Compatibility Layer

Layered atop Mach was a BSD-derived Unix environment, providing:

• POSIX-compliant APIs
• Standard Unix commands and utilities
• Familiar filesystem semantics
• TCP/IP networking

Object-Oriented Frameworks

NeXTSTEP pioneered object-oriented application development with:

• Objective-C: An object-oriented extension to C
• Application Kit: GUI widgets and event handling
• Foundation: Base classes for common operations
• Interface Builder: Visual interface design

These frameworks would evolve into today’s Cocoa.

Display PostScript

NeXTSTEP used PostScript for display rendering, ensuring that what appeared on screen matched printed output—a precursor to macOS’s Quartz.

The Acquisition

In December 1996, Apple announced it would acquire NeXT for $429 million. The deal brought:

1. NeXTSTEP technology: The foundation for the next-generation Mac OS
2. Steve Jobs: Who would eventually return as CEO
3. Key engineers: Including Avie Tevanian (who became Apple’s chief software architect)

From OPENSTEP to Rhapsody to Mac OS X

The transformation from NeXTSTEP to macOS occurred in stages:

OPENSTEP (1994-1997)

Before the Apple acquisition, NeXT had separated the operating system into:

• OPENSTEP: The application frameworks (portable to other OSes)
• NEXTSTEP/OPENSTEP for Mach: The complete OS

This separation facilitated the eventual Apple integration.

Rhapsody (1997-1998)

Apple’s first attempt at merging NeXT technology with Mac:

• Code-named “Rhapsody”
• NeXTSTEP renamed and modified for Mac hardware
• Included “Blue Box” for running classic Mac OS applications

Rhapsody was demonstrated publicly but never shipped as a consumer product.

Mac OS X Server 1.0 (1999)

The first released product combining NeXT and Apple technology:

• Based on Rhapsody
• Targeted at servers
• Introduced the Aqua interface concepts
• Still clearly NeXTSTEP under the hood

Mac OS X 10.0 (2001)

The first consumer release of the new operating system:

• Aqua interface: New visual design replacing NeXTSTEP’s look
• Carbon APIs: Allowing classic Mac applications to be ported
• Cocoa APIs: The renamed OPENSTEP frameworks
• Classic environment: Running Mac OS 9 in a compatibility layer
• Darwin foundation: The open-source Unix base

NeXTSTEP’s Legacy in Modern macOS

Many macOS characteristics trace directly to NeXTSTEP:

File Extensions and Bundles

NeXTSTEP popularized application bundles—directories that appear as single files:

$ ls -la /Applications/Safari.app/
total 0
drwxr-xr-x   3 root  wheel    96 Dec 11 11:23 .
drwxr-xr-x  88 root  wheel  2816 Jan 15 14:20 ..
drwxr-xr-x   7 root  wheel   224 Dec 11 11:23 Contents

$ file /Applications/Safari.app/
/Applications/Safari.app/: directory

The .app extension and bundle structure come directly from NeXTSTEP.

Property Lists

The .plist files used throughout macOS for configuration:

$ file /System/Library/LaunchDaemons/com.apple.metadata.mds.plist
/System/Library/LaunchDaemons/com.apple.metadata.mds.plist: XML 1.0 document text, ASCII text

Property lists originated in NeXTSTEP as a serialization format.

The Services Menu

The Services menu (under the application menu) allowing applications to provide functionality to other applications—pure NeXTSTEP.

/Library, ~/Library, and /System/Library

The three-tier Library structure:

• /System/Library: Apple’s system resources
• /Library: Local administrator resources
• ~/Library: User-specific resources

This hierarchy descends from NeXTSTEP’s organization.

launchd’s Ancestry

While launchd itself was created at Apple, its design philosophy—using property lists for service configuration—reflects NeXTSTEP’s approach to system management.

Objective-C and Cocoa

Until Swift’s introduction, Objective-C was the primary language for macOS development. Cocoa, the main application framework, is a direct evolution of OPENSTEP’s AppKit and Foundation.

The Mach Heritage

NeXTSTEP’s use of Mach brought several characteristics to macOS:

Mach Ports

Inter-process communication uses Mach ports:

$ sudo lsmp -p 1
Process (1) : launchd
  name      ipc-object    rights     flags   boost  reqs  recv  send sonce oref  qlimit  msgcount  context            identifier  type
---------   ----------    ------     -----   -----  ----  ----  ---- ----- ----  ------  --------  --------           ----------  ----
...

Memory Objects

Mach’s virtual memory system underlies macOS’s memory management:

$ vm_stat
Mach Virtual Memory Statistics: (page size of 16384 bytes)
Pages free:                               37103.
Pages active:                            385918.
Pages inactive:                          347108.
...

Task and Thread Model

macOS processes are Mach tasks containing threads—terminology that persists in debugging and profiling tools.

Understanding the Timeline

Here’s a consolidated timeline:

Why This History Matters

Understanding NeXTSTEP’s influence helps explain macOS peculiarities:

1. Why Objective-C? Because NeXTSTEP chose it in the 1980s
2. Why property lists? NeXTSTEP’s serialization format
3. Why application bundles? NeXTSTEP’s packaging model
4. Why Mach ports? NeXTSTEP’s kernel choice
5. Why is Darwin BSD-based? NeXTSTEP’s Unix compatibility layer

When you encounter something in macOS that seems different from Linux or traditional BSD, the answer often lies in this NeXTSTEP heritage. The next chapter examines the kernel architecture that emerged from this history.

The XNU Kernel Architecture

XNU—“X is Not Unix” (a recursive acronym in the tradition of GNU)—is the kernel at the heart of Darwin. XNU is neither a traditional monolithic kernel like Linux nor a pure microkernel like Mach was intended to be. It’s a hybrid that combines elements of both approaches.

What Is a Kernel?

Before diving into XNU’s specifics, let’s clarify what a kernel does:

• Process management: Creating, scheduling, and terminating processes
• Memory management: Allocating RAM, implementing virtual memory
• Device drivers: Interfacing with hardware
• System calls: Providing services to user-space programs
• IPC: Facilitating communication between processes

How these responsibilities are organized defines a kernel’s architecture.

Kernel Architecture Styles

Monolithic Kernels

Linux and traditional BSD use monolithic kernels:

• All kernel services run in kernel space
• Efficient: no context switches between kernel components
• Risk: a bug in any driver can crash the entire system

Microkernels

Pure microkernels (like MINIX or L4) take a different approach:

• Minimal kernel: only scheduling, IPC, basic memory management
• Services (filesystems, drivers) run as user-space processes
• Robust: service crashes don’t bring down the kernel
• Overhead: IPC between services adds latency

XNU: The Hybrid Approach

XNU combines both:

• Mach layer: Microkernel-derived component handling IPC, virtual memory, scheduling
• BSD layer: Monolithic component providing Unix APIs, networking, filesystems
• IOKit: Object-oriented driver framework in kernel space

┌─────────────────────────────────────────────────────────┐
│                    User Space                            │
│  Applications, Daemons, Shell commands                   │
├─────────────────────────────────────────────────────────┤
│                    System Calls                          │
├───────────────┬────────────────────────┬────────────────┤
│   BSD Layer   │      IOKit/Drivers     │                │
│  (POSIX APIs, │    (Device drivers,    │                │
│   networking, │   hardware abstraction)│                │
│  filesystems) │                        │                │
├───────────────┴────────────────────────┴────────────────┤
│                    Mach Layer                            │
│  (IPC, virtual memory, scheduling, timers)               │
├─────────────────────────────────────────────────────────┤
│                    Hardware                              │
└─────────────────────────────────────────────────────────┘

The Mach Layer

XNU’s Mach component derives from Mach 3.0, developed at Carnegie Mellon University. However, XNU doesn’t use Mach as a true microkernel—the BSD layer runs in kernel space, not as a user-space server.

Mach Abstractions

Mach provides several fundamental abstractions:

Tasks: The unit of resource ownership. A task contains threads and owns resources like memory and ports. In Unix terms, a task roughly corresponds to a process.

# View Mach task information
$ sudo launchctl procinfo 1
{
    "task_info" : {
        "suspend_count" : 0,
        "virtual_size" : 418099200,
        "resident_size" : 5423104,
        ...
    }
}

Threads: The unit of execution. Tasks contain one or more threads.

Ports: The fundamental IPC mechanism. Ports are endpoints for message passing.

# List Mach ports for a process
$ sudo lsmp -p $$

Messages: Data passed between ports. All Mach IPC occurs through message passing.

Memory Objects: Abstractions for memory that can be mapped into task address spaces.

Mach IPC

Mach’s message-passing IPC is powerful but complex:

// Simplified Mach message send
mach_msg_header_t msg;
msg.msgh_bits = MACH_MSGH_BITS(MACH_MSG_TYPE_COPY_SEND, 0);
msg.msgh_size = sizeof(msg);
msg.msgh_remote_port = destination_port;
msg.msgh_local_port = MACH_PORT_NULL;
mach_msg(&msg, MACH_SEND_MSG, sizeof(msg), 0,
         MACH_PORT_NULL, MACH_MSG_TIMEOUT_NONE, MACH_PORT_NULL);

While application developers rarely use Mach IPC directly (preferring higher-level APIs like XPC), it underpins many macOS facilities.

Virtual Memory

Mach’s virtual memory system provides:

• Sparse address spaces: Processes can have large virtual address spaces
• Copy-on-write: Efficient process forking and memory sharing
• Memory-mapped files: Files accessible as memory regions
• External pagers: Pluggable systems for backing memory with storage

# Examine Mach VM statistics
$ vm_stat
Mach Virtual Memory Statistics: (page size of 16384 bytes)
Pages free:                               32148.
Pages active:                            402844.
Pages inactive:                          373521.
Pages speculative:                        11276.
Pages throttled:                              0.
Pages wired down:                         96482.
Pages purgeable:                          23159.
"Translation faults":                 135941690.
Pages copy-on-write:                    4972318.
Pages zero filled:                     62938502.
Pages reactivated:                      4058508.
Pages purged:                            789235.
...

Scheduling

Mach handles thread scheduling with multiple scheduling policies:

• Timeshare: Standard interactive scheduling
• Fixed priority: Real-time scheduling with fixed priorities
• Round-robin: Equal time slices

# View thread scheduling information
$ sudo spindump -noProcesses 1 -stdout | head -50

The BSD Layer

The BSD layer provides the Unix personality that applications see:

POSIX APIs

Standard Unix system calls are implemented in the BSD layer:

• fork(), exec(), wait()
• open(), read(), write(), close()
• socket(), bind(), listen(), accept()
• kill(), signal handling

Process Model

The BSD layer maintains the traditional Unix process model atop Mach tasks:

# BSD process information
$ ps aux | head -5
USER   PID  %CPU %MEM      VSZ    RSS   TT  STAT STARTED      TIME COMMAND
root     1   0.0  0.1 410045680  18512   ??  Ss   Tue10AM   3:21.67 /sbin/launchd

# Underlying Mach task information
$ sudo taskinfo 1

Each BSD process corresponds to a Mach task, but the BSD layer provides the familiar Unix semantics.

Networking Stack

The BSD layer includes a complete TCP/IP networking stack derived from FreeBSD:

# View network statistics (BSD-style)
$ netstat -s | head -30
tcp:
    14548388 packets sent
        9522841 data packets (5765485687 bytes)
        96487 data packets (87239347 bytes) retransmitted
        0 resend initiated by MTU discovery
        3896370 ack-only packets (2451961 delayed)
...

Filesystems

Filesystem implementations live in the BSD layer:

• APFS: Apple File System (modern default)
• HFS+: Hierarchical File System Plus (legacy)
• NFS: Network File System
• SMB: Server Message Block
• devfs: Device filesystem
• Various FUSE filesystems: Via macFUSE

The VFS Layer

The Virtual File System layer abstracts filesystem differences:

# List mounted filesystems
$ mount
/dev/disk3s1s1 on / (apfs, sealed, local, read-only, journaled)
devfs on /dev (devfs, local, nobrowse)
/dev/disk3s6 on /System/Volumes/VM (apfs, local, noexec, journaled, noatime, nobrowse)
/dev/disk3s2 on /System/Volumes/Preboot (apfs, local, journaled, nobrowse)
/dev/disk3s4 on /System/Volumes/Update (apfs, local, journaled, nobrowse)
/dev/disk3s5 on /System/Volumes/Data (apfs, local, journaled, nobrowse, protect)

IOKit: The Driver Framework

IOKit is XNU’s object-oriented framework for device drivers:

Design Philosophy

• Written in a restricted subset of C++ (Embedded C++)
• Object-oriented: drivers inherit from base classes
• Uses a registry to match drivers with devices
• Power management integrated at the framework level

The IORegistry

IOKit maintains a database of devices and their relationships:

# View the IORegistry
$ ioreg -l | head -50
+-o Root  <class IORegistryEntry, id 0x100000100, retain 36>
  +-o MacBookPro18,3  <class IOPlatformExpertDevice, id 0x100000116, registered, matched, active, busy 0 (10112 ms), retain 61>
    | {
    |   "IOBusyInterest" = "IOCommand is not serializable"
    |   "IOPlatformSerialNumber" = "..."
    |   "IOPolledInterface" = "SMCPolledInterface is not serializable"
    |   "IOPlatformUUID" = "..."
...

# List USB devices through IOKit
$ ioreg -p IOUSB -l -w0

# List storage devices
$ ioreg -c IOMedia -l

Kernel Extensions (Kexts)

Traditionally, third-party drivers were loaded as kernel extensions:

# List loaded kexts
$ kextstat | head -10
Index Refs Address            Size       Wired      Name (Version) UUID <Linked Against>
    1  169 0                  0          0          com.apple.kpi.bsd (23.4.0)
    2   18 0                  0          0          com.apple.kpi.dsep (23.4.0)
    3  196 0                  0          0          com.apple.kpi.iokit (23.4.0)
...

However, modern macOS strongly discourages kexts in favor of:

• System Extensions: User-space drivers for many device types
• DriverKit: A framework for building drivers that run outside the kernel

This shift improves security and stability by moving driver code out of kernel space.

System Calls in XNU

XNU provides system calls through multiple interfaces:

Mach Traps

Low-level Mach operations:

// Example Mach trap numbers
#define MACH_MSG_TRAP         -31
#define MACH_REPLY_PORT       -26
#define THREAD_SELF_TRAP      -27
#define TASK_SELF_TRAP        -28

BSD System Calls

Standard Unix system calls:

# View system call numbers
$ grep -E "^#define\s+SYS_" /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/syscall.h | head -20
#define SYS_syscall        0
#define SYS_exit           1
#define SYS_fork           2
#define SYS_read           3
#define SYS_write          4
#define SYS_open           5
#define SYS_close          6
#define SYS_wait4          7
...

Machine-Dependent Calls

Architecture-specific operations.

Kernel Debugging and Introspection

Several tools expose XNU internals:

sysctl

The sysctl interface queries and sets kernel parameters:

# View kernel information
$ sysctl kern.ostype kern.osrelease kern.hostname
kern.ostype: Darwin
kern.osrelease: 23.4.0
kern.hostname: MacBook-Pro.local

# List all kernel parameters
$ sysctl -a | wc -l
    1847

# View memory parameters
$ sysctl vm
vm.loadavg: { 1.83 2.07 2.15 }
vm.swapusage: total = 0.00M  used = 0.00M  free = 0.00M  (encrypted)
vm.cs_validation: 1
...

DTrace (Where Available)

On systems where it’s available, DTrace provides deep kernel tracing:

# Trace system calls (requires SIP adjustment)
$ sudo dtrace -n 'syscall:::entry { @[execname] = count(); }'

Note: System Integrity Protection restricts DTrace on modern macOS.

Instruments

Apple’s Instruments application provides profiling using kernel trace facilities:

# Command-line profiling
$ xctrace record --template 'Time Profiler' --launch -- /path/to/app

XNU Source Code Structure

For those interested in exploring the code:

xnu/
├── bsd/           # BSD layer (processes, filesystems, networking)
├── iokit/         # IOKit driver framework
├── libkern/       # Kernel C++ runtime
├── libsa/         # Standalone library
├── osfmk/         # Mach layer (scheduling, IPC, VM)
├── pexpert/       # Platform expert (hardware abstraction)
└── security/      # Security policies (MAC framework)

The source is available at opensource.apple.com, though building it requires significant effort.

Why XNU’s Architecture Matters

Understanding XNU helps explain several macOS behaviors:

1. Why Mach-O binaries? XNU uses Mach-O format, not ELF like Linux
2. Why different process tools? BSD and Mach layers provide overlapping but different views
3. Why complex memory semantics? Mach’s VM model has unique characteristics
4. Why DriverKit? Moving away from kernel extensions reflects security priorities

The hybrid nature of XNU—combining Mach’s IPC and memory management with BSD’s Unix compatibility—creates a unique system that’s familiar yet different from both Linux and traditional BSD.

macOS vs Linux vs BSD: Key Differences

If you’re coming to macOS from Linux or BSD, many things will feel familiar—but the differences can be surprising and sometimes frustrating. This chapter catalogs the key differences you’ll encounter, explaining not just what differs but why.

Kernel and System Architecture

Kernel Type

Impact: macOS uses Mach-O binary format instead of ELF. Tools that work with binaries (nm, objdump, ldd) have different equivalents on macOS.

# Linux: View shared libraries
$ ldd /bin/ls

# macOS: View shared libraries (different tool)
$ otool -L /bin/ls
/bin/ls:
    /usr/lib/libutil.dylib (compatibility version 1.0.0, current version 1.0.0)
    /usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current version 5.4.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1336.61.1)

Init System

Impact: Service management is completely different on macOS. There’s no systemctl, no /etc/init.d/, no chkconfig.

# Linux (systemd)
$ sudo systemctl start nginx
$ sudo systemctl enable nginx

# macOS (launchd)
$ sudo launchctl load /Library/LaunchDaemons/homebrew.mxcl.nginx.plist
$ sudo launchctl enable system/homebrew.mxcl.nginx

Command-Line Tools

BSD vs GNU Commands

macOS ships with BSD versions of common commands. These often have different options than GNU versions:

sed

# GNU sed (Linux): In-place edit
$ sed -i 's/old/new/g' file.txt

# BSD sed (macOS): Requires backup extension (use '' for no backup)
$ sed -i '' 's/old/new/g' file.txt

grep

# GNU grep: Perl regex with -P
$ grep -P '\d{3}-\d{4}' file.txt

# BSD grep (macOS): No -P option, use -E for extended regex
$ grep -E '[0-9]{3}-[0-9]{4}' file.txt

ls

# GNU ls: Color output with --color
$ ls --color=auto

# BSD ls (macOS): Color with -G
$ ls -G

# GNU ls: Human-readable with -h requires -l
$ ls -lh

# BSD ls: -h works without -l (but usually used with -l anyway)
$ ls -lh

date

# GNU date: Specific date formatting
$ date -d "2024-01-15" +%s

# BSD date (macOS): Different syntax
$ date -j -f "%Y-%m-%d" "2024-01-15" +%s

stat

# GNU stat
$ stat --format="%s" file.txt

# BSD stat (macOS)
$ stat -f "%z" file.txt

readlink

# GNU readlink: -f to canonicalize
$ readlink -f /path/to/symlink

# BSD readlink (macOS): No -f, but realpath exists (macOS 12.3+)
$ realpath /path/to/symlink
# Or for older macOS:
$ python3 -c "import os; print(os.path.realpath('/path/to/symlink'))"

macOS-Specific Commands

macOS includes commands with no Linux/BSD equivalent:

# Copy to clipboard
$ echo "Hello" | pbcopy
$ pbpaste

# Open files with default application
$ open file.pdf
$ open -a Safari https://apple.com
$ open .  # Open current directory in Finder

# Spotlight search
$ mdfind "search term"
$ mdfind -name "filename"
$ mdfind "kMDItemContentType == 'public.jpeg'"

# Text-to-speech
$ say "Hello, world"

# Manage system preferences
$ open "x-apple.systempreferences:"

# Screen capture
$ screencapture -i screenshot.png

# Airport (Wi-Fi) utility
$ /System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport -s

# Disk utility
$ diskutil list
$ diskutil info disk0

Network Commands

Networking tools differ significantly:

# Linux: Modern networking
$ ip addr show
$ ip route
$ ss -tuln

# macOS: BSD networking
$ ifconfig
$ netstat -rn
$ netstat -an | grep LISTEN

Note: ip doesn’t exist on macOS (unless installed via Homebrew). macOS uses traditional BSD tools like ifconfig, netstat, and route.

# Linux: Network manager
$ nmcli device status

# macOS: networksetup
$ networksetup -listallhardwareports
$ networksetup -getinfo "Wi-Fi"

Filesystem Differences

Filesystem Types

Case Sensitivity

macOS is case-insensitive but case-preserving by default:

# On default macOS filesystem:
$ touch File.txt
$ ls file.txt
File.txt           # Finds it! Different from Linux

This can cause issues with Git repositories from case-sensitive systems.

Filesystem Hierarchy

Key differences from the Filesystem Hierarchy Standard (FHS):

Extended Attributes

macOS uses extended attributes heavily:

# View extended attributes
$ xattr file.txt
com.apple.quarantine

# List with values
$ xattr -l file.txt
com.apple.quarantine: 0083;5f4a7c14;Safari;...

# Remove quarantine attribute
$ xattr -d com.apple.quarantine file.txt

# Clear all attributes
$ xattr -c file.txt

Linux has extended attributes too, but they’re less pervasive:

# Linux extended attributes
$ getfattr -d file.txt
$ setfattr -n user.comment -v "test" file.txt

Resource Forks and Metadata

macOS preserves metadata when copying files:

# macOS: Copy with metadata
$ cp -p file.txt dest/          # Preserves basic metadata
$ ditto source/ dest/           # Preserves everything including resource forks

# Problem: copying to non-HFS+/APFS filesystems
$ cp file.txt /Volumes/USB_FAT32/
# Creates file.txt and ._file.txt (AppleDouble file)

Those ._* files you see on USB drives? That’s macOS preserving metadata on filesystems that don’t support it natively.

Package Management

No Native Package Manager

Unlike most Linux distributions and BSD systems, macOS doesn’t include a built-in package manager for third-party software:

Solutions for macOS:

# Homebrew (most popular)
$ brew install wget

# MacPorts (alternative)
$ sudo port install wget

User and Permission Differences

User IDs

Groups

macOS has different default groups:

$ id
uid=501(david) gid=20(staff) groups=20(staff),12(everyone),61(localaccounts),...

The primary group for users is staff (GID 20), not a user-specific group.

System Integrity Protection (SIP)

macOS restricts what even root can do:

# This fails even as root (SIP protected):
$ sudo rm /System/Library/CoreServices/Finder.app
rm: /System/Library/CoreServices/Finder.app: Operation not permitted

# Check SIP status
$ csrutil status
System Integrity Protection status: enabled.

Linux and BSD have no equivalent to SIP—root can do anything.

Sudo Configuration

macOS doesn’t use /etc/sudoers by default for most permissions. Instead, it uses admin group membership:

# Check who can sudo
$ grep -E "^%admin|^%wheel" /etc/sudoers
%admin		ALL = (ALL) ALL

Process and System Management

Process Listing

# Both work on macOS:
$ ps aux                    # BSD syntax
$ ps -ef                    # POSIX syntax

# Linux-specific options don't work:
$ ps --forest              # Error on macOS

System Information

# Linux
$ cat /proc/cpuinfo
$ free -h
$ lsb_release -a

# macOS (no /proc filesystem)
$ sysctl -n machdep.cpu.brand_string
$ vm_stat                   # Memory (in pages, not bytes)
$ sw_vers                   # macOS version

Signals

Signal numbers can differ:

# Check signal numbers
$ kill -l

Most common signals (SIGTERM=15, SIGKILL=9, SIGHUP=1) are the same.

Development Environment

Compiler

# Linux (usually GCC)
$ gcc --version
gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0

# macOS (Clang masquerading as gcc)
$ gcc --version
Apple clang version 15.0.0 (clang-1500.3.9.4)

The gcc command on macOS is actually Clang. This usually doesn’t matter, but some software with GCC-specific features may need adjustment.

Libraries

# Linux: Shared library extension
libfoo.so, libfoo.so.1, libfoo.so.1.0.0

# macOS: Shared library extension
libfoo.dylib, libfoo.1.dylib, libfoo.1.0.0.dylib

# Linux: Find library
$ ldconfig -p | grep libssl
$ ldd /usr/bin/openssl

# macOS: Find library
$ otool -L /usr/bin/openssl

Library Paths

# Linux
$ echo $LD_LIBRARY_PATH
$ cat /etc/ld.so.conf

# macOS
$ echo $DYLD_LIBRARY_PATH           # Runtime
$ echo $DYLD_FALLBACK_LIBRARY_PATH  # Fallback

Virtualization and Containers

Docker

# Linux: Docker runs natively
$ docker run alpine echo "Hello"

# macOS: Docker runs in a Linux VM
$ docker run alpine echo "Hello"
# (Actually runs in a hypervisor-backed Linux VM)

Docker Desktop on macOS uses a Linux virtual machine because containers are a Linux kernel feature.

Native Virtualization

# macOS: Virtualization.framework (Apple Silicon) or Hypervisor.framework
# No direct CLI, used by apps like UTM, Parallels, Docker Desktop

# Linux: KVM
$ virsh list --all

Summary Table

Understanding these differences helps you translate your Unix knowledge to macOS effectively. When something doesn’t work as expected, check whether it’s a BSD vs GNU difference, a filesystem assumption, or a macOS-specific security feature.

POSIX Compliance and Standards

When discussing Unix compatibility, you’ll often hear about “POSIX compliance.” macOS is notable for being both POSIX-compliant and UNIX-certified—distinctions that matter for software portability and system behavior.

What Is POSIX?

POSIX (Portable Operating System Interface) is a family of standards specified by the IEEE Computer Society for maintaining compatibility between operating systems. POSIX defines:

• System interfaces: C API functions like open(), fork(), pthread_create()
• Shell and utilities: Command-line tools and their expected behavior
• Shell language: The syntax and features of POSIX shell scripts
• Threads: The pthreads API for multi-threaded programming
• Real-time extensions: APIs for real-time computing

The goal: write code once, run it on any POSIX-compliant system.

POSIX vs UNIX Certification

These are related but different:

POSIX Compliance

A system that implements POSIX interfaces. Self-declared, no formal certification required.

• Linux: POSIX-compliant (mostly)
• FreeBSD: POSIX-compliant
• macOS: POSIX-compliant and certified

UNIX Certification

A formal certification from The Open Group, which owns the UNIX trademark. Requires:

1. Passing conformance tests
2. Paying certification fees
3. Regular re-certification

Currently certified UNIX systems:

• macOS (UNIX 03)
• IBM AIX
• HP-UX
• Oracle Solaris

Notable non-certified systems:

• Linux (too expensive/complex to certify each distribution)
• FreeBSD (philosophically opposed to the process)

What Certification Means

macOS’s UNIX 03 certification means it has passed The Open Group’s test suite demonstrating compliance with:

• Single UNIX Specification version 3 (SUS v3)
• POSIX.1-2001
• Related standards

You can verify this:

# Check POSIX version supported
$ getconf _POSIX_VERSION
200112

# Check UNIX version (SUS)
$ getconf _XOPEN_VERSION
600

POSIX on macOS in Practice

Standard Headers

macOS provides all required POSIX headers:

$ ls /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/ | head
_endian.h
_posix_availability.h
_pthread
_select.h
_structs.h
_symbol_aliasing.h
_types
_types.h
acct.h
acl.h

Feature Test Macros

To access POSIX-specific features in C code:

#define _POSIX_C_SOURCE 200112L  // Request POSIX.1-2001 features
#include <unistd.h>
#include <sys/types.h>
#include <sys/wait.h>

Or for all UNIX features:

#define _XOPEN_SOURCE 600  // Request SUS v3 features
#include <stdlib.h>

Checking POSIX Limits

POSIX defines various system limits that can be queried:

# Maximum arguments to exec()
$ getconf ARG_MAX
1048576

# Maximum filename length
$ getconf NAME_MAX /
255

# Maximum path length
$ getconf PATH_MAX /
1024

# Number of processors
$ getconf _NPROCESSORS_ONLN
10

# POSIX version
$ getconf _POSIX_VERSION
200112

In C code:

#include <unistd.h>
#include <limits.h>

long arg_max = sysconf(_SC_ARG_MAX);
long open_max = sysconf(_SC_OPEN_MAX);
long nprocessors = sysconf(_SC_NPROCESSORS_ONLN);

POSIX Shell Scripting

POSIX defines a portable shell language. Scripts targeting POSIX should:

Use /bin/sh

#!/bin/sh
# Not #!/bin/bash or #!/bin/zsh

On macOS, /bin/sh is actually bash running in POSIX mode (or zsh on newer versions in some contexts):

$ file /bin/sh
/bin/sh: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e]

$ /bin/sh --version
GNU bash, version 3.2.57(1)-release (arm64-apple-darwin23)

Avoid Bashisms

Common bash features that aren’t POSIX:

# Non-POSIX (bash-specific)
[[ $var == "test" ]]     # Double brackets
array=(one two three)     # Arrays
${var//old/new}          # Pattern substitution
$((var++))               # C-style increment
source file.sh           # source command

# POSIX equivalents
[ "$var" = "test" ]      # Single brackets, single =
# No arrays in POSIX
# Use sed for substitution
var=$((var + 1))         # POSIX arithmetic
. file.sh                # Dot command

Test Your Scripts

Check scripts for POSIX compliance:

# Install shellcheck
$ brew install shellcheck

# Check a script
$ shellcheck --shell=sh myscript.sh

Common POSIX-Related Issues

Shebang Paths

POSIX doesn’t specify interpreter locations:

#!/bin/bash              # Works on most Linux, macOS
#!/usr/bin/env bash      # More portable - finds bash in PATH

On some systems, bash might be in /usr/local/bin/bash or elsewhere.

Command Options

Even when commands exist, options may differ:

# POSIX requires these options for 'ls'
# -a, -A, -C, -d, -F, -g, -i, -l, -n, -o, -p, -q, -r, -R, -s, -t, -u, -1

# These are NOT required by POSIX:
ls --color     # GNU extension
ls -G          # BSD extension (macOS)

printf vs echo

echo behavior varies between systems. POSIX recommends printf:

# Portable
printf '%s\n' "Hello"
printf 'Value: %d\n' 42

# Not portable (echo -n, echo -e behavior varies)
echo -n "No newline"      # Not POSIX
echo -e "Tab:\tHere"     # Not POSIX

Regular Expressions

POSIX defines Basic Regular Expressions (BRE) and Extended Regular Expressions (ERE):

# BRE (default for grep, sed)
grep 'a\+b'              # + must be escaped
grep 'a\{2,3\}'          # Braces must be escaped

# ERE (grep -E, egrep, awk)
grep -E 'a+b'            # + doesn't need escaping
grep -E 'a{2,3}'         # Braces don't need escaping

Perl-compatible regex (PCRE) is NOT part of POSIX:

grep -P '\d+'            # GNU extension - NOT on macOS
grep -E '[0-9]+'         # POSIX ERE equivalent

Extensions Beyond POSIX

macOS includes extensions beyond POSIX requirements:

BSD Extensions

# sysctl - BSD system control
$ sysctl kern.hostname

# BSD-style ps flags
$ ps aux

# Disk management
$ diskutil list

Apple Extensions

# Spotlight search
$ mdfind "query"

# Clipboard
$ pbcopy < file.txt

# Open files
$ open document.pdf

# System configuration
$ defaults read com.apple.finder

These extensions are not portable to Linux or other systems.

Writing Portable Code

For Shell Scripts

1. Use #!/bin/sh and stick to POSIX features
2. Test with shellcheck --shell=sh
3. Avoid command options not listed in POSIX
4. Use printf instead of echo for complex output
5. Use [ ] instead of [[ ]]

#!/bin/sh
# Portable script example

set -e  # Exit on error (POSIX)

# Check if file exists
if [ -f "$1" ]; then
    printf 'Processing: %s\n' "$1"
    # Use cat (POSIX) not bashisms
    cat "$1" | while IFS= read -r line; do
        printf '%s\n' "$line"
    done
else
    printf 'File not found: %s\n' "$1" >&2
    exit 1
fi

For C Programs

1. Include _POSIX_C_SOURCE or _XOPEN_SOURCE appropriately
2. Check return values (POSIX mandates specific error codes)
3. Use configure scripts to detect platform differences
4. Provide fallbacks for non-POSIX features

#define _POSIX_C_SOURCE 200809L
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>

int main(void) {
    char *cwd = getcwd(NULL, 0);  // POSIX.1-2001
    if (cwd == NULL) {
        perror("getcwd");
        return EXIT_FAILURE;
    }
    printf("Current directory: %s\n", cwd);
    free(cwd);
    return EXIT_SUCCESS;
}

Autoconf and Portable Building

For larger projects, GNU Autotools help detect platform differences:

# Generate configure script
$ autoreconf -i

# Configure for the current platform
$ ./configure

# Build
$ make

The configure script detects macOS vs Linux differences and sets appropriate compiler flags.

Checking Standards Compliance

On macOS

# Compiler standards mode
$ cc -std=c11 -D_POSIX_C_SOURCE=200809L -Wall -pedantic program.c

# Check what POSIX version the system claims
$ getconf POSIX_VERSION
200112

# Check POSIX limits
$ getconf -a | grep POSIX

Useful References

• POSIX.1-2017 specification
• Apple’s UNIX Conformance documentation
• IEEE Std 1003.1 (requires IEEE subscription)

Summary

macOS’s POSIX compliance and UNIX certification mean:

1. Standard APIs work: POSIX C functions behave correctly
2. Basic commands exist: Core utilities are available
3. Shell scripts can be portable: With discipline
4. Not everything is standard: macOS and Apple add extensions

When writing portable code:

• Stick to POSIX-defined features when possible
• Test on multiple platforms
• Use tools like shellcheck and autoconf
• Be aware that BSD and GNU versions of tools differ

The standards provide a foundation, but real-world portability requires attention to detail and testing across target systems.

Understanding macOS Filesystems

Filesystems are where Unix heritage meets Apple innovation. While macOS supports familiar Unix filesystem concepts—files, directories, permissions, symbolic links—it implements them atop storage technologies that differ significantly from traditional Unix systems.

Understanding macOS filesystems means understanding three layers:

1. APFS: The modern storage technology providing advanced features like snapshots, clones, and space sharing
2. The VFS layer: How macOS presents a unified filesystem interface to applications
3. macOS conventions: Extended attributes, metadata, and organizational patterns unique to Mac

The Evolution of Mac Storage

The journey from original Macintosh to modern macOS represents a dramatic evolution:

1984-1998: The original Macintosh File System (MFS) and its successor HFS

• Resource forks and data forks as first-class concepts
• Case-insensitive by design
• No Unix permissions (Mac OS had no concept of users)

1998-2017: HFS+ (Mac OS Extended)

• Journaling for data integrity
• Support for Unix permissions (added for Mac OS X)
• Still case-insensitive by default
• Resource forks maintained for compatibility

2017-Present: APFS (Apple File System)

• Built for flash storage and SSDs
• Native encryption
• Snapshots and cloning
• Space sharing between volumes
• Case-sensitive option more practical

What You’ll Learn in This Part

APFS: Apple’s Modern Filesystem explains the architecture and capabilities of Apple’s current filesystem, including features like space sharing, snapshots, and encryption.

HFS+ Legacy and Migration covers the previous filesystem format, why you might still encounter it, and how to handle legacy volumes.

Case Sensitivity: Options and Implications addresses one of the most misunderstood aspects of macOS filesystems—why case-insensitivity is the default and when it matters.

Extended Attributes and Resource Forks explores macOS’s rich file metadata system, including quarantine attributes, Finder information, and the legacy of resource forks.

The Metadata Files: .DS_Store and ._AppleDouble explains those mysterious files that appear everywhere and how to manage them.

Filesystem Hierarchy: Where macOS Diverges maps out where things live on macOS compared to the Filesystem Hierarchy Standard used by Linux.

Disk Management from the Command Line teaches you to manage disks, partitions, and volumes using Terminal commands.

Volumes, Containers, and Snapshots dives deep into APFS’s container architecture and how to work with snapshots.

Why Filesystems Matter

As a Unix user, you might assume filesystems are interchangeable—files are files, directories are directories. On macOS, this assumption can lead to problems:

• Copy a file to a FAT32 USB drive and mysterious ._ files appear
• Git reports changes to files you didn’t modify (case sensitivity issues)
• Scripts fail because they assume /opt exists or /tmp persists across reboots
• Files downloaded from the internet won’t run (quarantine attributes)

Understanding macOS filesystems helps you avoid these pitfalls and leverage features like snapshots and clones that traditional Unix filesystems lack.

APFS: Apple’s Modern Filesystem

Apple File System (APFS) replaced HFS+ as macOS’s default filesystem in 2017. Designed for flash storage and modern security requirements, APFS brings capabilities that Unix veterans may find familiar from ZFS or Btrfs—but with Apple’s distinctive implementation choices.

Design Goals

APFS was designed to address HFS+’s limitations:

• Flash optimization: HFS+ was designed for spinning disks; APFS is optimized for SSDs
• Crash protection: Copy-on-write metadata ensures consistency without full-volume journaling
• Encryption: Native encryption integrated at the filesystem level
• Space efficiency: Space sharing eliminates wasted space from traditional partitioning
• Snapshots: Point-in-time filesystem states for backups and Time Machine
• Clones: Instant file copies that share underlying data

Core Concepts

Containers and Volumes

APFS introduces a two-level hierarchy that differs from traditional partitioning:

┌─────────────────────────────────────────────────┐
│                Physical Disk                     │
├─────────────────────────────────────────────────┤
│              APFS Container                      │
│  ┌─────────┐ ┌─────────┐ ┌─────────┐           │
│  │ Volume  │ │ Volume  │ │ Volume  │           │
│  │ (macOS) │ │ (Data)  │ │ (Preboot│           │
│  │         │ │         │ │         │           │
│  └─────────┘ └─────────┘ └─────────┘           │
│                                                  │
│       All volumes share container space          │
└─────────────────────────────────────────────────┘

Container: The APFS equivalent of a partition. A container occupies a contiguous region of the disk and can hold multiple volumes.

Volume: A logical filesystem within a container. Multiple volumes share the container’s space dynamically—no need to pre-allocate sizes.

View your system’s structure:

$ diskutil list
/dev/disk0 (internal):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *500.1 GB   disk0
   1:             Apple_APFS_ISC Container disk1         524.3 MB   disk0s1
   2:                 Apple_APFS Container disk3         494.4 GB   disk0s2
   3:        Apple_APFS_Recovery Container disk2         5.4 GB     disk0s3

/dev/disk3 (synthesized):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      APFS Container Scheme -                      +494.4 GB   disk3
   1:                APFS Volume Macintosh HD            9.8 GB     disk3s1
   2:              APFS Snapshot com.apple.os.update-... 9.8 GB     disk3s1s1
   3:                APFS Volume Preboot                 5.3 GB     disk3s2
   4:                APFS Volume Recovery                1.6 GB     disk3s3
   5:                APFS Volume Data                    256.3 GB   disk3s5
   6:                APFS Volume VM                      3.2 GB     disk3s6

Space Sharing

Unlike traditional partitions with fixed sizes, APFS volumes share space:

# View container space usage
$ diskutil apfs list
...
+-- Container disk3 494.4 GB
    ====================================================
    APFS Container Reference:     disk3
    Size (Capacity Ceiling):      494384795648 B (494.4 GB)
    Capacity In Use By Volumes:   275847008256 B (275.8 GB) (55.8%)
    Capacity Not Allocated:       218537787392 B (218.5 GB) (44.2%)
    |
    +-< Physical Store disk0s2 494.4 GB
    |
    +-> Volume disk3s1 Macintosh HD
    |   ---------------------------------------------------
    |   APFS Volume Disk (Role):   disk3s1 (System)
    |   Name:                      Macintosh HD
    |   Mount Point:               /
    |   Capacity Consumed:         9.8 GB
    ...

The “Capacity Not Allocated” is available to any volume in the container—no repartitioning needed.

Volume Roles

APFS assigns roles to volumes that affect their behavior:

# View volume roles
$ diskutil apfs listVolumes disk3
...
Role:              Data
Role:              System
Role:              Preboot
Role:              Recovery
Role:              VM

Key Features

Copy-on-Write

APFS uses copy-on-write (CoW) for metadata and optionally for data:

• Metadata: Always CoW; changes write to new locations, then atomically update references
• Data: CoW when cloning files; regular writes may overwrite in place

This provides crash protection without the overhead of journaling every write.

Clones: Instant File Copies

When you copy a file on APFS, the filesystem can create a clone:

# Create a clone (instant, regardless of file size)
$ cp -c largefile.iso largefile_copy.iso

# Both files share the same data blocks
$ ls -ls largefile.iso largefile_copy.iso
0 -rw-r--r--  1 user  staff  4700000000 Jan 15 10:30 largefile.iso
0 -rw-r--r--  1 user  staff  4700000000 Jan 15 10:30 largefile_copy.iso

Notice the 0 in the first column—the copy uses zero additional disk blocks. Modifications to either file write only the changed blocks.

Note: The -c flag requests cloning, but cp on macOS uses clones automatically when possible. The flag ensures failure if cloning isn’t possible (e.g., cross-volume copies).

Snapshots

Snapshots capture the state of a volume at a point in time:

# List snapshots
$ tmutil listlocalsnapshots /
Snapshots for volume group containing disk /:
com.apple.TimeMachine.2024-01-15-091234.local
com.apple.TimeMachine.2024-01-14-091234.local
...

# Or using diskutil
$ diskutil apfs listSnapshots disk3s1
Snapshots for disk3s1 (1 found)
|
+-- XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
    Snapshot Name:        com.apple.os.update-XXXXXXX
    Snapshot UUID:        XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
    Snapshot Sealed:      Yes

Time Machine creates local snapshots hourly. The system creates snapshots before updates for rollback capability.

Creating and managing snapshots:

# Create a snapshot (requires admin privileges)
$ sudo tmutil localsnapshot /
Created local snapshot with date: 2024-01-15-103045

# Delete a specific snapshot
$ sudo tmutil deletelocalsnapshots 2024-01-15-103045

# Delete all local snapshots
$ sudo tmutil deletelocalsnapshots /

Native Encryption

APFS supports encryption at multiple levels:

No encryption: Volume is unencrypted

Single-key encryption: Entire volume encrypted with one key

Multi-key encryption: Metadata encrypted with one key, file contents with per-file or per-extent keys

# Check encryption status
$ diskutil apfs list | grep -A2 "FileVault:"
    FileVault:                 Yes (Unlocked)

# Encrypt a volume
$ diskutil apfs encryptVolume disk3s5 -user disk

FileVault 2 on APFS uses native APFS encryption rather than the Core Storage layer used with HFS+.

Space Efficiency Features

Sparse files: Files with holes don’t consume space for zero-filled regions:

# Create a sparse file (1GB virtual, ~0 actual)
$ dd if=/dev/zero of=sparse.img bs=1 count=0 seek=1G 2>/dev/null
$ ls -ls sparse.img
0 -rw-r--r--  1 user  staff  1073741824 Jan 15 10:45 sparse.img

Fast directory sizing: APFS tracks directory sizes efficiently:

# This is fast on APFS
$ diskutil info disk3s5 | grep "Used"
   Container Total Space:    494.4 GB (494384795648 Bytes)...
   Volume Used Space:        256.3 GB (256300000000 Bytes)...

Atomic Safe-Save

APFS supports atomic rename operations that enable safe document saving:

1. Write new content to a temporary file
2. Atomically rename temporary file to replace original
3. Original is safely replaced without corruption risk

macOS applications use this pattern via NSDocument’s save methods.

APFS Limitations

No Built-in Compression

Unlike HFS+ (which supported transparent compression) or ZFS/Btrfs (which offer compression), APFS doesn’t compress data:

# Check if a file is compressed (HFS+ compression, may still exist)
$ ls -lO file.txt
-rw-r--r--  1 user  staff  compressed  1000 Jan 15 10:50 file.txt

# APFS doesn't add new compression; existing compressed files remain compressed

No Built-in Checksumming

APFS checksums metadata but not user data. Unlike ZFS, APFS cannot detect silent data corruption (bit rot) in file contents.

Case Sensitivity Trade-offs

APFS supports case-sensitive volumes, but the default (case-insensitive) maintains compatibility with existing macOS conventions and applications.

Fusion Drive Complexity

APFS with Fusion Drives (SSD + HDD combinations) works but was added later and lacks some optimizations present on pure SSD.

Working with APFS

Creating Volumes

# Add a volume to an existing container
$ sudo diskutil apfs addVolume disk3 APFS "MyVolume"
# or with case-sensitivity
$ sudo diskutil apfs addVolume disk3 "Case-sensitive APFS" "MyVolume"

# Add an encrypted volume
$ sudo diskutil apfs addVolume disk3 APFS "SecureVolume" -passphrase

Deleting Volumes

# Delete a volume (space returns to container)
$ sudo diskutil apfs deleteVolume disk3s7

Resizing Containers

# Resize a container
$ sudo diskutil apfs resizeContainer disk3 400GB

# Resize to fill available space
$ sudo diskutil apfs resizeContainer disk3 0

Converting from HFS+

# Non-destructive conversion (if supported)
$ sudo diskutil apfs convert disk2s1

# Check conversion eligibility
$ diskutil info disk2s1 | grep "APFS"

Checking Filesystem Health

# Verify an APFS volume
$ diskutil verifyVolume disk3s1
Started file system verification on disk3s1 Macintosh HD
Verifying storage system
...
Verified storage system
Storage system check exit code is 0
Finished file system verification on disk3s1 Macintosh HD

# Repair (requires booting to Recovery for system volume)
$ sudo diskutil repairVolume disk3s5

APFS vs Other Filesystems

Summary

APFS represents a significant advancement over HFS+, bringing macOS filesystems into the modern era with:

• Efficient space sharing through containers and volumes
• Instant file clones and volume snapshots
• Native encryption without external layers
• Crash protection through copy-on-write

For Unix users accustomed to ZFS or Btrfs, APFS will feel familiar in concept but different in implementation. Understanding these differences—and accepting that APFS is optimized for different priorities (consumer devices, flash storage, tight hardware integration)—helps you work effectively with macOS storage.

HFS+ Legacy and Migration

While APFS is now the default, HFS+ (Mac OS Extended) served macOS for nearly two decades and remains relevant. You’ll encounter it on older systems, external drives, Time Machine backups from before 2017, and in specific scenarios where APFS isn’t appropriate.

HFS+ History and Design

HFS+ debuted in 1998 as an evolution of the original Hierarchical File System (HFS) from 1985. Key improvements included:

• 32-bit allocation blocks: vs HFS’s 16-bit, allowing efficient use of large volumes
• Unicode filenames: Up to 255 UTF-16 characters
• Journaling: Added in Mac OS X 10.2.2 (2002) for crash recovery
• Case sensitivity option: Added for Unix compatibility

HFS+ Variants

HFS+ comes in several flavors, identified in diskutil:

# Identify HFS+ volumes
$ diskutil list
...
   2:                  Apple_HFS Backup                   1.0 TB     disk2s2
...

$ diskutil info disk2s2 | grep "Type"
   Type (Bundle):                  hfs
   File System Personality:        Mac OS Extended (Journaled)

Why HFS+ Still Exists

Compatibility Scenarios

Older Mac hardware: Macs that can’t run macOS High Sierra or later require HFS+.

Bootable external drives for old Macs: If you need a bootable drive for older hardware.

Windows cross-compatibility: Some tools read HFS+ but not APFS. (Note: Neither is truly cross-platform without third-party tools.)

Specific applications: Some virtualization or disk imaging tools may expect HFS+.

Time Machine Archives

Time Machine backups created before APFS store data on HFS+:

# Check Time Machine drive format
$ diskutil info /Volumes/TimeMachine | grep "File System"
   File System Personality:        Mac OS Extended (Journaled)

Modern Time Machine on APFS uses snapshots differently than the hardlink-based approach on HFS+.

HFS+ Features

Journaling

The journal records filesystem changes before they occur, enabling recovery after crashes:

# Check journaling status
$ diskutil info disk2s2 | grep -i journal
   File System Personality:        Mac OS Extended (Journaled)
   Journal:                        Journal size 40960 KB at offset 0x...

# Enable journaling
$ sudo diskutil enableJournal disk2s2

# Disable journaling (not recommended)
$ sudo diskutil disableJournal disk2s2

Transparent Compression

HFS+ supports transparent file compression (introduced in Snow Leopard):

# Check if a file is compressed
$ ls -lO /bin/ls
-rwxr-xr-x  1 root  wheel  compressed  38704 Jan  1  2020 /bin/ls

# The compressed flag indicates HFS+ compression
# Actual on-disk size may be smaller than reported size

# Get actual disk usage
$ stat -f "%z bytes (logical), %b blocks (physical)" /bin/ls

Note: While APFS carries forward compressed files, it doesn’t actively compress new files.

Resource Forks and Extended Attributes

HFS+ natively supports Apple’s extended file metadata:

# View resource fork (if present)
$ ls -l /path/to/file/..namedfork/rsrc

# View extended attributes
$ xattr -l file

Working with HFS+ Volumes

Creating HFS+ Volumes

# Format as HFS+ (Journaled)
$ sudo diskutil eraseDisk JHFS+ "BackupDrive" GPT disk2

# Format a single partition
$ sudo diskutil eraseVolume JHFS+ "NewVolume" disk2s2

# Create case-sensitive HFS+
$ sudo diskutil eraseDisk JHFS+X "CaseSensitive" GPT disk2

Verifying and Repairing

# Verify HFS+ volume
$ sudo diskutil verifyVolume disk2s2
Started file system verification on disk2s2 Backup
Checking Journaled HFS Plus volume
...
The volume Backup appears to be OK
Finished file system verification on disk2s2 Backup

# Repair HFS+ volume
$ sudo diskutil repairVolume disk2s2

# For more serious issues, use fsck_hfs directly
$ sudo fsck_hfs -fy /dev/disk2s2

Command-Line Formatting with newfs_hfs

For more control, use the underlying filesystem tools:

# Create HFS+ filesystem with specific options
$ sudo newfs_hfs -v "MyVolume" -J /dev/disk2s2

# Options:
#   -v: Volume name
#   -J: Enable journaling
#   -s: Case-sensitive
#   -b: Block size

Converting HFS+ to APFS

In-Place Conversion

Apple provides non-destructive HFS+ to APFS conversion:

# Check if conversion is possible
$ diskutil info disk2s2 | grep -i "APFS"

# Convert (non-destructive, but backup first!)
$ sudo diskutil apfs convert disk2s2
Converting HFS volume to APFS...
...

Conversion requirements:

• macOS High Sierra or later
• Journaled HFS+ (not plain HFS+)
• No Fusion Drive issues
• Sufficient free space for conversion process

When Conversion Isn’t Possible

If conversion fails or isn’t desired:

# Backup, erase, restore approach
# 1. Backup all data
# 2. Erase as APFS
$ sudo diskutil eraseDisk APFS "NewDrive" GPT disk2
# 3. Restore data

HFS+ Limitations

No Space Sharing

HFS+ requires fixed partition sizes:

# Partition must be resized explicitly
$ sudo diskutil resizeVolume disk2s2 500GB

No Native Snapshots

HFS+ lacks APFS’s snapshot capability. Time Machine on HFS+ uses hard links to directories, a workaround with its own complexities.

Performance on SSD

HFS+ wasn’t designed for flash storage:

• No TRIM support optimization
• Journaling patterns not ideal for SSD
• File operations less efficient than APFS

File Size and Volume Limits

Practical limits are lower due to macOS constraints.

Interacting with HFS+ from Unix Perspective

Mount Options

Hbash

View mount options for HFS+ volume

$ mount | grep hfs /dev/disk2s2 on /Volumes/Backup (hfs, local, nodev, nosuid, journaled)

Mount HFS+ volume read-only

$ sudo mount -t hfs -o ro /dev/disk2s2 /Volumes/Backup

### File System Attributes

```bash
# Get HFS+ specific information
$ /usr/sbin/fstyp /dev/disk2s2
hfs

# Detailed filesystem info
$ sudo diskutil info -plist disk2s2 | plutil -p -

Common Issues

Catalog File Corruption

HFS+’s B-tree catalog can become corrupted:

# Symptoms: Files disappear, disk errors
# Solution: Run Disk Utility or fsck_hfs
$ sudo fsck_hfs -fy /dev/disk2s2

Journal Overflow

On heavily used volumes, the journal can overflow:

# Symptoms: Slow performance, errors
# Solution: Recreate journal
$ sudo diskutil disableJournal disk2s2
$ sudo diskutil enableJournal disk2s2

Permissions Repair (Deprecated)

Older macOS versions had “Repair Disk Permissions.” This is no longer relevant on modern macOS where system files are protected by SIP.

When to Use HFS+ Today

Recommended for:

• External drives shared with older Macs
• Time Machine destinations for older Mac backups
• Virtual machine disk images (check VM software requirements)
• Bootable drives for pre-High Sierra Macs

Not recommended for:

• Primary macOS boot drives (use APFS)
• General storage on modern Macs
• SSD drives (APFS is better optimized)

Summary

HFS+ served macOS well for nearly 20 years, but APFS represents the future. Understanding HFS+ remains valuable for:

• Working with legacy systems and backups
• Troubleshooting older storage media
• Understanding the evolution of macOS storage
• Specific compatibility requirements

For new storage, prefer APFS unless you have a specific reason to use HFS+. The performance, reliability, and features of APFS make it the clear choice for modern Mac use.

Case Sensitivity: Options and Implications

One of the most surprising aspects of macOS for Unix users is that the filesystem is case-insensitive by default. File.txt and file.txt refer to the same file. This design choice dates back to the original Macintosh and persists today, though case-sensitive options exist.

Understanding Case Insensitivity

Case-Insensitive, Case-Preserving

macOS filesystems (APFS and HFS+) are case-insensitive but case-preserving:

# Create a file
$ echo "content" > Hello.txt

# Reference it with different case - same file!
$ cat hello.txt
content

# But the original case is preserved
$ ls
Hello.txt

# You cannot create a second file that differs only in case
$ touch HELLO.txt
touch: HELLO.txt: No such file or directory
# Actually, this silently succeeds but doesn't create a new file
$ ls
Hello.txt

Unicode Normalization

macOS also normalizes Unicode characters, treating different representations of the same character as identical:

# 'é' can be encoded as:
# - U+00E9 (precomposed: single character)
# - U+0065 U+0301 (decomposed: 'e' + combining acute accent)

# macOS treats these as the same filename

This affects filenames with accented characters, especially in cross-platform contexts.

Why Case Insensitivity?

Historical Reasons

The original Macintosh (1984) targeted non-technical users. Case sensitivity was seen as confusing—why should “Document” and “document” be different?

Practical Reasons

Many Mac applications, including Finder, assume case insensitivity:

• File extensions: .TXT should equal .txt
• Application bundles: Safari.app and safari.app shouldn’t be different
• User expectations: Most users don’t think about case

Compatibility

Changing to case-sensitive by default would break:

• Existing software assuming case insensitivity
• User workflows and muscle memory
• Cross-platform workflows where users expect Mac behavior

Case-Sensitive Volumes

macOS does support case-sensitive filesystems:

# Check if a volume is case-sensitive
$ diskutil info / | grep "Name (Bundle)"
   Name (Bundle):                  APFS
# Standard APFS is case-insensitive

$ diskutil info / | grep "File System Personality"
   File System Personality:        APFS
# vs "Case-sensitive APFS" for case-sensitive volumes

Creating Case-Sensitive Volumes

# Create a case-sensitive APFS volume
$ sudo diskutil apfs addVolume disk3 "Case-sensitive APFS" "CaseSensitive"

# Erase a disk with case-sensitive APFS
$ sudo diskutil eraseDisk "Case-sensitive APFS" "CaseSensitiveDisk" GPT disk2

# Create a case-sensitive HFS+
$ sudo diskutil eraseDisk JHFS+X "CaseSensitiveDisk" GPT disk2
# The X in JHFS+X denotes case-sensitive

Testing Case Sensitivity

# Check if current directory is case-sensitive
$ touch test_CASE test_case 2>/dev/null
$ count=$(ls test_* 2>/dev/null | wc -l | tr -d ' ')
$ rm -f test_CASE test_case 2>/dev/null
$ if [ "$count" = "2" ]; then
    echo "Case-sensitive"
else
    echo "Case-insensitive"
fi

Or more simply:

# This tells you the filesystem type
$ diskutil info "$(df . | tail -1 | awk '{print $1}')" | grep "Personality"

Problems with Case Insensitivity

Git and Version Control

This is the most common pain point. Repositories from case-sensitive systems (Linux) can have issues:

# On a Linux server
$ ls
Makefile
makefile

# Clone to case-insensitive Mac
$ git clone git@server:repo.git
# Only one file appears! Git sees both but filesystem conflates them

# Git status shows changed files that aren't "really" changed
$ git status
On branch main
Changes not staged for commit:
    modified:   Makefile

Workaround: Use a case-sensitive disk image for development:

# Create sparse disk image with case-sensitive filesystem
$ hdiutil create -size 50g -fs "Case-sensitive APFS" -type SPARSE -volname "DevWork" ~/dev.sparseimage

# Mount it
$ hdiutil attach ~/dev.sparseimage
/dev/disk4s1    Apple_APFS
/dev/disk5s1    APFS Volume    /Volumes/DevWork

# Clone repositories there
$ cd /Volumes/DevWork
$ git clone git@server:problematic-repo.git

Build Systems and Dependencies

Some software requires case sensitivity:

# Building software that expects case-sensitive filesystem
$ ./configure
checking for case-sensitive filesystem... no
configure: error: This package requires a case-sensitive filesystem.

Filename Conflicts

Files from case-sensitive systems may conflict:

# Archive from Linux might contain:
#   config.h
#   Config.h
#
# Extracting on macOS loses one file

# Check for conflicts before extracting
$ tar -tf archive.tar.gz | sort -f | uniq -di
# Lists filenames that would conflict

Problems with Case Sensitivity

Using a case-sensitive boot volume causes its own problems:

Adobe Software

Adobe applications (Photoshop, Illustrator, etc.) famously refuse to install on case-sensitive volumes:

"This product cannot be installed on a case-sensitive file system."

Steam and Games

Many games and game launchers assume case insensitivity:

# Game looks for "/path/to/data.dat"
# Actually stored as "/path/to/Data.dat"
# Works on Windows, fails on case-sensitive Mac

Apple’s Own Applications

Some older Apple software had issues with case-sensitive volumes. While modern macOS mostly works, testing revealed enough edge cases that Apple kept case-insensitive as default.

Best Practices

For Most Users

Keep the default case-insensitive filesystem. It’s what macOS is designed for and tested with.

For Developers

Options for handling case-sensitive requirements:

Option 1: Case-sensitive disk image (Recommended)

# Create sparse image that grows as needed
$ hdiutil create -size 100g -fs "Case-sensitive APFS" \
    -type SPARSE -volname "Code" ~/code.sparseimage

# Add to login items or create a mount script
$ cat > ~/mount-code.sh << 'EOF'
#!/bin/bash
if [ ! -d "/Volumes/Code" ]; then
    hdiutil attach ~/code.sparseimage
fi
EOF

Option 2: Separate case-sensitive volume

# Add volume to existing APFS container
$ sudo diskutil apfs addVolume disk3 "Case-sensitive APFS" "Code"

Option 3: Linux virtual machine or container

For maximum compatibility with Linux-centric projects:

# Use Docker for builds
$ docker run -v "$(pwd):/code" -w /code alpine make

For Cross-Platform Projects

When maintaining projects used on both case-sensitive and case-insensitive systems:

1. Establish naming conventions: Always use lowercase, or establish explicit conventions
2. Add CI checks: Detect case-only differences in commits

# Git hook to check for case conflicts
#!/bin/bash
git ls-files | sort -f | uniq -di && exit 1

3. Document requirements: If case sensitivity matters, document it clearly

Detecting and Handling Case Issues

Find Case Conflicts in a Directory

# Find files/directories that differ only in case
$ find . -maxdepth 1 -print | sort -f | uniq -di

# More thorough check
$ find . -print | sed 's|.*/||' | sort -f | uniq -di

Git Configuration for Case Issues

# Tell Git to notice case-only renames
$ git config core.ignorecase false

# This can cause issues on case-insensitive filesystems
# as Git may report phantom changes

Safe Renaming

On case-insensitive systems, renaming with case changes requires two steps:

# This fails (same file)
$ mv File.txt file.txt

# Two-step rename
$ mv File.txt File.txt.tmp
$ mv File.txt.tmp file.txt

# Or use git for tracked files
$ git mv File.txt file.txt
# Git handles the two-step internally

Summary

Case sensitivity on macOS is a trade-off:

Case-insensitive (default) provides:

• Maximum compatibility with Mac software
• Familiar behavior for most users
• Fewer surprises with existing workflows

Case-sensitive provides:

• Linux/Unix compatibility
• Correct handling of repositories from case-sensitive systems
• Required by some build systems

For most users, the default case-insensitive filesystem is correct. Developers working with cross-platform code should consider case-sensitive disk images or volumes for their development work, while keeping the boot volume case-insensitive for maximum application compatibility.

Extended Attributes and Resource Forks

macOS files carry metadata beyond the basic Unix permissions and timestamps. Extended attributes (xattrs) store arbitrary key-value pairs, while resource forks—a legacy from classic Mac OS—persist in modern macOS as a special extended attribute. Understanding this metadata system is essential for proper file handling.

Extended Attributes Overview

Extended attributes are named metadata associated with files:

# List extended attributes on a file
$ xattr file.txt
com.apple.quarantine

# List with values
$ xattr -l file.txt
com.apple.quarantine: 0083;65a12345;Safari;12345678-1234-1234-1234-123456789012

# View in hex
$ xattr -px com.apple.quarantine file.txt
00 38 33 3B 36 35 61 31 32 33 34 35 3B 53 61 66
...

Common Extended Attributes

The Quarantine Attribute

Files downloaded from the internet are “quarantined”:

# Download a file
$ curl -O https://example.com/file.zip

# Check quarantine status
$ xattr -l file.zip
com.apple.quarantine: 0083;65a12345;curl;12345678-...

# First time opening triggers Gatekeeper
$ open file.zip
# "file.zip" is from an unidentified developer...

Quarantine Fields

The quarantine attribute contains:

• Flags (hex): What checks have been performed
• Timestamp: When downloaded (hex Unix timestamp)
• Agent: Application that downloaded it
• UUID: Unique identifier

Removing Quarantine

# Remove quarantine from a single file
$ xattr -d com.apple.quarantine file.zip

# Remove from application bundle and contents
$ xattr -dr com.apple.quarantine /Applications/SomeApp.app

# Recursive removal for a directory
$ xattr -dr com.apple.quarantine ~/Downloads/extracted-folder/

Quarantine Flags

# Flag values:
# 0000: No quarantine (translocated app)
# 0001: Origin URL checked
# 0002: User consent obtained
# 0040: Notarization checked (macOS 10.15+)
# 0080: Hardened runtime checked

FinderInfo Attribute

Finder metadata is stored in com.apple.FinderInfo:

# View FinderInfo
$ xattr -px com.apple.FinderInfo file.txt

# This 32-byte structure contains:
# - File type and creator codes (legacy)
# - Finder flags (label color, extension hidden, etc.)
# - Position (for icon placement)

Setting Finder Tags/Labels

# Set color label using tag command (macOS 10.9+)
$ tag -a Red file.txt

# View tags
$ tag -l file.txt
file.txt    Red

# Using mdls to see color
$ mdls -name kMDItemUserTags file.txt
kMDItemUserTags = (
    "Red"
)

Note: The tag command is third-party. Install via brew install tag.

Resource Forks

Resource forks are a legacy concept from classic Mac OS where files had two “forks”:

• Data fork: The main content (what Unix sees as the file)
• Resource fork: Structured data (icons, code, dialog layouts, etc.)

Modern Resource Fork Access

Resource forks are stored in the com.apple.ResourceFork extended attribute:

# Check if a file has a resource fork
$ xattr file | grep ResourceFork
com.apple.ResourceFork

# Access resource fork via named fork syntax
$ ls -l file/..namedfork/rsrc

# Or via attribute
$ xattr -p com.apple.ResourceFork file | xxd | head

When Resource Forks Appear

Modern software rarely uses resource forks, but you’ll encounter them in:

• Classic Mac OS applications/files
• Some disk images
• Files created by older software
• Cross-platform transfers from old Mac files

# Get resource fork size (if any)
$ ls -l@ file | grep -A1 "ResourceFork"
    com.apple.ResourceFork       1234

Preserving Resource Forks

Standard Unix tools may not preserve resource forks:

# cp preserves extended attributes including resource forks
$ cp -p file /destination/

# tar can preserve extended attributes
$ tar --xattrs -cvf archive.tar file

# ditto preserves everything (recommended for Mac-to-Mac)
$ ditto source destination

# rsync requires special flags
$ rsync -avX source/ destination/
# -X preserves extended attributes on macOS

Working with Extended Attributes

Reading Attributes

# List attribute names
$ xattr file.txt
com.apple.quarantine
com.apple.metadata:kMDItemWhereFroms

# Print attribute value (human readable)
$ xattr -p com.apple.quarantine file.txt
0083;65a12345;Safari;...

# Print as hex
$ xattr -px com.apple.quarantine file.txt
00 38 33 3B 36 35 61 31 ...

Writing Attributes

# Set an attribute (string value)
$ xattr -w user.comment "My comment" file.txt

# Set an attribute (hex value)
$ xattr -wx user.binary "48454C4C4F" file.txt

# Verify
$ xattr -p user.comment file.txt
My comment

Removing Attributes

# Remove specific attribute
$ xattr -d com.apple.quarantine file.txt

# Remove all attributes
$ xattr -c file.txt

# Remove recursively
$ xattr -rc directory/

Copying Attributes

# Copy attributes from one file to another
$ xattr -l source.txt > /tmp/attrs
# (Manual process - no direct copy command)

# Or use ditto for full preservation
$ ditto source.txt dest.txt

Extended Attributes and Unix Tools

Tools That Preserve Attributes

Tools That Don’t Preserve Attributes

Attributes and SSH/SCP

When copying files to remote systems:

# Standard scp loses attributes
$ scp file.txt remote:~/

# Use rsync instead
$ rsync -avzX file.txt remote:~/

# Note: Remote must support xattrs
# Linux ext4 supports them; many remote systems do not

Metadata Spotlight Integration

Some extended attributes feed into Spotlight:

# View Spotlight metadata
$ mdls file.txt
kMDItemContentType         = "public.plain-text"
kMDItemContentTypeTree     = (
    "public.plain-text",
    "public.text",
    ...
)
kMDItemDisplayName         = "file.txt"
kMDItemFSCreatorCode       = ""
kMDItemFSFinderFlags       = 0
kMDItemFSHasCustomIcon     = (null)
...

# View WhereFroms (download URLs)
$ mdls -name kMDItemWhereFroms file.txt
kMDItemWhereFroms = (
    "https://example.com/file.txt",
    "https://example.com/"
)

Clearing Download History

# Remove WhereFroms metadata
$ xattr -d com.apple.metadata:kMDItemWhereFroms file.txt

AppleDouble Files

When copying files with extended attributes to filesystems that don’t support them (FAT32, SMB shares, etc.), macOS creates “AppleDouble” files:

# Copy file to FAT32 USB drive
$ cp file.txt /Volumes/USB_FAT32/

# Results in:
# /Volumes/USB_FAT32/file.txt        (data fork)
# /Volumes/USB_FAT32/._file.txt      (extended attributes)

The ._ files contain serialized extended attributes.

Preventing AppleDouble Files

# Use dot_clean to merge AppleDouble files
$ dot_clean /Volumes/USB_FAT32/

# Prevent creation (strip attributes during copy)
$ cp file.txt /Volumes/USB_FAT32/
$ xattr -c /Volumes/USB_FAT32/file.txt

Cleaning Up AppleDouble Files

# Find AppleDouble files
$ find /Volumes/USB_FAT32 -name '._*' -type f

# Remove them
$ find /Volumes/USB_FAT32 -name '._*' -type f -delete

# Or use dot_clean to merge them back
$ dot_clean /Volumes/USB_FAT32

Summary

Extended attributes on macOS provide:

• Quarantine tracking for security (Gatekeeper)
• Finder metadata for GUI integration
• Resource forks for legacy compatibility
• Spotlight integration for search
• Custom metadata for applications

When working with files:

1. Be aware that attributes exist and affect behavior
2. Use appropriate tools (ditto, rsync -X) when preservation matters
3. Know how to inspect (xattr -l) and remove (xattr -d) attributes
4. Understand AppleDouble files when working with non-Mac filesystems
5. Remember that quarantine affects executable files from the internet

Extended attributes are integral to the macOS experience, even when working from the command line.

The Metadata Files: .DS_Store and ._AppleDouble

Every Mac user eventually notices them: .DS_Store files appearing in directories, ._ prefix files cluttering USB drives, and mysterious metadata files showing up in archives. These files serve legitimate purposes but can be annoying when they leak into version control or cross-platform file sharing.

.DS_Store Files

What They Are

.DS_Store (Desktop Services Store) files are created by Finder to store custom folder attributes:

• Window position and size
• Icon positions (when using icon view)
• View settings (icon/list/column/gallery view)
• Background color or image
• Sort order and column widths
• Custom icons

# Find .DS_Store files
$ find ~/Documents -name '.DS_Store' -type f 2>/dev/null | head
/Users/david/Documents/.DS_Store
/Users/david/Documents/Projects/.DS_Store
...

# View basic info
$ file .DS_Store
.DS_Store: Apple Desktop Services Store

# Size is usually small
$ ls -la .DS_Store
-rw-r--r--@ 1 david  staff  6148 Jan 15 10:30 .DS_Store

.DS_Store Contents

The file format is proprietary binary, but tools exist to examine it:

# Install ds_store Python library
$ pip install ds_store

# Or use a basic hex dump
$ xxd .DS_Store | head
00000000: 0000 0001 4275 6431 0000 3000 0000 0800  ....Bud1..0.....
00000010: 0000 2000 0000 0008 0000 0000 0000 0000  .. .............
...

The file contains:

• File/folder names in the directory
• Per-item display settings
• Finder view preferences
• Spotlight comments (sometimes)

Preventing .DS_Store Creation

On Network Volumes

Finder can be configured to not create .DS_Store on network drives:

# Prevent on network volumes
$ defaults write com.apple.desktopservices DSDontWriteNetworkStores true

# Prevent on USB volumes
$ defaults write com.apple.desktopservices DSDontWriteUSBStores true

# Restart Finder to apply
$ killall Finder

This doesn’t prevent local .DS_Store creation—only network and USB volumes.

Globally (Not Recommended)

There’s no supported way to prevent local .DS_Store creation without breaking Finder functionality.

Cleaning Up .DS_Store

# Delete .DS_Store files in current directory tree
$ find . -name '.DS_Store' -type f -delete

# Dry run first (just show what would be deleted)
$ find . -name '.DS_Store' -type f -print

# Delete from a specific path, including hidden directories
$ find /path/to/folder -name '.DS_Store' -delete

.DS_Store and Git

These files should almost always be gitignored:

# Add to global gitignore
$ echo '.DS_Store' >> ~/.gitignore_global
$ git config --global core.excludesfile ~/.gitignore_global

# Add to project .gitignore
$ echo '.DS_Store' >> .gitignore

# Remove already-tracked .DS_Store files
$ git rm --cached $(git ls-files -i --exclude-standard '*.DS_Store')
$ git commit -m "Remove .DS_Store files"

AppleDouble Files (._prefix files)

What They Are

When macOS copies files to filesystems that don’t support extended attributes (FAT32, exFAT, SMB shares, etc.), it creates companion files with ._ prefix:

# Copy to FAT32 drive
$ cp photo.jpg /Volumes/USB_DRIVE/

# Results in:
$ ls -la /Volumes/USB_DRIVE/
-rwxr-xr-x  1 david  staff  1048576 Jan 15 10:30 photo.jpg
-rwxr-xr-x  1 david  staff     4096 Jan 15 10:30 ._photo.jpg

What They Contain

AppleDouble files store:

• Extended attributes that couldn’t be preserved
• Resource forks (if present)
• Finder metadata
• ACLs

# View what's in an AppleDouble file
$ xattr -l ._photo.jpg
# (Shows nothing - attributes are encoded in file content)

# Decode structure (basic view)
$ xxd ._photo.jpg | head
00000005 26 00 02 00 00 00 00 00  00 00 00 00 00 00 00 00  |&...............|
00000015 00 00 00 00 00 00 00 00  00 02 00 00 00 09 00 00  |................|
...

When They Appear

• Copying to FAT32/exFAT USB drives
• Copying to SMB/CIFS network shares
• Creating ZIP files (sometimes)
• Copying to NFS shares without xattr support
• Burning to data CDs/DVDs

The dot_clean Command

macOS provides dot_clean to merge AppleDouble files:

# Merge AppleDouble files back into their parent files
# (Only works when copying back to APFS/HFS+)
$ dot_clean /Volumes/USB_DRIVE/

# Merge and remove AppleDouble files
$ dot_clean -m /Volumes/USB_DRIVE/

# Verbose output
$ dot_clean -v /Volumes/USB_DRIVE/
Merging ._photo.jpg
...

Cleaning Up AppleDouble Files

# Find all AppleDouble files
$ find /Volumes/USB_DRIVE -name '._*' -type f

# Delete them
$ find /Volumes/USB_DRIVE -name '._*' -type f -delete

# Delete .DS_Store AND AppleDouble files
$ find /Volumes/USB_DRIVE \( -name '.DS_Store' -o -name '._*' \) -delete

Preventing AppleDouble Files

Use ditto with –norsrc

# Copy without resource forks/AppleDouble
$ ditto --norsrc source.txt /Volumes/USB_DRIVE/

# Copy entire directory
$ ditto --norsrc ~/Documents/folder /Volumes/USB_DRIVE/folder

Strip Attributes Before Copy

# Copy then strip
$ cp file.txt /Volumes/USB_DRIVE/
$ xattr -c /Volumes/USB_DRIVE/file.txt
# This removes the need for AppleDouble companion

Use Archive Formats

When sharing files, use formats that either:

• Don’t preserve Mac metadata (simple tar, zip)
• Explicitly preserve it (tar with –xattrs, ditto archives)

# Create cross-platform-friendly tar
$ tar -cvf archive.tar files/
# No metadata preserved, no ._files

# Or strip metadata first
$ find files/ -name '._*' -delete
$ find files/ -name '.DS_Store' -delete
$ tar -cvf archive.tar files/

__MACOSX Folders in ZIP Files

When Finder creates ZIP files, it may include a __MACOSX folder:

# Create ZIP with Finder
$ cd ~/Documents
# Right-click folder → Compress "folder"

# Extract on Windows/Linux reveals:
# folder/
# __MACOSX/
#   folder/
#     ._file1.txt
#     ._file2.txt

Creating Clean ZIPs

# Use zip command with exclusions
$ zip -r archive.zip folder/ -x "*.DS_Store" -x "__MACOSX/*" -x "._*"

# Or use ditto to create clean ZIP
$ ditto -c -k --sequesterRsrc --keepParent folder archive.zip
# --sequesterRsrc puts resource forks in __MACOSX (standard Mac behavior)

# For truly clean ZIP (no Mac metadata at all):
$ cd folder && zip -r ../archive.zip . -x "*.DS_Store" -x "*.git*" -x "._*"

Extracting and Cleaning

# Extract ZIP
$ unzip archive.zip

# Remove Mac metadata
$ find . -name '__MACOSX' -type d -exec rm -rf {} +
$ find . -name '._*' -type f -delete
$ find . -name '.DS_Store' -type f -delete

Best Practices

For Version Control

Standard .gitignore entries:

# macOS metadata
.DS_Store
.AppleDouble
.LSOverride

# Icon must end with two \r
Icon

# Thumbnails
._*

# Files that might appear in the root of a volume
.DocumentRevisions-V100
.fseventsd
.Spotlight-V100
.TemporaryItems
.Trashes
.VolumeIcon.icns
.com.apple.timemachine.donotpresent

# Directories potentially created on remote AFP share
.AppleDB
.AppleDesktop
Network Trash Folder
Temporary Items
.apdisk

For File Sharing

When sharing files with non-Mac users:

1. Clean before sharing:

find /path/to/share -name '.DS_Store' -delete
find /path/to/share -name '._*' -delete

2. Use appropriate tools:

# For cross-platform ZIP
zip -r archive.zip folder/ -x "*.DS_Store" -x "._*"

3. Configure Finder:

defaults write com.apple.desktopservices DSDontWriteNetworkStores true
defaults write com.apple.desktopservices DSDontWriteUSBStores true

For External Drives

If you regularly use drives with non-Mac systems:

# Configure once
defaults write com.apple.desktopservices DSDontWriteUSBStores true

# After use, clean up
dot_clean /Volumes/DRIVE_NAME
find /Volumes/DRIVE_NAME -name '.DS_Store' -delete
find /Volumes/DRIVE_NAME -name '._*' -delete

Summary

macOS metadata files serve real purposes but require management:

Key commands:

• find . -name '.DS_Store' -delete — Clean .DS_Store files
• find . -name '._*' -delete — Clean AppleDouble files
• dot_clean /path — Merge AppleDouble files back
• defaults write com.apple.desktopservices DSDontWriteUSBStores true — Prevent on USB

These files are a side effect of macOS’s rich metadata system—useful locally but often unwanted when files leave the Mac ecosystem.

Filesystem Hierarchy: Where macOS Diverges

If you’re coming from Linux, you expect files to be in certain places. /etc has configuration, /var has variable data, /opt has optional packages. macOS follows some of these conventions but diverges significantly in others. Understanding the macOS filesystem hierarchy helps you find files and understand where to put things.

The macOS Root Filesystem

$ ls -la /
total 17
drwxr-xr-x   20 root  wheel    640 Dec 11 11:23 .
drwxr-xr-x   20 root  wheel    640 Dec 11 11:23 ..
lrwxr-xr-x    1 root  wheel     36 Dec 11 11:23 System -> /System/Volumes/Data/System
drwxrwxr-x   45 root  admin   1440 Jan 15 14:20 Applications
drwxr-xr-x   70 root  wheel   2240 Dec 11 11:23 Library
drwxr-xr-x@   9 root  wheel    288 Dec 11 11:23 System
drwxr-xr-x    6 root  wheel    192 Dec 11 11:23 Users
drwxr-xr-x    4 root  wheel    128 Jan 17 09:00 Volumes
drwxr-xr-x@   2 root  wheel     64 Dec 11 11:23 bin
drwxr-xr-x    2 root  wheel     64 Dec 11 11:23 cores
dr-xr-xr-x    4 root  wheel   5207 Jan 15 10:30 dev
lrwxr-xr-x@   1 root  wheel     11 Dec 11 11:23 etc -> private/etc
lrwxr-xr-x    1 root  wheel     25 Dec 11 11:23 home -> /System/Volumes/Data/home
drwxr-xr-x    2 root  wheel     64 Dec 11 11:23 opt
drwxr-xr-x    6 root  wheel    192 Dec 11 11:23 private
drwxr-xr-x@  64 root  wheel   2048 Dec 11 11:23 sbin
lrwxr-xr-x@   1 root  wheel     11 Dec 11 11:23 tmp -> private/tmp
drwxr-xr-x@  11 root  wheel    352 Dec 11 11:23 usr
lrwxr-xr-x@   1 root  wheel     11 Dec 11 11:23 var -> private/var

Note the symbolic links: /etc, /tmp, and /var are links to /private/*.

Unix-Standard Directories

/bin and /sbin

Essential system binaries:

$ ls /bin
[        cp       df       ed       ksh      ls       mv       rm       stty     zsh
bash     csh      echo     expr     launchctl mkdir    pax      rmdir    sync
cat      dash     hostname kill     link     ln       pwd      sh       tcsh     unlink
chmod    date     sleep    test     wait4path

These are Apple-provided, BSD-derived utilities. On modern macOS (Catalina+), they’re read-only, protected by System Integrity Protection, and on a sealed system volume.

/usr

User utilities and libraries:

/usr/
├── bin/          # User commands (not essential for single-user mode)
├── include/      # Header files (symlink to SDK when Xcode installed)
├── lib/          # Libraries
├── libexec/      # Support binaries for system programs
├── local/        # Local additions (Homebrew on Intel Macs)
├── sbin/         # System administration commands
├── share/        # Architecture-independent data
└── standalone/   # Standalone resources

Important: /usr/local is writable and is where user-installed software traditionally goes. Homebrew uses it on Intel Macs.

/etc (→ /private/etc)

System configuration:

$ ls /etc | head
afpovertcp.cfg
apache2
asl
asl.conf
autofs.conf
auto_home
auto_master
bashrc
bashrc_Apple_Terminal
cups

Unlike Linux, many macOS services don’t use /etc for configuration—they use property lists in /Library/Preferences or /Library/LaunchDaemons.

/var (→ /private/var)

Variable data:

$ ls /var
agentx     db         empty      folders    install    log        mail       msgs       networkd   root       rpc        run        spool      tmp        vm         yp
audit      at         backups    db         empty      folders    install    jabberd    lib        log        mail       msgs       networkd   protected  root       rpc        run        screensharing  select     spool      tmp        vm         yp

Notable contents:

• /var/log — System logs (though modern macOS uses unified logging)
• /var/db — System databases
• /var/folders — Per-user temporary caches
• /var/tmp — Temporary files that survive reboots

/tmp (→ /private/tmp)

Temporary files:

$ ls -la /tmp
lrwxr-xr-x@ 1 root  wheel  11 Dec 11 11:23 /tmp -> private/tmp

Unlike some Unix systems, /tmp may not be cleaned on every reboot. Use the proper temp directory API:

# Get user-specific temp directory
$ echo $TMPDIR
/var/folders/xx/xxxxxxxxxx/T/

# This is per-user and properly permissioned

macOS-Specific Directories

/Applications

GUI applications:

$ ls /Applications
App Store.app
Automator.app
Calculator.app
...

Applications are bundles (directories that appear as files in Finder):

$ file /Applications/Safari.app
/Applications/Safari.app: directory

$ ls /Applications/Safari.app/Contents/
_CodeSignature  Frameworks      Info.plist      MacOS           PkgInfo         Resources       XPCServices

/Library

System-wide resources and configuration:

/Library/
├── Application Support/   # App-specific data (system-wide)
├── Audio/                 # Audio plug-ins and presets
├── Caches/               # System caches
├── ColorPickers/         # Color picker modules
├── Components/           # System components
├── Contextual Menu Items/
├── CoreMediaIO/          # Video device drivers
├── Desktop Pictures/     # System wallpapers
├── Documentation/        # System documentation
├── Extensions/           # Kernel extensions (deprecated)
├── Fonts/                # System-wide fonts
├── Frameworks/           # System-wide frameworks
├── Graphics/             # Graphics drivers
├── Image Capture/        # Scanner plug-ins
├── Input Methods/        # Input method editors
├── Internet Plug-Ins/    # Browser plug-ins
├── iTunes/               # iTunes-related resources
├── Java/                 # Java installations
├── Keyboard Layouts/     # Custom keyboard layouts
├── Keychains/            # System keychain
├── LaunchAgents/         # System-wide login agents
├── LaunchDaemons/        # System daemons
├── Logs/                 # Application logs
├── Mail/                 # Mail.app resources
├── OpenDirectory/        # Directory services
├── PDF Services/         # PDF workflow actions
├── Perl/                 # Perl modules
├── Preferences/          # System-wide preferences
├── Printers/             # Printer drivers
├── PrivilegedHelperTools/ # Helper tools running as root
├── Python/               # Python packages
├── QuickLook/            # QuickLook generators
├── QuickTime/            # QuickTime components
├── Receipts/             # Package receipts
├── Ruby/                 # Ruby gems
├── Sandbox/              # Sandboxing profiles
├── Screen Savers/        # Screen savers
├── ScriptingAdditions/   # AppleScript additions
├── Scripts/              # System scripts
├── Security/             # Security resources
├── Speech/               # Speech recognition/synthesis
├── Spelling/             # Spelling dictionaries
├── Spotlight/            # Spotlight importers
├── StartupItems/         # Legacy startup items (deprecated)
├── SystemMigration/      # Migration assistant data
├── SystemProfiler/       # System profiler plugins
├── Updates/              # Software update data
├── User Pictures/        # User account pictures
├── Video/                # Video drivers
└── WebServer/            # Apache web server files

/System/Library

Apple’s system resources—protected by SIP:

$ ls /System/Library | head
Accessibility
Accounts
Address Book Plug-Ins
Ambiances
AppleRAIDConsoleUI
Assistants
Audio
Automator
BridgeSupport
CFMSupport

Never modify files here. Changes are blocked by System Integrity Protection.

~/Library

Per-user resources (in your home directory):

~/Library/
├── Application Support/   # App-specific data
├── Caches/               # App caches (safe to delete)
├── Containers/           # Sandboxed app data
├── Cookies/              # Browser cookies
├── Fonts/                # User-installed fonts
├── Group Containers/     # Shared sandboxed data
├── Keychains/            # User keychains
├── LaunchAgents/         # User login agents
├── Logs/                 # User app logs
├── Mail/                 # Mail.app data
├── Messages/             # Messages.app data
├── Preferences/          # User preferences (plists)
├── Saved Application State/ # App state for restore
└── Services/             # User-installed services

This directory is hidden in Finder by default. Access it:

# From Terminal
$ open ~/Library

# Or in Finder: Go → Go to Folder → ~/Library
# Or hold Option while clicking Go menu

/System/Volumes

macOS Catalina+ uses multiple APFS volumes:

$ ls /System/Volumes
Data        Preboot     Recovery    Update      VM

• Data: User data, applications, mutable system data
• Preboot: Boot-time data
• Recovery: Recovery OS
• Update: Software update staging
• VM: Swap files

The system uses “firmlinks” to make these volumes appear as a unified filesystem.

/Volumes

Mount point for all volumes:

$ ls /Volumes
Macintosh HD            # Boot volume
Macintosh HD - Data     # Data volume (if visible)
ExternalDrive           # External drives appear here
DiskImage               # Mounted disk images

Comparison with FHS (Linux)

Homebrew Locations

Homebrew has different install locations:

Intel Macs

/usr/local/
├── bin/         # Symlinks to installed binaries
├── Cellar/      # Installed formula versions
├── etc/         # Configuration files
├── include/     # Header files
├── lib/         # Libraries
├── opt/         # Symlinks to latest versions
├── sbin/        # System binaries
├── share/       # Architecture-independent data
└── var/         # Variable data (logs, databases)

Apple Silicon Macs

/opt/homebrew/
├── bin/
├── Cellar/
├── etc/
├── include/
├── lib/
├── opt/
├── sbin/
├── share/
└── var/

The different location avoids conflicts with Rosetta 2 (Intel emulation).

Finding Files

Using locate (if enabled)

# Enable locate database
$ sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.locate.plist

# Update database
$ sudo /usr/libexec/locate.updatedb

# Find files
$ locate nginx.conf

Using mdfind (Spotlight)

# Find by name
$ mdfind -name "nginx.conf"

# Find by content
$ mdfind "error handling"

# Find by type
$ mdfind "kMDItemKind == 'Application'"

Using find

# Traditional Unix find
$ find /usr -name "*.h" -type f 2>/dev/null

Summary

The macOS filesystem hierarchy combines Unix traditions with Apple innovations:

Unix-familiar:

• /bin, /sbin, /usr, /etc, /var, /tmp work as expected
• /usr/local is available for local software

macOS-specific:

• /Applications for GUI applications
• /Library hierarchy for system/user resources
• /System protected by SIP
• /Volumes for all mounted volumes
• /Users instead of /home
• /private containing the actual /etc, /var, /tmp

Key differences from Linux:

• Configuration often in property lists, not /etc text files
• Three-tier Library structure (/System/Library, /Library, ~/Library)
• Homebrew in /opt/homebrew on Apple Silicon
• Multiple APFS volumes presenting as unified filesystem

Understanding this hierarchy helps you navigate macOS, find configuration files, and work effectively across Unix and macOS environments.

Disk Management from the Command Line

While Disk Utility provides a graphical interface for disk operations, the command line offers more power and precision. macOS provides diskutil as the primary disk management tool, along with lower-level utilities for specific tasks.

The diskutil Command

diskutil is macOS’s comprehensive disk management utility:

# Get help
$ diskutil
Disk Utility Tool
Utility to manage local disks and volumes
...

# List all verbs (commands)
$ diskutil listFilesystems
$ diskutil list
$ diskutil info
...

Listing Disks and Volumes

# List all disks and partitions
$ diskutil list
/dev/disk0 (internal):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *500.1 GB   disk0
   1:             Apple_APFS_ISC Container disk1         524.3 MB   disk0s1
   2:                 Apple_APFS Container disk3         494.4 GB   disk0s2
   3:        Apple_APFS_Recovery Container disk2         5.4 GB     disk0s3

/dev/disk3 (synthesized):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      APFS Container Scheme -                      +494.4 GB   disk3
   1:                APFS Volume Macintosh HD            10.1 GB    disk3s1
   2:              APFS Snapshot com.apple.os.update-... 10.1 GB    disk3s1s1
   3:                APFS Volume Preboot                 5.3 GB     disk3s2
   4:                APFS Volume Recovery                1.6 GB     disk3s3
   5:                APFS Volume Data                    258.2 GB   disk3s5
   6:                APFS Volume VM                      3.2 GB     disk3s6

# List with more detail
$ diskutil list -plist  # Machine-readable
$ diskutil list external # Only external disks
$ diskutil list internal # Only internal disks

Disk and Volume Information

# Detailed information about a disk
$ diskutil info disk0
   Device Identifier:         disk0
   Device Node:               /dev/disk0
   Whole:                     Yes
   Part of Whole:             disk0
   Device / Media Name:       APPLE SSD AP0512Q

   Volume Name:               Not applicable (no file system)
   Mounted:                   Not applicable (no file system)
   File System:               None

   Content (IOContent):       GUID_partition_scheme
   OS Can Be Installed:       No
   Media Type:                Generic
   Protocol:                  Apple Fabric
   SMART Status:              Verified
   Disk Size:                 500.1 GB (500107862016 Bytes)
   ...

# Information about a volume
$ diskutil info disk3s5
   Device Identifier:         disk3s5
   Device Node:               /dev/disk3s5
   Whole:                     No
   Part of Whole:             disk3

   Volume Name:               Data
   Mounted:                   Yes
   Mount Point:               /System/Volumes/Data

   File System Personality:   APFS
   Type (Bundle):             apfs
   Name (User Visible):       APFS
   Owners:                    Enabled
   ...

# Information about current volume
$ diskutil info /

Mounting and Unmounting

# Unmount a volume (leaves disk attached)
$ diskutil unmount disk3s5
Volume Data on disk3s5 unmounted

# Mount a volume
$ diskutil mount disk3s5
Volume Data on disk3s5 mounted

# Mount at specific location
$ diskutil mount -mountPoint /Volumes/MyMount disk3s5

# Unmount entire disk (all volumes)
$ diskutil unmountDisk disk2
Unmount of all volumes on disk2 was successful

# Force unmount (use with caution)
$ diskutil unmount force disk3s5

Ejecting Disks

# Eject (unmount and remove)
$ diskutil eject disk2
Disk disk2 ejected

# This is equivalent to ejecting in Finder

Formatting and Erasing

Erasing a Volume

# Erase and reformat a volume
$ sudo diskutil eraseVolume APFS "NewVolume" disk2s2

# Specify filesystem type:
#   APFS - Apple File System
#   JHFS+ - Mac OS Extended (Journaled)
#   ExFAT - Cross-platform
#   MS-DOS FAT32 - Legacy compatible

Erasing an Entire Disk

# Erase and partition entire disk with APFS
$ sudo diskutil eraseDisk APFS "MyDisk" GPT disk2
Started erase on disk2
Unmounting disk
Creating the partition map
Waiting for partitions to activate
Formatting disk2s2 as APFS with name MyDisk
...
Finished erase on disk2

# Options:
#   GPT - GUID Partition Table (modern, recommended)
#   MBR - Master Boot Record (legacy, limited)
#   APM - Apple Partition Map (PowerPC era)

Available Filesystems

$ diskutil listFilesystems
Formattable file systems

These file system personalities can be used for erasing and partitioning.
...
-------------------------------------------------------------------------------
PERSONALITY                      USER VISIBLE NAME
-------------------------------------------------------------------------------
ExFAT                            ExFAT
Free Space                       Free Space
(or) free
MS-DOS                           MS-DOS (FAT)
MS-DOS FAT12                     MS-DOS (FAT12)
MS-DOS FAT16                     MS-DOS (FAT16)
MS-DOS FAT32                     MS-DOS (FAT32)
(or) fat32
HFS+                             Mac OS Extended
Case-sensitive HFS+              Mac OS Extended (Case-sensitive)
Case-sensitive Journaled HFS+    Mac OS Extended (Case-sensitive, Journaled)
(or) jhfsx
Journaled HFS+                   Mac OS Extended (Journaled)
(or) jhfs+
APFS                             APFS
(or) apfs
Case-sensitive APFS              APFS (Case-sensitive)
(or) apfsx

APFS-Specific Operations

Working with Containers

# List APFS containers
$ diskutil apfs list
APFS Container (1 found)
|
+-- Container disk3 494.4 GB
    ====================================================
    APFS Container Reference:     disk3
    Size (Capacity Ceiling):      494384795648 B (494.4 GB)
    Capacity In Use By Volumes:   277712732160 B (277.7 GB) (56.2%)
    Capacity Not Allocated:       216672063488 B (216.7 GB) (43.8%)
    |
    +-< Physical Store disk0s2 494.4 GB
    |   ---------------------------------------------------
    |   APFS Physical Store Disk:   disk0s2
    |   Size:                       494384795648 B (494.4 GB)
    |
    +-> Volume disk3s1 Macintosh HD
    ...

# Create a new APFS container
$ sudo diskutil apfs createContainer disk2s2

# Resize a container
$ sudo diskutil apfs resizeContainer disk3 400GB

Working with Volumes

# Add volume to container
$ sudo diskutil apfs addVolume disk3 APFS "NewVolume"

# Add encrypted volume
$ sudo diskutil apfs addVolume disk3 APFS "SecureVolume" -passphrase

# Delete volume
$ sudo diskutil apfs deleteVolume disk3s7

# List volumes in container
$ diskutil apfs listVolumes disk3

Encryption

# Check encryption status
$ diskutil apfs list | grep -A5 "FileVault"

# Encrypt a volume
$ sudo diskutil apfs encryptVolume disk3s5 -user disk

# Decrypt a volume
$ sudo diskutil apfs decryptVolume disk3s5

# Change encryption password
$ sudo diskutil apfs changePassphrase disk3s5

Snapshots

# List snapshots
$ diskutil apfs listSnapshots disk3s1
Snapshots for disk3s1 (2 found)
|
+-- 8B6C8F4E-XXXX-XXXX-XXXX-XXXXXXXXXXXX
|   Snapshot Name:        com.apple.os.update-XXXX
|   Snapshot UUID:        8B6C8F4E-XXXX-XXXX-XXXX-XXXXXXXXXXXX
...

# Delete a snapshot
$ sudo diskutil apfs deleteSnapshot disk3s1 -uuid 8B6C8F4E-XXXX-XXXX-XXXX-XXXXXXXXXXXX

Partitioning

Creating Partitions

# Partition a disk with multiple partitions
$ sudo diskutil partitionDisk disk2 GPT \
    APFS "Data" 100GB \
    APFS "Backup" 100GB \
    "Free Space" "Available" R

# R = Remainder (use remaining space)

Resizing Partitions

# Resize a partition (grow or shrink)
$ sudo diskutil resizeVolume disk2s2 50GB

# Resize using limits (fill available space)
$ sudo diskutil resizeVolume disk2s2 R

Adding Partitions

# Add a partition using free space after existing partition
$ sudo diskutil addPartition disk2s2 APFS "NewPartition" 50GB

Merging Partitions

# Merge partitions (data on second partition is lost)
$ sudo diskutil mergePartitions JHFS+ "MergedVolume" disk2s2 disk2s3

Disk Images

Creating Disk Images

# Create empty disk image
$ hdiutil create -size 100m -fs APFS -volname "TestImage" test.dmg

# Create from folder
$ hdiutil create -srcfolder /path/to/folder -volname "Backup" backup.dmg

# Create encrypted disk image
$ hdiutil create -size 1g -fs APFS -encryption AES-256 -volname "Secure" secure.dmg

# Create sparse image (grows as needed)
$ hdiutil create -size 10g -fs APFS -type SPARSE -volname "Sparse" sparse.sparseimage

# Create sparse bundle (grows as bundle of files)
$ hdiutil create -size 10g -fs APFS -type SPARSEBUNDLE -volname "Bundle" bundle.sparsebundle

Mounting and Unmounting Images

# Mount disk image
$ hdiutil attach image.dmg
/dev/disk4          GUID_partition_scheme
/dev/disk4s1        Apple_APFS
/dev/disk5s1        Apple_APFS                      /Volumes/ImageVolume

# Mount read-only
$ hdiutil attach -readonly image.dmg

# Mount without Finder window
$ hdiutil attach -nobrowse image.dmg

# Unmount/detach
$ hdiutil detach /dev/disk4
# Or
$ hdiutil detach /Volumes/ImageVolume

Converting Images

# Convert to read-only compressed DMG
$ hdiutil convert source.dmg -format UDZO -o compressed.dmg

# Formats:
#   UDRO - Read-only
#   UDZO - Compressed (read-only)
#   UDBZ - bzip2 compressed
#   UDTO - DVD/CD master
#   UDSB - Sparsebundle

Resizing Images

# Resize sparse image
$ hdiutil resize -size 20g sparse.sparseimage

# Resize to minimum
$ hdiutil resize -size min sparse.sparseimage

# Compact sparse image (reclaim space)
$ hdiutil compact sparse.sparseimage

Verification and Repair

Verifying Filesystems

# Verify volume
$ diskutil verifyVolume disk3s5
Started file system verification on disk3s5 Data
Verifying storage system
Checking volume
...
Verified storage system
Storage system check exit code is 0
Finished file system verification on disk3s5 Data

# Verify disk
$ diskutil verifyDisk disk3

Repairing Filesystems

# Repair volume (must be unmounted for non-live repair)
$ sudo diskutil repairVolume disk3s5

# For boot volume, use Recovery Mode
# or First Aid in Disk Utility

Using fsck Directly

# Check APFS volume
$ sudo fsck_apfs -n /dev/disk3s5
# -n = no changes (dry run)

# Repair APFS
$ sudo fsck_apfs -y /dev/disk3s5

# Check HFS+
$ sudo fsck_hfs -n /dev/disk2s2

Secure Erase

Secure Erase Options

# Note: Secure erase is less effective on SSDs due to wear leveling
# For SSDs, use FileVault encryption instead

# Zero out free space on volume
$ diskutil secureErase freespace 0 /Volumes/DiskName
# Levels: 0=zero, 1=random, 2=US DoD 7-pass, 3=Gutmann 35-pass, 4=US DoE 3-pass

# Securely erase entire disk
$ diskutil secureErase 0 disk2

For SSDs

# Enable FileVault for secure deletion via encryption
# Then simply erase - data is unrecoverable without key

# Check TRIM status
$ system_profiler SPSerialATADataType | grep TRIM

Practical Examples

Format USB Drive for Cross-Platform Use

# ExFAT: Works on macOS, Windows, Linux
$ sudo diskutil eraseDisk ExFAT "USB_DRIVE" MBR disk2

# Use MBR for maximum compatibility with older systems

Create Bootable macOS Installer

# Download macOS from App Store, then:
$ sudo /Applications/Install\ macOS\ Sonoma.app/Contents/Resources/createinstallmedia \
    --volume /Volumes/USB_DRIVE

Clone a Volume

# Using asr (Apple Software Restore)
$ sudo asr restore --source /Volumes/Source --target /Volumes/Target --erase

# Using dd (byte-for-byte copy - use with extreme caution)
$ sudo dd if=/dev/rdisk2 of=/dev/rdisk3 bs=1m

# Note: Use 'r' prefix (rdisk) for raw device, faster

Check Disk Health

# SMART status
$ diskutil info disk0 | grep SMART
   SMART Status:              Verified

# More detailed SMART info
$ brew install smartmontools
$ sudo smartctl -a /dev/disk0

Summary

macOS disk management from the command line:

The diskutil command provides comprehensive disk management while hdiutil handles disk images. For most operations, these tools replace the need for lower-level utilities while providing macOS-appropriate behavior.

Volumes, Containers, and Snapshots

APFS introduces a new storage hierarchy that differs fundamentally from traditional partitioning. Understanding containers, volumes, and snapshots is essential for effective storage management on modern macOS.

The APFS Storage Hierarchy

Physical Disk
    │
    └── Partition (APFS Container)
            │
            ├── Volume A ─── Snapshot 1
            │                Snapshot 2
            ├── Volume B
            └── Volume C

Unlike traditional partitioning where each partition has a fixed size, APFS volumes share space within a container dynamically.

Understanding Containers

What Is a Container?

An APFS container is:

• Equivalent to a partition in terms of disk allocation
• A pool of storage that multiple volumes share
• The unit of encryption (container-level encryption wraps all volumes)

# View containers
$ diskutil apfs list
APFS Container (1 found)
|
+-- Container disk3 494.4 GB
    ====================================================
    APFS Container Reference:     disk3
    Size (Capacity Ceiling):      494384795648 B (494.4 GB)
    Capacity In Use By Volumes:   277848256512 B (277.8 GB) (56.2%)
    Capacity Not Allocated:       216536539136 B (216.5 GB) (43.8%)
    |
    +-< Physical Store disk0s2 494.4 GB

Container vs Partition

Multiple Physical Stores

A container can span multiple physical devices (like Fusion Drive):

# Fusion Drive example
+-< Physical Store disk0s2 (SSD) 128.0 GB
+-< Physical Store disk1s2 (HDD) 1.0 TB

This allows APFS to move data between fast and slow storage automatically.

Working with Volumes

Volume Roles

APFS assigns roles that define volume behavior:

# View volume roles
$ diskutil apfs list | grep "Role:"
    APFS Volume Disk (Role):   disk3s1 (System)
    APFS Volume Disk (Role):   disk3s5 (Data)
    APFS Volume Disk (Role):   disk3s6 (VM)

The System-Data Volume Pair

Modern macOS uses a “firmlinked” pair:

System Volume (read-only)     Data Volume (read-write)
/System                       /Users
/Library (system)             /Library (user)
/bin, /sbin, /usr            /Applications

Firmlinks make these appear as a single filesystem:

# These paths are on different volumes but appear unified
$ ls /
Applications  Library  System  Users  ...

# Check which volume a path is on
$ df /Users
Filesystem      512-blocks      Used Available Capacity  Mounted on
/dev/disk3s5   966871088 532846072 422952016    56%    /System/Volumes/Data

$ df /System
Filesystem     512-blocks     Used Available Capacity  Mounted on
/dev/disk3s1   966871088 19782344 422952016     5%    /

Creating Volumes

# Add a standard volume
$ sudo diskutil apfs addVolume disk3 APFS "DataVolume"
Exporting new APFS Volume "DataVolume" from APFS Container Reference disk3
Started APFS operation on disk3
Preparing to add APFS Volume to APFS Container disk3
Creating APFS Volume
...
Mounting APFS Volume
APFS Volume created and mounted at /Volumes/DataVolume
Finished APFS operation on disk3

# Add case-sensitive volume
$ sudo diskutil apfs addVolume disk3 "Case-sensitive APFS" "CaseSensitive"

# Add encrypted volume
$ sudo diskutil apfs addVolume disk3 APFS "Encrypted" -passphrase
Enter new passphrase:
Re-enter new passphrase:
...

# Add volume with quota (space limit)
$ sudo diskutil apfs addVolume disk3 APFS "Limited" -quota 10g

# Add volume with reserve (guaranteed minimum space)
$ sudo diskutil apfs addVolume disk3 APFS "Reserved" -reserve 5g

Volume Quotas and Reserves

# Set quota on existing volume (maximum space)
$ sudo diskutil apfs setQuota disk3s7 10g

# Set reserve on existing volume (guaranteed space)
$ sudo diskutil apfs setReserve disk3s7 5g

# Remove quota
$ sudo diskutil apfs setQuota disk3s7 0

# View settings
$ diskutil apfs list | grep -A10 "Volume disk3s7"
    +-> Volume disk3s7 DataVolume
        ---------------------------------------------------
        APFS Volume Disk (Role):   disk3s7 (No specific role)
        Quota:                     10.0 GB (10737418240 Bytes)
        Reserve:                   5.0 GB (5368709120 Bytes)

Deleting Volumes

# Delete a volume (space returns to container pool)
$ sudo diskutil apfs deleteVolume disk3s7
Started APFS operation on disk3s7 DataVolume
Deleting APFS Volume from APFS Container disk3
...
Finished APFS operation on disk3s7 DataVolume

Warning: This permanently destroys all data on the volume.

Renaming Volumes

# Rename a volume
$ sudo diskutil rename disk3s5 "New Name"

# Or
$ sudo diskutil apfs rename disk3s5 "New Name"

Working with Snapshots

What Are Snapshots?

Snapshots capture the state of a volume at a point in time:

• Read-only view of the filesystem
• Space-efficient (only stores changes)
• Created instantly
• Used by Time Machine and system updates

Viewing Snapshots

# List snapshots with tmutil
$ tmutil listlocalsnapshots /
Snapshots for volume group containing disk /:
com.apple.TimeMachine.2024-01-15-091234.local
com.apple.TimeMachine.2024-01-15-101234.local
com.apple.TimeMachine.2024-01-15-111234.local

# List with diskutil
$ diskutil apfs listSnapshots disk3s1
Snapshots for disk3s1 (2 found)
|
+-- 5A7C2E48-XXXX-XXXX-XXXX-XXXXXXXXXXXX
|   Snapshot Name:        com.apple.os.update-XXXX
|   Snapshot UUID:        5A7C2E48-XXXX-XXXX-XXXX-XXXXXXXXXXXX
|   Snapshot Sealed:      Yes
|
+-- 8B6C8F4E-XXXX-XXXX-XXXX-XXXXXXXXXXXX
    Snapshot Name:        com.apple.TimeMachine.2024-01-15-091234.local
    Snapshot UUID:        8B6C8F4E-XXXX-XXXX-XXXX-XXXXXXXXXXXX

Time Machine Local Snapshots

Time Machine creates hourly local snapshots:

# Create a local snapshot manually
$ sudo tmutil localsnapshot
Created local snapshot with date: 2024-01-15-143045

# View space used by local snapshots
$ tmutil listlocalsnapshots / | wc -l
      24

# Delete specific snapshot
$ sudo tmutil deletelocalsnapshots 2024-01-15-143045
Deleted local snapshot '2024-01-15-143045'

# Delete all local snapshots
$ sudo tmutil deletelocalsnapshots /
Deleted 24 Time Machine local snapshots

System Update Snapshots

macOS creates snapshots before system updates:

$ diskutil apfs listSnapshots disk3s1 | grep "update"
    Snapshot Name:        com.apple.os.update-XXXX

These allow rollback if an update causes problems.

Mounting Snapshots

# Mount a snapshot read-only
$ sudo mkdir /tmp/snapshot_mount
$ sudo mount_apfs -s com.apple.TimeMachine.2024-01-15-091234.local /dev/disk3s5 /tmp/snapshot_mount

# Browse the snapshot
$ ls /tmp/snapshot_mount

# Unmount
$ sudo umount /tmp/snapshot_mount

Snapshot Space Management

Snapshots don’t immediately use space, but they prevent space reclamation:

# View purgeable space (includes snapshot data)
$ diskutil apfs list | grep -A5 "Container disk3"
    Size (Capacity Ceiling):      494384795648 B (494.4 GB)
    Capacity In Use By Volumes:   277848256512 B (277.8 GB) (56.2%)
    Capacity Not Allocated:       216536539136 B (216.5 GB) (43.8%)

# When deleting files, space may not free until snapshots are removed

Deleting Snapshots

# Delete by name
$ sudo diskutil apfs deleteSnapshot disk3s5 -name "com.apple.TimeMachine.2024-01-15-091234.local"

# Delete by UUID
$ sudo diskutil apfs deleteSnapshot disk3s5 -uuid 8B6C8F4E-XXXX-XXXX-XXXX-XXXXXXXXXXXX

# Delete all Time Machine local snapshots (frees space)
$ sudo tmutil deletelocalsnapshots /

Sealed System Volumes

macOS Big Sur+ uses a “sealed” system volume:

$ diskutil apfs list | grep -A3 "Macintosh HD"
    +-> Volume disk3s1 Macintosh HD
        ---------------------------------------------------
        APFS Volume Disk (Role):   disk3s1 (System)
        Snapshot Sealed:           Yes

What Sealing Means

• Cryptographic verification of system files
• Any modification breaks the seal
• Ensures system integrity
• Snapshot contains known-good state

Sealed Snapshot Boot

The system actually boots from a sealed snapshot:

# Current boot is a snapshot
$ mount | grep "disk3s1"
/dev/disk3s1s1 on / (apfs, sealed, local, read-only, journaled)
#            ^^ Note the 's1' suffix indicating snapshot

Container Operations

Resizing Containers

# Shrink container to specific size
$ sudo diskutil apfs resizeContainer disk3 400GB

# Expand container to fill available space
$ sudo diskutil apfs resizeContainer disk3 0

Creating New Containers

# On a new disk or partition
$ sudo diskutil apfs createContainer disk2s2
Started APFS operation on disk2s2
Creating new APFS Container
...

Deleting Containers

# Delete container (destroys all volumes!)
$ sudo diskutil apfs deleteContainer disk3

Warning: This destroys all data in the container.

Practical Scenarios

Create a Development Volume

# Case-sensitive volume for cross-platform development
$ sudo diskutil apfs addVolume disk3 "Case-sensitive APFS" "Development" -quota 100g

# Will mount at /Volumes/Development
$ cd /Volumes/Development
$ git clone git@github.com:user/project.git

Create an Encrypted Work Volume

# Encrypted volume for sensitive data
$ sudo diskutil apfs addVolume disk3 APFS "WorkSecure" -passphrase

# Lock/unlock manually
$ diskutil apfs lockVolume disk3s7
$ diskutil apfs unlockVolume disk3s7

Recover Space from Snapshots

# Check space
$ df -h /
Filesystem       Size   Used  Avail Capacity  Mounted on
/dev/disk3s1s1  460Gi  256Gi  180Gi    59%    /

# Delete unnecessary snapshots
$ sudo tmutil deletelocalsnapshots /
$ sudo tmutil deletelocalsnapshots /System/Volumes/Data

# Verify space recovered
$ df -h /

Browse Past File Versions

# Find available snapshots
$ tmutil listlocalsnapshots /System/Volumes/Data

# Mount a snapshot
$ sudo mkdir /tmp/old_data
$ sudo mount_apfs -s com.apple.TimeMachine.2024-01-14-091234.local /dev/disk3s5 /tmp/old_data

# Browse old versions
$ ls /tmp/old_data/Users/david/Documents/

# Copy needed files
$ cp /tmp/old_data/Users/david/Documents/important.doc ~/Desktop/

# Unmount
$ sudo umount /tmp/old_data

Summary

APFS storage hierarchy:

Key advantages of APFS:

• Space sharing: No need to pre-size volumes
• Instant snapshots: Capture state without copying
• Fast clones: Duplicate files without using space
• Encryption: Built into the filesystem

Understanding this hierarchy helps you manage storage effectively, recover from issues, and take advantage of features that traditional Unix filesystems don’t offer.

Mastering the macOS Shell

The shell is your primary interface to macOS’s Unix power. While the graphical interface handles most daily tasks, the shell provides capabilities that no GUI can match: automation, text processing, remote access, and precise control over system operations.

macOS’s shell environment has evolved significantly, most notably with the transition from bash to zsh as the default shell in Catalina (2019). Understanding this environment—and its macOS-specific characteristics—is essential for effective command-line work.

Why the Shell Matters

For Unix users, the shell is familiar territory. But macOS’s shell environment has unique characteristics:

• Default shell change: zsh replaced bash as the default in 2019
• BSD commands: The underlying utilities differ from GNU/Linux
• macOS integration: Clipboard, Notification Center, and GUI apps accessible from shell
• Path complexity: Framework libraries, Homebrew, and Xcode tools all need PATH attention
• Security restrictions: SIP and privacy controls affect shell capabilities

Shell Basics for Mac Newcomers

If you’re new to Unix shells, here’s the essential context:

A shell is a program that interprets your commands and executes them. It provides:

• Command execution
• Script interpretation
• Environment variable management
• Input/output redirection
• Job control

macOS includes several shells:

$ cat /etc/shells
# List of acceptable shells
/bin/bash
/bin/csh
/bin/dash
/bin/ksh
/bin/sh
/bin/tcsh
/bin/zsh

What You’ll Learn in This Part

The Great Shell Transition: Bash to Zsh explains why Apple changed the default shell, what the differences are, and how to adapt your workflows.

Terminal.app Deep Dive covers Apple’s built-in terminal emulator, including its features, configuration, and capabilities that many users never discover.

iTerm2 and Alternative Terminals explores third-party terminal emulators that offer features beyond Terminal.app.

Shell Configuration on macOS teaches you to configure zsh and bash effectively, understanding which files are loaded when.

Environment Variables and PATH addresses one of the most common sources of confusion: getting your PATH right across different contexts.

macOS-Specific Shell Features introduces shell capabilities unique to macOS, from clipboard integration to speaking text.

Shell Integration with macOS Services shows how to connect command-line work with Finder, Spotlight, Keychain, and other system services.

Terminal Quick Start

If you’re new to the Mac terminal:

1. Open Terminal: Press Cmd+Space, type “Terminal”, press Enter
2. You’re now in zsh: The default shell since Catalina
3. Your home directory: Terminal starts in ~ (your home folder)
4. Basic commands work: ls, cd, pwd, cat, grep, etc.

# Where am I?
$ pwd
/Users/yourname

# What's here?
$ ls -la

# Go somewhere
$ cd Documents

# Run something
$ echo "Hello, macOS Unix!"
Hello, macOS Unix!

From here, the following chapters will help you master the macOS shell environment.

The Great Shell Transition: Bash to Zsh

In macOS Catalina (2019), Apple changed the default shell from bash to zsh. This was one of the most significant changes for command-line users in macOS history. Understanding why this happened, what’s different, and how to work with both shells is essential knowledge.

Why Apple Switched

The Licensing Issue

The primary reason was licensing. Bash 3.2 was the last version released under GPLv2. Starting with Bash 4.0 (released in 2009), bash is licensed under GPLv3.

Apple has consistently avoided GPLv3 software because:

• GPLv3 includes provisions about hardware restrictions (Tivoization)
• GPLv3 has patent licensing requirements
• Apple’s business model conflicts with some GPLv3 terms

This left Apple shipping decade-old Bash 3.2:

# macOS's built-in bash (still available)
$ /bin/bash --version
GNU bash, version 3.2.57(1)-release (arm64-apple-darwin23)
Copyright (C) 2007 Free Software Foundation, Inc.

# Modern bash (via Homebrew)
$ /opt/homebrew/bin/bash --version
GNU bash, version 5.2.26(1)-release (aarch64-apple-darwin23.2.0)

Zsh’s Advantages

Beyond licensing, zsh offers genuine improvements:

• Better completion system
• More flexible customization
• Improved array handling
• Better globbing and pattern matching
• More active development community
• Broad plugin ecosystem (Oh My Zsh, etc.)

What Changed

Default Shell

New user accounts use zsh. Existing accounts retain their shell:

# Check your current shell
$ echo $SHELL
/bin/zsh

# Check what shell is running
$ echo $0
-zsh

# See all configured shells
$ cat /etc/shells

Configuration Files

Zsh uses different configuration files than bash:

Startup Warnings

If you have bash configurations but use zsh, you may see:

The default interactive shell is now zsh.
To update your account to use zsh, please run `chsh -s /bin/zsh`.
For more details, please visit https://support.apple.com/kb/HT208050.

To silence this warning:

# In ~/.bash_profile or ~/.bashrc
export BASH_SILENCE_DEPRECATION_WARNING=1

Staying with Bash

You can continue using bash:

# Change default shell to bash
$ chsh -s /bin/bash

# Or install modern bash and use that
$ brew install bash
$ sudo sh -c 'echo /opt/homebrew/bin/bash >> /etc/shells'
$ chsh -s /opt/homebrew/bin/bash

Note: Using Homebrew’s bash gives you version 5.x features.

Key Differences

Array Indexing

This is a common gotcha:

# Bash: Arrays are 0-indexed
arr=(one two three)
echo ${arr[0]}  # "one"

# Zsh: Arrays are 1-indexed by default
arr=(one two three)
echo ${arr[1]}  # "one"
echo ${arr[0]}  # Empty!

For bash compatibility in zsh:

# In .zshrc - make arrays 0-indexed
setopt KSH_ARRAYS

# Or use zsh-native indexing
echo $arr[1]  # "one"

Word Splitting

# Bash: Variables split on whitespace
var="one two three"
for word in $var; do echo $word; done
# Output: one, two, three (three lines)

# Zsh: Variables don't split by default
var="one two three"
for word in $var; do echo $word; done
# Output: one two three (one line!)

# Zsh: Use explicit splitting
for word in ${=var}; do echo $word; done
# Or set option
setopt SH_WORD_SPLIT

Glob Patterns

# Bash: Unmatched globs pass through as literal
$ ls *.nonexistent
ls: *.nonexistent: No such file or directory

# Zsh: Unmatched globs are errors by default
$ ls *.nonexistent
zsh: no matches found: *.nonexistent

# Zsh: Bash-like behavior
setopt NULL_GLOB  # Unmatched patterns expand to nothing
# or
setopt NO_NOMATCH  # Unmatched patterns pass through literally

History

# Zsh has better history features
# These are often set by default or by Oh My Zsh

# Share history between sessions
setopt SHARE_HISTORY

# Append rather than overwrite
setopt APPEND_HISTORY

# Add timestamps
setopt EXTENDED_HISTORY

# Don't store duplicates
setopt HIST_IGNORE_DUPS

Prompts

# Bash PS1
export PS1='\u@\h:\w\$ '

# Zsh PROMPT (different escape sequences)
export PROMPT='%n@%m:%~%# '

# Zsh equivalents:
# %n = username (\u in bash)
# %m = hostname (\h in bash)
# %~ = current directory with ~ abbreviation (\w in bash)
# %# = # for root, % for users (\$ in bash)

Completion

Zsh’s completion system is more powerful:

# Enable completion system
autoload -Uz compinit
compinit

# Case-insensitive completion
zstyle ':completion:*' matcher-list 'm:{a-zA-Z}={A-Za-z}'

# Menu selection
zstyle ':completion:*' menu select

# Colored completion
zstyle ':completion:*:default' list-colors ${(s.:.)LS_COLORS}

Migration Guide

Step 1: Check Current Setup

# What shell files do you have?
ls -la ~/.bash* ~/.zsh* ~/.profile 2>/dev/null

Step 2: Create Zsh Configuration

Create ~/.zshrc with your settings:

# ~/.zshrc

# History configuration
HISTSIZE=10000
SAVEHIST=10000
HISTFILE=~/.zsh_history
setopt SHARE_HISTORY
setopt HIST_IGNORE_DUPS

# Directory navigation
setopt AUTO_CD
setopt AUTO_PUSHD

# Completion
autoload -Uz compinit
compinit

# Prompt (simple example)
PROMPT='%F{green}%n@%m%f:%F{blue}%~%f %# '

# Your aliases
alias ll='ls -la'
alias la='ls -A'

# Your PATH additions
export PATH="/opt/homebrew/bin:$PATH"

Step 3: Migrate Aliases and Functions

Most aliases work identically:

# These work in both bash and zsh
alias grep='grep --color=auto'
alias ll='ls -la'
alias ..='cd ..'

Functions may need adjustment:

# Bash function
function_name() {
    local var="$1"
    # ...
}

# Works in zsh too, but zsh allows:
function function_name {
    local var="$1"
    # ...
}

Step 4: Test Scripts

Scripts should specify their interpreter:

#!/bin/bash
# This script will run in bash regardless of user's default shell

#!/bin/zsh
# This script will run in zsh

#!/bin/sh
# This script runs in POSIX sh (which is bash on macOS)

Bash Compatibility Mode

For scripts that need to work in both:

# In zsh, enable bash-like behavior
emulate -L bash
# or individual options:
setopt BASH_REMATCH
setopt KSH_ARRAYS
setopt SH_WORD_SPLIT

Or create a compatibility shim in .zshrc:

# Source bash configuration if it exists
if [ -f ~/.bashrc ]; then
    emulate -L bash
    source ~/.bashrc
    emulate -L zsh
fi

Oh My Zsh

Many users adopt Oh My Zsh for easier zsh configuration:

# Install Oh My Zsh
$ sh -c "$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)"

Oh My Zsh provides:

• Sensible defaults
• Themes (prompt customization)
• Plugins (git, docker, etc.)
• Easier configuration

# Example .zshrc with Oh My Zsh
export ZSH="$HOME/.oh-my-zsh"
ZSH_THEME="robbyrussell"
plugins=(git docker brew macos)
source $ZSH/oh-my-zsh.sh

Alternative Frameworks

• Prezto: Faster than Oh My Zsh
• Zinit: Modern plugin manager
• Antibody: Lightweight plugin manager
• Plain zsh: No framework needed

Checking Shell Compatibility

Test Scripts

# Test script in different shells
$ bash script.sh
$ zsh script.sh
$ sh script.sh

ShellCheck

# Install shellcheck
$ brew install shellcheck

# Check a script
$ shellcheck script.sh

Portable Scripts

For maximum portability, use POSIX sh:

#!/bin/sh
# POSIX-compliant script

# Use [ ] not [[ ]]
if [ "$var" = "value" ]; then
    echo "Match"
fi

# Use $(command) not `command`
result=$(date +%Y)

# Avoid bash/zsh-specific features

Summary

The bash-to-zsh transition:

Key zsh differences to remember:

• Arrays are 1-indexed (not 0)
• Variables don’t word-split by default
• Unmatched globs are errors
• Different prompt escapes
• More powerful completion and globbing

Zsh is a capable shell that rewards learning its features. Whether you migrate fully or maintain dual proficiency, understanding both shells makes you effective on macOS.

Terminal.app Deep Dive

Terminal.app is macOS’s built-in terminal emulator. While many power users immediately install iTerm2, Terminal.app is surprisingly capable. Understanding its features helps you work effectively on any Mac, even when you can’t install third-party software.

Finding and Launching Terminal

# Terminal is in Utilities
/Applications/Utilities/Terminal.app

# Quick access:
# - Spotlight: Cmd+Space, type "Terminal"
# - Finder: Go → Utilities → Terminal
# - Launchpad: Other folder → Terminal

Creating Quick Access

Add Terminal to your Dock by dragging it from Applications/Utilities, or create a keyboard shortcut:

System Preferences → Keyboard → Shortcuts → Services

• Enable “New Terminal at Folder”
• Enable “New Terminal Tab at Folder”

Now you can right-click any folder and open Terminal there.

Terminal Preferences

Access with Cmd+, or Terminal → Preferences:

General Tab

On startup, open: Choose default behavior

• New window with profile
• New window with same profile (continues last session’s appearance)

Shells open with:

• Default login shell (/bin/zsh)
• Command (specify custom shell or command)

New tabs open with:

• Default Working Directory
• Same Working Directory as current tab

Profiles Tab

Terminal uses “profiles” for appearance and behavior settings. The default profile is “Basic.”

Creating Custom Profiles

1. Click + to add a new profile
2. Name it (e.g., “Development”)
3. Configure settings
4. Set as default with “Default” button

Profile Settings

Text Tab:

• Font: Choose monospace font (Menlo, SF Mono, etc.)
• Font size: Typically 12-14pt
• Text and bold colors
• Cursor: Block, underline, or vertical bar
• Cursor blink

# View current terminal size
$ echo "Columns: $COLUMNS, Rows: $LINES"
Columns: 80, Rows: 24

Window Tab:

• Window size (columns × rows)
• Background color/image
• Blur and transparency
• Title: What appears in title bar

Shell Tab:

• Startup command (run on shell start)
• Ask before closing (prevents accidental closure)
• Close if shell exited cleanly

Keyboard Tab:

• Use Option as Meta key (important for Emacs, many CLI tools)
• Function key behavior

Advanced Tab:

• Terminal type: xterm-256color (recommended for color support)
• International encoding: UTF-8
• Scroll behavior

Recommended Settings

For a good development experience:

1. Font: SF Mono or Menlo, 13pt
2. Window size: 120×35 or larger
3. Terminal type: xterm-256color
4. Option as Meta: Enable (for Alt key bindings)
5. Scroll: Enable “Scroll to bottom on input”

Working with Windows and Tabs

Keyboard Shortcuts

Window Groups

Save and restore window arrangements:

1. Arrange windows as desired
2. Window → Save Windows as Group
3. Name the group
4. Restore: Window → Open Window Group

Tab Bar

Show tab bar even with single tab:

• View → Show Tab Bar
• Or: Cmd+Shift+T toggles tab bar visibility

Text Selection and Editing

Selection Shortcuts

Copy and Paste

# Standard shortcuts
Cmd+C          # Copy
Cmd+V          # Paste
Cmd+Shift+V    # Paste escaped (safe for commands)

Paste Escaped (Cmd+Shift+V): Escapes special characters, preventing accidental command execution when pasting untrusted content.

Quick Look

Select text, press Cmd+Y to Quick Look (preview) URLs or file paths.

Marks and Navigation

Terminal maintains “marks” at each command prompt:

Using Marks

Searching

Colors and Themes

Built-in Profiles

Terminal includes several profiles:

• Basic (black on white)
• Grass (green on green)
• Homebrew (green on black, classic)
• Novel (warm, paper-like)
• Ocean (blue theme)
• Pro (semi-transparent dark)
• Red Sands (warm red tones)
• Silver Aerogel (metallic)
• Solid Colors (various)

Importing Themes

Download .terminal files and double-click to import, or:

1. Terminal → Preferences → Profiles
2. Click gear icon → Import
3. Select .terminal file

Popular theme sources:

• GitHub: lysyi3m/macos-terminal-themes
• Terminal.Sexy
• Gogh Themes

Creating Custom Colors

1. Preferences → Profiles → Text
2. Click color swatches to customize
3. For background: Window tab

256-Color Support

Ensure “Declare terminal as” is set to xterm-256color:

# Test 256-color support
$ for i in {0..255}; do printf "\e[48;5;${i}m  \e[0m"; done; echo

Shell Integration

macOS Terminal has built-in shell integration (since El Capitan):

What Shell Integration Provides

• Marks at each prompt (for navigation)
• Current directory tracking (new tabs open in same directory)
• Command exit status indication
• Notification on long-running command completion

Enabling Shell Integration

Shell integration is automatic with the default zsh configuration. For bash or custom configurations:

# In .zshrc or .bashrc
# This is usually automatic, but can be explicit:
if [[ "$TERM_PROGRAM" == "Apple_Terminal" ]]; then
    # Terminal is managing shell integration
fi

Directory Tracking

With shell integration, Terminal knows your working directory:

# New tab opens in same directory
# Cmd+T → new tab at ~/Documents (if that's where you were)

# Title bar shows current directory

Command Completion Notifications

Terminal can notify you when long commands finish:

1. Edit → Notify When Running Process Completes
2. Or: View → Allow Notifications

Touch Bar Support

On MacBook Pro with Touch Bar:

• Man page viewer
• Color pickers
• Autocomplete suggestions

Customize in System Preferences → Keyboard → Customize Touch Bar.

Secure Input

For entering passwords securely:

1. Terminal → Secure Keyboard Entry (or Cmd+Shift+S)
2. Prevents other applications from reading keystrokes
3. Disable when done to allow normal functionality

# Icon in title bar indicates secure input mode

Working with Files

Opening Files

# Open in default application
$ open document.pdf
$ open image.png

# Open in specific application
$ open -a Preview image.png
$ open -a "Visual Studio Code" file.txt

# Reveal in Finder
$ open -R file.txt

Drag and Drop

• Drag files to Terminal to insert their paths
• Paths are automatically quoted if they contain spaces

Services

Right-click selected text for Services menu:

• Open man page
• Search in Spotlight
• Look up in Dictionary

Printing

Print terminal content:

1. Make selection (optional—prints all if nothing selected)
2. File → Print or Cmd+P
3. Options: Include timestamp, color, etc.

Accessibility

VoiceOver Support

Terminal works with VoiceOver (screen reader):

• Cmd+F5 to enable VoiceOver
• Reads command output
• Navigate with VoiceOver commands

Font Smoothing

For better readability:

• System Preferences → Accessibility → Display
• Adjust contrast and transparency

Window Zoom

• Cmd++ / Cmd+- to zoom
• View → Reset Zoom to default

Automation

AppleScript

Terminal is scriptable:

tell application "Terminal"
    do script "echo 'Hello from AppleScript'"
    do script "cd ~/Documents" in front window
end tell

Opening URLs

Terminal recognizes URLs:

• Cmd+Click to open in browser
• URLs are underlined when hovering

Custom URL Schemes

Create custom schemes to open Terminal from other apps:

# URL: x-terminal://open?command=ls%20-la

Troubleshooting

Reset Terminal

If settings get corrupted:

# Reset to factory defaults
$ defaults delete com.apple.Terminal
# Restart Terminal

Slow Startup

If Terminal is slow to open:

# Clear ASL logs
$ sudo rm -rf /var/log/asl/*.asl

# Check shell startup files for slow operations
# Time your shell startup:
$ time zsh -i -c exit

TERM Variable Issues

If applications look wrong:

# Check TERM
$ echo $TERM
xterm-256color

# Set in Preferences → Profiles → Advanced
# Or in shell config:
export TERM=xterm-256color

Summary

Terminal.app capabilities:

Terminal.app is a solid terminal emulator that handles most needs. For advanced features like split panes, triggers, or extensive customization, consider iTerm2. But for many users, Terminal.app is sufficient and has the advantage of requiring no additional installation.

iTerm2 and Alternative Terminals

While Terminal.app is capable, many power users prefer third-party terminal emulators. iTerm2 is the most popular choice, offering features like split panes, extensive customization, and powerful automation. This chapter covers iTerm2 in depth and surveys alternatives.

iTerm2

iTerm2 is a free, open-source terminal emulator for macOS with an active community and extensive feature set.

Installation

# Via Homebrew (recommended)
$ brew install --cask iterm2

# Or download from: https://iterm2.com

Key Features

Split Panes

Create multiple panes in a single tab:

Hotkey Window

A terminal that slides down from the top (like Quake consoles):

1. Preferences → Keys → Hotkey
2. Create a Dedicated Hotkey Window
3. Assign a hotkey (e.g., Option+Space)
4. Configure to show/hide with hotkey

Search with Regex

Cmd+F opens find bar with regex support:

# Search for IP addresses
\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}

# Search for email-like patterns
\S+@\S+\.\S+

Shell Integration

iTerm2’s shell integration provides:

• Current directory in tab title
• Command history with timestamps
• Marks at prompts
• Captured command output
• Alerts on long command completion
• File download from remote servers

Install:

# Automatic installation
$ curl -L https://iterm2.com/shell_integration/install_shell_integration.sh | bash

# Or install for specific shell
$ curl -L https://iterm2.com/shell_integration/zsh -o ~/.iterm2_shell_integration.zsh
$ echo 'source ~/.iterm2_shell_integration.zsh' >> ~/.zshrc

After installation, right-click to:

• Select output of last command
• Download files from remote hosts
• Open Recent Directories

Triggers

Automatically perform actions when text matches patterns:

1. Preferences → Profiles → Advanced → Triggers
2. Add regex patterns
3. Choose actions: highlight, alert, run command, etc.

Examples:

• Highlight errors in red
• Alert on build completion
• Run command when pattern matches

Instant Replay

Scroll back through terminal history with time-based replay:

1. Press Cmd+Option+B
2. Use arrow keys to move through history
3. Press Escape to return to live view

Password Manager

Store frequently-used passwords:

1. Preferences → Profiles → Advanced → Triggers
2. Or: Window → Password Manager (Cmd+Option+F)
3. Store username/password pairs
4. Click to paste (use with Secure Keyboard Entry)

Image Display

iTerm2 can display images inline:

# Using imgcat (part of shell integration)
$ imgcat image.png

# For remote servers, install imgcat:
$ curl -L https://iterm2.com/utilities/imgcat -o ~/bin/imgcat
$ chmod +x ~/bin/imgcat

Broadcast Input

Type in multiple panes simultaneously:

1. Shell → Broadcast Input
2. Options: To All Panes, To All Tabs, etc.
3. Type once, appears everywhere

Useful for running the same command on multiple servers.

Profiles

Create different profiles for different contexts:

• Development (large font, dark theme)
• SSH (different colors per server)
• Presentation (huge font, high contrast)

Switch with Cmd+O (profile picker).

Configuration

Appearance

Preferences → Appearance:

• Theme: Dark, Light, Minimal, etc.
• Tab position: Top, Left, Bottom
• Status bar: Enable/configure

Keys

Preferences → Keys → Key Bindings:

• Global hotkey
• Custom key mappings
• Presets: Natural Text Editing (makes Option+Arrow work like other apps)

Natural Text Editing Preset:

1. Preferences → Profiles → Keys
2. Key Mappings → Presets → Natural Text Editing

This enables:

• Option+Left/Right - Move by word
• Option+Backspace - Delete word
• Cmd+Left/Right - Move to line start/end

Status Bar

Preferences → Profiles → Session → Status bar:

Components:

• Current directory
• Git branch
• CPU/Memory usage
• Battery
• Custom text

Scripting

iTerm2 supports Python scripting:

#!/usr/bin/env python3
# Requires: pip install iterm2

import iterm2

async def main(connection):
    app = await iterm2.async_get_app(connection)
    window = app.current_terminal_window
    if window:
        await window.current_tab.current_session.async_send_text('echo Hello\n')

iterm2.run_until_complete(main)

Recommended Settings

For productivity:

1. Enable hotkey window (Option+Space)
2. Install shell integration
3. Enable Natural Text Editing
4. Configure triggers for errors/completions
5. Set up profiles for different contexts

Alternative Terminal Emulators

Alacritty

GPU-accelerated, fast, cross-platform terminal:

$ brew install --cask alacritty

Characteristics:

• Very fast (GPU rendering)
• Configuration via TOML file
• No tabs (use tmux)
• Cross-platform (macOS, Linux, Windows)
• Minimal features by design

Configuration (~/.config/alacritty/alacritty.toml):

[font]
normal = { family = "SF Mono", style = "Regular" }
size = 13.0

[colors.primary]
background = "#1d1f21"
foreground = "#c5c8c6"

[window]
dimensions = { columns = 120, lines = 35 }
padding = { x = 5, y = 5 }

Best for: Users who want speed and use tmux for session management.

Kitty

Feature-rich, GPU-accelerated terminal:

$ brew install --cask kitty

Characteristics:

• GPU rendering (fast)
• Native tabs and splits
• Image display
• Ligature support
• Remote file access
• Extensive customization

Configuration (~/.config/kitty/kitty.conf):

font_family SF Mono
font_size 13.0
background #1a1a1a
foreground #d8d8d8
enable_audio_bell no

Best for: Users wanting features without sacrificing performance.

Warp

Modern terminal with AI features:

$ brew install --cask warp

Characteristics:

• Block-based command interface
• AI command suggestions
• Modern IDE-like editing
• Shared workflows
• Requires account (free tier available)

Best for: Users who want a modern take on terminals with AI assistance.

Hyper

Web-technology-based terminal:

$ brew install --cask hyper

Characteristics:

• Built on Electron (HTML/CSS/JS)
• Highly themeable
• Plugin ecosystem
• Cross-platform
• Higher memory usage than native apps

Configuration (~/.hyper.js):

module.exports = {
    config: {
        fontSize: 13,
        fontFamily: '"SF Mono", monospace',
        cursorShape: 'BLOCK',
        shell: '/bin/zsh',
    },
    plugins: ['hyper-dracula'],
};

Best for: Web developers who want to customize with JS/CSS.

WezTerm

GPU-accelerated, cross-platform, Lua-configured:

$ brew install --cask wezterm

Characteristics:

• GPU rendering
• Lua configuration
• Multiplexing built-in
• Cross-platform
• Active development

Configuration (~/.wezterm.lua):

local wezterm = require 'wezterm'
return {
    font = wezterm.font 'SF Mono',
    font_size = 13.0,
    color_scheme = 'Catppuccin Mocha',
    enable_tab_bar = true,
}

Best for: Users who want customization with a real programming language.

Feature Comparison

tmux: Terminal Multiplexer

Regardless of which terminal you choose, tmux provides powerful session management:

$ brew install tmux

tmux provides:

• Sessions that persist after disconnection
• Split panes (works in any terminal)
• Multiple windows
• Session sharing
• Scriptable

Basic tmux usage:

# Start new session
$ tmux new -s work

# Detach
Ctrl+b, d

# List sessions
$ tmux ls

# Reattach
$ tmux attach -t work

# Split panes
Ctrl+b, %    # Vertical
Ctrl+b, "    # Horizontal

# Navigate panes
Ctrl+b, arrow

Recommendation: If you use Alacritty, tmux is essential. For iTerm2 or Kitty users, tmux is optional but useful for remote work.

Choosing a Terminal

Use Terminal.app if:

• You want no additional software
• You’re working on shared/managed Macs
• Your needs are basic

Use iTerm2 if:

• You want split panes and hotkey window
• You need advanced features (triggers, shell integration)
• You prefer GUI configuration
• You’re a power user

Use Alacritty/Kitty if:

• You want maximum performance
• You prefer config-file configuration
• You use tmux
• You work cross-platform

Use Warp if:

• You want AI assistance
• You prefer modern IDE-like interfaces
• You’re comfortable with an account requirement

Summary

The terminal emulator choice is personal, but common paths:

1. Start with Terminal.app - Learn the basics
2. Graduate to iTerm2 - Most macOS power users land here
3. Consider alternatives - If you have specific needs (speed, cross-platform, etc.)

iTerm2’s combination of features, stability, and active development makes it the default recommendation for macOS power users. But any terminal that supports UTF-8 and 256 colors will work well for most tasks—the choice is about workflow optimization, not capability.

Shell Configuration on macOS

Getting your shell configured correctly is fundamental to a productive command-line experience. This chapter explains which configuration files are loaded when, how to structure your setup, and macOS-specific considerations.

Understanding Shell Invocation Modes

Shells can be invoked in different modes, and different configuration files are loaded depending on the mode:

Login vs Non-Login Shell

Login shell: First shell started when you log in. Characteristics:

• Reads “profile” files
• Sets up initial environment
• Terminal.app starts login shells by default

Non-login shell: Shells started from other shells or programs:

• Reads “rc” (run commands) files
• Inherits environment from parent
• Scripts typically run as non-login shells

Interactive vs Non-Interactive

Interactive: You’re typing commands at a prompt:

• Reads configuration files
• Sets up completion, prompts, aliases

Non-interactive: Running a script:

• Generally skips most configuration
• Faster startup

Zsh Configuration Files

Zsh reads files in this order:

All Shells

1. /etc/zshenv - System-wide, all shells
2. ~/.zshenv - User, all shells (including scripts)

Login Shells

3. /etc/zprofile - System-wide login
4. ~/.zprofile - User login

Interactive Shells

5. /etc/zshrc - System-wide interactive
6. ~/.zshrc - User interactive

Login Shell Completion

7. /etc/zlogin - System-wide, after zshrc
8. ~/.zlogin - User, after zshrc

Logout

9. ~/.zlogout - User logout cleanup
10. /etc/zlogout - System-wide logout

Diagram

                    ┌──────────────────┐
                    │  Shell Started   │
                    └────────┬─────────┘
                             │
                    ┌────────┴─────────┐
                    │    .zshenv       │ ← All zsh instances
                    └────────┬─────────┘
                             │
            ┌────────────────┼────────────────┐
            │                │                │
      ┌─────┴──────┐   ┌─────┴──────┐  ┌─────┴──────┐
      │ Non-Login  │   │   Login    │  │   Script   │
      │Interactive │   │Interactive │  │    (sh)    │
      └─────┬──────┘   └─────┬──────┘  └────────────┘
            │                │
            │         ┌──────┴──────┐
            │         │  .zprofile  │
            │         └──────┬──────┘
            │                │
      ┌─────┴────────────────┴──────┐
      │          .zshrc             │ ← Interactive shells
      └─────┬────────────────┬──────┘
            │                │
            │         ┌──────┴──────┐
            │         │   .zlogin   │
            │         └─────────────┘
            │
       (shell runs)

Recommended File Usage

Bash Configuration Files

For those still using bash:

Login Shells

1. /etc/profile - System-wide
2. First found of: ~/.bash_profile, ~/.bash_login, ~/.profile

Non-Login Interactive Shells

1. /etc/bash.bashrc (not on macOS by default)
2. ~/.bashrc

macOS Quirk

Terminal.app always starts login shells, but many expect .bashrc to be read. Common solution:

# In ~/.bash_profile
if [ -f ~/.bashrc ]; then
    source ~/.bashrc
fi

Example Zsh Configuration

~/.zshenv

Keep minimal—runs for all shells including scripts:

# ~/.zshenv
# Only put things here that ALL zsh instances need

# Skip global compinit for faster startup
skip_global_compinit=1

~/.zprofile

Login-specific PATH modifications:

# ~/.zprofile
# Runs for login shells (Terminal.app)

# Add Homebrew to PATH (Apple Silicon)
eval "$(/opt/homebrew/bin/brew shellenv)"

# Or Intel Mac
# eval "$(/usr/local/bin/brew shellenv)"

~/.zshrc

The main configuration file:

# ~/.zshrc
# Interactive shell configuration

#------------------
# History
#------------------
HISTSIZE=50000
SAVEHIST=50000
HISTFILE=~/.zsh_history

setopt EXTENDED_HISTORY       # Save timestamp
setopt HIST_EXPIRE_DUPS_FIRST # Expire duplicates first
setopt HIST_IGNORE_DUPS       # Don't store duplicates
setopt HIST_IGNORE_SPACE      # Ignore commands starting with space
setopt HIST_VERIFY            # Show expanded history before executing
setopt SHARE_HISTORY          # Share between sessions

#------------------
# Directory Navigation
#------------------
setopt AUTO_CD           # Type directory name to cd
setopt AUTO_PUSHD        # Push directories to stack
setopt PUSHD_IGNORE_DUPS # Ignore duplicate directories
setopt PUSHD_SILENT      # Silent pushd

#------------------
# Completion
#------------------
autoload -Uz compinit
compinit

# Case-insensitive completion
zstyle ':completion:*' matcher-list 'm:{a-zA-Z}={A-Za-z}'

# Menu selection
zstyle ':completion:*' menu select

# Verbose completion
zstyle ':completion:*' verbose yes

# Group completions by category
zstyle ':completion:*' group-name ''

#------------------
# Key Bindings
#------------------
bindkey -e  # Emacs key bindings (default)
# Or: bindkey -v for vi mode

# Better history search
bindkey '^[[A' history-search-backward  # Up arrow
bindkey '^[[B' history-search-forward   # Down arrow

# Word navigation (Option+Arrow)
bindkey "^[[1;3C" forward-word   # Option+Right
bindkey "^[[1;3D" backward-word  # Option+Left

# Delete word
bindkey "^[^?" backward-kill-word  # Option+Backspace

#------------------
# Prompt
#------------------
# Simple prompt with git info
autoload -Uz vcs_info
precmd_vcs_info() { vcs_info }
precmd_functions+=( precmd_vcs_info )
setopt prompt_subst
zstyle ':vcs_info:git:*' formats ' (%b)'
PROMPT='%F{green}%n@%m%f:%F{blue}%~%f%F{yellow}${vcs_info_msg_0_}%f %# '

# Right-side prompt (optional)
# RPROMPT='%F{gray}%T%f'  # Time

#------------------
# Aliases
#------------------
alias ll='ls -la'
alias la='ls -A'
alias l='ls -CF'
alias grep='grep --color=auto'
alias ..='cd ..'
alias ...='cd ../..'

# macOS specific
alias o='open'
alias finder='open -a Finder .'

#------------------
# Functions
#------------------
# Create directory and cd into it
mkcd() {
    mkdir -p "$1" && cd "$1"
}

# Extract various archive types
extract() {
    if [ -f "$1" ]; then
        case "$1" in
            *.tar.bz2) tar xjf "$1" ;;
            *.tar.gz)  tar xzf "$1" ;;
            *.bz2)     bunzip2 "$1" ;;
            *.gz)      gunzip "$1" ;;
            *.tar)     tar xf "$1" ;;
            *.tbz2)    tar xjf "$1" ;;
            *.tgz)     tar xzf "$1" ;;
            *.zip)     unzip "$1" ;;
            *.Z)       uncompress "$1" ;;
            *.7z)      7z x "$1" ;;
            *)         echo "'$1' cannot be extracted" ;;
        esac
    else
        echo "'$1' is not a file"
    fi
}

#------------------
# Tool Configuration
#------------------
# FZF (if installed)
[ -f ~/.fzf.zsh ] && source ~/.fzf.zsh

# Node Version Manager (if installed)
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && source "$NVM_DIR/nvm.sh"

# Python environment
export PYENV_ROOT="$HOME/.pyenv"
[[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"
command -v pyenv >/dev/null && eval "$(pyenv init -)"

#------------------
# Local Configuration
#------------------
# Machine-specific settings not in version control
[ -f ~/.zshrc.local ] && source ~/.zshrc.local

macOS-Specific Configuration

Homebrew Setup

# In ~/.zprofile for login shells, or ~/.zshrc

# Apple Silicon
if [[ -f /opt/homebrew/bin/brew ]]; then
    eval "$(/opt/homebrew/bin/brew shellenv)"
fi

# Intel
if [[ -f /usr/local/bin/brew ]]; then
    eval "$(/usr/local/bin/brew shellenv)"
fi

macOS Aliases

# Quick Look
ql() { qlmanage -p "$@" &>/dev/null; }

# Show/hide hidden files
alias showfiles='defaults write com.apple.finder AppleShowAllFiles YES; killall Finder'
alias hidefiles='defaults write com.apple.finder AppleShowAllFiles NO; killall Finder'

# Clipboard
alias pbp='pbpaste'
alias pbc='pbcopy'

# Empty trash
alias emptytrash='rm -rf ~/.Trash/*'

# Lock screen
alias lock='/System/Library/CoreServices/Menu\ Extras/User.menu/Contents/Resources/CGSession -suspend'

# Get macOS software updates
alias update='sudo softwareupdate -i -a'

PATH Configuration

Common PATH additions:

# In ~/.zprofile or ~/.zshrc

# Standard local binaries
export PATH="/usr/local/bin:$PATH"

# Homebrew (Apple Silicon)
export PATH="/opt/homebrew/bin:/opt/homebrew/sbin:$PATH"

# User binaries
export PATH="$HOME/bin:$HOME/.local/bin:$PATH"

# Development tools
export PATH="$HOME/go/bin:$PATH"              # Go
export PATH="$HOME/.cargo/bin:$PATH"           # Rust
export PATH="$HOME/.rbenv/bin:$PATH"           # Ruby

Configuration Management

Version Control

Keep your dotfiles in Git:

# Initialize dotfiles repo
$ cd ~
$ git init --bare ~/.dotfiles

# Create alias
alias dotfiles='git --git-dir=$HOME/.dotfiles --work-tree=$HOME'

# Add files
$ dotfiles add ~/.zshrc ~/.zprofile
$ dotfiles commit -m "Add shell config"

# Push to remote
$ dotfiles remote add origin git@github.com:username/dotfiles.git
$ dotfiles push -u origin main

Modular Configuration

Split configuration into separate files:

# In ~/.zshrc
# Load all configuration modules
for config in ~/.config/zsh/*.zsh; do
    source "$config"
done

~/.config/zsh/
├── aliases.zsh
├── completion.zsh
├── functions.zsh
├── history.zsh
├── keybindings.zsh
└── prompt.zsh

Local Overrides

Keep machine-specific settings separate:

# End of ~/.zshrc
# Load local configuration (not version controlled)
[ -f ~/.zshrc.local ] && source ~/.zshrc.local

Add to .gitignore:

.zshrc.local

Debugging Configuration

Startup Time

Profile shell startup:

# Time shell startup
$ time zsh -i -c exit
zsh -i -c exit  0.08s user 0.04s system 94% cpu 0.124 total

# Detailed profiling
$ zsh -xv 2>&1 | head -100

Trace Loading

# In ~/.zshrc, at the beginning:
zmodload zsh/zprof

# At the end:
zprof

Finding Slow Operations

Common slowdowns:

• compinit without caching
• nvm/rbenv initialization
• Plugin managers loading many plugins
• Network operations in prompts

Solutions:

# Cache compinit
autoload -Uz compinit
if [[ -n ${ZDOTDIR}/.zcompdump(#qN.mh+24) ]]; then
    compinit
else
    compinit -C
fi

# Lazy-load slow tools
nvm() {
    unfunction nvm
    export NVM_DIR="$HOME/.nvm"
    source "$NVM_DIR/nvm.sh"
    nvm "$@"
}

Summary

Shell configuration on macOS:

Best practices:

1. Keep .zshenv minimal — runs for all shells
2. Put PATH in .zprofile — for login shells
3. Put interactive config in .zshrc — aliases, prompts, completion
4. Use .zshrc.local — for machine-specific settings
5. Version control your dotfiles — but not secrets
6. Profile startup time — optimize if needed

The configuration files structure your shell experience. Take time to set them up well, and your command-line work becomes significantly more efficient.

Environment Variables and PATH

Environment variables on macOS present unique challenges. Between Homebrew, GUI applications, multiple shell contexts, and macOS-specific mechanisms like launchd, getting your PATH and environment right requires understanding how each context inherits (or doesn’t inherit) environment settings.

Environment Variable Basics

Environment variables are name-value pairs available to processes:

# View all environment variables
$ env
# or
$ printenv

# View specific variable
$ echo $HOME
/Users/david

# Set variable for current session
$ export MY_VAR="value"

# Set for a single command
$ MY_VAR="value" some_command

Important Built-in Variables

macOS-Specific Variables

Understanding PATH

PATH tells the shell where to find executables:

$ echo $PATH
/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin

# Shell searches directories left to right
# First match wins

Default macOS PATH

A fresh macOS installation has a basic PATH:

/usr/local/bin
/usr/bin
/bin
/usr/sbin
/sbin

PATH Sources

macOS PATH comes from multiple places:

1. System-wide PATH (/etc/paths):

$ cat /etc/paths
/usr/local/bin
/usr/bin
/bin
/usr/sbin
/sbin

2. PATH additions (/etc/paths.d/*):

$ ls /etc/paths.d
100-rvictl
MacGPG2

$ cat /etc/paths.d/MacGPG2
/usr/local/MacGPG2/bin

3. Shell configuration (~/.zprofile, ~/.zshrc):

export PATH="/opt/homebrew/bin:$PATH"

4. path_helper utility (/usr/libexec/path_helper):

# Combines /etc/paths and /etc/paths.d
$ /usr/libexec/path_helper -s
PATH="/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/MacGPG2/bin"; export PATH;

Setting PATH Correctly

For zsh (default shell):

# In ~/.zprofile (for login shells)
# Add to BEGINNING of PATH (searched first)
export PATH="/opt/homebrew/bin:$PATH"

# Add to END of PATH (searched last)
export PATH="$PATH:$HOME/bin"

Common PATH Additions

# Homebrew (Apple Silicon)
export PATH="/opt/homebrew/bin:/opt/homebrew/sbin:$PATH"

# Homebrew (Intel)
export PATH="/usr/local/bin:/usr/local/sbin:$PATH"

# User binaries
export PATH="$HOME/bin:$HOME/.local/bin:$PATH"

# Go
export PATH="$HOME/go/bin:$PATH"

# Rust
export PATH="$HOME/.cargo/bin:$PATH"

# Python (pyenv)
export PATH="$HOME/.pyenv/shims:$PATH"

# Node (nvm adds its own path)

Environment for GUI Applications

GUI applications do not inherit your shell environment. This is a common source of confusion.

The Problem

# Terminal: This works
$ echo $MY_VAR
my_value

# But GUI apps don't see it
# VS Code launched from Dock won't have MY_VAR

Solutions

1. Launch from Terminal

# Open VS Code with terminal's environment
$ code .

# Open any app
$ open -a "Visual Studio Code"

2. Use launchctl setenv

Set environment variables system-wide:

# Set for current boot
$ launchctl setenv MY_VAR "my_value"

# Verify
$ launchctl getenv MY_VAR
my_value

Note: This persists only until logout. For persistence, use Launch Agents.

3. Launch Agent for Persistent Environment

Create ~/Library/LaunchAgents/environment.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>my.environment</string>
    <key>ProgramArguments</key>
    <array>
        <string>sh</string>
        <string>-c</string>
        <string>
            launchctl setenv MY_VAR "my_value"
            launchctl setenv PATH "/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin"
        </string>
    </array>
    <key>RunAtLoad</key>
    <true/>
</dict>
</plist>

Load it:

$ launchctl load ~/Library/LaunchAgents/environment.plist

4. Per-Application Settings

Some apps have their own environment settings:

• VS Code: terminal.integrated.env.osx in settings
• IntelliJ: Run configurations → Environment variables
• Xcode: Scheme → Run → Arguments → Environment Variables

Checking GUI App Environment

# See what environment an app sees
$ open -a TextEdit
# In TextEdit, open: /dev/fd/1 shows stdout
# Or check in Activity Monitor → Process → Environment

SSH and Remote Sessions

SSH sessions need environment setup too:

~/.ssh/environment

If PermitUserEnvironment is enabled on the server:

# ~/.ssh/environment
MY_VAR=value
PATH=/usr/local/bin:/usr/bin:/bin

SendEnv and AcceptEnv

Configure which variables are sent over SSH:

# ~/.ssh/config
Host *
    SendEnv LANG LC_*
    SendEnv MY_*

Server must have AcceptEnv configured to receive them.

Remote PATH Issues

Remote servers won’t have your local Homebrew:

# Script that works locally may fail remotely
$ ssh server 'brew list'  # Fails - brew not in default PATH

# Solutions:
# 1. Use full path
$ ssh server '/opt/homebrew/bin/brew list'

# 2. Source profile
$ ssh server 'source ~/.profile && brew list'

# 3. Interactive shell
$ ssh -t server 'bash -l -c "brew list"'

Common Issues and Solutions

“Command Not Found” After Installing

$ brew install something
$ something
zsh: command not found: something

Solutions:

# Refresh PATH
$ source ~/.zprofile
# Or
$ source ~/.zshrc
# Or start a new terminal

# Check if Homebrew PATH is set
$ echo $PATH | grep homebrew
# Should see /opt/homebrew/bin

# If not, add to ~/.zprofile:
eval "$(/opt/homebrew/bin/brew shellenv)"

PATH Order Issues

# System Python instead of Homebrew Python
$ which python3
/usr/bin/python3  # Wrong!

# Check PATH order
$ echo $PATH | tr ':' '\n'

Fix: Ensure Homebrew path comes before system paths:

export PATH="/opt/homebrew/bin:$PATH"

Subshell Doesn’t Have Variables

$ export MY_VAR="value"
$ bash -c 'echo $MY_VAR'  # Works
$ zsh script.zsh          # May not work if script starts fresh

Solution: Use -l for login shell or source configuration:

$ bash -l -c 'echo $MY_VAR'

Scripts Don’t Have PATH

Scripts run with #!/bin/sh start with minimal environment:

#!/bin/sh
brew list  # May fail - brew not in PATH

Solutions:

# Use full path in scripts
#!/bin/sh
/opt/homebrew/bin/brew list

# Or set PATH in script
#!/bin/sh
export PATH="/opt/homebrew/bin:$PATH"
brew list

# Or use env with explicit path
#!/usr/bin/env -P /opt/homebrew/bin python3

Environment File Patterns

Per-Directory Environment (.env files)

Many tools support .env files:

# .env file
DATABASE_URL=postgres://localhost/mydb
API_KEY=secret123

Load manually:

$ export $(grep -v '^#' .env | xargs)

Or use tools like direnv:

$ brew install direnv
$ echo 'eval "$(direnv hook zsh)"' >> ~/.zshrc

# Now .envrc files load automatically

direnv for Automatic Environment

# Install
$ brew install direnv

# Add to ~/.zshrc
eval "$(direnv hook zsh)"

# Create .envrc in project
$ cd myproject
$ echo 'export MY_VAR=value' > .envrc
$ direnv allow

# Variables load when you enter directory
$ cd myproject
direnv: loading .envrc
direnv: export +MY_VAR

$ cd ..
direnv: unloading

Summary

Environment variable contexts on macOS:

Key points:

1. PATH order matters — first match wins
2. GUI apps don’t get shell environment — use launchctl or launch from terminal
3. Homebrew needs explicit PATH setup — add in .zprofile
4. Scripts have minimal environment — use full paths or set PATH explicitly
5. Use direnv for project-specific environment — automatic loading/unloading

Getting environment configuration right eliminates entire categories of “works on my machine” problems.

macOS-Specific Shell Features

macOS provides unique commands and capabilities that integrate the command line with the graphical system. These tools let you leverage macOS features—clipboard, Spotlight, text-to-speech, notifications—from your shell scripts and daily workflow.

Clipboard Integration

pbcopy and pbpaste

The pasteboard (clipboard) is accessible from the command line:

# Copy to clipboard
$ echo "Hello" | pbcopy

# Paste from clipboard
$ pbpaste
Hello

# Copy file contents
$ cat file.txt | pbcopy
# or
$ pbcopy < file.txt

# Paste to file
$ pbpaste > output.txt

Practical Uses

# Copy current directory path
$ pwd | pbcopy

# Copy file contents to clipboard
$ cat ~/.ssh/id_rsa.pub | pbcopy
echo "SSH key copied to clipboard"

# Generate and copy a password
$ openssl rand -base64 12 | pbcopy

# Copy command output
$ ls -la | pbcopy

# Pipe clipboard through a command
$ pbpaste | sort | uniq | pbcopy

# Convert clipboard contents
$ pbpaste | tr '[:lower:]' '[:upper:]' | pbcopy

Multiple Pasteboards

macOS has multiple pasteboards:

# General pasteboard (default)
$ pbcopy
$ pbpaste

# Find pasteboard (Cmd+E in many apps)
$ pbcopy -pboard find
$ pbpaste -pboard find

# Ruler pasteboard
$ pbcopy -pboard ruler

The open Command

open is one of the most versatile macOS commands:

Basic Usage

# Open file with default application
$ open document.pdf      # Opens in Preview
$ open image.png         # Opens in Preview
$ open file.html         # Opens in Safari

# Open directory in Finder
$ open .                 # Current directory
$ open ~/Documents       # Specific directory

# Open URL
$ open https://apple.com

Specifying Applications

# Open with specific application
$ open -a Safari https://google.com
$ open -a "Visual Studio Code" file.txt
$ open -a Preview *.png

# Open with application bundle identifier
$ open -b com.apple.Safari https://apple.com

Additional Options

# Open new instance of application
$ open -n -a Safari

# Reveal file in Finder (don't open it)
$ open -R file.txt

# Open in background
$ open -g document.pdf

# Wait for application to close before returning
$ open -W document.pdf

# Edit with TextEdit
$ open -e file.txt

# Read from stdin
$ ls -la | open -f    # Opens output in TextEdit

Opening System Preferences

# Open System Preferences
$ open "x-apple.systempreferences:"

# Open specific preference pane
$ open "x-apple.systempreferences:com.apple.preference.security"
$ open "x-apple.systempreferences:com.apple.preference.network"
$ open "x-apple.systempreferences:com.apple.preferences.Bluetooth"

# Apple Silicon: System Settings uses different IDs
$ open "x-apple.systempreferences:com.apple.settings.PrivacySecurity.extension"

Spotlight from Command Line

mdfind

Search using Spotlight:

# Basic search
$ mdfind "search term"

# Search by filename
$ mdfind -name "document.pdf"

# Search in specific directory
$ mdfind -onlyin ~/Documents "budget"

# Search by file type
$ mdfind "kMDItemContentType == 'public.jpeg'"

# Search by date
$ mdfind "kMDItemLastUsedDate > $time.today(-7)"

# Live query (continues watching)
$ mdfind -live "kMDItemContentType == 'public.jpeg'"

Common Spotlight Attributes

# Find all PDFs
$ mdfind "kMDItemContentType == 'com.adobe.pdf'"

# Find images
$ mdfind "kMDItemContentType == 'public.image'"

# Find by author
$ mdfind "kMDItemAuthors == 'John Smith'"

# Find large files
$ mdfind "kMDItemFSSize > 100000000"  # > 100MB

# Find files modified today
$ mdfind "kMDItemFSContentChangeDate > $time.today"

# Find apps
$ mdfind "kMDItemContentType == 'com.apple.application-bundle'"

mdls - Spotlight Metadata

View metadata for a file:

$ mdls document.pdf
kMDItemContentType             = "com.adobe.pdf"
kMDItemContentTypeTree         = (
    "com.adobe.pdf",
    "public.data",
    "public.item",
    "public.content"
)
kMDItemDisplayName             = "document.pdf"
kMDItemFSContentChangeDate     = 2024-01-15 10:30:00 +0000
kMDItemFSCreationDate          = 2024-01-10 08:00:00 +0000
kMDItemFSName                  = "document.pdf"
kMDItemFSSize                  = 1048576
...

# Get specific attribute
$ mdls -name kMDItemFSSize document.pdf
kMDItemFSSize = 1048576

mdutil - Spotlight Indexing

Control Spotlight indexing:

# Check indexing status
$ mdutil -s /
/:
    Indexing enabled.

# Disable indexing for volume
$ sudo mdutil -i off /Volumes/External

# Erase and rebuild index
$ sudo mdutil -E /

# Enable indexing
$ sudo mdutil -i on /

Text-to-Speech

say Command

# Basic speech
$ say "Hello, world"

# Specify voice
$ say -v Samantha "Hello"
$ say -v Daniel "Hello"  # British
$ say -v Kyoko "こんにちは"  # Japanese

# List available voices
$ say -v '?'

# Speak file contents
$ say -f document.txt

# Save to audio file
$ say "Hello" -o hello.aiff
$ say "Hello" -o hello.m4a --data-format=aac

# Speak at different rate (words per minute)
$ say -r 200 "Fast speech"
$ say -r 100 "Slow speech"

Practical Uses

# Notify when long command finishes
$ make build; say "Build complete"

# Read log file
$ tail -f /var/log/system.log | while read line; do say "$line"; done

# Speak clipboard
$ pbpaste | say

# Alarm
$ sleep 300; say "Time's up"

Screen and Window Management

screencapture

Take screenshots from command line:

# Capture entire screen
$ screencapture screenshot.png

# Capture selection (interactive)
$ screencapture -i screenshot.png

# Capture specific window (click to select)
$ screencapture -W screenshot.png

# Capture with delay (seconds)
$ screencapture -T 5 screenshot.png

# Capture to clipboard
$ screencapture -c

# Capture without shadow
$ screencapture -o screenshot.png

# Capture in different format
$ screencapture -t jpg screenshot.jpg
$ screencapture -t pdf screenshot.pdf

screen Recording

# Basic screen recording (stop with Ctrl+C)
$ screencapture -v recording.mov

# Record specific display
$ screencapture -D 1 -v recording.mov

# Record specific area
$ screencapture -R 0,0,800,600 -v recording.mov

Notifications

osascript for Notifications

# Display notification
$ osascript -e 'display notification "Hello" with title "My Script"'

# With subtitle and sound
$ osascript -e 'display notification "Task complete" with title "Build" subtitle "Project X" sound name "Glass"'

terminal-notifier (Third-party)

# Install
$ brew install terminal-notifier

# Use
$ terminal-notifier -message "Hello" -title "My Script"

# With action
$ terminal-notifier -message "Click to open" -title "Alert" -open "https://apple.com"

# With sound
$ terminal-notifier -message "Done" -title "Build" -sound default

System Information

sw_vers

macOS version information:

$ sw_vers
ProductName:		macOS
ProductVersion:		14.0
BuildVersion:		23A344

system_profiler

Detailed system information:

# Full report (very verbose)
$ system_profiler

# Specific data types
$ system_profiler SPHardwareDataType
$ system_profiler SPSoftwareDataType
$ system_profiler SPNetworkDataType
$ system_profiler SPStorageDataType

# List available data types
$ system_profiler -listDataTypes

# Machine-readable output
$ system_profiler -json SPHardwareDataType

sysctl

Kernel parameters:

# CPU info
$ sysctl -n machdep.cpu.brand_string
Apple M1 Pro

# Memory size
$ sysctl -n hw.memsize
17179869184

# Number of CPUs
$ sysctl -n hw.ncpu
10

# Kernel version
$ sysctl -n kern.osrelease
23.0.0

Power Management

pmset

Power management settings:

# View current settings
$ pmset -g
System-wide power settings:
Currently in use:
 standby              1
 Sleep On Power Button 1
 hibernatefile        /var/vm/sleepimage
...

# Prevent sleep for a command
$ caffeinate -i long_running_command

# Prevent sleep for duration (seconds)
$ caffeinate -t 3600

# Prevent sleep until process exits
$ caffeinate -w $(pgrep -f "my_process")

# Schedule sleep
$ sudo pmset schedule sleep "01/20/24 22:00:00"

# Schedule wake
$ sudo pmset schedule wake "01/20/24 06:00:00"

User Management

dscl (Directory Service command line)

# List users
$ dscl . -list /Users

# User info
$ dscl . -read /Users/david

# Check group membership
$ dscl . -read /Groups/admin GroupMembership

# List groups
$ dscl . -list /Groups

id

$ id
uid=501(david) gid=20(staff) groups=20(staff),12(everyone),...

$ id david
$ groups david

Networking Commands

networksetup

# List network services
$ networksetup -listallnetworkservices
An asterisk (*) denotes that a network service is disabled.
Wi-Fi
Thunderbolt Ethernet

# Get current Wi-Fi network
$ networksetup -getairportnetwork en0
Current Wi-Fi Network: MyNetwork

# Get IP address
$ networksetup -getinfo "Wi-Fi"

airport

# Scan for networks
$ /System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport -s

# Current connection info
$ /System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport -I

# Create alias for convenience
alias airport='/System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport'

Disk Commands

# List disks
$ diskutil list

# Disk info
$ diskutil info disk0

# Eject disk
$ diskutil eject /Volumes/External

# Mounting/unmounting
$ diskutil mount disk2s1
$ diskutil unmount /Volumes/External

Summary

macOS-specific shell commands:

These commands bridge the gap between command-line efficiency and macOS’s graphical features, enabling powerful workflows that combine the best of both worlds.

Shell Integration with macOS Services

The command line on macOS doesn’t exist in isolation—it can interact with Finder, Spotlight, Keychain, and other system services. This integration enables workflows that combine the precision of shell commands with the convenience of macOS features.

Finder Integration

Opening Finder from Terminal

# Open current directory
$ open .

# Open specific directory
$ open ~/Documents

# Reveal file in Finder
$ open -R /path/to/file.txt

Getting Finder Selection

Get the currently selected files in Finder:

# Using AppleScript
$ osascript -e 'tell application "Finder" to get selection as alias list'

# More useful: get paths
$ osascript -e 'tell application "Finder"
    set sel to selection as alias list
    set output to ""
    repeat with f in sel
        set output to output & POSIX path of f & "\n"
    end repeat
    return output
end tell'

Create a shell function:

# Add to ~/.zshrc
finder_selection() {
    osascript -e 'tell application "Finder"
        set sel to selection as alias list
        set output to ""
        repeat with f in sel
            set output to output & POSIX path of f & "\n"
        end repeat
        return output
    end tell' 2>/dev/null | sed '/^$/d'
}

# Usage
$ finder_selection
/Users/david/Documents/file1.txt
/Users/david/Documents/file2.txt

Creating Finder Aliases

# Create Finder alias (different from symlink)
$ osascript -e 'tell application "Finder" to make new alias file at desktop to POSIX file "/path/to/original"'

Tags and Comments

# Get tags (using mdls)
$ mdls -name kMDItemUserTags file.txt
kMDItemUserTags = (
    Red,
    Important
)

# Set tags (using xattr - complex, better to use tools)
# Install 'tag' for easier tag management
$ brew install tag

# Add tag
$ tag -a Red file.txt

# Remove tag
$ tag -r Red file.txt

# List tags
$ tag -l file.txt

# Find files with tag
$ tag -f Red
$ mdfind "kMDItemUserTags == 'Red'"

Finder Comments

# Set Finder comment
$ osascript -e 'tell application "Finder" to set comment of (POSIX file "/path/to/file" as alias) to "My comment"'

# Get Finder comment
$ mdls -name kMDItemFinderComment file.txt

Keychain Access

security Command

Access Keychain from the command line:

# Find a password
$ security find-generic-password -a "account_name" -s "service_name" -w
# -w prints just the password

# Find internet password
$ security find-internet-password -a "username" -s "server.com" -w

# Add a password
$ security add-generic-password -a "account_name" -s "service_name" -w "password123"

# Delete a password
$ security delete-generic-password -a "account_name" -s "service_name"

Practical Keychain Usage

Store and retrieve secrets safely:

# Store API key
$ security add-generic-password -a "$USER" -s "my_api_key" -w "secret123"

# Retrieve in scripts
API_KEY=$(security find-generic-password -a "$USER" -s "my_api_key" -w 2>/dev/null)

# Use in environment (in ~/.zshrc.local - not version controlled)
export GITHUB_TOKEN=$(security find-generic-password -a "$USER" -s "github_token" -w 2>/dev/null)

SSH Keys in Keychain

macOS can store SSH passphrases in Keychain:

# Add SSH key to agent with Keychain storage
$ ssh-add --apple-use-keychain ~/.ssh/id_ed25519

# Configure SSH to use Keychain (~/.ssh/config)
Host *
    AddKeysToAgent yes
    UseKeychain yes
    IdentityFile ~/.ssh/id_ed25519

AppleScript and JavaScript for Automation

osascript Command

Run AppleScript from the shell:

# Basic dialog
$ osascript -e 'display dialog "Hello" buttons {"OK"}'

# Get input
$ osascript -e 'text returned of (display dialog "Enter name:" default answer "")'

# Alert
$ osascript -e 'display alert "Warning" message "Something happened"'

# Choose from list
$ osascript -e 'choose from list {"Option 1", "Option 2", "Option 3"}'

JavaScript for Automation (JXA)

Modern alternative to AppleScript:

# Using JXA
$ osascript -l JavaScript -e 'Application("Finder").selection().map(f => f.url())'

# Multi-line JXA
$ osascript -l JavaScript << 'EOF'
const app = Application.currentApplication();
app.includeStandardAdditions = true;
const result = app.displayDialog("Enter your name:", {
    defaultAnswer: "",
    withTitle: "Input"
});
result.textReturned;
EOF

Controlling Applications

# Control iTunes/Music
$ osascript -e 'tell application "Music" to playpause'
$ osascript -e 'tell application "Music" to next track'
$ osascript -e 'tell application "Music" to get name of current track'

# Control Safari
$ osascript -e 'tell application "Safari" to get URL of current tab of window 1'

# Control System Events
$ osascript -e 'tell application "System Events" to get name of every process'

Automator and Shortcuts

Running Automator Workflows

# Run workflow
$ automator /path/to/workflow.workflow

# Run workflow with input
$ automator -i "input text" /path/to/workflow.workflow

Shortcuts (macOS Monterey+)

# List shortcuts
$ shortcuts list

# Run a shortcut
$ shortcuts run "My Shortcut"

# Run with input
$ echo "input text" | shortcuts run "My Shortcut"

# View shortcut
$ shortcuts view "My Shortcut"

Quick Look

Preview Files

# Quick Look from command line
$ qlmanage -p file.pdf

# Multiple files
$ qlmanage -p *.png

# Generate thumbnail
$ qlmanage -t -s 256 file.pdf -o /tmp/thumbnails/

# Generate preview
$ qlmanage -p -s 1024 file.pdf -o /tmp/previews/

Quick Look Generator Info

# List Quick Look generators
$ qlmanage -m

# Debug Quick Look for a file
$ qlmanage -d4 -p file.pdf

Services Menu Integration

Create shell-accessible services:

Creating a Service

1. Open Automator
2. Create new “Quick Action”
3. Add “Run Shell Script” action
4. Write your script
5. Save with descriptive name

The service appears in:

• Right-click context menu
• Application menu → Services

Running Services from Shell

# Services aren't directly callable, but you can:
# 1. Use Automator to create a workflow
# 2. Run the workflow with automator command
$ automator /path/to/service.workflow

Notification Center

Built-in Notification

# Using osascript
$ osascript -e 'display notification "Message" with title "Title" subtitle "Subtitle" sound name "Glass"'

terminal-notifier

More powerful notification tool:

# Install
$ brew install terminal-notifier

# Basic notification
$ terminal-notifier -title "Build" -message "Complete"

# With actions
$ terminal-notifier -title "Alert" -message "Click me" \
    -open "https://apple.com"

# Execute command on click
$ terminal-notifier -title "Alert" -message "Click to run" \
    -execute "echo 'clicked' >> /tmp/log"

# With buttons (macOS 10.9+)
$ terminal-notifier -title "Question" -message "Choose:" \
    -actions "Yes,No" -closeLabel "Maybe"

# Notify when command completes
$ long_command; terminal-notifier -message "Done" -title "Task Complete"

Calendar and Reminders

Calendar Events

# List calendars
$ osascript -e 'tell application "Calendar" to get name of every calendar'

# Create event
$ osascript << 'EOF'
tell application "Calendar"
    tell calendar "Work"
        make new event with properties {
            summary: "Meeting",
            start date: (current date) + 1 * days,
            end date: (current date) + 1 * days + 1 * hours
        }
    end tell
end tell
EOF

Reminders

# Create reminder
$ osascript -e 'tell application "Reminders" to make new reminder with properties {name: "Buy milk"}'

# Create with due date
$ osascript << 'EOF'
tell application "Reminders"
    tell list "Tasks"
        make new reminder with properties {
            name: "Important task",
            due date: (current date) + 2 * days
        }
    end tell
end tell
EOF

Location Services

Core Location

# Get current location (requires permission)
# Create and compile a Swift snippet or use:
$ osascript -e 'tell application "System Events" to get the current location'
# Note: Often restricted

# Alternative: Use whereami tool
$ brew install whereami
$ whereami
Latitude: 37.7749
Longitude: -122.4194

Integration Scripts

Complete Example: Project Opener

Create a script that integrates multiple services:

#!/bin/zsh
# project-open: Open a project with all necessary tools

project_dir="$1"

if [[ ! -d "$project_dir" ]]; then
    terminal-notifier -title "Error" -message "Directory not found"
    exit 1
fi

cd "$project_dir"

# Open in VS Code
open -a "Visual Studio Code" .

# Open Finder
open .

# Open terminal tab
osascript -e "tell application \"Terminal\"
    do script \"cd '$project_dir' && git status\"
end tell"

# Notify
terminal-notifier -title "Project Opened" \
    -message "$(basename $project_dir)" \
    -open "file://$project_dir"

# Play sound
afplay /System/Library/Sounds/Glass.aiff

Clipboard Workflow

#!/bin/zsh
# Process clipboard contents

# Get clipboard
content=$(pbpaste)

# Transform (example: convert to uppercase)
transformed=$(echo "$content" | tr '[:lower:]' '[:upper:]')

# Put back
echo "$transformed" | pbcopy

# Notify
osascript -e 'display notification "Clipboard transformed" with title "Done"'

File Watcher with Actions

#!/bin/zsh
# Watch directory and act on changes

watch_dir="$HOME/Downloads"

fswatch -0 "$watch_dir" | while read -d "" event; do
    filename=$(basename "$event")

    case "$filename" in
        *.pdf)
            # Move PDFs to Documents
            mv "$event" ~/Documents/PDFs/
            terminal-notifier -message "PDF filed: $filename"
            ;;
        *.zip)
            # Extract ZIPs
            unzip -d "${event%.zip}" "$event"
            terminal-notifier -message "Extracted: $filename"
            ;;
    esac
done

Summary

macOS shell integration enables powerful workflows:

Key integration patterns:

1. Use osascript for GUI application control
2. Use security for credential management
3. Use terminal-notifier for user feedback
4. Combine tools in shell scripts for workflows

The command line on macOS isn’t separate from the graphical system—it’s deeply integrated, enabling automation that bridges both worlds.

Package Management on macOS

Unlike most Unix systems, macOS doesn’t include a native package manager for command-line software. The App Store handles GUI applications, but for developer tools, libraries, and Unix utilities, you need third-party solutions.

This gap has been filled by community projects, with Homebrew emerging as the dominant solution. Understanding how to install, manage, and troubleshoot packages on macOS is essential for any developer or system administrator.

The Package Management Landscape

macOS offers several package management approaches:

Homebrew

The most popular choice. Homebrew provides:

• 6,000+ formulae (packages)
• 5,000+ casks (GUI applications)
• Active community maintenance
• Simple brew install workflow

MacPorts

An older, more Unix-traditional approach:

• Builds everything from source by default
• Extensive package collection
• More isolation from system
• Uses /opt/local prefix

Native Installers

Apple’s standard installation methods:

• .pkg files for command-line tools
• .dmg disk images for applications
• Xcode Command Line Tools
• App Store for sandboxed applications

Language-Specific Managers

Development often requires language-specific tools:

• pip / pyenv for Python
• gem / rbenv / rvm for Ruby
• npm / nvm for Node.js
• cargo for Rust
• go get for Go

Why Package Management Matters

Coming from Linux, you might expect:

$ apt install nginx       # Ubuntu
$ dnf install nginx       # Fedora
$ pacman -S nginx         # Arch

On macOS, there’s no equivalent system command. You need Homebrew:

$ brew install nginx

Understanding macOS package management helps you:

• Install development dependencies reliably
• Keep software updated
• Manage conflicts between versions
• Work across Apple Silicon and Intel Macs
• Troubleshoot installation issues

What You’ll Learn in This Part

Homebrew: Architecture and Internals explains how Homebrew works—formulae, casks, taps, bottles, and the Cellar—giving you the knowledge to troubleshoot effectively.

Homebrew Best Practices covers daily usage patterns, maintenance routines, and avoiding common pitfalls.

MacPorts: An Alternative Approach presents MacPorts for those who prefer its philosophy or need packages not available in Homebrew.

Native Installers: pkg and dmg covers Apple’s native installation formats and when to use them.

Managing Python Environments addresses the complexity of Python on macOS—system Python, Homebrew Python, pyenv, and virtual environments.

Managing Ruby Environments covers Ruby version management with rbenv or rvm, essential for Rails development.

Managing Node.js Environments explains nvm and other tools for managing multiple Node.js versions.

Comparing Package Managers provides a feature comparison to help you choose the right tool for different situations.

Quick Start: Installing Homebrew

For those eager to get started:

# Install Homebrew
$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

# Follow the post-install instructions to add to PATH
# For Apple Silicon, add to ~/.zprofile:
eval "$(/opt/homebrew/bin/brew shellenv)"

# Verify installation
$ brew doctor

# Install something
$ brew install wget

The following chapters explain what’s happening under the hood and how to use these tools effectively.

Homebrew: Architecture and Internals

Homebrew describes itself as “The Missing Package Manager for macOS.” To use it effectively—especially when things go wrong—you need to understand its architecture: how formulae define packages, how the Cellar stores installations, how taps extend the available packages, and how bottles provide pre-compiled binaries.

Installation Locations

Homebrew’s location depends on your Mac’s architecture:

Apple Silicon (M1/M2/M3)

/opt/homebrew/
├── bin/           → Symlinks to installed binaries
├── Cellar/        → Installed packages (versioned)
├── etc/           → Configuration files
├── include/       → Header files
├── lib/           → Libraries
├── opt/           → Symlinks to latest versions
├── sbin/          → System binaries
├── share/         → Architecture-independent data
└── var/           → Variable data (logs, databases)

Intel Macs

/usr/local/
├── bin/
├── Cellar/
├── etc/
...

Why Different Locations?

Apple Silicon Macs use /opt/homebrew to:

• Avoid conflicts with Rosetta 2 (Intel emulation)
• Allow running both native and Intel Homebrew
• Follow Unix convention (/opt for optional software)

Core Concepts

Formulae

A formula is a Ruby script defining how to install a package:

# View a formula
$ brew cat wget

class Wget < Formula
  desc "Internet file retriever"
  homepage "https://www.gnu.org/software/wget/"
  url "https://ftp.gnu.org/gnu/wget/wget-1.21.4.tar.gz"
  sha256 "81542f5cefb8faacc39bbbc6c82ded..."

  depends_on "openssl@3"
  depends_on "pkg-config" => :build

  def install
    system "./configure", "--prefix=#{prefix}",
                          "--with-ssl=openssl",
                          "--with-openssl-dir=#{Formula["openssl@3"].opt_prefix}"
    system "make", "install"
  end

  test do
    system "#{bin}/wget", "-O", "-", "https://example.com"
  end
end

Key elements:

• desc: Package description
• homepage: Project website
• url: Source code download location
• sha256: Checksum for verification
• depends_on: Dependencies
• install: Build and installation commands
• test: Verification test

The Cellar

Installed packages live in the Cellar, organized by name and version:

$ ls /opt/homebrew/Cellar/wget/
1.21.4/

$ ls /opt/homebrew/Cellar/wget/1.21.4/
bin/        etc/        lib/        share/

$ ls /opt/homebrew/Cellar/wget/1.21.4/bin/
wget

This versioned structure allows:

• Multiple versions installed simultaneously
• Easy rollback (brew switch wget 1.21.3)
• Clean uninstallation

Symlinks and opt

Homebrew creates symlinks for easy access:

# Symlink in bin
$ ls -la /opt/homebrew/bin/wget
lrwxr-xr-x  1 user  admin  32 Jan 15 10:00 /opt/homebrew/bin/wget -> ../Cellar/wget/1.21.4/bin/wget

# Stable reference in opt
$ ls -la /opt/homebrew/opt/wget
lrwxr-xr-x  1 user  admin  22 Jan 15 10:00 /opt/homebrew/opt/wget -> ../Cellar/wget/1.21.4

The opt directory provides stable paths regardless of version:

• /opt/homebrew/opt/openssl always points to current OpenSSL
• Useful for configuring build systems

Bottles

Bottles are pre-compiled binary packages:

# Install from bottle (default)
$ brew install wget
==> Downloading https://ghcr.io/v2/homebrew/core/wget/manifests/1.21.4
==> Pouring wget--1.21.4.arm64_sonoma.bottle.tar.gz

Without bottles, Homebrew compiles from source:

# Force compilation from source
$ brew install --build-from-source wget
==> Downloading https://ftp.gnu.org/gnu/wget/wget-1.21.4.tar.gz
==> ./configure --prefix=/opt/homebrew/Cellar/wget/1.21.4 ...
==> make install

Bottles are:

• Pre-compiled for specific macOS versions
• Built for both Intel and Apple Silicon
• Stored on GitHub Container Registry
• Much faster than source compilation

Taps

Taps are third-party formula repositories:

# List tapped repositories
$ brew tap
homebrew/core
homebrew/cask
homebrew/services

# Add a tap
$ brew tap hashicorp/tap

# Now you can install from it
$ brew install hashicorp/tap/terraform

# View tap contents
$ brew tap-info hashicorp/tap

Default taps (pre-installed):

• homebrew/core: Main formula repository
• homebrew/cask: GUI application installers

Tap mechanics:

# Taps clone Git repositories
$ ls ~/Library/Homebrew/Library/Taps/
homebrew/
hashicorp/

# You can create your own tap
$ brew tap-new myname/mytap

Casks

Casks install macOS GUI applications:

# Install application
$ brew install --cask visual-studio-code

# Casks typically:
# 1. Download .dmg or .zip
# 2. Extract .app bundle
# 3. Move to /Applications

Cask definition example:

cask "visual-studio-code" do
  version "1.85.0"
  sha256 "abc123..."

  url "https://update.code.visualstudio.com/#{version}/darwin-arm64/stable"
  name "Visual Studio Code"
  homepage "https://code.visualstudio.com/"

  app "Visual Studio Code.app"

  zap trash: [
    "~/Library/Application Support/Code",
    "~/.vscode",
  ]
end

Package Lifecycle

Installation

$ brew install python@3.11

# What happens:
# 1. Resolve dependencies
# 2. Download bottle (or source)
# 3. Verify checksum
# 4. Extract to Cellar
# 5. Create symlinks
# 6. Run post-install script

Upgrading

$ brew upgrade python@3.11

# What happens:
# 1. Download new version
# 2. Install to new Cellar directory
# 3. Update symlinks
# 4. Keep old version (until cleanup)

Uninstallation

$ brew uninstall python@3.11

# What happens:
# 1. Remove symlinks
# 2. Remove Cellar directory
# 3. Optional: remove unused dependencies

Dependencies

Viewing Dependencies

# Direct dependencies
$ brew deps wget
gettext
libidn2
openssl@3

# Full dependency tree
$ brew deps --tree wget
wget
├── gettext
├── libidn2
│   ├── gettext
│   └── libunistring
└── openssl@3
    └── ca-certificates

# Why is something installed?
$ brew uses --installed openssl@3
curl
python@3.11
wget

Dependency Types

• Required: Must be installed
• Build: Only needed during compilation
• Optional: Enables additional features
• Recommended: Suggested but not required

# In formula
depends_on "cmake" => :build          # Build-time only
depends_on "openssl@3"                # Runtime required
depends_on "readline" => :optional    # Optional feature

Services

Homebrew can manage background services:

# Start a service
$ brew services start postgresql@14

# Stop a service
$ brew services stop postgresql@14

# Restart
$ brew services restart postgresql@14

# List services
$ brew services list
Name          Status  User File
postgresql@14 started user ~/Library/LaunchAgents/homebrew.mxcl.postgresql@14.plist

# Service files are launchd plists
$ cat ~/Library/LaunchAgents/homebrew.mxcl.postgresql@14.plist

Important Files and Directories

# Homebrew itself
/opt/homebrew/                          # Prefix (Apple Silicon)
/opt/homebrew/bin/brew                  # Main executable
/opt/homebrew/Homebrew/                 # Homebrew Git repository

# Installed packages
/opt/homebrew/Cellar/                   # All installations
/opt/homebrew/opt/                      # Stable symlinks

# Configuration
/opt/homebrew/etc/                      # Config files

# Cache
~/Library/Caches/Homebrew/              # Downloaded files
~/Library/Caches/Homebrew/downloads/    # Bottles and sources

# Logs
~/Library/Logs/Homebrew/                # Build logs

# Taps
$(brew --repository)/Library/Taps/      # Tap repositories

Under the Hood

Git-Based Design

Homebrew itself is a Git repository:

# View Homebrew version
$ brew --version
Homebrew 4.2.0

# Update Homebrew (git pull)
$ brew update
==> Updating Homebrew...

# View homebrew commits
$ cd $(brew --repository) && git log --oneline -5

Formula repositories are also Git-based:

# View formula history
$ brew log wget

Keg-Only Packages

Some packages are “keg-only”—installed but not symlinked:

$ brew info openssl@3
==> openssl@3: stable 3.2.0 (bottled)
...
openssl@3 is keg-only, which means it was not symlinked into /opt/homebrew,
because macOS provides LibreSSL.

# Access keg-only packages
$ /opt/homebrew/opt/openssl@3/bin/openssl version

# Or add to PATH
export PATH="/opt/homebrew/opt/openssl@3/bin:$PATH"

Keg-only reasons:

• Conflicts with macOS system software
• Multiple versions available
• Not meant for direct use

Linking and Unlinking

# Unlink (remove symlinks but keep installed)
$ brew unlink python@3.11

# Link (create symlinks)
$ brew link python@3.11

# Force link keg-only formula (use carefully)
$ brew link --force openssl@3

Diagnostic Commands

# Check Homebrew health
$ brew doctor
Your system is ready to brew.

# Show configuration
$ brew config
HOMEBREW_VERSION: 4.2.0
ORIGIN: https://github.com/Homebrew/brew
HEAD: abc1234
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
...

# Debug information
$ brew --env
HOMEBREW_CC: clang
HOMEBREW_CXX: clang++
...

# List all installed packages
$ brew list

# List with versions
$ brew list --versions

# Show what would be upgraded
$ brew outdated

Summary

Homebrew architecture key concepts:

Understanding this architecture helps you:

• Troubleshoot installation issues
• Manage multiple versions
• Create your own formulae
• Work with keg-only packages
• Understand why things are where they are

Homebrew Best Practices

Homebrew is straightforward to use but benefits from disciplined practices. This chapter covers daily usage patterns, maintenance routines, managing upgrades, and avoiding common pitfalls.

Daily Usage

Installing Packages

# Search for packages
$ brew search postgresql
==> Formulae
postgresql@11    postgresql@14    postgresql@15    postgresql@16

# Get information before installing
$ brew info postgresql@16
==> postgresql@16: stable 16.1 (bottled)
Object-relational database system
...

# Install
$ brew install postgresql@16

# Install specific version
$ brew install postgresql@14

# Install from source (if needed)
$ brew install --build-from-source package

Installing GUI Applications

# Search casks
$ brew search --cask firefox

# Install cask
$ brew install --cask firefox

# Casks go to /Applications by default
# Verify installation
$ ls /Applications/Firefox.app

Updating and Upgrading

# Update Homebrew itself and formula definitions
$ brew update

# See what's outdated
$ brew outdated

# Upgrade all packages
$ brew upgrade

# Upgrade specific package
$ brew upgrade wget

# Upgrade casks
$ brew upgrade --cask

# Pin a package (prevent upgrades)
$ brew pin postgresql@16
$ brew list --pinned
$ brew unpin postgresql@16

Uninstalling

# Uninstall a package
$ brew uninstall wget

# Uninstall with dependencies check
$ brew uninstall --zap firefox  # For casks: removes preferences too

# Remove unused dependencies
$ brew autoremove

Maintenance Routine

Regular Maintenance Commands

Run these periodically (weekly or monthly):

# Update everything
$ brew update && brew upgrade

# Clean up old versions
$ brew cleanup

# Check for problems
$ brew doctor

# Remove unused dependencies
$ brew autoremove

Cleanup Details

# Preview what cleanup would remove
$ brew cleanup --dry-run

# Clean up older than default (120 days)
$ brew cleanup --prune=30  # 30 days

# Clean specific package
$ brew cleanup wget

# See disk usage before/after
$ du -sh $(brew --cache)
1.2G    /Users/david/Library/Caches/Homebrew
$ brew cleanup
$ du -sh $(brew --cache)
256M    /Users/david/Library/Caches/Homebrew

Dealing with Doctor Warnings

$ brew doctor
Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: Some installed formulae are deprecated or disabled.
...

Common warnings and fixes:

Unbrewed files in Homebrew directories:

Warning: Unbrewed dylibs were found in /opt/homebrew/lib
# Usually safe to ignore, or remove manually if you know they're orphaned

Outdated Xcode Command Line Tools:

Warning: A newer Command Line Tools release is available.
$ softwareupdate --list
$ softwareupdate -i "Command Line Tools for Xcode-15.0"
# Or just update Xcode from App Store

Broken symlinks:

Warning: Broken symlinks were found
$ brew doctor --list-checks | xargs brew
# Or manually remove listed broken symlinks

Managing Multiple Versions

Version-Specific Packages

# Install specific versions
$ brew install python@3.11
$ brew install python@3.12

# Both are installed but one is linked
$ brew list | grep python
python@3.11
python@3.12

# Switch versions
$ brew unlink python@3.11
$ brew link python@3.12

# Check which is active
$ python3 --version

Using Unlinked Versions

Access unlinked versions directly:

# Direct path
$ /opt/homebrew/opt/python@3.11/bin/python3 --version
Python 3.11.7

# Or add to PATH for session
$ export PATH="/opt/homebrew/opt/python@3.11/bin:$PATH"

Dealing with Dependencies

Understanding Dependency Problems

# See why a package is installed
$ brew uses --installed openssl@3
curl
git
python@3.11
wget

# See what a package needs
$ brew deps wget
gettext
libidn2
openssl@3

# Full dependency tree
$ brew deps --tree --installed

Dependency Conflicts

When upgrades break dependencies:

# Reinstall package and dependencies
$ brew reinstall wget

# Rebuild all dependents of a package
$ brew reinstall $(brew uses --installed openssl@3)

Services Management

Using brew services

# List all services
$ brew services list

# Start service (runs at login)
$ brew services start postgresql@16

# Run once (doesn't persist across reboot)
$ brew services run postgresql@16

# Stop service
$ brew services stop postgresql@16

# Restart service
$ brew services restart postgresql@16

# View service status
$ brew services info postgresql@16

Service Files

# Location of service plists
$ ls ~/Library/LaunchAgents/homebrew.*
homebrew.mxcl.postgresql@16.plist

# View service configuration
$ cat ~/Library/LaunchAgents/homebrew.mxcl.postgresql@16.plist

# Manual service management
$ launchctl list | grep homebrew

Bundler: Reproducible Environments

Creating a Brewfile

# Generate Brewfile from current installations
$ brew bundle dump

# Creates Brewfile:
$ cat Brewfile
tap "homebrew/bundle"
tap "homebrew/cask"
tap "homebrew/core"
brew "git"
brew "node"
brew "python@3.11"
cask "visual-studio-code"
cask "docker"

Installing from Brewfile

# Install everything in Brewfile
$ brew bundle

# Check what would be installed
$ brew bundle check

# List what's in Brewfile but not installed
$ brew bundle list

Brewfile Best Practices

# Brewfile with options and documentation

# Taps
tap "homebrew/bundle"
tap "homebrew/cask-fonts"

# Development tools
brew "git"
brew "gh"  # GitHub CLI
brew "neovim"

# Languages
brew "python@3.11"
brew "node"
brew "go"

# Databases
brew "postgresql@16", restart_service: true
brew "redis"

# Applications
cask "visual-studio-code"
cask "docker"
cask "iterm2"

# Fonts
cask "font-fira-code"

# Mac App Store apps (requires mas)
# mas "Xcode", id: 497799835

Troubleshooting

Common Issues

“Error: No available formula with the name…”

$ brew install nonexistent
Error: No available formula with the name "nonexistent"

# Solution: Update and search
$ brew update
$ brew search partial-name

“Error: Cannot install in Homebrew on ARM…”

# This happens when mixing architectures
# Ensure you're using the right Homebrew
$ which brew
/opt/homebrew/bin/brew  # Should be this on Apple Silicon

Permission errors:

$ brew install something
Error: Permission denied @ rb_sysopen

# Fix permissions
$ sudo chown -R $(whoami) $(brew --prefix)/*

Bottle not available:

# Install from source when bottle unavailable
$ brew install --build-from-source package

Resetting Homebrew

When things are really broken:

# Nuclear option: Uninstall Homebrew
$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/uninstall.sh)"

# Then reinstall
$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Less nuclear:

# Reset to clean state
$ brew update-reset

# Fix all symlinks
$ brew link --overwrite --dry-run $(brew list --formula)
$ brew link --overwrite $(brew list --formula)

Security Considerations

Verifying Packages

# Homebrew verifies checksums automatically
# View checksum
$ brew info --json wget | jq '.[0].versions.bottle_sha256'

# Audit a formula
$ brew audit wget

Avoiding Untrusted Taps

# List your taps
$ brew tap

# Only use trusted taps
# Official: homebrew/core, homebrew/cask
# Verify third-party taps before adding

Cask Security

# Some casks require passwords - be cautious
# Check what a cask does before installing
$ brew cat --cask suspicious-app

# Review artifacts and scripts

Performance Tips

Faster Updates

# Use parallel downloads
$ export HOMEBREW_PARALLEL_FETCH=4

# Skip analytics
$ brew analytics off

Reducing Disk Usage

# Regular cleanup
$ brew cleanup --prune=all

# Check what's using space
$ brew list --formula | xargs -I{} sh -c 'echo -n "{}: " && du -sh $(brew --cellar)/{} | cut -f1'

# Remove old versions aggressively
$ brew cleanup --prune=0

Cache Management

# View cache
$ ls ~/Library/Caches/Homebrew/

# Clear cache
$ rm -rf ~/Library/Caches/Homebrew/*

# Or selectively
$ brew cleanup --prune=all

CI/CD Usage

GitHub Actions Example

- name: Install Homebrew dependencies
  run: |
    brew update
    brew install cmake ninja

# Or with Brewfile
- name: Install from Brewfile
  run: brew bundle

Caching in CI

- name: Cache Homebrew
  uses: actions/cache@v3
  with:
    path: |
      ~/Library/Caches/Homebrew
      /opt/homebrew/Cellar
    key: ${{ runner.os }}-brew-${{ hashFiles('Brewfile.lock.json') }}

Summary

Best practices checklist:

Golden rules:

1. Update before installing — brew update first
2. Use bottles when possible — Much faster than source
3. Run cleanup regularly — Saves disk space
4. Use Brewfile for teams — Reproducible environments
5. Don’t sudo brew — Never run Homebrew with sudo
6. Check doctor warnings — Not all need fixing, but awareness helps

MacPorts: An Alternative Approach

MacPorts predates Homebrew and takes a different philosophical approach to package management on macOS. While Homebrew focuses on ease of use and leveraging macOS system libraries, MacPorts prioritizes isolation, reproducibility, and Unix tradition. Understanding MacPorts helps you make informed choices about which tool to use—and some software is only available in MacPorts.

MacPorts Philosophy

MacPorts follows principles from the BSD ports tradition:

1. Self-contained: Installs entirely under /opt/local, separate from system
2. Build from source: Compiles everything by default
3. Explicit variants: Fine-grained control over build options
4. Full dependency trees: Doesn’t rely on macOS system libraries

This approach offers predictability and control at the cost of longer installation times.

Installation

Prerequisites

MacPorts requires Xcode or Command Line Tools:

$ xcode-select --install

Installing MacPorts

Download the installer from macports.org for your macOS version, or install manually:

# Download (check macports.org for current version)
$ curl -O https://distfiles.macports.org/MacPorts/MacPorts-2.8.1.tar.bz2
$ tar xf MacPorts-2.8.1.tar.bz2
$ cd MacPorts-2.8.1

# Build and install
$ ./configure
$ make
$ sudo make install

Post-Installation

Add to your PATH in ~/.zprofile:

export PATH="/opt/local/bin:/opt/local/sbin:$PATH"
export MANPATH="/opt/local/share/man:$MANPATH"

Verify:

$ source ~/.zprofile
$ port version
Version: 2.8.1

Basic Usage

Searching for Ports

# Search by name
$ port search postgresql
postgresql15 @15.4 (databases)
    PostgreSQL is a highly-extensible, object-relational database...

# Search descriptions too
$ port search --description "web server"

# List all ports
$ port list

Getting Information

$ port info nginx
nginx @1.24.0 (www)
Variants:             debug, gzip_static, http2, mail, realip...

Description:          Nginx is a high performance HTTP server
Homepage:             https://nginx.org/

Build Dependencies:   pcre
Library Dependencies: openssl, pcre, zlib
Platforms:            darwin
License:              BSD
Maintainers:          Email: ...

Installing Ports

# Install (requires sudo)
$ sudo port install nginx

# Install with variants
$ sudo port install nginx +http2 +realip

# Install specific version
$ sudo port install postgresql15 @15.4

# Dry run (see what would happen)
$ port -y install nginx

Updating

# Update port definitions
$ sudo port selfupdate

# Upgrade all outdated ports
$ sudo port upgrade outdated

# Upgrade specific port
$ sudo port upgrade nginx

# See what's outdated
$ port outdated

Uninstalling

# Uninstall a port
$ sudo port uninstall nginx

# Uninstall with dependents
$ sudo port uninstall --follow-dependents nginx

# Remove inactive ports (old versions)
$ sudo port uninstall inactive

Variants

Variants are build options that customize port installation:

# List variants for a port
$ port variants vim
vim has the variants:
   athena: Use Athena widgets for GUI
   big: Build big features
   cscope: Enable cscope interface
   gtk2: Enable GTK2 GUI
   gtk3: Enable GTK3 GUI
   huge: Build huge features
   ...

# Install with variant
$ sudo port install vim +huge +python311

# Install without default variant
$ sudo port install vim -x11

# View installed variants
$ port installed vim
  vim @9.0.1000_0+huge+python311 (active)

Default Variants

Configure default variants in /opt/local/etc/macports/variants.conf:

# Always build with these variants
+no_x11
+quartz

# Never build with these
-debug

Port Lifecycle

Version Management

# List installed versions
$ port installed
  nginx @1.24.0_0 (active)
  nginx @1.22.0_0

# Activate specific version
$ sudo port activate nginx @1.22.0_0

# Deactivate (without uninstalling)
$ sudo port deactivate nginx

Cleaning

# Clean build artifacts for a port
$ sudo port clean nginx

# Clean all ports
$ sudo port clean --all all

# Clean work directories, distribution files, and archives
$ sudo port clean --all --dist nginx

File Locations

/opt/local/
├── bin/              # Binaries
├── etc/              # Configuration files
├── include/          # Header files
├── lib/              # Libraries
├── libexec/          # Support programs
├── sbin/             # System binaries
├── share/            # Architecture-independent files
├── var/              # Variable data (logs, databases)
│   └── macports/     # MacPorts-specific
│       ├── build/    # Build directories
│       └── sources/  # Downloaded sources
└── Library/
    └── Frameworks/   # macOS frameworks

Maintenance

Regular Maintenance

# Full update cycle
$ sudo port selfupdate
$ sudo port upgrade outdated
$ sudo port uninstall inactive
$ sudo port clean --all all

Reclaiming Disk Space

# Remove inactive ports
$ sudo port uninstall inactive

# Remove unrequested ports (installed as dependencies but no longer needed)
$ sudo port uninstall leaves

# Clean all working directories and sources
$ sudo port clean --all installed
$ sudo rm -rf /opt/local/var/macports/distfiles/*

Checking Installation

# Verify installed ports
$ port -v installed

# Check for broken ports
$ port diagnose

# List dependencies
$ port deps nginx
$ port rdeps nginx  # Recursive
$ port dependents nginx  # What depends on this

MacPorts vs Homebrew

Key Differences

When to Use MacPorts

• Need specific build options: Variants offer fine-grained control
• Isolation is important: Complete separation from system
• Package only in MacPorts: Some software not in Homebrew
• Reproducible builds: Same configuration everywhere
• Server environments: Predictable, isolated installations

When to Use Homebrew

• Quick installation: Bottles are fast
• macOS integration: Works with system libraries
• Casual use: Simpler command syntax
• GUI apps: Casks are convenient
• Most common packages: Available and well-maintained

Coexistence

You can run both MacPorts and Homebrew:

# Different paths, different prefixes
/opt/local/bin/python3          # MacPorts
/opt/homebrew/bin/python3        # Homebrew

# Adjust PATH to prefer one
export PATH="/opt/homebrew/bin:$PATH"       # Prefer Homebrew
# or
export PATH="/opt/local/bin:$PATH"          # Prefer MacPorts

Caveats:

• Don’t mix libraries from both in builds
• Be explicit about which version you’re using
• Can cause confusion with shared dependencies

Creating Custom Ports

Portfile Basics

# /opt/local/var/macports/sources/.../myport/Portfile

PortSystem          1.0

name                myapp
version             1.0.0
categories          devel
platforms           darwin
maintainers         {example.com:admin}
description         My application
long_description    ${description} with more details

homepage            https://example.com/myapp
master_sites        https://example.com/downloads/

checksums           rmd160  abc123... \
                    sha256  def456...

depends_lib         port:openssl

configure.args      --with-ssl=${prefix}

post-destroot {
    xinstall -d ${destroot}${prefix}/share/doc/${name}
    xinstall -m 644 ${worksrcpath}/README ${destroot}${prefix}/share/doc/${name}
}

Local Portfile Repository

# Create local repository
$ mkdir -p ~/ports/mycat/myport
$ cp Portfile ~/ports/mycat/myport/

# Add to sources.conf
$ sudo nano /opt/local/etc/macports/sources.conf
# Add line:
file:///Users/username/ports

# Update index
$ cd ~/ports
$ portindex

# Now install
$ sudo port install myport

Troubleshooting

Common Issues

Port fails to build:

# View build log
$ port logfile nginx
$ less /opt/local/var/macports/logs/_opt_local_var_macports_sources_.../nginx/main.log

# Try clean build
$ sudo port clean nginx
$ sudo port install nginx

Dependency conflicts:

# See dependency graph
$ port deps nginx

# Force rebuild of dependencies
$ sudo port -n upgrade --force nginx

Stuck port:

# Force uninstall
$ sudo port -f uninstall nginx

# Clean state
$ sudo port clean nginx
$ sudo rm -rf /opt/local/var/macports/build/*nginx*

Recovering from Problems

# Selfupdate when things are broken
$ sudo port -d selfupdate

# Restore ports from backup
$ sudo port -f restore /path/to/backup.tar.gz

# Migration after macOS upgrade
$ sudo port migrate

Summary

MacPorts provides:

• Self-contained installation in /opt/local
• Build-from-source with variants for customization
• Large package collection
• Traditional Unix approach

Best for users who:

• Need specific build configurations
• Prefer isolation from system
• Have time for source compilation
• Need packages not in Homebrew

Commands reference:

Native Installers: pkg and dmg

macOS has its own installation formats that predate third-party package managers. Understanding .pkg installers and .dmg disk images helps you manage software that doesn’t come through Homebrew or MacPorts, including Xcode Command Line Tools and many commercial applications.

Package Types Overview

.pkg Installers

What pkg Files Contain

A .pkg file is an archive containing:

• Payload (files to install)
• Scripts (pre/post install)
• Bill of Materials (file manifest)
• Package info

# Examine pkg contents
$ pkgutil --payload-files /path/to/package.pkg
$ pkgutil --bom /path/to/package.pkg

Installing pkg Files

# GUI installation
$ open package.pkg

# Command-line installation (requires sudo)
$ sudo installer -pkg package.pkg -target /
installer: Package name is My Package
installer: Installing at base path /
installer: The install was successful.

# Install to specific volume
$ sudo installer -pkg package.pkg -target /Volumes/OtherDisk

# Verbose installation
$ sudo installer -pkg package.pkg -target / -verbose

Managing Installed Packages

# List all installed packages
$ pkgutil --pkgs
com.apple.pkg.CLTools_Executables
com.apple.pkg.CLTools_SDK_macOS14
...

# Filter by pattern
$ pkgutil --pkgs | grep -i xcode

# Package information
$ pkgutil --pkg-info com.apple.pkg.CLTools_Executables
package-id: com.apple.pkg.CLTools_Executables
version: 15.0.0.0.1.1694021235
volume: /
location: /
install-time: 1694000000

# Files installed by a package
$ pkgutil --files com.apple.pkg.CLTools_Executables | head
Library/Developer/CommandLineTools/
Library/Developer/CommandLineTools/SDKs/
Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
...

# Find which package owns a file
$ pkgutil --file-info /usr/bin/git
volume: /
path: /usr/bin/git
pkgid: com.apple.pkg.CLTools_Executables
...

Removing Packages

macOS doesn’t have a built-in package uninstaller. Removal requires:

# 1. Find installed files
$ pkgutil --files com.example.package > /tmp/files.txt

# 2. Review the file list
$ cat /tmp/files.txt

# 3. Remove files (carefully!)
$ cd /
$ sudo rm -rf $(pkgutil --files com.example.package)

# 4. Forget the package receipt
$ sudo pkgutil --forget com.example.package

Warning: Be very careful with rm -rf. Always review file lists first.

Xcode Command Line Tools

The most common .pkg on developer Macs:

# Check if installed
$ xcode-select -p
/Library/Developer/CommandLineTools

# Install
$ xcode-select --install
# Opens GUI prompt

# Or via softwareupdate
$ softwareupdate --list | grep "Command Line Tools"
$ softwareupdate -i "Command Line Tools for Xcode-15.0"

# Switch between Xcode and CLI tools
$ sudo xcode-select --switch /Applications/Xcode.app
$ sudo xcode-select --switch /Library/Developer/CommandLineTools

# Reset
$ sudo xcode-select --reset

Creating pkg Files

For distributing your own software:

# Create package from directory
$ pkgbuild --root /path/to/files \
           --identifier com.example.myapp \
           --version 1.0.0 \
           --install-location /Applications \
           myapp.pkg

# With scripts
$ pkgbuild --root /path/to/files \
           --identifier com.example.myapp \
           --version 1.0.0 \
           --scripts /path/to/scripts \
           --install-location /Applications \
           myapp.pkg

# Create distribution (combines multiple packages)
$ productbuild --distribution distribution.xml \
               --package-path /path/to/packages \
               final-installer.pkg

.dmg Disk Images

Working with DMG Files

# Mount disk image
$ hdiutil attach application.dmg
/dev/disk4          GUID_partition_scheme
/dev/disk4s1        Apple_APFS
/dev/disk5s1        Apple_APFS                   /Volumes/Application

# List mounted images
$ hdiutil info | grep image-path
image-path      : /path/to/application.dmg

# Unmount
$ hdiutil detach /Volumes/Application
# or
$ hdiutil detach /dev/disk5

Installing from DMG

Typical DMG workflow:

# Mount
$ hdiutil attach MyApp.dmg

# Copy app to Applications
$ cp -R /Volumes/MyApp/MyApp.app /Applications/

# Unmount
$ hdiutil detach /Volumes/MyApp

# Optional: delete dmg
$ rm MyApp.dmg

DMG Scripted Installation

#!/bin/bash
# install-from-dmg.sh

DMG_PATH="$1"
APP_NAME="$2"

# Mount
MOUNT_POINT=$(hdiutil attach "$DMG_PATH" -nobrowse | grep "/Volumes" | tail -1 | cut -f3)

# Copy
cp -R "$MOUNT_POINT"/*.app /Applications/ 2>/dev/null || \
cp -R "$MOUNT_POINT/$APP_NAME" /Applications/

# Unmount
hdiutil detach "$MOUNT_POINT"

echo "Installed to /Applications"

Creating DMG Files

# Create DMG from folder
$ hdiutil create -srcfolder /path/to/MyApp.app \
                 -volname "MyApp" \
                 -format UDZO \
                 MyApp.dmg

# Create empty DMG
$ hdiutil create -size 100m \
                 -fs HFS+ \
                 -volname "MyApp" \
                 MyApp.dmg

# Create read-write DMG (for customization)
$ hdiutil create -srcfolder /path/to/MyApp.app \
                 -volname "MyApp" \
                 -format UDRW \
                 MyApp-rw.dmg

# Convert to compressed read-only
$ hdiutil convert MyApp-rw.dmg -format UDZO -o MyApp.dmg

Internet-Enabled DMG

DMGs can auto-extract:

# Make DMG internet-enabled
$ hdiutil internet-enable -yes MyApp.dmg

When downloaded via Safari, the app is extracted and DMG deleted automatically.

Application Bundles (.app)

Bundle Structure

$ ls -la /Applications/Safari.app/Contents/
total 24
drwxr-xr-x   7 root  wheel   224 Dec 11 11:23 .
drwxr-xr-x   3 root  wheel    96 Dec 11 11:23 ..
drwxr-xr-x   3 root  wheel    96 Dec 11 11:23 _CodeSignature
-rw-r--r--   1 root  wheel  8572 Dec 11 11:23 Info.plist
drwxr-xr-x   3 root  wheel    96 Dec 11 11:23 MacOS
-rw-r--r--   1 root  wheel     8 Dec 11 11:23 PkgInfo
drwxr-xr-x  73 root  wheel  2336 Dec 11 11:23 Resources
drwxr-xr-x   5 root  wheel   160 Dec 11 11:23 XPCServices

# Key files
/Contents/Info.plist         # Application metadata
/Contents/MacOS/             # Executable
/Contents/Resources/         # App resources
/Contents/_CodeSignature/    # Code signature

App Metadata

# View Info.plist
$ plutil -p /Applications/Safari.app/Contents/Info.plist | head -20
{
  "BuildMachineOSBuild" => "23A344"
  "CFBundleDevelopmentRegion" => "English"
  "CFBundleExecutable" => "Safari"
  "CFBundleIdentifier" => "com.apple.Safari"
  "CFBundleInfoDictionaryVersion" => "6.0"
  "CFBundleName" => "Safari"
  "CFBundlePackageType" => "APPL"
  ...
}

# Get specific values
$ defaults read /Applications/Safari.app/Contents/Info.plist CFBundleVersion
17617.1.17.11.9

Installing Applications

# Copy to Applications
$ cp -R /path/to/MyApp.app /Applications/

# Or per-user
$ cp -R /path/to/MyApp.app ~/Applications/

# Remove quarantine (for trusted apps)
$ xattr -d com.apple.quarantine /Applications/MyApp.app

Uninstalling Applications

# Remove app bundle
$ rm -rf /Applications/MyApp.app

# Also remove:
# - ~/Library/Application Support/MyApp
# - ~/Library/Preferences/com.example.myapp.plist
# - ~/Library/Caches/com.example.myapp

# Find related files
$ mdfind "kMDItemCFBundleIdentifier == 'com.example.myapp'"
$ mdfind -name "myapp"

Software Update

Command-Line Software Updates

# List available updates
$ softwareupdate --list
Software Update found the following new or updated software:
* Label: macOS Sonoma 14.2.1-23C71
...

# Install all updates
$ sudo softwareupdate --install --all

# Install specific update
$ sudo softwareupdate --install "macOS Sonoma 14.2.1-23C71"

# Download only (don't install)
$ sudo softwareupdate --download --all

# Install and restart if needed
$ sudo softwareupdate --install --all --restart

Managing Update Behavior

# Check automatic update settings
$ defaults read /Library/Preferences/com.apple.SoftwareUpdate

# Enable automatic updates
$ sudo defaults write /Library/Preferences/com.apple.SoftwareUpdate AutomaticCheckEnabled -bool true
$ sudo defaults write /Library/Preferences/com.apple.SoftwareUpdate AutomaticDownload -bool true

# Disable automatic updates
$ sudo defaults write /Library/Preferences/com.apple.SoftwareUpdate AutomaticCheckEnabled -bool false

App Store from Command Line

mas (Mac App Store CLI)

# Install mas
$ brew install mas

# Sign in (may require App Store sign-in first)
$ mas signin email@example.com

# Search
$ mas search Xcode
497799835  Xcode (15.0)

# Install by ID
$ mas install 497799835

# List installed
$ mas list

# Outdated apps
$ mas outdated

# Upgrade all
$ mas upgrade

Summary

Native macOS installation methods:

Key commands:

Native installers complement package managers for software that requires system-level installation or comes directly from vendors.

Managing Python Environments

Python on macOS is notoriously confusing. There’s the system Python (don’t use it), Homebrew Python (convenient), pyenv (version management), and the official python.org installers. Add virtual environments and you have a recipe for confusion. This chapter untangles the mess.

Python Versions on macOS

System Python

macOS historically included Python, but:

• macOS 12.3+: No Python pre-installed (finally!)
• Older macOS: Python 2.7 at /usr/bin/python (deprecated)
• Never modify system Python: It’s for macOS internals

# Check if system Python exists
$ /usr/bin/python --version
zsh: /usr/bin/python: bad interpreter: No such file or directory

# Modern macOS has no /usr/bin/python
# But may have /usr/bin/python3 from Xcode CLT
$ /usr/bin/python3 --version
Python 3.9.6

Which Python Should You Use?

Homebrew Python

Installation

# Install latest Python
$ brew install python

# Or specific version
$ brew install python@3.11
$ brew install python@3.12

# Check installation
$ brew info python

Location and Linking

# Where is it?
$ which python3
/opt/homebrew/bin/python3

$ ls -la /opt/homebrew/bin/python3*
lrwxr-xr-x  1 user  admin  42 Jan 15 10:00 /opt/homebrew/bin/python3 -> ../Cellar/python@3.12/3.12.0/bin/python3
lrwxr-xr-x  1 user  admin  49 Jan 15 10:00 /opt/homebrew/bin/python3.12 -> ../Cellar/python@3.12/3.12.0/bin/python3.12

# Actual location
$ /opt/homebrew/opt/python@3.12/bin/python3 --version

Managing Multiple Homebrew Pythons

# Install multiple versions
$ brew install python@3.11 python@3.12

# By default, one is linked
$ python3 --version
Python 3.12.0

# Use specific version directly
$ python3.11 --version
$ python3.12 --version

# Or switch linked version
$ brew unlink python@3.12
$ brew link python@3.11

Homebrew pip

# pip is version-specific
$ pip3 --version
pip 23.3.1 from /opt/homebrew/lib/python3.12/site-packages/pip (python 3.12)

# Install packages
$ pip3 install requests

# Where packages go
$ python3 -c "import site; print(site.getsitepackages())"
['/opt/homebrew/lib/python3.12/site-packages']

pyenv: Version Management

pyenv lets you install and switch between Python versions easily.

Installing pyenv

# Install with Homebrew
$ brew install pyenv pyenv-virtualenv

# Or from source
$ curl https://pyenv.run | bash

Shell Configuration

Add to ~/.zprofile and ~/.zshrc:

# ~/.zprofile
export PYENV_ROOT="$HOME/.pyenv"
[[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init --path)"

# ~/.zshrc
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"

Restart your shell or source ~/.zshrc.

Using pyenv

# List available versions
$ pyenv install --list | grep "^  3\."
  3.10.13
  3.11.7
  3.12.0
  ...

# Install a version
$ pyenv install 3.11.7
Downloading Python-3.11.7.tar.xz...
Installing Python-3.11.7...
Installed Python-3.11.7 to /Users/david/.pyenv/versions/3.11.7

# List installed versions
$ pyenv versions
  system
  3.11.7
  3.12.0
* 3.12.0 (set by /Users/david/.pyenv/version)

# Set global default
$ pyenv global 3.11.7

# Set local version (per directory)
$ cd myproject
$ pyenv local 3.11.7
# Creates .python-version file

# Set for current shell only
$ pyenv shell 3.10.13

pyenv Build Dependencies

pyenv compiles Python from source. Install dependencies first:

$ brew install openssl readline sqlite3 xz zlib tcl-tk

# Set build flags (in ~/.zshrc)
export LDFLAGS="-L$(brew --prefix openssl@3)/lib -L$(brew --prefix readline)/lib -L$(brew --prefix sqlite3)/lib -L$(brew --prefix zlib)/lib"
export CPPFLAGS="-I$(brew --prefix openssl@3)/include -I$(brew --prefix readline)/include -I$(brew --prefix sqlite3)/include -I$(brew --prefix zlib)/include"

Virtual Environments

Virtual environments isolate project dependencies.

Built-in venv

# Create virtual environment
$ python3 -m venv myenv

# Activate
$ source myenv/bin/activate
(myenv) $

# Now pip installs to this env
(myenv) $ pip install requests
(myenv) $ which python
/path/to/myenv/bin/python

# Deactivate
(myenv) $ deactivate
$

pyenv-virtualenv

Combined version and environment management:

# Create virtualenv with specific Python
$ pyenv virtualenv 3.11.7 myproject-env

# List virtualenvs
$ pyenv virtualenvs

# Activate
$ pyenv activate myproject-env
(myproject-env) $

# Or set local (auto-activates in directory)
$ pyenv local myproject-env

# Deactivate
$ pyenv deactivate

# Delete virtualenv
$ pyenv virtualenv-delete myproject-env

Poetry

Modern dependency management:

# Install Poetry
$ curl -sSL https://install.python-poetry.org | python3 -
# Or
$ brew install poetry

# New project
$ poetry new myproject
$ cd myproject

# Add dependencies
$ poetry add requests

# Install from existing pyproject.toml
$ poetry install

# Run in environment
$ poetry run python script.py

# Activate shell
$ poetry shell

pipenv

Alternative to Poetry:

# Install
$ brew install pipenv

# Create environment and install packages
$ pipenv install requests

# Activate
$ pipenv shell

# Run command
$ pipenv run python script.py

Recommended Setup

For Most Developers

1. Install pyenv:

$ brew install pyenv pyenv-virtualenv
# Add shell configuration

2. Install Python versions:

$ pyenv install 3.11.7
$ pyenv install 3.12.0
$ pyenv global 3.11.7

3. Use virtual environments per project:

$ cd myproject
$ pyenv virtualenv 3.11.7 myproject
$ pyenv local myproject

For Data Science

Consider Conda/Miniconda for scientific packages:

# Install Miniconda
$ brew install --cask miniconda

# Initialize
$ conda init zsh

# Create environment
$ conda create -n datascience python=3.11 numpy pandas scikit-learn

# Activate
$ conda activate datascience

Troubleshooting

“python: command not found”

# Check what's available
$ which python3
$ pyenv which python

# Create alias if needed (in ~/.zshrc)
alias python=python3
alias pip=pip3

Wrong Python Version

# Check which Python
$ which python3
$ python3 --version

# Check pyenv
$ pyenv version
$ pyenv which python

# Common issue: Homebrew Python overriding pyenv
# Ensure pyenv is early in PATH
$ echo $PATH | tr ':' '\n' | head

pip Install Permission Errors

# Never use sudo pip!
# Use virtual environment instead
$ python3 -m venv myenv
$ source myenv/bin/activate
$ pip install package

SSL Certificate Errors

# Ensure certificates are installed
$ /Applications/Python\ 3.x/Install\ Certificates.command
# Or
$ pip install certifi
$ python -c "import certifi; print(certifi.where())"

pyenv install Fails

# Install build dependencies
$ brew install openssl readline sqlite3 xz zlib

# Set compiler flags
$ CFLAGS="-I$(brew --prefix openssl)/include -I$(brew --prefix readline)/include" \
  LDFLAGS="-L$(brew --prefix openssl)/lib -L$(brew --prefix readline)/lib" \
  pyenv install 3.11.7

Summary

Python management strategy:

Key commands:

Golden rules:

1. Never modify system Python
2. Always use virtual environments
3. Pick one version manager (pyenv recommended)
4. Document Python version in projects (.python-version or pyproject.toml)

Managing Ruby Environments

macOS includes a system Ruby, but like system Python, you shouldn’t rely on it for development. Ruby version managers solve the problem of needing different Ruby versions for different projects. This chapter covers rbenv (recommended), rvm, and chruby—plus managing gems effectively.

System Ruby

macOS includes Ruby, but it’s outdated and restricted:

$ /usr/bin/ruby --version
ruby 2.6.10p210 (2022-04-12 revision 67958) [universal.arm64e-darwin23]

# System Ruby is:
# - Old (2.6.x)
# - Requires sudo for gem install
# - Modified by macOS updates
# - Not suitable for development

Rule: Never use system Ruby for development.

rbenv: Recommended Approach

rbenv is lightweight and follows Unix philosophy—it does one thing well.

Installation

# Install rbenv and ruby-build
$ brew install rbenv ruby-build

# Verify installation
$ rbenv --version
rbenv 1.2.0

Shell Configuration

Add to ~/.zshrc:

# Initialize rbenv
eval "$(rbenv init - zsh)"

Restart shell or source ~/.zshrc.

Installing Ruby Versions

# List available versions
$ rbenv install -l
3.2.2
3.3.0
jruby-9.4.5.0
truffleruby-23.1.2
...

# Install a version
$ rbenv install 3.2.2
Downloading ruby-3.2.2.tar.gz...
Installing ruby-3.2.2...
Installed ruby-3.2.2 to /Users/david/.rbenv/versions/3.2.2

# List installed versions
$ rbenv versions
  system
* 3.2.2 (set by /Users/david/.rbenv/version)

Switching Versions

# Set global default
$ rbenv global 3.2.2

# Set local version (per directory)
$ cd myproject
$ rbenv local 3.1.4
# Creates .ruby-version file

# Set for current shell
$ rbenv shell 3.0.6

# Check current version
$ rbenv version
3.2.2 (set by /Users/david/.rbenv/version)

$ ruby --version
ruby 3.2.2 (2023-03-30 revision e51014f9c0) [arm64-darwin23]

How rbenv Works

rbenv uses shims—wrapper scripts that intercept Ruby commands:

# Shims directory
$ ls ~/.rbenv/shims
bundle  erb  gem  irb  rake  rdoc  ri  ruby

# Shim intercepts and redirects
$ which ruby
/Users/david/.rbenv/shims/ruby

# Actual Ruby
$ rbenv which ruby
/Users/david/.rbenv/versions/3.2.2/bin/ruby

Rehashing

After installing gems with executables, rehash:

$ rbenv rehash
# Creates shims for new executables

# Modern rbenv auto-rehashes, but manual rehash if needed

chruby: Minimal Alternative

chruby is even simpler than rbenv—no shims, just environment modification.

Installation

$ brew install chruby ruby-install

Configuration

Add to ~/.zshrc:

source /opt/homebrew/opt/chruby/share/chruby/chruby.sh
source /opt/homebrew/opt/chruby/share/chruby/auto.sh  # Auto-switching

Usage

# Install Ruby
$ ruby-install ruby 3.2.2

# List Rubies
$ chruby
   ruby-3.2.2

# Switch
$ chruby ruby-3.2.2

# Auto-switching reads .ruby-version
$ echo "ruby-3.2.2" > .ruby-version
$ cd .  # Triggers auto-switch

rvm: Full-Featured Alternative

RVM (Ruby Version Manager) is feature-rich but more complex.

Installation

# Install rvm
$ \curl -sSL https://get.rvm.io | bash -s stable

# Restart shell or source
$ source ~/.rvm/scripts/rvm

Usage

# Install Ruby
$ rvm install 3.2.2

# List versions
$ rvm list

# Switch versions
$ rvm use 3.2.2

# Set default
$ rvm --default use 3.2.2

# Create gemset (isolated gem environment)
$ rvm gemset create myproject
$ rvm use 3.2.2@myproject

# Project-specific .rvmrc
$ echo "rvm use 3.2.2@myproject" > .rvmrc

rvm vs rbenv

Gem Management

Understanding Gems

# Gem environment
$ gem env
RubyGems Environment:
  - RUBYGEMS VERSION: 3.4.10
  - RUBY VERSION: 3.2.2
  - INSTALLATION DIRECTORY: /Users/david/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0
  - USER INSTALLATION DIRECTORY: /Users/david/.gem/ruby/3.2.0
  - GEM PATHS:
     - /Users/david/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0
     - /Users/david/.gem/ruby/3.2.0

# Install gem
$ gem install rails

# List installed gems
$ gem list

# Uninstall
$ gem uninstall rails

Bundler: Project Dependencies

Bundler manages gems per-project:

# Install bundler
$ gem install bundler

# Create Gemfile
$ bundle init

# Edit Gemfile
$ cat Gemfile
source 'https://rubygems.org'
gem 'rails', '~> 7.0'
gem 'puma'

# Install dependencies
$ bundle install

# Run command with bundled gems
$ bundle exec rails server

# Update gems
$ bundle update

# Lock to specific versions
$ bundle lock

Gemfile Best Practices

# Gemfile
source 'https://rubygems.org'

# Specify Ruby version
ruby '3.2.2'

# Specify gem versions
gem 'rails', '~> 7.0.8'
gem 'pg', '~> 1.5'

# Development-only gems
group :development do
  gem 'rubocop'
  gem 'pry'
end

group :test do
  gem 'rspec'
end

Gem Configuration

# ~/.gemrc
gem: --no-document

# Skip documentation for faster installs
$ gem install rails --no-document

Rails Development Setup

Complete setup for Rails development:

# 1. Install rbenv
$ brew install rbenv ruby-build

# 2. Configure shell
$ echo 'eval "$(rbenv init - zsh)"' >> ~/.zshrc
$ source ~/.zshrc

# 3. Install Ruby
$ rbenv install 3.2.2
$ rbenv global 3.2.2

# 4. Install Bundler
$ gem install bundler

# 5. Install Rails
$ gem install rails

# 6. Verify
$ rails --version
Rails 7.1.2

# 7. Create project
$ rails new myapp
$ cd myapp
$ bundle install
$ bin/rails server

Troubleshooting

“gem install” Permission Denied

# Don't use sudo!
# Ensure you're using rbenv Ruby, not system
$ which ruby
/Users/david/.rbenv/shims/ruby  # Good

$ which ruby
/usr/bin/ruby  # Bad - system Ruby

# Fix: Check rbenv setup
$ rbenv versions
$ rbenv global 3.2.2

Command Not Found After gem install

# Rehash to create shims
$ rbenv rehash

# Or check if gem's bin is in PATH
$ gem env | grep "EXECUTABLE DIRECTORY"

Build Failures

# Install build dependencies
$ brew install openssl readline libyaml

# Set build flags
$ RUBY_CONFIGURE_OPTS="--with-openssl-dir=$(brew --prefix openssl@3)" rbenv install 3.2.2

SSL Certificate Errors

# Update certificates
$ brew install openssl
$ rbenv install 3.2.2
# openssl from Homebrew is used automatically

# Or update system certificates
$ security find-certificate -a -p /Library/Keychains/System.keychain > certs.pem

Wrong Ruby Version in Project

# Check .ruby-version
$ cat .ruby-version
3.1.4

# Install if missing
$ rbenv install 3.1.4

# cd out and back in
$ cd .. && cd myproject
$ ruby --version

Summary

Ruby version management:

Key commands (rbenv):

Best practices:

1. Never use system Ruby
2. Use rbenv for version management
3. Use Bundler for project dependencies
4. Commit Gemfile.lock to version control
5. Specify Ruby version in .ruby-version

Managing Node.js Environments

Node.js development often requires different versions for different projects—an older LTS for production, the latest for new features, or specific versions matching your CI environment. Version managers like nvm, fnm, and volta solve this problem, while package managers like npm, yarn, and pnpm handle dependencies.

Node.js Installation Options

nvm: Node Version Manager

nvm is the most widely used Node.js version manager.

Installation

# Install nvm
$ curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash

# Or with Homebrew (note: Homebrew version has caveats)
$ brew install nvm

Shell Configuration

Add to ~/.zshrc:

export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"  # Load nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"  # Completion

Restart shell or source ~/.zshrc.

Using nvm

# List available versions
$ nvm ls-remote
        v18.18.0   (LTS: Hydrogen)
        v18.19.0   (Latest LTS: Hydrogen)
        v20.10.0   (LTS: Iron)
        v21.5.0
...

# Install a version
$ nvm install 20  # Installs latest v20.x
$ nvm install 18.19.0  # Specific version
$ nvm install --lts  # Latest LTS

# List installed versions
$ nvm ls
->     v20.10.0
       v18.19.0
default -> 20 (-> v20.10.0)
...

# Switch versions
$ nvm use 18
Now using node v18.19.0 (npm v10.2.3)

# Set default
$ nvm alias default 20

# Use in current directory (.nvmrc)
$ echo "20" > .nvmrc
$ nvm use
Found '/path/to/project/.nvmrc' with version <20>
Now using node v20.10.0 (npm v10.2.3)

Auto-switching

Automatically switch when entering directories:

# Add to ~/.zshrc
autoload -U add-zsh-hook
load-nvmrc() {
  local nvmrc_path="$(nvm_find_nvmrc)"
  if [ -n "$nvmrc_path" ]; then
    local nvmrc_node_version=$(nvm version "$(cat "${nvmrc_path}")")
    if [ "$nvmrc_node_version" = "N/A" ]; then
      nvm install
    elif [ "$nvmrc_node_version" != "$(nvm version)" ]; then
      nvm use
    fi
  elif [ -n "$(PWD=$OLDPWD nvm_find_nvmrc)" ] && [ "$(nvm version)" != "$(nvm version default)" ]; then
    echo "Reverting to nvm default version"
    nvm use default
  fi
}
add-zsh-hook chpwd load-nvmrc
load-nvmrc

nvm Performance

nvm can slow shell startup. Mitigate with lazy loading:

# Lazy load nvm (in ~/.zshrc)
export NVM_DIR="$HOME/.nvm"

nvm() {
  unset -f nvm node npm npx
  [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
  nvm "$@"
}

node() {
  unset -f nvm node npm npx
  [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
  node "$@"
}

npm() {
  unset -f nvm node npm npx
  [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
  npm "$@"
}

npx() {
  unset -f nvm node npm npx
  [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
  npx "$@"
}

fnm: Fast Node Manager

fnm is a Rust-based alternative that’s significantly faster.

Installation

# Via Homebrew
$ brew install fnm

# Or curl
$ curl -fsSL https://fnm.vercel.app/install | bash

Configuration

# Add to ~/.zshrc
eval "$(fnm env --use-on-cd)"

Usage

# Install Node
$ fnm install 20
$ fnm install --lts

# List installed
$ fnm list
* v20.10.0 default
  v18.19.0

# Use version
$ fnm use 18

# Set default
$ fnm default 20

# Auto-switching via .nvmrc or .node-version
$ fnm use  # Reads from .nvmrc

fnm vs nvm

volta: Toolchain Manager

Volta manages Node.js plus npm/yarn versions per project.

Installation

$ curl https://get.volta.sh | bash

Usage

# Install Node
$ volta install node@20
$ volta install node@lts

# Install global tools
$ volta install yarn
$ volta install typescript

# Pin version in project
$ volta pin node@18
# Updates package.json with volta field

# Now anyone with volta uses same versions

How volta Differs

Volta pins tool versions in package.json:

{
  "name": "myproject",
  "volta": {
    "node": "18.19.0",
    "npm": "10.2.3"
  }
}

This ensures consistent tooling across team members.

Package Managers

npm (default)

npm comes with Node.js:

# Check version
$ npm --version

# Install dependencies
$ npm install

# Add package
$ npm install lodash

# Add dev dependency
$ npm install --save-dev jest

# Global install
$ npm install -g typescript

# Update packages
$ npm update

# Audit security
$ npm audit
$ npm audit fix

yarn

Yarn offers performance improvements and workspaces:

# Install yarn
$ npm install -g yarn
# Or with corepack (Node 16.10+)
$ corepack enable
$ corepack prepare yarn@stable --activate

# Install dependencies
$ yarn install

# Add package
$ yarn add lodash
$ yarn add --dev jest

# Update
$ yarn upgrade

# Workspaces (monorepo)
$ yarn workspaces list

pnpm

pnpm saves disk space via hard links:

# Install pnpm
$ npm install -g pnpm
# Or with corepack
$ corepack enable
$ corepack prepare pnpm@latest --activate

# Install dependencies
$ pnpm install

# Add package
$ pnpm add lodash
$ pnpm add -D jest

# Disk savings
$ pnpm store path
$ pnpm store prune

Choosing a Package Manager

Global Packages

Where They Go

# npm global location
$ npm root -g
/Users/david/.nvm/versions/node/v20.10.0/lib/node_modules

# yarn global location
$ yarn global dir
/Users/david/.config/yarn/global

Managing Globals

# List globals
$ npm list -g --depth=0
$ yarn global list

# Install global
$ npm install -g typescript
$ yarn global add typescript

# Update globals
$ npm update -g
$ yarn global upgrade

Globals with Version Managers

Global packages are per Node version:

# Install in Node 20
$ nvm use 20
$ npm install -g typescript

# Switch to Node 18 - typescript not available
$ nvm use 18
$ which tsc
tsc not found

# Need to install again
$ npm install -g typescript

For shared globals, use npx instead:

# Run without installing
$ npx typescript --version
$ npx create-react-app myapp

Project Configuration

.nvmrc / .node-version

# Create version file
$ echo "20" > .nvmrc
# Or specific
$ echo "20.10.0" > .nvmrc

# Works with nvm, fnm, and others
$ nvm use

package.json engines

{
  "engines": {
    "node": ">=18.0.0",
    "npm": ">=9.0.0"
  }
}

Enforced with npm install --engine-strict.

Corepack

Node 16.10+ includes Corepack for package manager management:

# Enable corepack
$ corepack enable

# Specify in package.json
{
  "packageManager": "yarn@4.0.2"
}

# Now yarn is auto-installed at correct version
$ yarn --version
4.0.2

Troubleshooting

“node: command not found” After nvm install

# Check nvm is loaded
$ command -v nvm

# Reload shell config
$ source ~/.zshrc

# Set default
$ nvm alias default 20
$ nvm use default

Permission Errors

# Never use sudo with nvm-installed node
# If you have permission issues, nvm isn't set up right
$ which node
/Users/david/.nvm/versions/node/v20.10.0/bin/node  # Good
/usr/local/bin/node  # Bad - not nvm

Wrong Node in Scripts

# Scripts may use different PATH
# Add to script or use full path
#!/usr/bin/env node

# Or ensure PATH includes nvm
export PATH="$NVM_DIR/versions/node/$(nvm version)/bin:$PATH"

node-gyp Build Errors

# Install build tools
$ xcode-select --install

# For Python dependency
$ brew install python

Summary

Node.js management strategy:

Key commands (nvm):

Best practices:

1. Use a version manager (nvm or fnm)
2. Include .nvmrc in projects
3. Set engines in package.json
4. Don’t sudo npm install -g
5. Use npx for one-off commands

Comparing Package Managers

With multiple package management options on macOS, choosing the right tool matters. This chapter provides a comprehensive comparison to help you make informed decisions for different scenarios.

General Package Managers

Homebrew vs MacPorts

When to Choose Homebrew

• Quick installations: Bottles install in seconds
• macOS GUI apps: Casks make it easy
• Developer workflows: Most tutorials assume Homebrew
• Casual use: Simple commands, minimal configuration
• Standard packages: Common tools well-supported

When to Choose MacPorts

• Build customization: Variants offer fine-grained control
• Isolation: Complete separation from system
• Reproducibility: Same build everywhere
• Obscure packages: Larger package collection
• Server environments: Predictable configurations

Language Version Managers

Python: pyenv vs Others

Recommendation: pyenv for most Python development. Conda for data science.

Ruby: rbenv vs rvm vs chruby

Recommendation: rbenv for most users. chruby for minimalists. rvm if you need gemsets.

Node.js: nvm vs fnm vs volta

Recommendation: fnm for speed. nvm for compatibility. volta for teams needing consistent tooling.

Universal Version Managers

asdf: One Tool for Everything

asdf manages multiple languages with plugins:

# Install asdf
$ brew install asdf

# Add plugins
$ asdf plugin add nodejs
$ asdf plugin add python
$ asdf plugin add ruby

# Install versions
$ asdf install nodejs 20.10.0
$ asdf install python 3.11.7
$ asdf install ruby 3.2.2

# Set versions
$ asdf global nodejs 20.10.0
$ asdf local python 3.11.7

# .tool-versions file
nodejs 20.10.0
python 3.11.7
ruby 3.2.2

Pros:

• Single tool for all languages
• Consistent interface
• One configuration file

Cons:

• Plugin quality varies
• Another abstraction layer
• May lag behind native managers

mise (formerly rtx)

Modern asdf alternative in Rust:

$ brew install mise

# Compatible with asdf plugins and .tool-versions
$ mise install node@20
$ mise use node@20

Pros: Faster than asdf, compatible with asdf plugins.

Decision Framework

For Individual Developers

Need GUI apps?
├─ Yes → Homebrew (casks)
└─ No → Either works

Need build customization?
├─ Yes → MacPorts
└─ No → Homebrew

Multiple Python versions?
├─ Yes → pyenv
└─ No → Homebrew Python

Multiple Ruby versions?
├─ Yes → rbenv
└─ No → Homebrew Ruby

Multiple Node versions?
├─ Yes → fnm or nvm
└─ No → Homebrew Node

Many languages?
├─ Yes → Consider asdf or mise
└─ No → Use language-specific managers

For Teams

• Consistency: Use Brewfile and version files
• Documentation: Document required tools in README
• CI/CD: Match local and CI environments

# Brewfile for team
tap "homebrew/bundle"
brew "git"
brew "node"
brew "python@3.11"

# .tool-versions for asdf users
nodejs 20.10.0
python 3.11.7
ruby 3.2.2

# Or individual version files
# .nvmrc, .ruby-version, .python-version

For Servers/Production

Package Manager Combinations

Common setups that work well together:

Minimal Setup

# Just Homebrew
brew install python node ruby
# Single versions, simple

Standard Developer Setup

# Homebrew for system tools
brew install git wget jq

# Language version managers
brew install pyenv rbenv nvm

# Configure each
pyenv install 3.11.7
rbenv install 3.2.2
nvm install 20

Full-Featured Setup

# Homebrew for tools and casks
brew install git wget
brew install --cask visual-studio-code docker

# asdf for all languages
brew install asdf
asdf plugin add python nodejs ruby golang

# Or individual managers if preferred

Avoiding Conflicts

PATH Priority

# Check what's first in PATH
$ echo $PATH | tr ':' '\n' | head -5

# Typical priority order:
# 1. Version manager shims (~/.pyenv/shims)
# 2. Homebrew (/opt/homebrew/bin)
# 3. Local (/usr/local/bin)
# 4. System (/usr/bin)

Detecting Conflicts

# Which python am I using?
$ which python3
$ python3 --version

# Are there multiple?
$ type -a python3
python3 is /Users/david/.pyenv/shims/python3
python3 is /opt/homebrew/bin/python3
python3 is /usr/bin/python3

Resolving Conflicts

# Remove Homebrew version if using version manager
$ brew uninstall python

# Or unlink
$ brew unlink python

# Ensure version manager is in PATH first
export PATH="$HOME/.pyenv/shims:$PATH"

Summary

Quick Reference

Best Practices

1. Don’t mix Homebrew and MacPorts for the same package
2. Use version managers for languages you develop in
3. Document your setup in project READMEs
4. Use Brewfile for reproducible tool installation
5. Keep things simple - more tools ≠ better
6. Match CI environment to local development

Recommended Stack

For most macOS developers:

# Package manager
Homebrew

# Languages (pick what you need)
pyenv + pyenv-virtualenv  # Python
rbenv                      # Ruby
fnm                        # Node.js

# Or if using many languages
asdf or mise

# Project configuration
.python-version, .ruby-version, .nvmrc
Brewfile for team tools

This combination provides flexibility, isolation, and reproducibility without excessive complexity.

System Services on macOS

If you’re coming from Linux, you’ll search for systemctl and find nothing. macOS uses launchd, a fundamentally different approach to service management that Apple introduced in 2005. Understanding launchd is essential for managing background processes, scheduling tasks, and running services on macOS.

What Is launchd?

launchd is macOS’s unified service management framework. It replaces:

• init (process 1)
• cron (scheduled tasks)
• inetd (network services)
• xinetd (extended inetd)
• rc.d scripts (startup scripts)

launchd is always PID 1:

$ ps aux | head -2
USER   PID  %CPU %MEM      VSZ    RSS   TT  STAT STARTED      TIME COMMAND
root     1   0.1  0.1 410148544  18624   ??  Ss   10:00AM   0:05.23 /sbin/launchd

Key Concepts

Jobs vs Services

In launchd terminology:

• Job: Any task managed by launchd (one-time or recurring)
• Daemon: A background process running as root
• Agent: A background process running as a user

Property Lists (plists)

launchd jobs are configured via property list files:

# System-wide daemons
/Library/LaunchDaemons/

# System-wide agents
/Library/LaunchAgents/

# Per-user agents
~/Library/LaunchAgents/

# Apple's daemons (don't modify)
/System/Library/LaunchDaemons/

# Apple's agents (don't modify)
/System/Library/LaunchAgents/

launchctl

launchctl is the command-line interface to launchd:

# List all services
$ launchctl list

# Load/start a service
$ launchctl load /Library/LaunchDaemons/com.example.daemon.plist

# Unload/stop a service
$ launchctl unload /Library/LaunchDaemons/com.example.daemon.plist

What You’ll Learn in This Part

launchd: The Heart of macOS provides a comprehensive overview of launchd architecture, how it boots the system, and its role in process management.

Understanding Property Lists (plists) explains the XML and binary formats used for launchd configuration and how to create and edit them.

Creating Launch Agents and Daemons walks through creating your own background services step by step.

launchctl Command Reference is a comprehensive guide to the launchctl command and all its subcommands.

Process Management: CLI vs GUI covers managing processes from both Terminal and Activity Monitor.

Background Task Scheduling explains how to schedule recurring tasks using launchd (the macOS way) and cron (the traditional Unix way).

XPC Services and IPC introduces Apple’s modern inter-process communication framework.

Comparing launchd to systemd and init helps Linux users understand how macOS’s approach differs from what they know.

Quick Example

Here’s a simple launch agent that runs a script every hour:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.example.hourly</string>
    <key>ProgramArguments</key>
    <array>
        <string>/usr/local/bin/myscript.sh</string>
    </array>
    <key>StartInterval</key>
    <integer>3600</integer>
</dict>
</plist>

Save as ~/Library/LaunchAgents/com.example.hourly.plist and load:

$ launchctl load ~/Library/LaunchAgents/com.example.hourly.plist

The following chapters explain this and much more in detail.

launchd: The Heart of macOS

launchd is the first process started by the macOS kernel and the ancestor of all other processes. Understanding launchd is fundamental to understanding how macOS manages services, schedules tasks, and controls the lifecycle of processes.

launchd’s Role

launchd serves multiple functions that are separate on other Unix systems:

The Boot Process

When macOS boots:

1. Firmware (iBoot on Apple Silicon) loads the kernel
2. Kernel starts and launches /sbin/launchd as PID 1
3. launchd reads system configuration from:
   • /System/Library/LaunchDaemons/ (Apple services)
   • /Library/LaunchDaemons/ (third-party system services)
4. launchd starts essential system services
5. loginwindow appears (GUI login)
6. User session launches per-user launchd
7. Per-user launchd loads:
   • /System/Library/LaunchAgents/ (Apple user agents)
   • /Library/LaunchAgents/ (system-wide user agents)
   • ~/Library/LaunchAgents/ (user agents)

Kernel
    │
    └── launchd (PID 1, root)
            │
            ├── System daemons
            │   ├── mds (Spotlight)
            │   ├── configd (network)
            │   ├── bluetoothd
            │   └── ...
            │
            ├── loginwindow
            │   │
            │   └── User session launchd (per user)
            │           │
            │           ├── User agents
            │           │   ├── Dock
            │           │   ├── Finder
            │           │   └── ...
            │           │
            │           └── User applications
            │
            └── Background daemons

Daemons vs Agents

Daemons

• Run as root (or specified user)
• Start at boot, before user login
• System-wide scope
• Located in LaunchDaemons directories
• Can’t access GUI or user session

# System daemons location
$ ls /Library/LaunchDaemons/
com.apple.installer.osmessagetracing.plist
homebrew.mxcl.postgresql@14.plist
...

Agents

• Run as the logged-in user
• Start at user login
• Per-user scope
• Located in LaunchAgents directories
• Can access GUI, user session

# User agents location
$ ls ~/Library/LaunchAgents/
com.example.myagent.plist
...

Which to Use?

Job States

launchd jobs have several states:

# View job status
$ launchctl list | grep com.example
-       0       com.example.myjob
# Columns: PID, Exit Status, Label

# PID "-" means not running
# Exit Status "0" means last run succeeded

States:

• Loaded: Job definition is registered
• Running: Process is executing
• Waiting: Waiting for trigger (time, socket, path)
• Stopped: Not running, can be started

On-Demand Loading

launchd’s key innovation is on-demand loading. Services can be:

Always Running (KeepAlive)

<key>KeepAlive</key>
<true/>

launchd restarts the job if it exits.

Socket Activated

<key>Sockets</key>
<dict>
    <key>Listeners</key>
    <dict>
        <key>SockServiceName</key>
        <string>http</string>
    </dict>
</dict>

launchd listens on the socket; starts service when connection arrives.

Time-Based

<key>StartInterval</key>
<integer>3600</integer>

Runs every 3600 seconds.

Path-Based

<key>WatchPaths</key>
<array>
    <string>/path/to/watch</string>
</array>

Runs when file/directory changes.

launchd Domains

Modern launchctl organizes jobs into domains:

# List jobs in user domain
$ launchctl print user/$(id -u)

# List jobs in GUI domain
$ launchctl print gui/$(id -u)

Working with launchd

Basic Commands

# List all loaded jobs
$ launchctl list

# Load a job
$ launchctl load /path/to/job.plist

# Unload a job
$ launchctl unload /path/to/job.plist

# Start a loaded job
$ launchctl start com.example.myjob

# Stop a running job
$ launchctl stop com.example.myjob

Modern Commands (macOS 10.10+)

# Bootstrap (load and enable)
$ launchctl bootstrap gui/$(id -u) /path/to/job.plist

# Bootout (unload)
$ launchctl bootout gui/$(id -u)/com.example.myjob

# Enable job to run at load
$ launchctl enable user/$(id -u)/com.example.myjob

# Disable job
$ launchctl disable user/$(id -u)/com.example.myjob

# Kick (force immediate run)
$ launchctl kickstart gui/$(id -u)/com.example.myjob

# Kill and restart
$ launchctl kickstart -k gui/$(id -u)/com.example.myjob

Debugging

# Print detailed job info
$ launchctl print gui/$(id -u)/com.example.myjob

# Print domain info
$ launchctl print gui/$(id -u)

# Check why job isn't loading
$ launchctl print-disabled user/$(id -u)

# View launchd logs
$ log show --predicate 'subsystem == "com.apple.launchd"' --last 1h

Common launchd Keys

Log Output

By default, launchd job output goes to the unified log. To capture output:

<key>StandardOutPath</key>
<string>/tmp/myjob.stdout.log</string>
<key>StandardErrorPath</key>
<string>/tmp/myjob.stderr.log</string>

Or view in Console.app / unified log:

$ log show --predicate 'process == "myjob"' --last 1h

Security Considerations

Disabled Jobs

macOS can disable jobs for security:

# Check disabled status
$ launchctl print-disabled user/$(id -u)
"com.example.suspicious" => disabled

Code Signing

System integrity checks may prevent unsigned jobs from loading.

Sandbox

launchd jobs can be sandboxed:

<key>EnableSandboxing</key>
<true/>

Summary

launchd key concepts:

launchd is more than an init system—it’s a comprehensive process management framework that enables macOS’s responsive behavior by starting services only when needed.

Understanding Property Lists (plists)

Property lists are macOS’s standard format for structured data. launchd configurations, application preferences, and system settings all use plists. Understanding how to read, create, and modify plists is essential for macOS system administration.

Plist Formats

Property lists can be stored in multiple formats:

# Check plist format
$ file /Library/LaunchDaemons/com.apple.example.plist
/Library/LaunchDaemons/com.apple.example.plist: XML 1.0 document text, ASCII text

$ file /System/Library/LaunchDaemons/com.apple.mds.plist
/System/Library/LaunchDaemons/com.apple.mds.plist: Apple binary property list

XML Plist Structure

Basic Example

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.example.myjob</string>
    <key>ProgramArguments</key>
    <array>
        <string>/usr/local/bin/myscript</string>
        <string>--flag</string>
        <string>argument</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>StartInterval</key>
    <integer>300</integer>
</dict>
</plist>

Data Types