# GoldenGateAlly Menu Bar Capture Flow

`GoldenGateAlly` contains the original menu bar scanning and image capture path. Its job is to return a list of menu bar items with:

- the resolved display name
- the AX metadata used to identify the item
- the menu bar frame in global screen coordinates
- an optional captured image of the item
- the capture source and matched `CGWindowID`, when one exists

The recommended entry point is `GoldenGateMenuItemManager`. It is an `@Observable` `@MainActor` class that owns the scanner, snapshot cache, sorting, and refresh state. The lower-level entry point is `MenuBarScanner.scanRunningApps(...)` if an integrating app wants to wire its own state container.

## High-Level Flow

1. Call `GoldenGateMenuItemManager.refresh(promptForAccessibility:)`.
2. The manager checks Accessibility permission with `MenuBarScanner.requestAccessibilityIfNeeded(prompt:)`.
3. The scanner enumerates running apps via `NSWorkspace.shared.runningApplications`.
4. For each app, create an AX app element with `AXUIElementCreateApplication(pid)`.
5. Ask `AXMenuBarElementResolver` for menu bars and menu bar items.
6. Read each item frame and metadata from AX.
7. Resolve a human-friendly item name with `MenuBarDisplayNameResolver`.
8. Try to match the AX frame to a real menu bar `CGWindowID`.
9. Capture an image using the selected `MenuBarSnapshotCaptureMethod`.
10. The manager sorts the resulting `[MenuBarItem]` left-to-right and publishes it through `items`.

The manager sorts returned items by `frame.minX`, then `frame.minY`, so the left-most item appears first in UI lists or grids.

## Manager

`GoldenGateMenuItemManager` is the integration facade.

It exposes:

- `items`: the current sorted menu bar items
- `state`: `idle`, `scanning`, `unauthorized`, or `loaded`
- `lastScanDuration`: duration of the last refresh
- `lastRefreshFinishedAt`: completion timestamp for the last refresh
- `captureMethod`: `legacy`, `screenCapture`, or `appIcons`
- `capturingSnapshots`: whether image capture should be attempted

Typical SwiftUI usage:

```swift
@State private var manager = GoldenGateMenuItemManager()

var body: some View {
    List(manager.items) { item in
        Text(item.appName)
    }
    .task {
        await manager.refresh()
    }
}
```

For non-SwiftUI code, keep the manager on the main actor and observe it through the Observation framework.

## Scanning

`MenuBarScanner` is the coordinator.

It fetches menu bar window candidates first:

```swift
CGWindowBridge.getWindowList(option: [.menuBarItems])
```

Those IDs come from the private CoreGraphics Services menu bar window list. They are later used either as exact per-item matches or as a fallback set for region capture.

For every running app, the scanner:

- skips terminated apps
- creates the app AX element
- asks `AXMenuBarElementResolver.menuBars(in:)` for `AXMenuBar` children
- scans all menu bars for `MenuBarAgent`, because Apple system extras can be exposed as unresolved wrapper groups there
- otherwise uses `menuBars[1]` when an app has multiple menu bars
- uses `menuBars[0]` for accessory apps with one menu bar
- falls back to scanning all menu bars if the selected bar produces no items
- skips the app if no target menu bar is found

Then it asks:

```swift
AXMenuBarElementResolver.menuBarItems(
    in: menuBar,
    includeUnresolvedParents: true
)
```

`includeUnresolvedParents: true` matters for apps that expose a wrapper element instead of a direct status menu child. If a real nested status menu is found, the resolver returns that child. If not, the scanner can still keep the wrapper and capture by frame.

## AX Resolution

`AXMenuBarElementResolver` is the shared AX helper.

It handles three important cases:

- direct `AXMenuBarItem` elements
- elements whose `AXRoleDescription` normalizes to `status menu`
- wrapper/group elements containing a nested status menu descendant

For nested items it searches these child attributes:

- `AXChildren`
- `AXVisibleChildren`
- `AXContents`

The search depth is capped at `4`.

For each resolved item, metadata comes from:

- `AXLabel`
- `AXTitle`
- `AXDescription`
- `AXIdentifier`
- `AXRoleDescription`
- `AXSubrole`

The frame comes from `AXPosition` plus `AXSize`.

## Window Matching

AX gives reliable menu item frames, but it often does not give a useful `CGWindowID`.

The scanner matches AX frames against `CGWindowBridge.getWindowList(option: [.menuBarItems])` results. A candidate is accepted only when:

- the candidate window is at most `400` points wide
- `minX`, `minY`, `width`, and `height` are all within `5` points of the AX frame

The width limit prevents matching the raw full-width menu bar window instead of a menu bar item.

When a match is found, `MenuBarItem.matchedWindowID` is set. When no match is found, it remains `nil`, but the item can still be captured by region.

## Capture

`MenuBarSnapshotCapturer` wraps the available capture backends behind three operations:

- `captureComposite(for:)`
- `captureWindow(for:)`
- `captureRegion(windowIDs:screenBounds:debugAppName:)`

The scanner uses this order:

1. If there is a matched `CGWindowID`, capture that single window.
2. If that fails or there is no match, capture the item frame using all known menu bar window IDs as the source set.
3. If live capture fails, load the last cached snapshot for the item.

The region fallback is the key behavior for items whose `CGWindowID` is `nil`. It captures the menu bar window set with the AX item frame as `screenBounds`, so the image is produced by coordinates even without a per-item window ID.

### Legacy Capture

`MenuBarSnapshotCaptureMethod.legacy` uses:

```swift
CGWindowListCreateImageFromArray(...)
```

The Swift side goes through `CGWindowBridge`, which calls the Objective-C shim in `WindowCapture.m` because that API is easier to use from Objective-C.

This is the most important path for the current menu bar item captures.

### ScreenCaptureKit Capture

`MenuBarSnapshotCaptureMethod.screenCapture` uses `ScreenCaptureKit`.

For a matched window, it tries to find a corresponding `SCWindow` from `SCShareableContent.windows` by:

- exact `windowID` plus plausible coordinates
- closest plausible coordinates
- exact `windowID` as a final fallback

It ignores `SCWindow` candidates wider than `400` points for the same reason as the legacy matcher.

Capture then tries:

- desktop-independent `SCContentFilter(desktopIndependentWindow:)`
- display-scoped window capture if desktop-independent capture fails
- display-region capture for region fallback

`filter.includeMenuBar = true` is set on ScreenCaptureKit filters.

ScreenCaptureKit is best-effort for this use case because macOS may not expose real menu bar items as separate `SCWindow` objects.

### App Icon Mode

`MenuBarSnapshotCaptureMethod.appIcons` intentionally does not capture snapshots. It lets `MenuBarItem` fall back to the owning app icon.

## Output

The scanner returns `[MenuBarItem]`.

Important fields:

- `appName`: resolved display name for the menu bar item
- `bundleIdentifier`: owning app bundle identifier
- `axSourceAppName`: app that exposed the AX element
- `axSourceBundleIdentifier`: bundle that exposed the AX element
- `axSourcePID`: source app PID
- `axLabel`, `axTitle`, `axDescription`: original AX naming data
- `axIdentifier`: AX identifier, useful for Apple menu extras
- `axRoleDescription`: usually `status menu` for resolved menu extras
- `icon`: owning app icon fallback
- `itemImage`: captured menu bar item image, if available
- `imageSource`: `legacy`, `screencapture-filter`, `screencapture-display`, `cache`, `appicon`, or `none`
- `frame`: global screen-coordinate frame from AX
- `itemIndex`: index within the source app's resolved menu bar items
- `didMatchWindowID`: whether a strict `CGWindowID` match was found
- `matchedWindowID`: matched `CGWindowID`, or `nil`

Coordinates can be negative on secondary displays. Do not clamp or normalize them before capture.

## Supporting Objects

`MenuBarDisplayNameResolver`

Maps raw AX names and identifiers to friendlier names. This handles cases like Apple menu extras where the raw item may be an identifier such as `com.apple.menuextra.wifi`.

`MenuBarDisplayImage`

Converts app icons and PNG snapshot data into displayable `NSImage` values.

`MenuBarSnapshotImageCache`

Stores PNG snapshot data under Application Support using a hashed key based on owner identity, item index, and item size. It is used only after live capture succeeds, and as a fallback when live capture fails.

`MenuBarSnapshotModels`

Contains shared transport models:

- `RunningApplicationInfo`
- `MenuBarWindowSnapshotInfo`

`CGWindowBridge`

Wraps private CGS calls for menu bar window IDs and window frames, and wraps the Objective-C capture shim.

`WindowCapture.h` / `WindowCapture.m`

Objective-C bridge around `CGWindowListCreateImageFromArray`.

## Integration Notes

To move this into another app, copy the `GoldenGateAlly` folder and make sure the new app also has:

- an Objective-C bridging header importing `GoldenGateAlly/WindowCapture.h`
- the private CGS declarations currently in `private.swift`, or equivalent declarations
- AppKit, ApplicationServices, CoreGraphics, CryptoKit, ImageIO, and ScreenCaptureKit available to the target
- Accessibility permission before AX scanning
- Screen Recording permission before live image capture

Lower-level usage:

```swift
let manager = GoldenGateMenuItemManager(
    captureMethod: .legacy,
    capturingSnapshots: true
)

await manager.refresh()
let displayItems = manager.items
```

For display, prefer `item.itemImage` when present, then fall back to `item.icon`.

## Current Behavior To Preserve

- AX is the source of truth for item identity and frames.
- `CGWindowID` is optional; many good captures have `matchedWindowID == nil`.
- Region capture by AX coordinates is expected and valid.
- Full-width menu bar windows should not be matched as item windows.
- ScreenCaptureKit should include the menu bar on filters, but it should not be assumed to expose every menu item as an `SCWindow`.
