Note

    The eye and the hand: Skysight and Computer Use

    Skysight and Computer Use appear to share part of the same local Sky substrate, but they give an AI opposite relationships to the desktop: durable ambient context versus task-scoped perception and action.

    Updated Aug 22, 2026ai opscomputer historycomputer useskysightmemoryagencycontext systemsdata visualization
    Markdown file

    Imported markdown

    The eye and the hand: Skysight and Computer Use

    I initially understood the difference between Skysight and Computer Use too superficially: Skysight remembers things, while Computer Use can take screenshots and click things. That is true, but the filesystem makes the architecture more interesting.

    The installed Computer History and Computer Use plugins ship the same computer-use-client-launcher binary. They invoke it differently. Computer History starts the launcher in a history-specific mode; Computer Use starts the ordinary live-control mode. Skysight event segments are also written inside the com.openai.sky.CUAService app-group container—the same local service family used by Computer Use.

    That does not prove that every internal implementation detail is shared. It does show that these are not best understood as a recorder over here and an unrelated robot over there. They are neighboring contracts over a common local perception substrate.

    System diagram showing one shared Sky and Accessibility substrate splitting into Skysight, which turns eligible desktop activity into event segments and durable summaries, and Computer Use, which repeatedly reads current app state and performs task-scoped actions. Solid lines are observed contracts; a dashed enclosure marks the inferred shared substrate boundary.

    Figure 1. One substrate, two contracts. The solid behaviors are documented by the installed plugins. The enclosing shared-substrate interpretation is an inference from the identical launcher, shared service identity, and Skysight storage location—not an announced OpenAI architecture diagram.

    The deeper distinction is the direction of agency

    Skysight turns the desktop into context for the model.

    Computer Use turns model intent into changes on the desktop.

    Skysight is an ambient, asynchronous path: eligible activity becomes rolling interaction-event segments, then ten-minute summaries, then six-hour rollups and longer-lived memory. Its job is to make the past available later. It answers questions such as “What was I doing?” or “Where did I leave off?”

    This makes the cleanest mental model less like memory versus screenshots and more like eye versus hand:

    • Skysight is the long-horizon eye. It observes and compresses eligible experience, but it has no action interface.
    • Computer Use is the short-horizon eye-and-hand loop. It sees enough current state to act, but its product contract is not a durable history store.
    • A Computer Use action can still leave traces elsewhere: in the changed application, in the Codex session, and—if that activity is eligible for observation—in later Skysight context. That is an ecosystem effect, not Computer Use quietly becoming Skysight.

    They also put consent in different places

    The two systems are separated as much by their safety boundary as by their data shape.

    Computer History asks, in effect, what may become memory? Its controls sit at the observation boundary: running, paused, or stopped; application and URL policies; allowlists and blocklists; and a hard exclusion for private browsing. The important moment is before activity enters the historical stream.

    Computer Use asks what may the agent do? Its controls sit at the action boundary. Reading current state is different from clicking a button, transmitting private information, deleting something, accepting an agreement, or making a purchase. The important moment is before an action changes state or creates a consequence.

    Two-lane process diagram. The Skysight lane places its consent gate before observation and retention. The Computer Use lane places its confirmation or handoff gate immediately before consequential action. Both lanes include explicit no-action or stop states.

    Figure 2. Two gates at two different moments. Skysight governs eligibility for observation; Computer Use governs authority for action. Neither permission should be treated as permission for the other.

    This distinction matters because “the AI can see my computer” collapses two very different questions:

    1. Can a system retain eligible activity so it can be recalled later?
    2. Can an agent inspect and manipulate a live interface for this task?

    The first is a memory policy. The second is an agency policy.

    .codex is a constellation, not one giant memory

    Reviewing the full memory working tree and the broader .codex folder also makes another boundary visible. Local context is distributed across separate truth surfaces with different jobs and retention patterns.

    At the August 22 snapshot, the memory working tree contained 565 files, including 396 ten-minute Skysight summaries, 23 six-hour rollups, and 128 rollout summaries. The memory database held 129 stage-one thread outputs. Separately, the current Skysight cache held 74 raw event-segment directories, while Ambient Suggestions contained 29 saved suggestion bundles with 680 suggestion records.

    Those counts change while the app is running. More importantly, adjacency is not dependency. The existence of an Ambient Suggestions record next to Memories, sessions, visualizations, and Computer Use does not prove that Skysight produced that suggestion. In one earlier inspected configuration, Ambient Suggestions operated with Memories/Chronicle disabled. Each surface needs its own provenance.

    Constellation diagram of separate local Codex truth surfaces. Nodes show a snapshot of 565 memory working-tree files, 419 Skysight summary files split into 396 ten-minute and 23 six-hour summaries, 128 rollout summaries, 129 stage-one memory database rows, 74 current raw event segments, and 29 Ambient Suggestions bundles containing 680 records. Computer Use is shown as an execution runtime rather than a memory store. Dashed connectors mean possible context relationships, not proven data flow.

    Figure 3. A forensic inventory, not a pipeline diagram. Solid containment lines represent observed local storage; dashed lines mean that two surfaces can be contextually related without proving direct ingestion.

    This is why I now think “Skysight versus Computer Use” is more revealing than a feature comparison. Together, they expose a larger agent architecture:

    • perception can be ambient or task-scoped;
    • context can be ephemeral or durable;
    • memory can inform agency without granting it;
    • agency can create new history without owning the history system;
    • nearby local artifacts can form an ecosystem without collapsing into a single database or a single permission.

    Skysight is not passive Computer Use. Computer Use is not live Skysight. They appear to share some of the same nervous system, but one turns experience into context and the other turns intent into action.

    Evidence and limits

    The public naming remains precise: Computer History is the current user-facing feature, Skysight is the local history/memory namespace observed in storage, and Computer Use is the task-scoped Mac control capability. “Shared substrate” is a strong local inference, not a claim that OpenAI has published this internal architecture.

    Related context: From Chronicle's visual mosaics to the Computer History/Skysight event stream. The internal follow-on work is tracked separately in Project 1153 for the Skysight reporting-system transition and Project 1155 for data-visualization and reporting standards.

    Dataset Preview

    • Raw CSV row/table content is available in the source artifact.