# ComfyUI

> How a ComfyUI run is recorded in OWG. One generative task whose interior is the executed graph, the models and media it used as assets, and an output labeled in IPTC terms.

**Status:**
- **Today:** buildable on v0.92 with no schema change.
- **v0.93 draft (for review):** the input keys, asset roles, identifier scopes and recording rules below are version 1.1 of the [generative-AI profile](/schemas/profiles/generative-ai/v1.1.json), and the output's labels move to a typed [`asset.iptc`](/iptc).

## What ComfyUI records

[ComfyUI](https://github.com/Comfy-Org/ComfyUI) is an open-source node-graph tool for building and running generative image, video and audio pipelines. Its Save Image node writes two text chunks into every PNG it saves:

- **`prompt`** is the graph as it was executed, in ComfyUI's API format: every node, the settings it ran with, and the links between nodes. Bypassed and muted nodes are already gone, and a seed set to randomize is stored as the value it drew.
- **`workflow`** is the editor's layout of the same graph, including the package and version of each node. Dragging the PNG back into ComfyUI reopens the run from this chunk.

Together they are the most complete record of a single generation in common use. A production needs two more things from them:
- **The record has to outlive the file.** Most platforms strip these chunks when they re-encode or upload an image.
- **It has to be connected to the rest of the work.** Nothing in the chunks says who ran the graph, for which production, from which approved plate, or what happened to the output afterwards.

## The hook

One ComfyUI run is one OWG task with the `comfyui_graph` executor. The executed graph is carried whole as the task's interior, so nothing a node did is lost because a reader did not understand it.

```json
{
  "id": "generate",
  "type": "work",
  "tool": "comfyui_graph",
  "executor": { "type": "comfyui_graph", "ref": "comfyui:prompt/sha256:f61f0cb6…" },
  "ai_role": "generative",
  "performed_by": "comfyui",
  "used": ["cui_checkpoint_sd_xl_base_1_0_safetensors"],
  "produced": ["cui_out_lighthouse_00001_png"],
  "inputs": {
    "graph_ref": "comfyui:prompt/sha256:f61f0cb6…",
    "prompt": "a lighthouse on a sea cliff at golden hour, 35mm film, cinematic",
    "negative_prompt": "blurry, text, watermark",
    "seed": 156680208700286,
    "model": "sd_xl_base_1.0.safetensors",
    "passes": [
      { "node": "3", "class_type": "KSampler", "seed": 156680208700286, "steps": 25, "cfg": 7,
        "sampler": "dpmpp_2m", "scheduler": "karras", "denoise": 1, "latent": "empty" }
    ],
    "size": { "width": 1024, "height": 1024, "batch": 1 }
  },
  "subgraph": { "ref": "comfyui:prompt/sha256:f61f0cb6…", "format": "comfyui.api_prompt", "graph": { "…": "the executed graph" } },
  "metadata": {
    "comfyui": {
      "graph_hash": "sha256:f61f0cb6…",
      "packages": [{ "id": "comfy-core", "version": "0.37.0" }],
      "frontend_version": "1.53.6",
      "workflow": { "…": "the editor layout" }
    }
  }
}
```

The output is an asset the task produced. It carries its labels in IPTC terms, shown here in the v0.92 form. In v0.93 the same object is the typed `iptc` field; see [IPTC](/iptc).

```json
{
  "id": "cui_out_lighthouse_00001_png",
  "role": "output", "type": "image", "content_type": "image/png",
  "produced_by": "generate", "state": "draft", "assurance": "asserted",
  "identifiers": [{ "scope": "filename", "id": "lighthouse_00001_.png" }],
  "metadata": {
    "iptc": {
      "DigitalSourceType": "http://cv.iptc.org/newscodes/digitalsourcetype/trainedAlgorithmicMedia",
      "AISystemUsed": "ComfyUI (sd_xl_base_1.0.safetensors)",
      "AIPromptInformation": "a lighthouse on a sea cliff at golden hour, 35mm film, cinematic\nNegative: blurry, text, watermark",
      "AIPromptWriterName": "Dana Ortiz"
    }
  }
}
```

## How it maps

| ComfyUI | OWG |
|---|---|
| The executed graph (`prompt` chunk) | The task's `subgraph`: `{ ref, format: "comfyui.api_prompt", graph }`. `executor.ref` and `inputs.graph_ref` name it by content hash |
| The editor layout (`workflow` chunk) | `metadata.comfyui.workflow` on the task, so the run reopens in ComfyUI |
| Node packages and versions | `metadata.comfyui.packages` on the task |
| Checkpoint, diffusion model, VAE, text encoders, ControlNet, style and upscale models | Assets in the task's `used[]`: `type: "model"`, `role` naming the kind, identifier scope `comfyui.model` |
| LoRAs | Assets of `type: "lora"` in `used[]`, with their strengths in `inputs.loras[]` |
| Loaded images, masks and video | Assets in `used[]` with `role` `init_image`, `mask`, `control_image` or `reference_image`, identifier scope `comfyui.input` |
| Sampler settings | `inputs.passes[]`, one entry per sampling pass |
| ControlNet, IP-Adapter and style application | `inputs.controls[]` |
| The saved file | The produced asset, identifier scope `filename` (or `comfyui.output` when no file name is known), labeled in [`metadata.iptc`](/iptc) |
| An edit of a loaded image (inpainting, image-to-image below full strength) | The output's `predecessor` is the starting image, with `version_relation: "derivation"` |

### Input keys

The first six are the inputs version 1.0 of the generative-AI profile names for `comfyui_graph`; the rest are added in version 1.1.

| Key | Holds |
|---|---|
| `graph_ref` | The executed graph's identifier, `comfyui:prompt/sha256:<hash>` |
| `prompt`, `negative_prompt` | The prompt text of the pass that created the image |
| `seed` | That pass's seed |
| `model` | The base model's file name |
| `lora` | The first LoRA in effect, for readers of the current profile |
| `loras[]` | Every LoRA loaded: `{ name, strength_model, strength_clip }` |
| `passes[]` | Every sampling pass: `node`, `class_type`, `seed`, `steps`, `cfg`, `sampler`, `scheduler`, `denoise`, `guidance`, `start_step`, `end_step`, `latent` (`empty`, `encoded`, `inpaint`, `upstream`), and `hosted: true` for a hosted-model node |
| `controls[]` | Every control applied: `{ kind, strength, start, end }` |
| `size` | `{ width, height, batch }` of a generation that started from an empty latent |

Prompts are user text. A literal `$` inside a string input is written `$$`, per the [reference syntax](/reference-syntax).

## Recording rules

1. **Record what ran.** Every value comes from the executed graph, never from a value typed into a later step.
2. **A graph with no negative prompt has none to record.** Some model families take no negative prompt, and their graphs pass a zeroed copy of the positive conditioning in its place. That zeroed conditioning is not a negative prompt.
3. **Record every pass; lead with the one that created the image.** A refinement or upscale pass that runs last at low denoise changes the image without creating it. `prompt`, `negative_prompt` and `seed` describe the creating pass, and `passes[]` holds them all.
4. **A LoRA at zero strength stays out of `used[]`.** It was loaded and had no effect, so it appears in `inputs.loras[]` only.
5. **A hosted-model node is a generation step.** A node that calls a hosted model from inside the graph generates content. It is recorded as a pass, and the output's IPTC labels name it and its prompt alongside the local model.
6. **When AI involvement cannot be established, leave `ai_role` unset.** Nodes nobody can interpret are recorded by class name and package, and the task is left for review under [rule R1](/compliance#the-baseline-rule-pack). They are never recorded as `none`.
7. **A file name is not a model's identity.** ComfyUI identifies models by file name, and the same name can be retrained or replaced. When the model file is held, put its content hash in the model asset's `identity`.

## Reference workflow

[`comfyui-text-to-image.owg.json`](/corpus/comfyui-text-to-image.owg.json) is one SDXL text-to-image run. It was read from [`comfyui-text-to-image.png`](/corpus/comfyui-text-to-image.png), a PNG written by ComfyUI 0.37.0's own Save Image node after ComfyUI validated the graph. The run is synthetic: the metadata is ComfyUI's, and the pixels are a placeholder.

```bash
curl -O https://openworkflowgraph.org/corpus/comfyui-text-to-image.owg.json
owg-validate comfyui-text-to-image.owg.json --json
```

**What to take from it:**

- The task's interior is the whole executed graph, and the editor layout travels with it. A reader can reopen the run in ComfyUI from the OWG document alone.
- The checkpoint is an asset in `used[]`, so the lineage of the output runs back to the model that made it.
- The output says how it came to be in the vocabulary that C2PA and IPTC metadata already use.
- `ComfyUI` is an `agent` participant `operated_by` a named person. The AI disclosure falls out of the graph, as it does for [C2PA](/c2pa).

The article [Recording a ComfyUI run](/recording-comfyui-runs) walks through this run and a more complex composite.

## What stays in ComfyUI

OWG does not run ComfyUI graphs, and it does not redefine ComfyUI's node vocabulary. The graph is carried as ComfyUI wrote it. ComfyUI remains the authority on what its nodes mean, and a reader that recognizes more nodes over time can re-read the same record.

## Known gaps

- **Profile 1.1 is a draft.** The input keys, asset roles and scopes on this page are version 1.1 of the generative-AI profile, published with the v0.93 draft for review.
- **The required input under 1.0.** Version 1.0 lists `graph` as a required input of `comfyui_graph`. A document following this page carries the graph as the task's `subgraph` and names it in `graph_ref`. Version 1.1 accepts either.
- **Profiles are not yet enforced.** The reference validator does not yet check a document against its profiles, so the 1.1 input types are guidance until it does. See [Profiles](/profiles).
- **One task, one performer.** A graph that calls another organization's hosted model still has a single `performed_by`. Naming that provider as its own participant would mean expanding the interior into child tasks through a [subgraph document](/subgraphs-and-turnovers). That is not yet specified.
- **Only a still image so far.** Video and audio outputs map the same way, but the reference workflow covers a still image only.
- **Custom nodes can't all be listed.** ComfyUI's custom nodes are unbounded. The binding requires a reader to record what it does not interpret; it cannot list them.

See also: [IPTC](/iptc), [C2PA](/c2pa), [Compliance](/compliance), [Profiles](/profiles), [Subgraphs and turnovers](/subgraphs-and-turnovers).