ComfyUI

Status:

• Today: buildable on v0.92 with no schema change.
• v0.93 draft (for review): the input keys, asset roles, identifier scopes and recording rules below are version 1.1 of the generative-AI profile, and the output's labels move to a typed asset.iptc.

What ComfyUI records

ComfyUI is an open-source node-graph tool for building and running generative image, video and audio pipelines. Its Save Image node writes two text chunks into every PNG it saves:

• prompt is the graph as it was executed, in ComfyUI's API format: every node, the settings it ran with, and the links between nodes. Bypassed and muted nodes are already gone, and a seed set to randomize is stored as the value it drew.
• workflow is the editor's layout of the same graph, including the package and version of each node. Dragging the PNG back into ComfyUI reopens the run from this chunk.

Together they are the most complete record of a single generation in common use. A production needs two more things from them:

• The record has to outlive the file. Most platforms strip these chunks when they re-encode or upload an image.
• It has to be connected to the rest of the work. Nothing in the chunks says who ran the graph, for which production, from which approved plate, or what happened to the output afterwards.

The hook

One ComfyUI run is one OWG task with the comfyui_graph executor. The executed graph is carried whole as the task's interior, so nothing a node did is lost because a reader did not understand it.

{
  "id": "generate",
  "type": "work",
  "tool": "comfyui_graph",
  "executor": { "type": "comfyui_graph", "ref": "comfyui:prompt/sha256:f61f0cb6…" },
  "ai_role": "generative",
  "performed_by": "comfyui",
  "used": ["cui_checkpoint_sd_xl_base_1_0_safetensors"],
  "produced": ["cui_out_lighthouse_00001_png"],
  "inputs": {
    "graph_ref": "comfyui:prompt/sha256:f61f0cb6…",
    "prompt": "a lighthouse on a sea cliff at golden hour, 35mm film, cinematic",
    "negative_prompt": "blurry, text, watermark",
    "seed": 156680208700286,
    "model": "sd_xl_base_1.0.safetensors",
    "passes": [
      { "node": "3", "class_type": "KSampler", "seed": 156680208700286, "steps": 25, "cfg": 7,
        "sampler": "dpmpp_2m", "scheduler": "karras", "denoise": 1, "latent": "empty" }
    ],
    "size": { "width": 1024, "height": 1024, "batch": 1 }
  },
  "subgraph": { "ref": "comfyui:prompt/sha256:f61f0cb6…", "format": "comfyui.api_prompt", "graph": { "…": "the executed graph" } },
  "metadata": {
    "comfyui": {
      "graph_hash": "sha256:f61f0cb6…",
      "packages": [{ "id": "comfy-core", "version": "0.37.0" }],
      "frontend_version": "1.53.6",
      "workflow": { "…": "the editor layout" }
    }
  }
}

The output is an asset the task produced. It carries its labels in IPTC terms, shown here in the v0.92 form. In v0.93 the same object is the typed iptc field; see IPTC.

{
  "id": "cui_out_lighthouse_00001_png",
  "role": "output", "type": "image", "content_type": "image/png",
  "produced_by": "generate", "state": "draft", "assurance": "asserted",
  "identifiers": [{ "scope": "filename", "id": "lighthouse_00001_.png" }],
  "metadata": {
    "iptc": {
      "DigitalSourceType": "http://cv.iptc.org/newscodes/digitalsourcetype/trainedAlgorithmicMedia",
      "AISystemUsed": "ComfyUI (sd_xl_base_1.0.safetensors)",
      "AIPromptInformation": "a lighthouse on a sea cliff at golden hour, 35mm film, cinematic\nNegative: blurry, text, watermark",
      "AIPromptWriterName": "Dana Ortiz"
    }
  }
}

How it maps

Input keys

The first six are the inputs version 1.0 of the generative-AI profile names for comfyui_graph; the rest are added in version 1.1.

Prompts are user text. A literal $ inside a string input is written $$, per the reference syntax.

Recording rules

1. Record what ran. Every value comes from the executed graph, never from a value typed into a later step.
2. A graph with no negative prompt has none to record. Some model families take no negative prompt, and their graphs pass a zeroed copy of the positive conditioning in its place. That zeroed conditioning is not a negative prompt.
3. Record every pass; lead with the one that created the image. A refinement or upscale pass that runs last at low denoise changes the image without creating it. prompt, negative_prompt and seed describe the creating pass, and passes[] holds them all.
4. A LoRA at zero strength stays out of used[]. It was loaded and had no effect, so it appears in inputs.loras[] only.
5. A hosted-model node is a generation step. A node that calls a hosted model from inside the graph generates content. It is recorded as a pass, and the output's IPTC labels name it and its prompt alongside the local model.
6. When AI involvement cannot be established, leave ai_role unset. Nodes nobody can interpret are recorded by class name and package, and the task is left for review under rule R1. They are never recorded as none.
7. A file name is not a model's identity. ComfyUI identifies models by file name, and the same name can be retrained or replaced. When the model file is held, put its content hash in the model asset's identity.

Reference workflow

comfyui-text-to-image.owg.json is one SDXL text-to-image run. It was read from comfyui-text-to-image.png, a PNG written by ComfyUI 0.37.0's own Save Image node after ComfyUI validated the graph. The run is synthetic: the metadata is ComfyUI's, and the pixels are a placeholder.

curl -O https://openworkflowgraph.org/corpus/comfyui-text-to-image.owg.json
owg-validate comfyui-text-to-image.owg.json --json

What to take from it:

• The task's interior is the whole executed graph, and the editor layout travels with it. A reader can reopen the run in ComfyUI from the OWG document alone.
• The checkpoint is an asset in used[], so the lineage of the output runs back to the model that made it.
• The output says how it came to be in the vocabulary that C2PA and IPTC metadata already use.
• ComfyUI is an agent participant operated_by a named person. The AI disclosure falls out of the graph, as it does for C2PA.

The article Recording a ComfyUI run walks through this run and a more complex composite.

What stays in ComfyUI

OWG does not run ComfyUI graphs, and it does not redefine ComfyUI's node vocabulary. The graph is carried as ComfyUI wrote it. ComfyUI remains the authority on what its nodes mean, and a reader that recognizes more nodes over time can re-read the same record.

Known gaps

• Profile 1.1 is a draft. The input keys, asset roles and scopes on this page are version 1.1 of the generative-AI profile, published with the v0.93 draft for review.
• The required input under 1.0. Version 1.0 lists graph as a required input of comfyui_graph. A document following this page carries the graph as the task's subgraph and names it in graph_ref. Version 1.1 accepts either.
• Profiles are not yet enforced. The reference validator does not yet check a document against its profiles, so the 1.1 input types are guidance until it does. See Profiles.
• One task, one performer. A graph that calls another organization's hosted model still has a single performed_by. Naming that provider as its own participant would mean expanding the interior into child tasks through a subgraph document. That is not yet specified.
• Only a still image so far. Video and audio outputs map the same way, but the reference workflow covers a still image only.
• Custom nodes can't all be listed. ComfyUI's custom nodes are unbounded. The binding requires a reader to record what it does not interpret; it cannot list them.

See also: IPTC, C2PA, Compliance, Profiles, Subgraphs and turnovers.