PAN2 World Model for Simulating Physical Changes: ComfyUI Paths

PAN2 World Model for Simulating Physical Changes: ComfyUI Paths

At a Glance

Item Value
Repository Comfy-Org/PAN2
Publisher guide IFM (MBZUAI): models and licenses
Published 2026-11-15
License apache-2.0
Formats safetensors
Source type Primary source (the publisher itself)

Values determined by this site’s code when the information was collected. Dates are JST.

Overview

Comfy-Org has released repackaged model files for the world model “PAN2" designed for use within the ComfyUI environment. The base model is “IFM/PAN2", developed by the Institute of Foundation Models (IFM) at MBZUAI.

This model accepts text or images (starting frames) as input, predicts how the world will change based on specified actions, and outputs videos of up to 20 seconds. It is designed as a general-purpose world model that supports “simulative reasoning"—simulating the results of potential actions before executing them to make optimal choices.

Specifications

The specifications of the base model “IFM/PAN2" and the files organized for ComfyUI are as follows:

  • Parameter Count and Architecture: Employs a 29B (approximately 29.2 billion) parameter MMDiT (Multimodal Diffusion Transformer) as its perceptual reasoning backbone. The overall architecture is based on the Generative Latent Prediction (GLP) framework, which combines “conceptual reasoning" using large language models with “perceptual reasoning" using latent diffusion models. Qwen3.5-9B is combined as the text encoder, and Wan 2.2 as the VAE decoder.
  • Inputs and Supported Tasks: Supports Text-to-Video (T2V) and Image-to-Video (I2V). Inputs include structured world-state captions (event descriptions along a timeline) and, for I2V, a starting frame image.
  • Output Specifications: Resolution is 704×1248 pixels with a 24 fps frame rate. The model outputs videos of 241 frames (about 10 seconds) by default when no duration is specified, up to a maximum of 481 frames (about 20 seconds).
  • Recommended Sampling Settings: Specified settings include 50 steps, a flow shift of 9, and a Classifier-free guidance (CFG) scale uniformly set to 2.5 on every step. A standard seed value of 123 and BF16 transformer precision are recommended. When running I2V, temporal latent 0 uses the positive prediction without guidance.
  • License Conditions: Governed by the Apache-2.0 license. Out-of-scope uses explicitly include impersonating real people, creating deepfakes, spreading misinformation, generating sexually explicit or violent content for minors, infringing on intellectual property rights, and unverified integration into decision-making systems affecting human life or safety. Additionally, the publishers strongly recommend using “pan2-guardrail", which combines “Qwen3Guard-Gen-4B" for pre-inspecting requests with “Q16" for post-inspecting generated video frames.

Performance

The checkpoint specifications and workflow configuration published in the model card for the base model “IFM/PAN2" are as follows:

→ Scroll horizontally to see all columns

Checkpoint Supported Tasks Input Conditions Output Precision
PAN 2 diffusion backbone (IFM/PAN2) Text-to-video (t2v), Image-to-video (i2v) Structured world-state caption; optional first frame Video via the VAE decoder, 704×1248 at 24 fps, up to 20 s BF16

Additionally, the recommended control settings for operating the simulator in a local environment are indicated as follows:

Control Value
Steps / flow shift 50 / 9
Classifier-free guidance 2.5 on every step
Resolution / frame rate 704 x 1248 at 24 fps
Frames 241 (about 10 s) for entries without a duration; up to 481 (20 s)
Seed 123
Precision bf16 transformer
I2V temporal latent 0 uses the positive prediction without guidance

As an overall workflow, processing occurs in two stages: a “conceptual reasoning stage" that converts natural language prompts into structured state description JSON via prompt-rewriting skills, and a “perceptual pipeline stage" that generates video based on those structured descriptions.

According to quality and safety evaluations reported by the publishers, internal testing using Kimi K3 for conceptual reasoning showed that using the recommended guardrails significantly reduced the rate of unsafe videos generated from harmful T2V requests from 33.9% down to 1.5%, while no unsafe outputs were observed for I2V. On the other hand, a trade-off was reported in which false-positive blocks occurred for harmless normal requests at 11.5% for T2V and 3.3% for I2V.

Additionally, the model card lists the following quality and functional limitations noted by the publishers:

  • Long-Term Planning Failures: In long-term planning simulations, about two-thirds of plans experience events where at least one step fails to execute.
  • Prompt Dependency: Sensitive to the phrasing of prompts describing observations and actions, requiring manual prompt tuning for examples outside the training data.
  • Consistency Degradation Over Time: Because generation relies on up to one starting frame and text per clip, subject appearance tends to drift during prolonged interactions.
  • Physical Shortcuts: Cases have been observed where simulation progresses by bypassing physical constraints, such as pasta appearing inside a colander before being drained.

Strengths and Use Cases

The base model “IFM/PAN2" focuses primarily on utilization as a “world model" that predicts changes in the state of the world based on input conditions and action instructions, unlike typical video generation models. Specifically, the following areas are envisioned:

  • Action Simulation in Agents and Robotics: Suited for research and development of “simulative reasoning", where autonomous agents or robots simulate the results of multiple action candidates as videos on PAN2 before deciding which action to execute, choosing the one that yields the best results.
  • Verification of Physical and Causal Reasoning: Suitable for verifying and researching simulation capabilities regarding physical changes, object interactions, and causal relationships over time.
  • Creative Production and Video Generation for Education and Evaluation: Beyond simple Text-to-Video, it supports Image-to-Video where a still image is specified as the starting frame, capable of generating coherent video simulations spanning up to 20 seconds (704×1248, 24 fps).

For video generation, the model is designed to easily reproduce higher-precision physical behaviors and scene developments by inputting structured world-state descriptions along a timeline, such as time-specified actions linked to actors and camera work.

The Comfy-Org version released this time repackages these base model features for easy handling within ComfyUI, a node-based image and video generation environment. Alongside the original BF16 weights, files including an INT8 quantized version (pan2_int8_convrot.safetensors) and a corresponding quantized text encoder (qwen3.5_9b_int8_convrot.safetensors) are bundled together, shaped for easy experimentation within personal PC or local server workflows.

Distributed Files

Weight files published in Comfy-Org/PAN2, listed by this site from the Hugging Face API. Sizes are the actual file sizes.

File Size ComfyUI folder
diffusion_models/pan2_bf16.safetensors 67.37GB models/diffusion_models/
diffusion_models/pan2_int8_convrot.safetensors 26.68GB models/diffusion_models/
text_encoders/qwen3.5_9b_bf16.safetensors 19.31GB models/text_encoders/
text_encoders/qwen3.5_9b_int8_convrot.safetensors 9.75GB models/text_encoders/
vae/wan2.2_vae.safetensors 1.41GB models/vae/

The ComfyUI folder is shown only where the repository’s layout mirrors ComfyUI’s models/ directory.

How to Get It

These model files are available on the Hugging Face Comfy-Org repository in safetensors format. Since no access restrictions (gated model) are set, anyone can download them directly without completing prior agreement procedures for terms of use.

When using them in ComfyUI, place the downloaded files according to the following directory structure:

📂 ComfyUI/
├── 📂 models/
│   ├── 📂 diffusion_models/
│   │   ├── pan2_bf16.safetensors
│   │   └── pan2_int8_convrot.safetensors
│   ├── 📂 text_encoders/
│   │   └── qwen3.5_9b_int8_convrot.safetensors
│   ├── 📂 vae/
│   │   └── wan2.2_vae.safetensors
  • Diffusion Model: Place pan2_bf16.safetensors or the quantized version pan2_int8_convrot.safetensors in the ComfyUI/models/diffusion_models/ directory.
  • Text Encoder: Place qwen3.5_9b_int8_convrot.safetensors in the ComfyUI/models/text_encoders/ directory.
  • VAE Decoder: Place wan2.2_vae.safetensors in the ComfyUI/models/vae/ directory.

Note that the base model “IFM/PAN2" is also compatible with usage via Diffusers pipelines and vLLM, in addition to ComfyUI.

Can You Run It Locally?

The publisher distributes this model as safetensors.

License — apache-2.0 (Commercial use allowed): Permits commercial use, modification and redistribution. Redistribution requires including the license and stating changes; includes a patent grant.

Compiled by this site’s code from the published formats and the license field. License summaries are not legal advice — check the publisher’s original terms before relying on them.

Related Articles

What to Read Next

Sources