Methodology

Visual AI Lab compares AI image and video generation models under the same conditions, records reproducibility, and applies final human review to determine which models work in real Japan-market production.

How models are selected

We compare T2I, I2I, T2V and I2V models that are practically accessible via public API or official/representative platforms.
Only models with "ACTIVE" status appear in comparison pages and rankings.

Same-condition comparison

We generate outputs from each model with the same prompt, same key image and same settings.
For images, we align resolution, aspect ratio and quality parameters as closely as possible.
For video, we record duration, fps, resolution, inference steps and guidance scale.
Cost is estimated from each model's official pricing or measured API call, converted to USD per image or per second.

Evaluation types

Every comparison page shows its evaluation type.

Baseline / Common Prompt: models are compared on a shared prompt and shared settings to measure baseline capabilities.
Model-Optimized: each model uses recommended prompts or settings for that model. These are not labeled as a "fair test".
Production Workflow: includes post-processing, rerolls, upscaling, ComfyUI or other production steps.

Scoring criteria

Outputs are scored on prompt fidelity, subject consistency, motion quality, temporal stability, background integrity and commercial usability on a 5-point scale.
Final verdicts are classified as "first_run_usable", "reroll_needed" or "unusable".
Failure tags (face collapse, hand collapse, object morphing, ignored camera direction, background collapse, temporal flicker, prompt mismatch, etc.) visualize failure patterns.

Human review and automation

Automation is fine for production and first-pass sorting, but the following are always reviewed by humans:

Output images and video
Scores and failure tags
Article conclusions
Model names, pricing and versions
Legal/commercial terms
Sponsor disclosures
Comparison template consistency

Published evaluation scores come from manual evaluation in our admin tool. Articles containing images or video are not published until human review is complete.

Sample transparency

We disclose the number of prompts, generations per prompt and how representative outputs were selected.
We record failed outputs, API route, model version and generation date.
Resolution, duration, quality setting and reroll count are shown.
When only the best output is shown, the selection criteria are stated.

Sponsor and advertising policy

Sponsors, article production fees, free credits and affiliate links are clearly marked so readers can recognize them.
We do not change scores or rankings in exchange for advertising fees.
When a sponsor is being compared, we disclose it and apply the same generation conditions to non-sponsored competitors.
Exclusion and failure conditions are defined before testing.

Updates and corrections

Articles and comparison pages show the last updated date.
Pricing and terms show a confirmation date.
Corrections are recorded in a changelog.
Old comparisons are labeled as "old version" and articles that are no longer updated may be archived or set to noindex.

This site is not a rigorous academic benchmark; it is a practical comparison review based on production use. Published content may change as models are updated or prices change. Please check each provider's official documentation for the latest information.