ImagenHub
I
Imagenhub
Overview :
ImagenHub is a one-stop repository for standardizing the inference and evaluation of all conditional image generation models. The project first defines seven prominent tasks and creates high-quality evaluation datasets. Second, we build a unified inference pipeline to ensure fair comparisons. Third, we design two human evaluation metrics, semantic consistency and perceptual quality, and establish comprehensive guidelines for evaluating generated images. We train expert reviewers to evaluate model outputs based on the proposed metrics. This human evaluation achieved high inter-rater consistency on 76% of the models. We comprehensively evaluated around 30 models and observed three key findings: (1) The performance of existing models is generally unsatisfactory, with 74% of models scoring lower than 0.5 overall except for text-guided image generation and theme-driven image generation. (2) We examined claims made in published papers and found 83% of the claims to be accurate. (3) Apart from theme-driven image generation, existing automatic evaluation metrics have no Spearman correlation coefficient higher than 0.2. In the future, we will continue to evaluate newly released models and update the rankings to track the progress of the conditional image generation field.
Target Users :
ImagenHub is a platform for standardized conditional image generation model evaluation. Researchers and developers can use it to fairly compare the performance of different models and track the progress of the field.
Total Visits: 12.9K
Top Region: US(38.30%)
Website Views : 51.1K
Use Cases
ImagenHub collects seven major conditional image generation tasks, including text-guided image generation, mask-guided image editing, theme-driven image generation, providing researchers with a comprehensive evaluation dataset.
ImagenHub establishes a unified inference pipeline, ensuring different models are compared fairly under the same evaluation process.
ImagenHub designs two human evaluation metrics, semantic consistency and perceptual quality, and trains expert reviewers to evaluate model outputs based on these metrics, achieving high inter-rater consistency.
Features
Defines seven major conditional image generation tasks
Builds high-quality evaluation datasets
Establishes a unified inference pipeline
Designs semantic consistency and perceptual quality human evaluation metrics
Trains expert reviewers for evaluation
Comprehensively evaluates around 30 conditional image generation models
Updates rankings to track field progress
AIbase
Empowering the Future, Your AI Solution Knowledge Base
© 2025AIbase