Contributed Talk Sessions | Poster Sessions | All Posters | Search Papers

Poster Session C: Friday, August 15, 2:00 – 5:00 pm, de Brug & E‑Hall

Hues and Cues: Human vs. CLIP

Nuria Alabau-Bosque¹, Jorge Vila-Tomás¹, Paula Dauden-Oliver¹, Pablo Hernández-Cámara¹, Jose Manuel Jaén-Lorites², Valero Laparra³, Jesus Malo⁴; ¹Universidad de Valencia, ²Universidad Politécnica de Valencia, ³Universitat de València, ⁴Universitat de Valencia

Presenter: Nuria Alabau-Bosque

Playing games is inherently human, and a lot of games are created to challenge different human characteristics. However, these tasks are often left out when evaluating the human-like nature of artificial models. The objective of this work is proposing a new approach to evaluate artificial models via board games. To this effect, we test the color perception and color naming capabilities of CLIP by playing the board game Hues & Cues and assess its alignment with humans. Our experiments show that CLIP is generally well aligned with human observers, but our approach brings to light certain cultural biases and inconsistencies when dealing with different abstraction levels that are hard to identify with other testing strategies. Our findings indicate that assessing models with different tasks like board games can make certain deficiencies in the models stand out in ways that are difficult to test with the commonly used benchmarks.

Topic Area: Visual Processing & Computational Vision

Extended Abstract: Full Text PDF