QCRI

This paper introduces BloomBench, a new bilingual (English-Arabic), cognitively-informed benchmark based on Bloom's Taxonomy to systematically evaluate the reasoning abilities of Vision-Language Models across different hierarchical cognitive levels.

Global-Local Uncertainty

Evaluation and inference scripts for DisasterVQA — a benchmark dataset for assessing Vision-Language Models (VLMs) on disaster-response visual question answering. Contains 1,395 real-world disaster images and 4,405 expert-curated QA pairs across floods, wildfires, and earthquakes.

ALT research group publications

Source code for LLMxCPG paper

Nahw is a comprehensive benchmark for evaluating Arabic grammar understanding in large language models. It includes multiple-choice grammar questions (Nahw-MCQ), annotated passages for grammatical error detection, correction, and explanation (Nahw-Passage), and synthetic training data.