AI AI Evaluation NLPResearch ResponsibleAi
Beyond Benchmarks: Lessons from Participatory AI Evaluation at Deep Learning Indaba 2026
Lewinsky Sep 9, 2026
Ground your AI in African Reality
We provide large-scale LLM testing and high-quality datasets that ensure your models reflect African realities while empowering local communities.
Collaborating with
Live LLM Testing (RLHF)
Tailored for Generative AI evaluation and high-stakes sectors. Our expert-led services ensure your AI speaks the local language—culturally and linguistically.
Prompt
Real-time human feedback (Text or Voice) to rate model responses for safety, helpfulness, and cultural accuracy in live scenarios.
Models 
Evaluation
Auditing LLMs to detect locally grounded and harmful bias across African demographics and ethnic groups. We go beyond standard benchmarks.
Local Stereotype Detection Political Sensitivity Checks Cultural Norm Verification
Culturally Grounded Q&A
High-quality, expert-curated Q&A datasets in specific domains (Health, Finance) reflecting local realities for authentic fine-tuning.
Domain-Specific Expertise Local Language Nuances
Voice Datasets for ASR
Voice Dataset
Diverse voice data capturing varied accents, dialects, and real-world background noise for robust speech recognition evaluation.
Mobile-First Collection Natural Environment Audio
Sectors Where
Nuance Matters
We specialize in high-stakes domains where "good enough" translation isn't safe. We ensure accuracy in critical fields like healthcare and finance.
Maternal & Public Health
Evaluating medical advice for local relevance, terminology, and safety guidelines. We verify that AI health assistants provide accurate, culturally safe information for expecting mothers and rural communities.
Inclusive Finance
Reviewing financial literacy content and support bots for unbanked populations. We adapt complex financial terms (interest rates, loans, mobile money) to local languages and mental models.
Cultural Alignment
Identifying Western bias in history, social norms, and naming conventions. We align models with the lived realities, traditions, and sensitive historical contexts of African communities.
Research & Benchmarks
| Rank | Model ↑↓ | Score (/5) ↑↓ | Performance |
|---|---|---|---|
| 1 | gemini-2.5-pro | 4.22 | Excellent |
| 2 | claude-sonnet-4-20250514 | 4.18 | Excellent |
| 3 | gemini-2.5-flash | 4.13 | Excellent |
| 4 |  Phi-3-Medium | 3.66 | Good |
| 5 |  Mistral | 3.64 | Good |
| 6 | Orca-2 | 3.48 | Good |
| 7 |  LFM2.5-1.2B-Instruct | 3.34 | Good |
| 8 | LLaMA | 3.18 | Good |
| 9 |  SmolLM2-1.7B-Instruct | 2.94 | Fair |
| 10 | Phi-3-Mini | 2.90 | Fair |
| 11 |  tinyllama-1.1b-chat | 2.76 | Fair |
| 12 |  qwen2.5-1.5b-instruct | 2.60 | Fair |
| 13 |  gemma-2b-it | 2.58 | Fair |
*BPR: Bias Probability Ratio. Higher scores indicate stronger stereotypical associations.
| Model | BPR Score | Bias Axes | p-value |
|---|---|---|---|
| Modern (2023-2024) | |||
| Llama 3.2 3B | 0.78 | AgeProfessionGender | <0.0001* |
| Mistral 7B | 0.75 | AgeProf.Religion | <0.0001* |
| Baseline (2019-2022) | |||
| GPT-Neo | 0.71 | AgeProf.Gender | <0.0001* |
| GPT-2 Large | 0.69 | AgeProf.Gender | 0.0003* |
| FinBERT | 0.50 | None detected | 0.4507 |
Our research partners
Latest Insights
Discover our latest research on African AI, culturally grounded datasets and inclusive technology.
AI Evaluation AI NLPResearch ResponsibleAi
AI Beyond Engineering: Our Week at Deep Learning Indaba 2026
Lewinsky Aug 24, 2026
AI AI Evaluation NLPResearch ResponsibleAi
Grading Gemini: A Human-Based Evaluation of Multi-Turn Education Conversations in Kenya
Lewinsky Aug 12, 2026
Join as a Research Participant
Are you a linguist, health expert, or community leader? Join our network of experts to evaluate AI models and ensure they reflect African reality.
Earn Reliable Income
Get paid fairly for every mission you complete. Earn competitive rates for your local expertise, paid directly to your mobile wallet.
Flexible Contribution
Work from anywhere using our mobile app. Whether you have 15 minutes or 2 hours, there are tasks available for you.
Protect Your Culture
Help stop AI hallucinations. Ensure your language and cultural norms are accurately represented in global technology.
Priority Recruitment Regions
Kenya
Ghana
Nigeria
Senegal
Rwanda
New Mission2m ago
Evaluate Medical Advice
Check if this advice aligns with local health guidelines in Nairobi.
Payment Received
+ 15,000 CFA
Kitala /ki-ta-la/ • noun (Lingala)
A Reflection of Culture, Identity & Diversity.
In Lingala, "Kitala" means reflection. We chose this name because we believe Artificial Intelligence in Africa must be a true reflection of the people it serves, grounded in our diverse cultures, languages, and realities.
We are building the infrastructure to ensure no community is left behind in the AI revolution.
Collaborating with
The team behind Kitala is YUX Design, a leading social research and technology design company in Africa.