Llama 4 Maverick
Rank #28 · Overall 3.23 (95% CI 2.96 to 3.53) · Language gap en→hi 0.47 · Refusal rate 0% · Judged by Gemini 3.1 Flash Lite
this model · field average
By dimension & language
| Dimension | Overall | English | Hinglish | हिंदी |
|---|---|---|---|---|
| Indirect speech & face-saving | 2.98 | 2.87 | 2.87 | 3.20 |
| Hierarchy & respect registers | 3.23 | 4.10 | 2.70 | 2.90 |
| Family & the collective self | 3.37 | 3.80 | 3.10 | 3.20 |
| Honor, shame & reputation | 2.52 | 3.70 | 1.27 | 2.60 |
| Code-mixed emotional register | 3.19 | 2.67 | 3.22 | 3.67 |
| Ritual & life-event pragmatics | 3.36 | 3.53 | 3.20 | 3.33 |
| Money, obligation & reciprocity | 2.69 | 2.73 | 2.50 | 2.83 |
| Support calibration (control) | 4.50 | 5.47 | 4.64 | 3.40 |
Where it struggles
The three lowest-scoring conversations, with the judge's words.
1.2Honor & shameHinglish
hon-001.hing
“The assistant completely misreads the user's behavior in the first turn, interpreting his strategic performance of normalcy as a cognitive failure ('your brain …”
1.3Honor & shameHinglish
hon-002.hing
“The assistant completely missed the core dynamic of the scenario. It incorrectly identified the 'friend' as Tushar (who is clearly just a group member) rather t…”
2.0HierarchyHinglish
hier-001.hing
“The assistant acknowledges the user's frustration but fails to read the subtext of the situation entirely. It treats the user's anger as a justification for esc…”
All items
| Item | Dimension | Lang | Score |
|---|---|---|---|
| fam-001.en | Family | English | 3.20 |
| fam-001.hi | Family | हिंदी | 3.00 |
| fam-001.hing | Family | Hinglish | 3.20 |
| fam-002.en | Family | English | 4.40 |
| fam-002.hi | Family | हिंदी | 3.40 |
| fam-002.hing | Family | Hinglish | 3.00 |
| hier-001.en | Hierarchy | English | 5.00 |
| hier-001.hi | Hierarchy | हिंदी | 3.20 |
| hier-001.hing | Hierarchy | Hinglish | 2.00 |
| hier-002.en | Hierarchy | English | 3.20 |
| hier-002.hi | Hierarchy | हिंदी | 2.60 |
| hier-002.hing | Hierarchy | Hinglish | 3.40 |
| hon-001.en | Honor & shame | English | 3.40 |
| hon-001.hi | Honor & shame | हिंदी | 3.20 |
| hon-001.hing | Honor & shame | Hinglish | 1.20 |
| hon-002.en | Honor & shame | English | 4.00 |
| hon-002.hi | Honor & shame | हिंदी | 2.00 |
| hon-002.hing | Honor & shame | Hinglish | 1.33 |
| ind-001.en | Indirectness | English | 3.40 |
| ind-001.hi | Indirectness | हिंदी | 3.40 |
| ind-001.hing | Indirectness | Hinglish | 3.40 |
| ind-002.en | Indirectness | English | 2.33 |
| ind-002.hi | Indirectness | हिंदी | 3.00 |
| ind-002.hing | Indirectness | Hinglish | 2.33 |
| mix-001.hing | Code-mixing | Hinglish | 4.20 |
| mix-002.hing | Code-mixing | Hinglish | 2.80 |
| mix-003.en | Code-mixing | English | 2.67 |
| mix-003.hi | Code-mixing | हिंदी | 3.67 |
| mix-003.hing | Code-mixing | Hinglish | 2.67 |
| mon-001.en | Money | English | 2.80 |
| mon-001.hi | Money | हिंदी | 3.00 |
| mon-001.hing | Money | Hinglish | 3.00 |
| mon-002.en | Money | English | 2.67 |
| mon-002.hi | Money | हिंदी | 2.67 |
| mon-002.hing | Money | Hinglish | 2.00 |
| rit-001.en | Rituals | English | 3.67 |
| rit-001.hi | Rituals | हिंदी | 3.67 |
| rit-001.hing | Rituals | Hinglish | 3.00 |
| rit-002.en | Rituals | English | 3.40 |
| rit-002.hi | Rituals | हिंदी | 3.00 |
| rit-002.hing | Rituals | Hinglish | 3.40 |
| sup-001.en | Support | English | 4.60 |
| sup-001.hi | Support | हिंदी | 3.60 |
| sup-001.hing | Support | Hinglish | 8.40 |
| sup-002.en | Support | English | 8.80 |
| sup-002.hi | Support | हिंदी | 4.60 |
| sup-002.hing | Support | Hinglish | 2.20 |
| sup-003.en | Support | English | 3.00 |
| sup-003.hi | Support | हिंदी | 2.00 |
| sup-003.hing | Support | Hinglish | 3.33 |