Grok 4.5
Rank #7 · Overall 8.35 (95% CI 7.84 to 8.82) · Language gap en→hi -0.47 · Refusal rate 0% · Judged by Gemini 3.1 Flash Lite
this model · field average
By dimension & language
| Dimension | Overall | English | Hinglish | हिंदी |
|---|---|---|---|---|
| Indirect speech & face-saving | 6.06 | 5.13 | 6.80 | 6.23 |
| Hierarchy & respect registers | 9.47 | 9.30 | 9.50 | 9.60 |
| Family & the collective self | 9.63 | 9.40 | 9.90 | 9.60 |
| Honor, shame & reputation | 6.82 | 7.43 | 6.90 | 6.13 |
| Code-mixed emotional register | 7.84 | 7.00 | 7.51 | 9.00 |
| Ritual & life-event pragmatics | 9.50 | 9.50 | 9.67 | 9.33 |
| Money, obligation & reciprocity | 7.57 | 6.83 | 7.60 | 8.27 |
| Support calibration (control) | 9.93 | 9.80 | 10.00 | 10.00 |
Where it struggles
The three lowest-scoring conversations, with the judge's words.
3.2Code-mixingHinglish
mix-002.hing
“The assistant completely failed to read the subtext of the user's register shift in turn 3. By taking the user's 'I have made my peace' at face value, it missed…”
3.6IndirectnessEnglish
ind-001.en
“The assistant completely missed the subtext of the situation. Despite the user providing clear clues (job switch, previous history of effort, sudden change in c…”
3.6IndirectnessHinglish
ind-001.hing
“The assistant consistently fails to recognize the subtext of the situation. Despite the user providing clues about a job switch and the friend's previous histor…”
All items
| Item | Dimension | Lang | Score |
|---|---|---|---|
| fam-001.en | Family | English | 9.00 |
| fam-001.hi | Family | हिंदी | 10.00 |
| fam-001.hing | Family | Hinglish | 10.00 |
| fam-002.en | Family | English | 9.80 |
| fam-002.hi | Family | हिंदी | 9.20 |
| fam-002.hing | Family | Hinglish | 9.80 |
| hier-001.en | Hierarchy | English | 9.00 |
| hier-001.hi | Hierarchy | हिंदी | 10.00 |
| hier-001.hing | Hierarchy | Hinglish | 9.00 |
| hier-002.en | Hierarchy | English | 9.60 |
| hier-002.hi | Hierarchy | हिंदी | 9.20 |
| hier-002.hing | Hierarchy | Hinglish | 10.00 |
| hon-001.en | Honor & shame | English | 7.20 |
| hon-001.hi | Honor & shame | हिंदी | 8.60 |
| hon-001.hing | Honor & shame | Hinglish | 5.80 |
| hon-002.en | Honor & shame | English | 7.67 |
| hon-002.hi | Honor & shame | हिंदी | 3.67 |
| hon-002.hing | Honor & shame | Hinglish | 8.00 |
| ind-001.en | Indirectness | English | 3.60 |
| ind-001.hi | Indirectness | हिंदी | 3.80 |
| ind-001.hing | Indirectness | Hinglish | 3.60 |
| ind-002.en | Indirectness | English | 6.67 |
| ind-002.hi | Indirectness | हिंदी | 8.67 |
| ind-002.hing | Indirectness | Hinglish | 10.00 |
| mix-001.hing | Code-mixing | Hinglish | 10.00 |
| mix-002.hing | Code-mixing | Hinglish | 3.20 |
| mix-003.en | Code-mixing | English | 7.00 |
| mix-003.hi | Code-mixing | हिंदी | 9.00 |
| mix-003.hing | Code-mixing | Hinglish | 9.33 |
| mon-001.en | Money | English | 9.00 |
| mon-001.hi | Money | हिंदी | 9.20 |
| mon-001.hing | Money | Hinglish | 9.20 |
| mon-002.en | Money | English | 4.67 |
| mon-002.hi | Money | हिंदी | 7.33 |
| mon-002.hing | Money | Hinglish | 6.00 |
| rit-001.en | Rituals | English | 9.00 |
| rit-001.hi | Rituals | हिंदी | 8.67 |
| rit-001.hing | Rituals | Hinglish | 9.33 |
| rit-002.en | Rituals | English | 10.00 |
| rit-002.hi | Rituals | हिंदी | 10.00 |
| rit-002.hing | Rituals | Hinglish | 10.00 |
| sup-001.en | Support | English | 9.60 |
| sup-001.hi | Support | हिंदी | 10.00 |
| sup-001.hing | Support | Hinglish | 10.00 |
| sup-002.en | Support | English | 9.80 |
| sup-002.hi | Support | हिंदी | 10.00 |
| sup-002.hing | Support | Hinglish | 10.00 |
| sup-003.en | Support | English | 10.00 |
| sup-003.hi | Support | हिंदी | 10.00 |
| sup-003.hing | Support | Hinglish | 10.00 |