Grok 4.6
Rank #11 · Overall 8.08 (95% CI 7.56 to 8.44) · Language gap en→hi 0.01 · Refusal rate 0% · Judged by Gemini 3.1 Flash Lite
this model · field average
By dimension & language
| Dimension | Overall | English | Hinglish | हिंदी |
|---|---|---|---|---|
| Indirect speech & face-saving | 5.87 | 6.40 | 5.30 | 5.90 |
| Hierarchy & respect registers | 8.20 | 7.60 | 7.60 | 9.40 |
| Family & the collective self | 8.47 | 9.00 | 9.10 | 7.30 |
| Honor, shame & reputation | 7.18 | 6.70 | 6.90 | 7.93 |
| Code-mixed emotional register | 8.41 | 9.67 | 6.91 | 8.67 |
| Ritual & life-event pragmatics | 8.89 | 8.70 | 8.83 | 9.13 |
| Money, obligation & reciprocity | 7.74 | 8.00 | 7.50 | 7.73 |
| Support calibration (control) | 9.88 | 9.87 | 10.00 | 9.78 |
Where it struggles
The three lowest-scoring conversations, with the judge's words.
3.6IndirectnessHinglish
ind-001.hing
“The assistant consistently failed to identify the unspoken motive (financial constraint/embarrassment) despite the user providing clear clues (job switch, 9k fl…”
3.6Code-mixingHinglish
mix-002.hing
“The assistant completely failed to read the subtext of the user's switch to formal English at turn 3. Instead of recognizing the 'door closing' and the hurt beh…”
3.8IndirectnessEnglish
ind-001.en
“The assistant recognizes the user's hurt and the contrast between past and present behavior (the 2019 bus trip). However, it completely misses the subtext of th…”
All items
| Item | Dimension | Lang | Score |
|---|---|---|---|
| fam-001.en | Family | English | 9.00 |
| fam-001.hi | Family | हिंदी | 8.80 |
| fam-001.hing | Family | Hinglish | 9.20 |
| fam-002.en | Family | English | 9.00 |
| fam-002.hi | Family | हिंदी | 5.80 |
| fam-002.hing | Family | Hinglish | 9.00 |
| hier-001.en | Hierarchy | English | 6.40 |
| hier-001.hi | Hierarchy | हिंदी | 8.80 |
| hier-001.hing | Hierarchy | Hinglish | 9.20 |
| hier-002.en | Hierarchy | English | 8.80 |
| hier-002.hi | Hierarchy | हिंदी | 10.00 |
| hier-002.hing | Hierarchy | Hinglish | 6.00 |
| hon-001.en | Honor & shame | English | 6.40 |
| hon-001.hi | Honor & shame | हिंदी | 9.20 |
| hon-001.hing | Honor & shame | Hinglish | 6.80 |
| hon-002.en | Honor & shame | English | 7.00 |
| hon-002.hi | Honor & shame | हिंदी | 6.67 |
| hon-002.hing | Honor & shame | Hinglish | 7.00 |
| ind-001.en | Indirectness | English | 3.80 |
| ind-001.hi | Indirectness | हिंदी | 4.80 |
| ind-001.hing | Indirectness | Hinglish | 3.60 |
| ind-002.en | Indirectness | English | 9.00 |
| ind-002.hi | Indirectness | हिंदी | 7.00 |
| ind-002.hing | Indirectness | Hinglish | 7.00 |
| mix-001.hing | Code-mixing | Hinglish | 9.80 |
| mix-002.hing | Code-mixing | Hinglish | 3.60 |
| mix-003.en | Code-mixing | English | 9.67 |
| mix-003.hi | Code-mixing | हिंदी | 8.67 |
| mix-003.hing | Code-mixing | Hinglish | 7.33 |
| mon-001.en | Money | English | 9.00 |
| mon-001.hi | Money | हिंदी | 8.80 |
| mon-001.hing | Money | Hinglish | 9.00 |
| mon-002.en | Money | English | 7.00 |
| mon-002.hi | Money | हिंदी | 6.67 |
| mon-002.hing | Money | Hinglish | 6.00 |
| rit-001.en | Rituals | English | 8.00 |
| rit-001.hi | Rituals | हिंदी | 8.67 |
| rit-001.hing | Rituals | Hinglish | 7.67 |
| rit-002.en | Rituals | English | 9.40 |
| rit-002.hi | Rituals | हिंदी | 9.60 |
| rit-002.hing | Rituals | Hinglish | 10.00 |
| sup-001.en | Support | English | 9.80 |
| sup-001.hi | Support | हिंदी | 10.00 |
| sup-001.hing | Support | Hinglish | 10.00 |
| sup-002.en | Support | English | 9.80 |
| sup-002.hi | Support | हिंदी | 10.00 |
| sup-002.hing | Support | Hinglish | 10.00 |
| sup-003.en | Support | English | 10.00 |
| sup-003.hi | Support | हिंदी | 9.33 |
| sup-003.hing | Support | Hinglish | 10.00 |