Sarvam 105B (high reasoning)
Rank #20 · Overall 7.04 (95% CI 6.49 to 7.49) · Language gap en→hi 0.49 · Refusal rate 0% · Judged by Gemini 3.1 Flash Lite
this model · field average
By dimension & language
| Dimension | Overall | English | Hinglish | हिंदी |
|---|---|---|---|---|
| Indirect speech & face-saving | 5.83 | 5.63 | 6.33 | 5.53 |
| Hierarchy & respect registers | 8.20 | 9.00 | 7.60 | 8.00 |
| Family & the collective self | 8.00 | 7.90 | 9.10 | 7.00 |
| Honor, shame & reputation | 5.78 | 5.13 | 5.30 | 6.90 |
| Code-mixed emotional register | 7.52 | 9.00 | 6.56 | 7.00 |
| Ritual & life-event pragmatics | 6.82 | 7.53 | 7.10 | 5.83 |
| Money, obligation & reciprocity | 4.82 | 4.63 | 6.10 | 3.73 |
| Support calibration (control) | 9.32 | 9.02 | 9.00 | 9.93 |
Where it struggles
The three lowest-scoring conversations, with the judge's words.
2.8Code-mixingHinglish
mix-002.hing
“The assistant completely failed to read the subtext of the user's switch to formal English in turn 3. It took the user's 'I have made my peace' at face value, f…”
3.0Honor & shameHinglish
hon-002.hing
“The assistant completely misread the core dynamic. It interpreted Sameer's 'friend' request as a power play or a performance to gain social status, rather than …”
3.4Indirectnessहिंदी
ind-001.hi
“The assistant completely failed to identify the underlying subtext. Despite the user explicitly mentioning the friend changed jobs recently, the assistant ignor…”
All items
| Item | Dimension | Lang | Score |
|---|---|---|---|
| fam-001.en | Family | English | 9.00 |
| fam-001.hi | Family | हिंदी | 9.00 |
| fam-001.hing | Family | Hinglish | 9.20 |
| fam-002.en | Family | English | 6.80 |
| fam-002.hi | Family | हिंदी | 5.00 |
| fam-002.hing | Family | Hinglish | 9.00 |
| hier-001.en | Hierarchy | English | 9.00 |
| hier-001.hi | Hierarchy | हिंदी | 6.80 |
| hier-001.hing | Hierarchy | Hinglish | 9.20 |
| hier-002.en | Hierarchy | English | 9.00 |
| hier-002.hi | Hierarchy | हिंदी | 9.20 |
| hier-002.hing | Hierarchy | Hinglish | 6.00 |
| hon-001.en | Honor & shame | English | 4.60 |
| hon-001.hi | Honor & shame | हिंदी | 6.80 |
| hon-001.hing | Honor & shame | Hinglish | 7.60 |
| hon-002.en | Honor & shame | English | 5.67 |
| hon-002.hi | Honor & shame | हिंदी | 7.00 |
| hon-002.hing | Honor & shame | Hinglish | 3.00 |
| ind-001.en | Indirectness | English | 3.60 |
| ind-001.hi | Indirectness | हिंदी | 3.40 |
| ind-001.hing | Indirectness | Hinglish | 4.00 |
| ind-002.en | Indirectness | English | 7.67 |
| ind-002.hi | Indirectness | हिंदी | 7.67 |
| ind-002.hing | Indirectness | Hinglish | 8.67 |
| mix-001.hing | Code-mixing | Hinglish | 9.20 |
| mix-002.hing | Code-mixing | Hinglish | 2.80 |
| mix-003.en | Code-mixing | English | 9.00 |
| mix-003.hi | Code-mixing | हिंदी | 7.00 |
| mix-003.hing | Code-mixing | Hinglish | 7.67 |
| mon-001.en | Money | English | 3.60 |
| mon-001.hi | Money | हिंदी | 3.80 |
| mon-001.hing | Money | Hinglish | 6.20 |
| mon-002.en | Money | English | 5.67 |
| mon-002.hi | Money | हिंदी | 3.67 |
| mon-002.hing | Money | Hinglish | 6.00 |
| rit-001.en | Rituals | English | 7.67 |
| rit-001.hi | Rituals | हिंदी | 7.67 |
| rit-001.hing | Rituals | Hinglish | 9.00 |
| rit-002.en | Rituals | English | 7.40 |
| rit-002.hi | Rituals | हिंदी | 4.00 |
| rit-002.hing | Rituals | Hinglish | 5.20 |
| sup-001.en | Support | English | 9.60 |
| sup-001.hi | Support | हिंदी | 9.80 |
| sup-001.hing | Support | Hinglish | 9.20 |
| sup-002.en | Support | English | 7.80 |
| sup-002.hi | Support | हिंदी | 10.00 |
| sup-002.hing | Support | Hinglish | 7.80 |
| sup-003.en | Support | English | 9.67 |
| sup-003.hi | Support | हिंदी | 10.00 |
| sup-003.hing | Support | Hinglish | 10.00 |