Is your feature request related to a problem?
We encountered issues during STT Evaluation in staging with ATREE, where several samples returned an error stating {'message': 'Cannot fetch content from the provided URL.', 'code': 3}. This inconsistency affected the reliability of the evaluation process.
These evaluation runs by ATREE are in staging (sample language: Kannada):
| Evaluation Run | Run ID | Samples | Failed |
|---|
| STT Evals 6 | 260 | 49 | 20 |
| STT Evals 5 | 259 | 49 | 16 |
| STT Evals 5 | 252 | 49 | 11 |
| STT Evals 4 | 252 | 49 | 13 |
Describe the solution you'd like
- Debug the current STT evaluation process to identify the root cause of the errors.
- Consider altering our approach for multi-modal batch operations using Gemini.
Screenshot
Is your feature request related to a problem?
We encountered issues during STT Evaluation in staging with ATREE, where several samples returned an error stating
{'message': 'Cannot fetch content from the provided URL.', 'code': 3}. This inconsistency affected the reliability of the evaluation process.These evaluation runs by ATREE are in staging (sample language: Kannada):
Describe the solution you'd like
Screenshot