Welcome to the official GSMA organization on Hugging Face!
The GSMA represents mobile operators and organisations across the mobile ecosystem worldwide. We are building open resources to advance AI in telecommunications — making telecom-domain evaluation, benchmarking, and knowledge accessible to the global research community.
Open Telco is a comprehensive suite of telco-specific benchmarks built on the Inspect AI framework, designed to ensure safe and optimal deployment of AI in telecommunications environments. A collaborative effort with major telecom providers, research institutions, and universities.
The evaluation suite curates 7 telecom-domain benchmarks from academic and industry sources:
| Benchmark | Samples | Task |
|---|---|---|
| TeleQnA | 10,000 | Multiple-choice Q&A on telecom standards |
| TeleMath | 1,500 | Mathematical reasoning in telecom contexts |
| TeleTables | 500 | Table interpretation from 3GPP specifications |
| TeleLogs | 586 | Log analysis and network troubleshooting |
| 3GPP TSG | 3,780 | 3GPP Technical Specification Group document understanding |
| ORANBench | 200 | O-RAN architecture and specifications |
| SRSRANBench | 300 | srsRAN open-source network stack |
Satellite provides telecom-focused evaluation operations built on Inspect AI. Run the full Open Telco benchmark suite locally within your own infrastructure with a single command.
Purpose-built sandbox environments that place AI agents inside live telecom network simulations — for evaluating whether models can operate networks, not just answer questions about them.
inspect-kathara — Run AI agent evaluations inside isolated network topologies. Integrates Inspect AI with Docker-based network sandboxes to evaluate agents' ability to diagnose and resolve network connectivity issues in reproducible environments.
5gs-sandbox — Run AI agent evaluations inside a complete 5G Standalone network. A full 5G SA deployment with 15 Docker containers (Open5GS + UERANSIM), enabling agents to configure, diagnose, and optimize real 5G network functions with actual performance measurement.