The AI Observatory
Investigate a mysterious bandwidth anomaly at a remote research station by instrumenting its AI system with OpenTelemetry, OpenLLMetry, and Jaeger.
Challenges
Calibrating the Lens
Something is eating 847% of your station's bandwidth and nobody knows what. Instrument HubSystem with OpenLLMetry, send traces to the OpenTelemetry Collector, and use Jaeger to uncover what the AI is doing behind the scenes.
Start ChallengeThe Distracted Pilot
ART's RAG pipeline is retrieving entertainment data instead of navigation coordinates and won't calculate your jump. Instrument the full retrieval pipeline with OpenLLMetry, build a custom OTel metric to quantify the distraction, and write a Prometheus recording rule to prove it.
Start ChallengeThe Noise Filter
ART is flooding Jaeger with 40,000 non-standard spans an hour. Fix the chat span to follow OpenTelemetry GenAI semantic conventions with proper token usage attributes, then configure tail sampling in the Collector to keep only traces that contain errors or exceed 5 seconds.
Start ChallengeThe Story
You are stationed at Perimeter Alpha, a research outpost on the newly discovered planet HB-7742. The station is run by HubSystem, a central AI that manages everything from life support to data analysis.
Recently, the station's bandwidth usage has spiked to 847% above baseline, but no one knows why. As the systems engineer, it's your job to instrument the AI, trace its activities, and uncover the root cause of the anomaly.
Your mission: bring visibility to the station's AI and solve the mystery.