Can agents learn useful skills from frozen logs that people and agents already leave behind? 🚨Introducing TeleTune: We turn noisy offline telemetry into agent skills w/o access to a live environment during training! +6.7% on WorkArena & +7.7% on Mind2Web 👉 up to 75x fewer tokens used for optimization. 🧵 Activity logs like user telemetry capture how people use software, but often (1) lack explicit task goals, (2) cannot be replayed to test each skill update, and (3) can be a noisy mix of multiple tasks. We address these jointly in TeleTune, where we propose skill-guided progress, an offline metric that uses action-prediction errors to improve agent skills. Not only does optimizing it improve downstream performance, but it also saves token usage by 5 to 75x compared to validating skill updates in live environments! Key takeaways: 1️⃣ TeleTune achieves 77.1% average success on WorkArena and 80.6% on Online-Mind2Web, outperforming the strongest baselines by +6.7% and +7.7% absolute 2️⃣ Optimizing offline skill-guided progress transfers to improvements in downstream task success 3️⃣ Offline optimization uses 5 to 75x fewer tokens than validating the skill edits in a live environment