A computer-use agent that can only click buttons in tidy web forms is of limited use. Games are a harder test: busy 3D scenes, quests that last hours, and controls that need precise timing. Gamer, a preprint from the Honkai AI team at miHoYo with several Chinese universities and the National University of Singapore, is a model that plays them using only what appears on screen and the same keyboard and mouse a person uses.
The training happens in stages. First the team continued pre-training on tens of thousands of hours of device-level keyboard and mouse recordings from games and simulated environments, so the model learns how actions change what it sees. They then built about 1,000 hours of instruction data from curated play of Honkai: Star Rail, using a language model to add task goals, reasoning at key decisions and running memory summaries. A step they call sparse-thinking tuning cuts unnecessary reasoning, and a multi-turn reinforcement learning method handles long sessions, limited context windows and rare rewards. The base model is Qwen3-VL-8B.
Speed is as important as accuracy, since the game keeps moving while the model thinks. The team built Streaming-vLLM, a serving engine that runs many game sessions at once, reuses cached state between turns and keeps response times low, tested on Nvidia H200 and A800 hardware. Over roughly 50 hours of evaluation, the agent played on its own for more than 95% of the time. The authors report performance comparable to skilled players on several 3D game tasks and competitive scores on standard 2D game benchmarks.
The design choice matters beyond games. Using raw pixels and ordinary input devices, rather than a game's internal data, is the same bet that general computer-use agents such as ChatGPT Work and Claude in Chrome make. Gamer suggests that large amounts of plain keyboard and mouse logs are useful training data for that kind of agent.
The caveats are significant. This is a preprint and not peer reviewed. Most of the detailed evaluation is on miHoYo's own game, where the team has unusual access to data. "Comparable to skilled players" is the authors' own description, and the paper notes that reasoning remains a bottleneck in games that need deeper strategy. Serving costs also limit wider use for now.