Dev log 002 — Teaching a bot to play Spider-Man

I'm teaching a bot to play Spider-Man in Marvel Rivals by watching me play. It only gets the screen, and it plays through a virtual Xbox controller. It doesn't read game memory or get around anti-cheat, and it only plays in the practice range, never against people.

There are two steps to the plan:

1. Copy me. I record myself with an exact log of every key and mouse movement, then train a model to predict what I'd do next from what's on screen.
2. Copy better players. Train a second model that watches any gameplay video and works out which inputs produced it. If that works, replays of expert players become training data.

This post looks back at the first stretch of work. Step one is starting to work, and step two isn't there yet. I'll take it one visual at a time: what each one shows, and why it matters.

Can it guess what I'd press?

What you're looking at: thirty seconds of me playing, shown three times. All three panels are the same recording. None of them is the bot playing. On the left is what I actually pressed. The middle panel is what an older model, H1, guessed I'd press at each moment, and the right panel is the current model, NitroGen. An amber flash is a press, blue is a held key, and the yellow line traces how the camera turned left and right.

Why it matters: the more a panel's lights match the left one, the better that model knows when to act. I pressed 77 times in this clip. H1 guessed one press, so it had basically learned to sit still. NitroGen guessed 65. That's one clip, and a count isn't a score; the proper test is further down.

What the model actually sees

The game runs at 2560×1440. Shrink the whole screen enough for a small model and the crosshair, which is where the target is, turns into a smudge. So the original design cuts every frame into pieces at different zoom levels and stacks them:

• The whole screen, 10× smaller. It shows the walls, roughly where the dummies are, and whether I'm in the air.
• The crosshair close-up, only 2× smaller, so the target stays readable.
• The ability bar and web-charge box, which show whether a move is available at all.

The current model keeps the idea but only uses the first two views, fed through a pretrained image encoder. The viewer below shows the original stacked layout.

What you're looking at: eight moments from one practice session. Pick one and press play or scrub. The large video is the real recording, and beside it is the stacked, shrunken view the model gets. Underneath is the answer key: which of the 15 actions I was holding (bars) or pressing (ticks), plus how far I turned the camera.

Why it matters:

• The answer key is the target. Every 33 ms, 30 times a second, the model has to guess this row: which buttons, and how many degrees to turn. I converted mouse movement to degrees by timing full 360° turns, which works out to 0.0331° per mouse count. Left and right turns agreed within 0.08%, so the measurement held up.
• Camera only: no key is held, but the view swings about 75° in a second. Turning the camera is an action too, and a bot that only learns buttons can't aim.
• Cut by intake: menus, Alt+Tabs, and pauses are removed before training. The model can't learn to play from a blurred menu.
• The shape of the timeline: long held bars with rare one-step press ticks between them. Presses are rare, and that's why they're hard to predict. It also explains what went wrong next.

Why the old model learned to do nothing

The original model got two inputs: the screen, and what I'd done a moment earlier. The second input was a trap. If I'm holding forward now, I'm almost certainly still holding it 33 ms later, so "repeat the last action" is nearly always right. The model learned that shortcut instead of learning to read the game.

That looked fine in training, because the model was fed my real previous actions. When it had to feed on its own guesses, one idle guess led to another and it stopped doing anything.

What you're looking at: in the old design, the arrow loops back, so the model's own guess becomes its next input. The current design is the fix. There's no action history at all, just the last few screens run through a pretrained image encoder. The only way to predict a press is to notice something on screen.

One quick check made the problem obvious: I blanked out the image input on the history-fed models and the training score barely moved. They weren't looking at the screen.

Did removing it help?

I trained the no-history model and an identical version with history, three times each, and scored them on the same held-out practice sessions.

What you're looking at: a score for predicting presses on six abilities. It's called F1: 1.0 means every press was found at the right moment with nothing made up, and 0 means useless. Each dot is one training run. The dashed line is the old H1 model, and the dotted line is what random guessing scores.

Why it matters: the no-history model scores about 0.30, three times H1, and all three runs agree. The same model with history scores 0.004, below random, because it almost stopped pressing. That confirms the shortcut was the problem. The new model also predicts 93% as many presses as I actually made, so it isn't gaming the score by pressing constantly.

Two caveats. These sessions were reused throughout development, so this shows the result repeats, not that it holds on new footage. And 0.30 is a big improvement, not a good player.

But it still can't aim

What you're looking at: the average camera error per step, split into left/right (yaw) and up/down (pitch). Lower is better. Grey is a lazy baseline that always predicts "don't move the camera." Green is the model.

Why it matters: the model beats the baseline on up/down. On left/right, which means turning toward a target and is most of aiming, it's worse than not moving at all. Averaged together, the model looks 3% better than the baseline. Split apart, the real problem shows up, and it's the main thing between this model and a fight.

Making experiments fit a weekend

Every idea above cost a training run, and a batch of nine runs took just over nine hours on my Mac. That's slow for a weekend project, so I benchmarked the same training recipe on rented GPUs:

The surprise is the two L40S rows: it's the same GPU, and one is 2.4× faster. The AWS machine only had four CPUs around the card, and the rest of the machine was the bottleneck. On Modal a full run was projected at about 36 minutes for about $1.54. That's a projection, not a measured full run. Starting lots of runs at once then hit rate limits and timeouts, so launches needed pacing before parallel runs actually saved time.

Faster compute is what made the three-run confirmation above practical. It couldn't tell me which question was worth another run.

Will it be fast enough to play?

The game doesn't wait. Each decision takes the model about 40–42 ms on my RTX 4080 SUPER. The full loop in a test harness takes 70–72 ms, with a stand-in controller, so real screen capture and pad delivery aren't measured yet. The target is 33 ms, one frame of the model's clock. The controller still has to be mapped to my exact in-game settings too.

Learning from better players

Step two needs a different kind of model: an inverse-dynamics model, or IDM. It watches a short clip and works out which inputs produced it. The player model can only use the past. The IDM is only labelling footage after the fact, so it's allowed to peek a few frames ahead.

What you're looking at: my recorded play trains both models. The labeller's guesses only reach the player if they pass a check on replay footage, and they're stored separately from the real input logs, so a bad guess can't quietly become "truth."

What you're looking at: cyan is where my mouse actually moved the camera, from the log. Amber is the labeller's guess from the pixels alone. When the two arrows point the same way, it's reading my hands from video. Around 3 seconds it gets up/down wrong. Around 9 seconds it shows unknown, meaning it isn't confident enough to guess. That doesn't mean it predicted no movement.

Why it matters: this is the tool that could turn expert replays into training data. The clip is from footage the model trained on, so it's a demonstration, not a test. On held-out footage, its camera error was 0.28° / 0.26° (left-right / up-down), against 0.87° / 0.36° for guessing "no movement." Buttons are much weaker, and it doesn't recover movement keys yet.

The replay problem

Expert footage comes from replays, not from the player's live screen. So before trusting the labeller on it, I need to know whether it gives the same answer for a replay of the same play.

What you're looking at: I recorded 35 seconds of live play, recorded the game's replay of the same 35 seconds, and ran the labeller on both. Each line shows how much the two answers disagree, second by second, and zero would mean identical. The three colours are the replay nudged one frame earlier, lined up, and one frame later.

Why it matters: if the labeller transferred cleanly, these lines would sit near zero. Instead they average 0.82° left/right and 0.26° up/down per step, and nudging the timing doesn't help, so it isn't just a sync error. A small correction trained on top didn't close the gap either. Until this passes, expert replays can't be used as labels. This is one segment I'd already examined, and it measures whether the two readings agree, not which one is right.

Where the first chunk leaves it

An earlier live test proved a learned decision can make it through the controller. A small learned model chose when to fire a web, and it killed the Luna bot in the range, but aim, targeting, and movement were scripted. A whole-session policy has to handle much more of the fight than that.

Nothing learned has played a whole fight yet. The next live step is a small, reviewed check that the scripted controller works with my current account settings. That tests the controller, not learned gameplay. Before I can claim the bot finishes a fight, it needs a working camera-to-controller path, measured in the range.

The comparison at the top is why I'm keeping going: the model is starting to pick up when to act from the screen. Getting it to turn toward the right thing, in time to do something about it, is the next chunk. I'll write that one up when there's something to show.

Numbers and caveats, for the detail-minded

Data. 180.57 admitted range minutes across 10 sessions: 166.93 for fitting, 13.64 frozen for development, and 15.58 in a separate validation take. A sealed test take stays unopened until the final score. Every recording is hash-pinned, and a different model family independently reviewed admission. The video and input logs are synced with a measured +21 ms anchor.

Actions. 15 named actions (forward, left, back, right, jump, web swing, Get Over Here, Amazing Combo, ultimate, melee, Spider-Power, Web Cluster, team-up, pull targeting, simple swing), each held, pressed, or released per 33.3 ms step. Yaw uses the measured 0.0331°/count. Pitch assumes the same sensitivity until the live controller is calibrated.

Views. The original cache stacks 256×144 whole screen, 128×128 crosshair crop, a 200×50 ability row, and a 40×30 web-charge box into 256×352, using a pinned ffmpeg area filter with accurate_rnd+bitexact. The viewer's compact video is a compressed illustration, not the exact tensor. The current candidate uses a frozen NitroGen encoder on a whole-screen view and a crosshair view (256×256 each, a 4×4 spatial grid per view), with temporal heads and no HUD branch. My UID and player name are blurred in the recordings; at model resolution they're a couple of pixels tall anyway.

Press score. Macro F1 over Spider-Power, Web Cluster, Get Over Here, Amazing Combo, Web Swing, and Jump, with ±1-step tolerance, thresholds chosen on training data, and the median camera decoder. The no-history mean is 0.3026 (runs 0.289–0.314), the history-enabled control 0.0041, the older H1 reference 0.1017, and the rate-matched random floor 0.0276. Combined camera error was 1.1831° versus 1.3273° for the control, about 3.4% better than predicting zero motion. The single clip at the top scored 0.370 (NitroGen) versus 0.022 (H1). An exploratory check on starting a press after a quiet period scored 0.134 versus 0.004. Movement onsets stayed weak at 0.03–0.06, and a stricter from-rest slice had too few events to judge.

Compute. Training-only epoch times for one benchmark recipe, not a general GPU ranking. The 13-epoch costs ($1.54 on L40S, $1.43 on A10) are historical projections, not measured runs or current prices.

Latency. CUDA decisions took 40–42 ms, and send age was 70–72 ms in a recorded-input harness with a mock sender. Live capture latency, pad delivery, and the game's frame-rate cost aren't measured yet.

IDM. Offline inference uses ±133 ms of context. The demo video compares movement over the preceding 0.10 s and omits buttons. Held-out button precision at a 0.5 threshold was 0.198 for Combo and 0.120 for Jump, and it still lost to a persistence baseline given the true action history. For replay transfer, a correction fitted on frozen features failed both axes' predeclared agreement and movement checks, and those errors are measured against the live prediction, not ground truth.