And that's it from Glasgow! It was a fun conference, with a lively community & interesting presentations. While games are not common at ACII, still, there were interesting applications in the performing arts! Thank you to all collaborators at @InDigitalGames for their hard work!

By rewarding the generator when it matches an "ideal" arousal curve, the generator learns what to place in different parts of the level. A level that is constantly non-arousing is boring, but so is one constantly arousing. Pacing can be a reward. Paper: matthewbarthet.com/files/PCG…

The generator puts down one track segment at a time, and repairs the track after some actions (making a loop). Then, a simple AI agent plays the track: based on the AI game-states, "arousal" is measured in every part of the track from human arousal labels in similar game-states.

Finally, Matthew Barthet closed the conference on the last day by presenting how we can generate racing tracks that would increase a player's arousal (or follow a specific arousal progression). Again, a corpus of annotated human gameplay sessions was used to "assess" arousal.

We tried to predict engagement in an unseen video of the same game as the game we trained the model on, annotated by unseen participants. Results were mixed: some games were easier to predict than others. Battle Royale games were especially hard. Paper: antoniosliapis.com/papers/va…

We modeled engagement in an ordinal fashion, trying to predict if engagement would increase or decrease in the next time window of this video. We used pre-trained computer vision and audio models and fused them, finally training an SVM on the annotators' consensus on engagement.

At the main conference, Kosmas Pinitas presented his work on predicting viewers' engagement from gameplay videos and audio alone (without face cams or sensors). To do this, 20 participants annotated their engagement when watching 2 hours of First Person Shooter gameplay videos.

As expected, maximizing arousal (predicted or not) didn't work well. When combined with in-game rewards it 𝘴𝘰𝘮𝘦𝘵𝘪𝘮𝘦𝘴 did 𝘰𝘬. Pushing the agent to explore more states (not get stuck in one "arousing" game-state) is an important next step. Paper: antoniosliapis.com/papers/af…

We can use this predicted emotion (based on humans' reported arousal at similar game-states) as a reward for a reinforcement learning agent. While this doesn't work alone, we can combine it with an in-game reward (such as going quickly in the right direction for a racing game).

The 3 games feature short game sessions (max 2 minutes) but have over 120 sessions from different players who annotated their arousal levels while playing (on a recording of their play). We match game-state and arousal to find how an AI agent would "feel" in a similar game-state.