Cinnamon_Antonyms_Shot2.PNG

CINNAMON &
ANTONYMS.

Leveraging Brain Activity for Combat Systems

 

How can the player’s cognitive states contribute to game feel?

 

Cinnamon & Antonyms is a top-down, melee-driven roguelike that features layered combat systems. The game interfaces with an electroencephalogram (EEG) to modulate the avatar’s abilities based on the player’s level of concentration.

 

I began this project to apply my background in neuroscience to game design and memorialize my first foray into combat systems. Cinnamon & Antonyms was inspired by two talks at the 2022 Game Developers Conference: Ian Cofino’s “Dreamscaper: Killer Combat on an Indie Budget” and John Muñoz’s “Future Realities Summit: How NASA Has Translated Aerospace Research into Biofeedback Game Experiences.”




Timeline

May 2022 — [Current]

[X] months




My Contribution

Solo development




Tools

Unreal Engine 4
Muse 2
OpenVibe
BlueMuse
Mind Monitor




Contributors

Playtesters
Katherine Buse
Brad Bolman

Audio
Dreamscaper (2020)

OVERVIEW.

WHAT KIND OF INFORMATION CAN WE GET OUT OF BRAIN ACTIVITY?

Understanding What You Can Do Versus What You Should Do

 

In my eleven years (!!) as an aspiring neuroscientist, there have only been a handful of moments when I'm reminded why I care so much about the brain—or rather, why my brain cares so much about herself. I was lucky to capture one of those moments during the early stages of this project, which I’ve clipped below.

A Side Note
for Nerds

It goes without saying that I tried a couple (read: a ton) of signal processing schemes before landing on this one. EEG headsets typically have built-in dimensionality reduction, which means that we’re aggregating signals from billions of neurons into a modest five channels. The main architecture of the processor involves a high-frequency band filter and fast Fourier transform to isolate spike waveforms. The resultant signal is a 5 x t matrix, where t represents the neural manifold over which the Kalman filter estimates the current state. As a general rule, smaller t values allow for higher fidelity but are more computationally intensive and susceptible to noise. Given that the Muse 2 headset samples at a rate of 256 Hz, it’s safe to choose a t value well into the double digits. We can make our math a little easier by multiplying by an empirically-determined 5 x t matrix of decoder weights, so we’re left with a single time-varying float value. And that’s brain control, folks!

Brain control is a longstanding fantasy within our cultural imagination, capturing the attention of tech bros with the budding field of ubiquitous computing and Trekkies with the near-future promise of psychokinesis. It speaks to a level of authority over ourselves and our environment that will likely remain unresolved within our lifetimes.

 

So, why didn’t I make a fully brain-controlled game? Whenever I consider a new feature, I ask myself two questions. Firstly, what portion of the product’s resources (time, money, computational power, or otherwise) does this feature consume? Secondly, how much of my vision for this product can I achieve with this feature?

 

Cinnamon & Antonyms aims to deliver a visceral power fantasy that rewards strategy and improvision. Based on that goal, I'd place brain control in the low-impact range with an astronomically high cost. Even when approaching new games, hardcore gamers have already accumulated a mental model for how they should interface with combat systems. We can capitalize on that model by using genre conventions—such as standard movement controls—instead of incurring the cost of a tutorial level. Peripheral devices also tend to have high latencies and rarely offer the fidelity for fast-paced gameplay. I'm not arguing that brain control in games is always bad—think about how it could be used as an awesome accessibility feature—but it’s not right for this product.

Capturing Brain Activity

To establish a wireless connection to the Muse 2 headset, I used three acquisition servers that leverage different forms of serial communication to successively process the incoming data. BlueMuse streams real-time EEG data through the Lab Streaming Layer (LSL) protocol. The OpenVibe acquisition client filters and writes the input stream to a Transmission Control Protocol (TCP) socket, which feeds into the UE4-BCI plugin.

DataScheme.PNG
Inside the Decoder

Now that we’ve gotten our signal into the engine, we have to decide how we want to interpret it. As shown on the right, the headset’s conductive sensor collects electrical signals from outside of our frontal lobe. Because we’re working with a non-invasive device, the fidelity of our signal is limited to stable, large-scale patterns of activity.

One such EEG pattern is the index of engagement, which describes a user’s level of focus. Researchers at NASA quantified the index of engagement by the formula β / (α + θ), for which beta, alpha, and theta describe discrete wave frequency bands.

HOW CAN WE INDUCE CERTAIN COGNITIVE STATES THROUGH COMBAT?

 

Theory of Flow

To leverage the player’s brain activity as a mechanic, the primary objective of Cinnamon and Antonyms’ gameplay should revolve around inducing changes in the player’s index of engagement. Psychologist Mihaly Csikszentmihalyi characterized the phenomenon of flow as a sense of immersion and energized focus that arises from certain forms of gameplay. He theorized that flow occurs when a game offers focus, freedom, feedback, and challenge. Using the index of engagement as a proxy for flow, I derived six design pillars that capitalize on the player’s cognitive resources to encourage flow states.

Purposeful

Players should be able to
understand the situation, tactically
assess options, and make
strategic decisions on the fly

Improvisational

Combat should feel fluid and
allow for multiple approaches
that can all work intuitively

Tough, But Fair

The players' fate is in their hands,
not the hands of the RNG system.
Player actions allow for correction and enemies feel balanced

Dynamic

Combat should drive variety and replayability through overlapping
systems

Feedback-Driven

Players should understand and
enjoy how their actions impact the
world

Personalized

Players should feel like the
emotional and cognitive state they
bring into the game is reflected by
their avatar

Framing Combat

Genre conventions around the camera position inform players’ expectations of the speed and snappiness of combat. As a rule of thumb, the fidelity of gameplay typically decreases with camera distance—a farther camera reduces the weight of combat and often feels less cinematic, but the macroview provides the player with more information on their positioning. To balance the immediacy of melee combat and macroview required for ranged combat, the camera’s field of view decreases when enemies cluster around Cinnamon and increases when she draws her ranged weapon.

2022-04-30 12-57-30.gif
2022-05-13 15-54-56.gif
2022-05-14 00-13-01.gif
Game Feel and Animation

Four techniques emphasize the weight and readability of attacks.

Favoring Player Input - The player may cancel all attack animations to defensively break out of inputs

 

Secondary Motion - The human eye is attuned to unnatural movement. The cloth simulation on Cinnamon’s scarf alleviates the robotic movement of her base animations

 

Translate Forward - Adding forward momentum connects chains of attacks, gives the impression of powerful impulses, and feels more committal

 

Instant Reactions - The distance of the camera creates a need to convey large differences between hits. Instead of blending between animations, hit reactions jump to the middle of the montage, which allows for clear telegraphing of movement. Hit flashes also provide feedback on collisions