poodleface

That was my question as well. The only thing I found via search is references to a press release.

It's fine to be excited about a potential breakthrough, but "solve" is putting it very strongly. Test this with players of varying experience in VR and I'll believe you.

I used to work in an academic research lab doing game research and you correctly identify the lack of quantitative data (e.g. examining gameplay logs across players) to compliment qualitative findings (interviews with players). The interviews and other qualitative measures are important because they give context to what actually happened, but you need both parts to get the whole picture.

In games research it is difficult to analyze gameplay data unless you build the game yourself (which I did a few times) or use a game with a rich replay file that can be reverse engineered to gather player data (there is a lot of research on the original Starcraft for this reason).

My main criticism of games research is that many experimenters frequently don’t understand the mechanical nuances of the games they are using to test (the OPs here are not part of this statement, they know what they are doing!).

An older study might do something like test Rise of Nations to determine cognitive gains and then come up with a generalized finding like “playing action-based video games over sustained time lead to cognitive gains”. It doesn’t get to what aspects of the gameplay led to the benefit (is it managing multiple units in off-screen spaces, is it forward projection of outcomes of actions, the amount of active attentional demand, etc). I think there is a path for hard findings, but that takes a lot more work that is honestly much more time consuming to do.

Thanks for doing this. Big fan of your updates on Facebook.

Can you give any insight as to how much people generally pay when they can “name your own price”? Anything surprising you learned from going in this direction?

If you were the only gamer in the lab, can you shed some light on the process of choosing which particular "action video game" to use in a study? Focus groups? Prior work using the same games?

This is a good call-out. It’s definitely an engineering challenge. Ultimately, what we usually did was being smart about the variables we were capturing. While a game was running, we would frequently capture aggregate calculations (the result of a series of actions) as a single event rather than capture the raw input stream and generating these measures later (we’d batch them in the background and push them to a server periodically when cycles were available). That made our capture lossy, but it was more performative and it made analysis much, much easier. If we missed something it was usually more cost-effective to run more participants (the benefits of an academic recruiting pool, class credit is cheap and students usually are excited to do “games research”).