AI and Gaming Research Summit

While musical AI has only recently begun to enter industry, AI-based systems have been creating music for over 70 years. We will showcase some of the most exciting musical AI systems from industry and academia, spanning composition and improvisation. Opportunities at the intersection of musical AI and gaming will be discussed.

Dr. Maya Ackerman is an award-winning Artificial Intelligence expert, named “Woman of influence” by the Silicon Valley Business Journal. Ackerman is a Computer Science and Engineering Professor at Santa Clara University and the CEO/Co-founder of WaveAI. Her research, focusing on Computational Creativity and Cluster Analysis, has earned awards from the Association for Computational Creativity, US Office of Naval Research, Natural Sciences and Engineering Research Council of Canada, and many more. A sought-after speaker, Dr. Ackerman had been an invited speaker at the United Nations, Google, IBM Research, and Stanford University, amongst others. She earned her PhD from the University of Waterloo and held postdoctoral fellowships at Caltech and UC San Diego. Maya is also a singer and music producer.

Many people use online gaming to socialize, both to relieve stress in a casual setting and to nurture lasting relationships. These goals are best met in healthy online communities that foster trust among their members, as gamers must weigh risks and benefits using their understanding of the community. In this talk, we examine the relationship between trust and social interaction across different gaming communities, grounding the discussion in both a large, on-platform survey of gamers and an automatic analysis of self-disclosure on English-language gamer forums. We contrast in-game experiences with activity in fan forums, where public perceptions of games are often formed and shared. For online gaming to effectively bridge geographic and demographic differences, and thus sustain healthy social bonds, platforms need to offer better affordances that both support different modes of self-disclosure and help gamers manage the possible impacts of doing so.

Maria Antoniak is a researcher and data scientist working in natural language processing, computational social science, and cultural analytics She is currently a PhD Candidate in Information Science at Cornell University and has a master’s degree in Computational Linguistics from the University of Washington. Her work focuses on modeling how people communicate about their subjective experiences through text, especially when these communications occur in online communities.

One of the big concerns around remote and hybrid work is that companies and the people who work there may be losing “social capital,” as their social ties weaken. Gaming is one way that co-workers can maintain informal social ties that support their relationships. This talk reports on an X-Box console survey of approximately 11,000 people who say they game with co-workers, asking how they think this affects their job satisfaction and their productivity. We find people do say gaming with coworkers helps them build their social capital, especially those who report gaming more since the pandemic began.

Nancy Baym studies how people understand and act with new communication technologies in their relationships. A pioneer in the field of internet research, Baym wrote some of the first articles about online community in the early 1990s. With Jean Burgess, she is the author of Twitter: A Biography (2020, NYU). Other books include Playing to the Crowd: Musicians, Audiences, and the Intimate Work of Connection (2018, NYU), Personal Connections in the Digital Age (2010, Second Edition 2014, Polity), Internet Inquiry: Conversations About Method (co-edited with Annette Markham, 2010, Sage), and Tune In, Log On: Soaps, Fandom and Online Community (2000, Sage).

She was a co-founder of the Association of Internet Researchers and served as its second president. She has been recognized with the Frederick Williams Prize for Contributions to the Study of Communication and Technology awarded by the International Communication Association, the naming of the Nancy Baym Book Award by the Association of Internet Researchers, and an Honorary Doctorate from the Faculty of Information Technology at the University of Gothenburg. Most of her papers and more information are available at nancybaym.com.

Welcome to day 2

Christopher Bishop is a Microsoft Technical Fellow and Laboratory Director of the Microsoft Research Lab in Cambridge, UK.

He is also Professor of Computer Science at the University of Edinburgh, and a Fellow of Darwin College, Cambridge. In 2004, he was elected Fellow of the Royal Academy of Engineering, in 2007 he was elected Fellow of the Royal Society of Edinburgh, and in 2017 he was elected Fellow of the Royal Society.

At Microsoft Research, Chris oversees a world-leading portfolio of industrial research and development, with a strong focus on machine learning and AI, and creating breakthrough technologies in cloud infrastructure, security, workplace productivity, computational biology, and healthcare.

Chris obtained a BA in Physics from Oxford, and a PhD in Theoretical Physics from the University of Edinburgh, with a thesis on quantum field theory. After his PhD he joined the Theoretical Physics Division of Culham Laboratory where he conducted research into the physics of magnetically confined fusion plasmas. During this time he developed an interest in machine learning, and became Head of the Applied Neurocomputing Centre at AEA Technology. He was subsequently elected to a Chair in the Department of Computer Science and Applied Mathematics at Aston University, where he set up and led the Neural Computing Research Group.

Chris is the author of two highly cited and widely adopted machine learning text books: Neural Networks for Pattern Recognition (1995) and Pattern Recognition and Machine Learning (2006). He has also worked on a broad range of applications of machine learning in domains ranging from computer vision to healthcare. Chris is a keen advocate of public engagement in science, and in 2008 he delivered the prestigious Royal Institution Christmas Lectures, established in 1825 by Michael Faraday, and broadcast on national television.

Chris is a member of the UK AI Council. He was also recently appointed to the Prime Minister’s Council for Science and Technology.

I am a Senior Researcher at Microsoft Research New England. My research focuses on developing systems that expand access to information for people with disabilities, in particular sign language users and low-vision readers. My work is interdisciplinary, combining Accessibility, Human-Computer Interaction, and Applied Machine Learning. I take data-driven approaches to address accessibility problems, helping to make the world a more equitable place. I recently completed my PhD in Computer Science at the University of Washington advised by Richard Ladner, and hold an AB in Applied Mathematics from Harvard.

My diverse past research projects have spanned data visualization, computational biology, computer music, applied mathematics, and network protocols.

Incorporating machine learning and reinforcement learning into videogames is a huge opportunity and a reality for a number of top titles today. However, these techniques produce black-box AI agents whose behavior is often inscrutable, and challenging for designers to understand or customize. This talk will describe three projects that study this gap between learned AI agents and game design. First, we present tools inspired by interactive programming and generative testing to visualize learned agent behaviors. Second, we describe algorithms to allow designers to calibrate agents while learning to express desired behaviors (presented at ICML 2020). Third, we showcase new environments (built on the RTS game 0 A.D.) to study how game developers can explore the game design/balance space when using AI agents. Throughout we will highlight opportunities for open collaboration with Microsoft Research in developing designer-friendly learning techniques for use in videogames.

Brian Broll is a Research Scientist at the Institute for Software Integrated Systems at Vanderbilt University. His research interests include artificial intelligence, model integrated computing, and computer science education. His current research projects include AI-assisted design of cyber-physical systems and leveraging computational thinking and other disciplines. After completing his Ph.D. in Computer Science from Vanderbilt University in 2018, he was an AI Resident at Microsoft Research where he was largely focused on topics around imitation and reinforcement learning. Upon completion of the residency, he returned to Vanderbilt to expand on his prior work.

This talk looks at how game developers and researchers frame procedural content generation (PCG) as automation of creativity in the games industry. By producing scalable results with combinatorial diversity, PCG are touted as the future of content, yet flouted as the harbinger of technological unemployment in the future of creative work. However, the real danger of automating creativity is not job loss per se, but the constitution of an underclass of technical artists, writers, and musicians, whose cognitive work are deemed “manual” and doomed to maintain and be managed by algorithms. This cognitive underclass of creative work—disproportionately performed by racialized, minoritized, feminized, and outsourced labour—is seen as derivative, expendable, and interchangeable with automatic processes. If the future of digital creativity is algorithmic, this project asks how futures thinking can be reoriented through plural valuations of human capability towards social justice.

Aleena Chia is an ethnographer of gaming cultures. She is an assistant professor in the School of Communication at Simon Fraser University. She researches digital labour in video game play and production cultures. She received her PhD from Indiana University Bloomington in 2017 and was a postdoctoral researcher at the Academy of Finland’s Centre of Excellence in Game Culture studies in 2018.

Fireside chat between Peter Lee (Corporate Vice-President, Research and Incubations, Microsoft) & Kareem Choudhry (Corporate Vice-President, Cloud Gaming, Microsoft)

Kareem Choudhry is a Microsoft Corporate Vice President in the Xbox division, leading xCloud Gaming. The project’s mission is to put gamers at the center enabling them to play anywhere, from any device.

Kareem graduated from the University of Michigan with a Computer Engineering degree. He joined Microsoft in 1998 as a Software Engineer on the Outlook team. Gaming is his life-long passion. That led him to work on Windows & DirectX to deliver game platforms. He later joined Xbox in 2005 and has been an instrumental leader in Xbox Engineering and in driving the evolution of gaming at Microsoft. Kareem prides himself on being at the edge of technology and consumer innovation. Gaming and Project xCloud represents the perfect opportunity and intersection of leading-edge technology and pushing new customer/business scenarios beyond what’s there today. His leadership style is guided by his core values of integrity, empowerment, & inclusion.

Game development involves lots of languages that we both speak with and think in: languages for players, for testers, for producers, for programmers, for designers, and for every other job title you can think of. If autonomously creative AI start joining the games industry, we might end up adding even more new languages to that list. But what if our AI design assistants could speak and write in a language we already understand, like program code? In this talk, we’ll look at what it means to have an AI that reads code that you’ve written, and writes its own. We’ll see how that might be a useful bridge between AI game designers and real-world modern game development, and we’ll also explore the weird new skills we might have to learn to write code that an AI can read.

Michael Cook is an AI researcher and game designer, currently working at Queen Mary University of London, where he holds a Royal Academy of Engineering Research Fellowship. His research lies at the intersection of computational creativity, generative systems and game design; building AI that design games autonomously, and AI that help people design games themselves. He is the developer of the game-creating AI ANGELINA, the founder of the Procedural Generation Jam, PROCJAM, and co-author of the MIT Press book Twitterbots. He very narrowly missed out on holding a speedrun world record once.

In collaboration with Ninja Theory, we are exploring multi-agent learning in their latest game, Bleeding Edge, which is a perfect testing ground for reinforcement learning agents trained to collaborate in teams. Whilst past work has often assumed we will be in control of all agents, if we want our agents to play well with any human they need to adapt quickly online. In this talk we will formalise the problem of ad-hoc teamwork and present our proposed approach to meta-learn policies robust to a given set of possible future collaborators.

My long term goal is to create autonomous agents capable of intelligible decision making in a wide range of complex environments with real world applications. In particular, my passion is using digital games to push boundaries in the capabilities of modern AI and making state of the art methods accessible to encourage a new generation of intelligent games. To achieve these goals, my research currently focuses on machine learning, artificial intelligence, digital games and player experience.

Previously, I received an MEng degree in Computer Systems and Software Engineering from the University of York in 2009 including a year working with the human factors team at BAE Systems. After completing this degree I worked on traditional commercial game AI, integrating behaviour trees and nav mesh generation into the open-source game engine CrystalSpace as part of the Google Summer of Code program in 2009 and again in 2012. In 2013, I completed my PhD on multi-agent reinforcement learning at the University of York and visited Oregon State University funded by a Santander International Connections Award. Before starting my current role at Microsoft Research, I was a Research Associate from 2013-2015 on the EPSRC funded New Economic Models and Opportunities for digital Games project working on data mining for collective game intelligence, and then a research fellow/lecturer in the Digital Creativity Labs where I led a team focussed on integrating decision making AI (including deep reinforcement learning and Monte Carlo tree search) into modern commercial videogames.

We know how to make AI agents that optimize a specified reward. But when AI agents need to interact with people, the way we formulate the problem needs to change, and, with it, the algorithms we use to generate the agent’s behavior. First, interactive AI agents take actions not just in isolation, but in spaces where we, humans, act as well. Agents thus have to choose their actions in a way to coordinates well with ours. Second, agents that assist us need to do what we want them to do: help us with what we want, how and how much we want to be helped. My research group’s missing is to formalize and algorithmically solve the problem of AI action not in isolation, but in coordination with and assistance of people — AI action for and around people.

I am an Assistant Professor in the EECS Department at UC Berkeley. My goal is to enable robots to work with, around, and in support of people. I run the InterACT Lab, where we focus on algorithms for human-robot interaction — algorithms that move beyond the robot’s function in isolation, and generate robot behavior that coordinates well with people, and is aligned with what we actually want the robot to do. We work across different applications, from assistive arms, to quadrotors, to autonomous cars, and draw from optimal control, game theory, reinforcement learning, Bayesian inference, and cognitive science. I also helped found and serve on the steering committee for the Berkeley AI Research (BAIR) Lab, and am a co-PI of the Center for Human-Compatible AI. I’ve been honored by the Sloan Fellowship, MIT TR35, the Okawa award, an NSF CAREER award, and the PECASE award.

I currently teach Data Structures and Parallel Programming (GPGPU) and have previously taught 3D graphics, statistics and programming to a range of cohorts. I hold a BSc in Physics and PhD in Ecological Modelling and my ongoing research interests are the development of modelling and visualization frameworks to understand complex systems. My innovations include developing the first theoretical model and visualization framework for indeterminate organisms. These modelling and visualization skills have been applied across disciplines to develop interactive visual simulations of urban sustainability, heat loss and gain from built environment and precision agriculture. I am interested in applying games technology including games engines, graphics hardware and related infrastructure, to develop playable models of complex systems. Current areas of application are the Water-Energy-Food-nexus, Microbial Ecology and Health and Social Care. I am interested in the broader applications of game technology to develop intuitive systems models. I am an editorial board member for Scientific Reports, have over 55 peer reviewed articles and have secured research income over 1M. I am a Co-I and PI on several RCUK and charity-funded grants.

In recent years we have seen fast progress on a number of zero-sum benchmark problems in AI, e.g. Go, Poker and Dota. In contrast, success in the real world requires humans to collaborate and communicate with others, in settings that are, at least partially, cooperative. Recently, the card game Hanabi has been established as a new benchmark environment to fill this gap. In particular, Hanabi is interesting to humans since it is entirely focused on theory of mind, i.e., the ability to reason over the intentions, beliefs and point of view of other agents when observing their actions. This is particularly important in applications such as communication, assistive technologies and autonomous driving.

We start out by introducing the zero-shot coordination setting as a new frontier for multi-agent research, which is partially addressed by Other-Play, a novel learning algorithm which biases learning towards more human compatible policies.

Jakob Foerster received a CIFAR AI chair in 2019 and is starting as an Assistant Professor at the University of Toronto and the Vector Institute in the academic year 20/21. During his PhD at the University of Oxford, he helped bring deep multi-agent reinforcement learning to the forefront of AI research and interned at Google Brain, OpenAI, and DeepMind. He has since been working as a research scientist at Facebook AI Research in California, where he will continue advancing the field up to his move to Toronto. He was the lead organizer of the first Emergent Communication (EmeCom) workshop at NeurIPS in 2017, which he has helped organize ever since.

Minecraft has embarked on a two-year long project to ramp up its data science capabilities and close the ‘last mile gap’ between algorithms and personalized experiences in one of the world’s largest video games. We will cover key focus areas and approaches to enable a modern Machine Learning function, as well as the strategic mechanisms that align Minecraft’s personalization efforts to the studio philosophy that “Minecraft is for everyone”.

Abby has been playing video games since she was a small child demanding that people “play!” King’s Quest on the Tandy 1000. While getting her PhD she engaged in epic Warcraft III competitions versus the other graduate students in her department. Now she’s found a way to combine her love of analysis and her love of video games as the manager of the Data Science team at Minecraft.

In collaboration with Ninja Theory, we are exploring multi-agent learning in their latest game, Bleeding Edge, which is a perfect testing ground for reinforcement learning agents trained to collaborate in teams. Whilst past work has often assumed we will be in control of all agents, if we want our agents to play well with any human they need to adapt quickly online. In this talk we will formalise the problem of ad-hoc teamwork and present our proposed approach to meta-learn policies robust to a given set of possible future collaborators.

I am a Research Software Development Engineer based in the Microsoft Research Lab in Cambridge, UK. I take a keen interest in projects related to the field of reinforcement learning.

I have completed my undergraduate studies in computer science, followed by a Masters degree in big data analytics at UCL in London in 2017. During my Masters, I nurtured my interest in machine learning, taking on modules in computer vision, natural language processing, information retrieval, supervised learning, as well as applied machine learning and optimization. In my Masters thesis I focused on distributing the training of a computer vision algorithm using the Azure cloud technologies, as part of a summer internship with the MSR Lab in Cambridge, UK. During my studies, I have also taken a one year software engineering internship as part of the query formulation team within Bing, London.

Although deep reinforcement learning has led to breakthroughs in many difficult domains, these successes have required an ever-increasing number of samples, affording only a shrinking segment of the AI community access to their development. Resolution of these limitations requires new, sample-efficient methods. To facilitate research in this direction, we propose and ran the second iteration of the MineRL Competition on Sample Efficient Reinforcement Learning using Human Priors. The primary goal of the competition is to foster the development of algorithms which can efficiently leverage human demonstrations to drastically reduce the number of samples needed to solve complex, hierarchical, and sparse environments. To that end, participants compete under a limited environment sample-complexity budget to develop systems which solve the MineRL ObtainDiamond task in Minecraft, a sequential decision making environment requiring long-term planning, hierarchical control, and efficient exploration methods. The competition is structured into two rounds in which competitors are provided several paired versions of the dataset and environment with different game textures and shaders. At the end of each round, competitors submit containerized versions of their learning algorithms to the AIcrowd platform where they are trained from scratch on a hold-out dataset-environment pair for a total of 4-days on a pre-specified hardware platform. In this follow-up iteration to the NeurIPS 2019 MineRL Competition, we implement new features to expand the scale and reach of the competition. In response to the feedback of the previous participants, we introduced a second minor track focusing on solutions without access to environment interactions of any kind except during test-time. Further we aimed to prompt domain agnostic submissions by implementing several novel competition mechanics including action-space randomization and desemantization of observations and actions. In this talk we will summarize the results and insights developed over the course of the competition, present on the top competitors’ solutions, and make suggestions for RL competitions more broadly.

I am a computer scientist and mathematician from Salt Lake City, Utah. I’m currently doing my PhD in deep learning theory and deep reinforcement learning at Carnegie Mellon University under Ruslan Salakhutdinov. A large portion of my time is spent working on MineRL, a project I co-founded at CMU, with the broad goal of developing general AI in Minecraft using human priors.

Matthew Guzdial is an Assistant Professor in the Computing Science department, a Canadian CIFAR Chair, and an Alberta Machine Intelligence Institute (Amii) fellow. His research focuses on the intersection of machine learning, and creativity, including machine learning-based automated game generation, human-AI design collaboration, and novel transfer learning methods. His work has been supported through Mitacs and NSERC, and has been featured in the BBC, WIRED, and Popular Science.

Research at the intersection of Gaming and AI is extremely timely and fruitful. Not only are games an important testbed for driving AI development, but advances in this space also have the potential to impact how we develop and play games, and a wide range of real-world applications that go beyond gaming. In this talk, I will give an overview of the many ways in which research done at, and in collaboration with, Microsoft Research is leading to novel research insights, as well as their current and future potential for impact in gaming and beyond.

I am a Principal Researcher and lead of Game Intelligence at Microsoft Research Cambridge. My team and I advance the state of the art in reinforcement learning, driven by current and future applications in video games. We share the belief that games will drive a transformation of how we interact with AI technology. My long-term goal is to develop AI systems that learn to collaborate with people, to empower their users and help solve complex real-world problems.

Human-AI co-creation offers exciting new possibilities for games and game creators. If we consider co-creation directly between players and non-player characters (NPCs), co-creating the game-play together, we can imagine novel game mechanics like improvisation with NPCs. Improvisation is supremely challenging for many reasons, but we demonstrate some initial successes improvising with a virtual agent in dance and improv theatre games. On the other hand, at a meta level, if we consider co-creation between game creators and AI systems, we can imagine game development and design workflows where industry professionals and AI systems co-create game agents together. In order to better understand the opportunities for AI to support game agent creators in this fledgling space, we interviewed developers and designers from the game industry and research labs. Based on the challenges and opportunities we surfaced, we also discuss early research to focalise the agent creator as a central part of a co-creation workflow using reinforcement learning.

I am fascinated by the role artificial intelligence (AI) can play to enable and augment human creativity. I practice a human-centered AI research methodology — applying a combination of design, human-computer interaction, and AI/machine learning methods — to explore questions about AI and creativity. My current research focuses on understanding how AI can best support designers and developers with the creation of engaging game agents (non-player characters and bots) in real-world commercial games.

I received a Ph.D. in Computer Science (2019) from the Georgia Institute of Technology (Atlanta, USA) with the Expressive Machinery Lab. My dissertation investigated the effects of creative arc negotiation — a novel real-time decision-making paradigm for improvisation between people and computers — on player experience within VR games for improvisational theatre. I have previously studied the application of human-computer co-creativity in problems ranging from improvisational dance and pretend play to music recommendation. I received an M.S. in Computer Science (2013) from the Georgia Institute of Technology and a B.E. in Computer Science Engineering (2011) from the Manipal Institute of Technology (Manipal, India).

Fireside chat between Peter Lee (Corporate Vice-President, Research and Incubations, Microsoft) & Kareem Choudhry (Corporate Vice-President, Cloud Gaming, Microsoft)

Dr. Peter Lee is Corporate Vice President, Research and Incubations, at Microsoft. He leads Microsoft Research across its eight laboratories around the world. He also oversees several incubation teams for new research-powered lines of business, the largest of which today is Microsoft’s growing healthcare and life sciences effort. Dr. Lee has extensive experience in managing fundamental research to commercial impact in a range of areas, spanning artificial intelligence, to quantum computing, to biotechnology, and more. Before joining Microsoft in 2010, he was at DARPA, where he established a new technology office that created operational capabilities in machine learning, data science, and computational social science. From 1987 to 2005 he was a Professor at Carnegie Mellon University, and from 2005 to 2008 the Head of the university’s computer science department. Today, in addition to his management responsibilities, Dr. Lee speaks and writes widely on technology trends and policies. He is a member of the National Academy of Medicine. He serves on the Boards of Directors of the Allen Institute for Artificial Intelligence, the Brotman Baty Institute for Precision Medicine, and the Kaiser Permanente Bernard J. Tyson School of Medicine. In public service, Dr. Lee was a commissioner on President Obama’s Commission on Enhancing National Cybersecurity and led several studies for both PCAST and the National Academies on the impact of federal research investments on economic growth. He has testified before both the US House Science and Technology Committee and the US Senate Commerce Committee.

Mahjong is a very popular game worldwide, especially in Asia Pacific region. As a multi-player imperfect game, it is more complicated than Go and Texas hold’em poker. We designed several new technologies, which enabled our Mahjong AI Super Phoenix (or Suphx) to outperform top human professional players.

Suphx was tested on Tenhou, the most famous online Mahjong platform in Japan. After playing over 5000 games against human-players in the expert room, which is the only room AI is allowed to play in, Suphx successfully achieved 10 DAN, the highest level in that room, and becomes the first and the only Mahjong AI achieving 10 DAN. So far it is the strongest Mahjong AI in the world and ranked higher than 99.9% players in Tenhou.

Suphx has been highly recognized by the best human players. One of the highest-ranking players wrote a book “Suphx’s Impact” dedicated to Suphx, which analyzes and summarizes Suphx’s novel style and is one of the best-selling books about Mahjong tactics.

On one hand, Suphx can teach and help human players to improve their Mahjong skills; one the other hand, our techniques designed in Suphx can also be applied in real-world complex decision applications such as financial investment.

Junjie Li is a Senior Research SDE in Machine Learning Group, Microsoft Research Asia. His recent work focused on game AI with deep reinforcement learning.

Concluding Remarks

I lead the global outreach team at Microsoft Research (MSR) – our mission is to amplify the impact of research at Microsoft and to advance the cause of science and technology research around the world. Together, we drive strategy and execution for MSR engagement with the rest of Microsoft and with the broader science and technology community. We invest in high-impact collaborative research projects on behalf of the company, create pipelines for diverse, world-class talent, and generate awareness of the current and envisioned future impact of science and technology research.

Prior to joining Microsoft, I served as the founding chief of the Intelligent Systems Center at the Johns Hopkins Applied Physics Laboratory (APL), where I directed research and development in artificial intelligence (AI), robotics and neuroscience and created APL’s first enterprise-wide AI strategy and technology roadmap. During my two decades at APL, I led interdisciplinary teams in developing novel AI technologies from concept to real-world application with a focus on autonomous systems. My background is in machine learning and signal processing and current research interests include reinforcement learning for real-world systems, machine decision-making under uncertainty, human-machine teaming, and practical AI safety.

As a subject matter expert in AI and autonomous systems, I’ve served on advisory boards and strategic studies for the U.S. Department of Defense, the U.S. Department of Energy and the National Academy of Sciences. I was recently nominated by the White House Office of Science and Technology Policy to serve as an AI expert on the Global Partnership on AI and was elected to serve as the Science Representative on its inaugural steering committee. Alongside my career in engineering, I have pursued a parallel career as a hip-hop artist and serve as a voting member of the Recording Academy, the institution that organizes the Grammys.

Creating inclusive and positive experiences for players in the metaverse requires safety by design which creates opportunities to apply artificial intelligence (AI) solutions. In this talk we will look at how AI can be applied to help foster a safe and civil community. We will also discuss how to ensure that AI implementations are aligned with content policy evolution and moderation processes.

Remy Malan serves as both VP of Trust & Safety and Chief Privacy Officer at Roblox. In addition to implementing best practices to ensure a civil environment for players and creators, he leads Roblox’s customer care, content moderation, and online safety initiatives, developing industry-leading technology to promote positive online behavior among kids and teens. Remy also leads the company’s Privacy Office, promoting its mission of protecting the privacy of the Roblox community.

Remy brings over 25 years of industry experience to the executive team, having built his reputation at Silicon Valley organizations like SugarCRM, Teleplace, and AOL Time Warner. Remy is passionate about creating great experiences for customers and finding new ways to engage with them.

Remy received his Bachelor of Science (S.B.) and Master of Science (S.M.) from the Massachusetts Institute of Technology.

Simulating the decision-making behavior of diverse groups of humans within a shared environment has long been a grand challenge of artificial intelligence, as well as of great interest for game design and social science research. However, recent advances in data-driven machine intelligence have not brought much new to bear on this set of important challenges, in part because of the diversity of contexts — the traffic patterns of people commuting to work is a very different context (with correspondingly different prediction problems) from how people formulate and revise opinions of one another through conversation, for example. In this talk, we present an analysis and taxonomy of social simulation systems, and outline our ongoing work on distilling a set of common, reusable abstractions for social behaviors in virtual environments.

Chris Martens is an assistant professor of Computer Science at NC State University, where they lead the Principles of Expressive Machines (POEM) Laboratory in research on digital games and interactive narrative. Their research advances the ability for creative thinkers of all backgrounds to build and use interactive systems as “tools for thought,” which in practice means advancing programming language design, procedural content generation, social AI, and our understanding of the relationships between human cognition and the formal affordances of media that simulates familiar environments and situations. Their work has been supported by the NSF, the Laboratory for Analytic Sciences, and the Air Force Office of Scientific Research, and has been recognized with a Best Paper award at the International Conference on Interactive Digital Storytelling and an NSF CAREER Award.

Matchmaking has changed little since it was first automated. The general approach is to look for an ideal match across one or more preferred metrics, and then expand the ideal until a match is found. Issues with this approach make it difficult to trade-off the importance of each metric, hard to customize to specific player populations including geographic ones, and expanding forces players to wait longer than necessary. This session will present TrueMatch, a new matchmaking approach that allows developers to more intuitively express the value of each metric, and then uses machine learning to automatically optimize over the desired metrics in real-time. The results give better matches in less time and are customized to each player’s characteristics and each region’s real-time concurrency as it changes over time.

Josh Menke has spent the last 15+ years designing skill, matchmaking, and ranking systems, primarily for the Halo, Blizzard, and Call of Duty franchises. He is now working as an engagement designer on Halo Infinite, designing the flows and systems that will help players find and engage with what they will most enjoy. He is also running the Live team for Halo 5. Josh holds a PhD in Computer Science specializing in skill systems and neural networks.

Although deep reinforcement learning has led to breakthroughs in many difficult domains, these successes have required an ever-increasing number of samples, affording only a shrinking segment of the AI community access to their development. Resolution of these limitations requires new, sample-efficient methods. To facilitate research in this direction, we propose and ran the second iteration of the MineRL Competition on Sample Efficient Reinforcement Learning using Human Priors. The primary goal of the competition is to foster the development of algorithms which can efficiently leverage human demonstrations to drastically reduce the number of samples needed to solve complex, hierarchical, and sparse environments. To that end, participants compete under a limited environment sample-complexity budget to develop systems which solve the MineRL ObtainDiamond task in Minecraft, a sequential decision making environment requiring long-term planning, hierarchical control, and efficient exploration methods. The competition is structured into two rounds in which competitors are provided several paired versions of the dataset and environment with different game textures and shaders. At the end of each round, competitors submit containerized versions of their learning algorithms to the AIcrowd platform where they are trained from scratch on a hold-out dataset-environment pair for a total of 4-days on a pre-specified hardware platform. In this follow-up iteration to the NeurIPS 2019 MineRL Competition, we implement new features to expand the scale and reach of the competition. In response to the feedback of the previous participants, we introduced a second minor track focusing on solutions without access to environment interactions of any kind except during test-time. Further we aimed to prompt domain agnostic submissions by implementing several novel competition mechanics including action-space randomization and desemantization of observations and actions. In this talk we will summarize the results and insights developed over the course of the competition, present on the top competitors’ solutions, and make suggestions for RL competitions more broadly.

Stephanie Milani is a Ph.D. student in the Machine Learning Department at Carnegie Mellon University. She is advised by Dr. Fei Fang and her research interests include sequential decision-making problems, with an emphasis on reinforcement learning. In 2019, she completed her B.S. in Computer Science and her B.A. in Psychology at the University of Maryland, Baltimore County, and she co-organized the 2019 MineRL competition. Since 2016, she has worked to increase the participation of underrepresented groups in CS and AI at the local and state level. For these efforts, she has been nationally recognized through a Newman Civic Fellowship.

This talk will describe how Microsoft Flight Simulator generates a 3D model of Earth using AI to generate a digital twin. Using over 2PB of data from Bing Maps, developers use ML and a grammar system to essentially re-populate the planet every 72 hours, procedurally planting somewhere in the realm of 2 trillion trees and creating 2 billion buildings. Additional emphasis will be how the 3D representation of Earth is transformed into a 4D digital twin model via a dynamic time-of-day / time-of-year system that is augmented with a sophisticated weather forecast system.

Jorg Neumann is the lead for Microsoft Flight Simulator.

Many people use online gaming to socialize, both to relieve stress in a casual setting and to nurture lasting relationships. These goals are best met in healthy online communities that foster trust among their members, as gamers must weigh risks and benefits using their understanding of the community. In this talk, we examine the relationship between trust and social interaction across different gaming communities, grounding the discussion in both a large, on-platform survey of gamers and an automatic analysis of self-disclosure on English-language gamer forums. We contrast in-game experiences with activity in fan forums, where public perceptions of games are often formed and shared. For online gaming to effectively bridge geographic and demographic differences, and thus sustain healthy social bonds, platforms need to offer better affordances that both support different modes of self disclosure and help gamers manage the possible impacts of doing so.

Alexandra Olteanu is a computational social science and social computing researcher. Currently, she is a Principal Researcher in the Fairness, Accountability, Transparency and Ethics (FATE) Group. Prior to joining the FATE group, she was a Social Good Fellow at the IBM T.J. Watson Research Center, NY. She is interested in how data biases and methodological limitations delimit what we can learn from online social traces, and how we can make the systems that leverage such data safer, fairer, and generally less biased. The problems she tackles are often motivated by existing societal challenges such as hate speech, racial discrimination, climate change, and disaster relief. Her work has won two best paper awards (WISE 2014, Eurosys’ SNS workshop 2012), and has been featured in the UN OCHA’s “World Humanitarian Data and Trends” and in media outlets such as The Washington Post, VentureBeat, and ZDNet. Alexandra has served on the program committees of the main social media and web conferences, including ICWSM, WWW, WebSci, CIKM, and SIGIR, on the steering committee of the new ACM Conference on Fairness, Accountability, and Transparency (FAT*), and as the Tutorial Co-chair for ICWSM 2018 and FAT* 2019. Alexandra holds a PhD (2016) from École Polytechnique Fédérale de Lausanne (EPFL), Switzerland. She draws her experience from academic institutions and research labs across 5 different countries.

Embark Studios procedural artist Anastasia Opara shares a sneak-peek into Embark’s experimental project Kittiwake, where our team set out to explore a feeling of co-creation with a procedural system in a dialogue-like interaction. We embodied the example-based procedural placement algorithm into a little creature, whom we named Kittus. Kittus tries to assist you by mimicking the way you setdress a scene: the agent analyses neighborhoods of the user-placed objects and finds similar-looking locations in the level. We will briefly talk about the UX goals of example-based synthesis and dive deep into the technical details of how example-based procedural placement works.

Anastasia is a Procedural Artist at Embark Studios in Stockholm. Before that, she was a part of SEED, an R&D group at Electronic Arts.

Anastasia comes from a family of artists, and studied graphic design, photography and 3D visual arts, aspiring to become an artist too. She then discovered the power of programing, mathematics, and statistics, which allowed her to capture her own artistic processes and thinking and imbue them into algorithms. Anastasia still thinks of herself as an artist, but her brushes are mostly code nowadays.

Diego Pérez-Liébana is a Senior Lecturer in Computer Games and Artificial Intelligence at Queen Mary University of London (UK). He holds a Ph.D in Computer Science from the University of Essex (2015). His research is centred in the application of Artificial Intelligence to games, Tree Search and Evolutionary Computation. He is especially interested in the application of Statistical Forward Planning methods (such as Monte Carlo Tree Search and Rolling Horizon Evolutionary Algorithms) to modern games, General Video Game Playing, and strategy games. He has a long experience in organizing competitions for the Game AI community in recent years, including the Physical Traveling Salesman Problem, the GVGAI and the MARLO challenge.021

The recent growth of accessibility in the video game industry has enabled play for thousands of people with disabilities. With major titles across the game industry now incorporating a diverse range of options, more players are having accessible player experiences (APX) than ever before. With game developers now taking action to improve their games, and hundreds of them applying the AbleGamers APX Design Patterns to innovate new accessible designs, there are opportunities for AI research to further improve our player experiences even further. In this talk, I will introduce the APX Design patterns, their structure and their current applications in games. I will use these patterns to contextualize the opportunities that are available for AI researchers to help players customize their settings and fine-tune their experiences so that everyone can enjoy games together.

Dr. Christopher Power is an Associate Professor at the University of Prince Edward Island on the east coast of Canada. He has been fortunate enough to spend over 20 years working in accessibility with people with disabilities and is the Vice President of the AbleGamers Charity. He is one of the inventors of the Accessible Player Experiences Design Patterns and has helped train hundreds of developers across the game industry in how to use this data-driven design language to make more accessible games.

The video game Minecraft is being used to engage people in conversations about race, equity, social justice, and environmental sustainability while also supporting remote and distance learning during the global COVID-19 pandemic. Hear how developers, educators, content creators, and curriculum designers work together to bring learning from computer science to social-emotional learning to millions of students who are suddenly home from school, and how the gaming community is rallying around the popular social platform to connect virtually and bridge differences.

Deirdre Quarnstrom started the program at Mojang Studios to bring Minecraft to mainstream education which now reaches millions of students and educators around the world. She currently leads the teams responsible for Minecraft Education, Minecraft Hour of Code, Minecraft’s China business and the work to make Minecraft more accessible for players of all abilities. She also serves as a Director on the board of Block by Block, a non-profit partnership between Mojang and UN-Habitat, and is a Trustee on the Board for the French American School of Puget Sound.

Computational Creativity is the art, science, and engineering of computational systems which, by taking on particular responsibilities, exhibit behaviors that unbiased observers would deem to be creative. Computational creativity in games is both applied and fundamental research; it can be used to generate gameplay experiences, and games can help us explore fundamental questions about algorithms that express creativity. This talk will explore computational creativity in the context of generating interactive text worlds. While text games appear simple, the generation of the worlds involve the challenges pertaining to natural language, commonsense reasoning, and procedural knowledge in order to produce sensible, coherent, and playable structures. I will use interactive world generation to probe broader implications for artificial intelligence in games.

Dr. Mark Riedl is an Associate Professor in the Georgia Tech School of Interactive Computing and Associate Director of the Georgia Tech Machine Learning Center. Dr. Riedl’s research focuses on human-centered artificial intelligence-the development of artificial intelligence and machine learning technologies that understand and interact with human users in more natural ways. Dr. Riedl’s recent work has focused on story understanding and generation, computational creativity, explainable AI, and teaching virtual agents to behave safely. His research is supported by the NSF, DARPA, ONR, the U.S. Army, U.S. Health and Human Services, Disney, and Google. He is the recipient of a DARPA Young Faculty Award and an NSF CAREER Award.

Minecraft has embarked on a two-year long project to ramp up its data science capabilities and close the ‘last mile gap’ between algorithms and personalized experiences in one of the world’s largest video games. We will cover key focus areas and approaches to enable a modern Machine Learning function, as well as the strategic mechanisms that align Minecraft’s personalization efforts to the studio philosophy that “Minecraft is for everyone”.

Francisco is the Head of Data Science and Data Engineering at Minecraft. He has led data science and analytics efforts at several AAA videogames, and is passionate about the intersection between games and science, where studios can intelligently use data, insights, and algorithms to improve player experiences and player outcomes.

The AI Settlement Generation challenge is a competition that asks participants to write code that can generate interesting and believable Minecraft settlements. It was designed to foster interest in procedural content generation – with a particular focus on adaptive and holistic generation – touching on co-creativity and open endedness. In this talk I will outline why the GDMC poses an important challenge as well as a great opportunity for outreach. It currently brings together both the general public and academic researchers, and has their ideas compete on a levelled playing field. I will give an update on the results of the competition, which is now in its fourth year, and sketch out our plans for the future.

Dr. Christoph Salge is a tenured Research Fellow at the University of Hertfordshire, where he has just finished a Marie-Curie Fellowship focussed on using Intrinsic Motivation to evaluate Procedural Content Generation for Games. He spends two year’s working at the NYU Game Innovation Lab, where he started the GDMC AI Settlement Generation competition in 2018. His research interests include the application of AI to various elements of games, and the modelling of intrinsic motivation to better understand the interaction between AIs and humans. He is also an avid board and computer gamer, and has built a life size replica of Isengard while writing up his Phd on Information Theoretic Models of Social Interaction.

Games and interactive narratives are emerging as platforms used for entertainment, education, training, and health. The potential social impact of these environments elevates the importance of developing novel methods that can evaluate and enhance their designs. Towards that goal, I will discuss my current work integrating machine learning approaches with visualization to develop novel methods that can aid in effectively capturing the player experience. I start by discussing a novel methodology called Data-Driven Retrospective Interviewing, which enabled us to use, process and visualize player logs and combine them with annotations from interviews to shed light on design problems that impacted players’ engagement, adherence, and motivation. I then discuss our latest project aiming at understanding and modeling the behaviors of esports players. For this project, we developed a new methodology called Interactive Behavior Analytics, where we focused on deeply modeling players’ behaviors using a combination of machine learning and qualitative coding to capture strategic and tactical aspects. In addition to enhancing game designs, the method has demonstrated utility as a tool for esports players, spectators, and coaches to diagnose strategic behaviors as well as for diagnosing progression and pacing issues.

Dr. Seif El-Nasr earned her Ph.D. degree from Northwestern University in Computer Science. Her research focuses on two goals (a) developing automated tools and techniques for authoring, adapting, and personalizing virtual environments (e.g., interactive narrative, believable characters, and visuals), and (b) developing evidence-based methodologies to measure the effectiveness of game environments through the development of novel in-depth behavior mining and visual analytics tools. She published the first book on the subject of game analytics, called Game Analytics: Maximizing the Value of Player Data. Her work is internationally known and cited in several game industry books. Additionally, she has received several awards and recognition within the game research community. Notably, she received four Best Paper Awards and one honorable mention. Further, she was named as a HEVGA (Higher Education Video Game Alliance) Fellow. She also serves as an associate editor for IEEE Transactions on Games and IEEE Transactions on Affective Computing.

Microsoft plenary session

Phil Spencer is executive vice president, Gaming at Microsoft. In this role, Spencer is accountable for leading Microsoft’s gaming business across all devices and services. With his team and game development partners, Spencer continues to push the boundaries of creativity, technical innovation and fun across gaming genres, audiences and devices.

Spencer is both a passionate gamer and seasoned gaming executive serving more than 15 years in the gaming industry leading global business, creative and engineering teams. Spencer has held various roles across Microsoft including Head of Xbox, Corporate Vice President, Microsoft Studios, and GM, Microsoft Game Studios EMEA. In these roles, Spencer led the Xbox organization with the launches of Xbox One S and Xbox One X, the acquisition of Minecraft and influenced blockbuster game franchises from “Halo,” “Gears of War,” “Forza Motorsport,” as well as Microsoft Publishing. He’s also led the expansion of cross-platform gaming with Xbox Live, which now counts more than 50 million monthly active users.

Before beginning his career as an intern with Microsoft in 1988, Spencer earned his bachelor’s degree from the University of Washington. He currently serves on the board of Entertainment Software Association and of The Paley Center for Media. Spencer has two daughters in college and lives with his wife in the Seattle area.

Incorporating machine learning and reinforcement learning into videogames is a huge opportunity and a reality for a number of top titles today. However, these techniques produce black-box AI agents whose behavior is often inscrutable, and challenging for designers to understand or customize. This talk will describe three projects that study this gap between learned AI agents and game design. First, we present tools inspired by interactive programming and generative testing to visualize learned agent behaviors. Second, we describe algorithms to allow designers to calibrate agents while learning to express desired behaviors (presented at ICML 2020). Third, we showcase new environments (built on the RTS game 0 A.D.) to study how game developers can explore the game design/balance space when using AI agents. Throughout we will highlight opportunities for open collaboration with Microsoft Research in developing designer-friendly learning techniques for use in videogames.

Adith Swaminathan is a senior researcher in the Reinforcement Learning Group at Microsoft Research in Redmond. He studies principles and algorithms that can improve human-centered systems using machine learning and counterfactual reasoning.

Adith spent the 2015-16 academic year visiting the Information and Language Processing Systems group at the University of Amsterdam, interned with the Machine Learning group at Microsoft Research NYC during the summer of 2015, Computer Human Interactive Learning group (now called Machine Teaching Group) at Microsoft Research Redmond during the summer of 2013, Search Labs at Microsoft Research during the summer of 2012, and worked as a strategist with Tower Research Capital for 14 months from June 2010 – July 2011.

Welcome and concluding remarks

Dr Kenji Takeda is Director of Health and AI Partnerships (Academic) for Microsoft Research, Cambridge, UK. He is working to empower academic researchers to develop and deploy human-centric AI and machine learning to transform healthcare by exploiting data in the cloud, empowering those at the frontline of healthcare, and moving towards precision medicine. This includes work in medical imaging on Project InnerEye and working with the global healthcare data research community. He regularly advises funding agencies and research organisations on innovation and technology strategy. He is a visiting industry fellow at the Alan Turing Institute and visiting associate professor at the University of Southampton, UK.

He was previously global lead for Microsoft’s Azure for Research program, helping researchers take best advantage of cloud computing, including through data science, high-performance computing, and the internet of things.

He has a passion for developing novel computational and system-wide approaches to tackle fundamental and applied problems in science, engineering, and healthcare. He has extensive experience in aeronautics and astronautics, aerodynamics, aeroacoustics, flight simulation, cloud computing, high performance and high productivity computing, data science, scientific workflows, scholarly communication, engineering and educational outreach.

He has received numerous awards, including Royal Aeronautical Society Silver Award, Royal Academy of Engineering/ExxonMobil Gold medal for excellence in engineering teaching, and inaugural Royal Academy of Engineering Innovation prize.

Dr. Tommy Thompson is a game AI developer, researcher, and consultant whose interests lie in non-player character design and exploration of procedural content generation for video games. He is largely known for his work at his company ‘AI and Games’, popularised by a YouTube show that explores research and applications of artificial intelligence in video games. In addition, Tommy is a lecturer in computer science at King’s College London and on advisory board to the GDC AI Summit.

Agents of various kinds can do much more than “just” play games. There are several ways they can help in game development. For example, they can generate and test level design. In particular, I will discuss recent work on using reinforcement learning to train agents to generate levels, and building agents based on quality-diversity algorithms to find visual and functional glitches in levels.

I’m working on artificial intelligence techniques for making computer games more fun, and on games for making artificial intelligence smarter. I ask what AI can do for games, and what games can do for AI.

I want to make computer games adapt to their players through finding out what players want (whether they know it or not) and creating new game levels, challenges or rules that suit the players. Related to this is the challenge of making sense of large amounts of data generated by computer games, and on assisting human game designers in creating great game experiences. I also want to make opponents and collaborators in games more intelligent and believable, research that has applications far outside of computer games. I believe games, in particular video games, are perfect testbeds for AI methods. But it is important that you test your algorithms not just on a single game, but on many games, so you focus on general intelligence and not just solving a single problem.

In the gaming industry, AI can often be perceived as a luxury good that studios only invest in with executive sponsorship. Without a CTO to protect the time of data scientists, it is crucial to conceptualize machine learning applications that fit hand-in-hand with the business and design goals of the games themselves. This talk will outline how to consider the final product at each phase of AI research and development. We will reflect on how best to discuss these topics with non-technical audiences and get everyone invested in what AI can do for the players.

Cat began her career as an economist at the Bureau of Labor Statistics before realizing that video game data is much more interesting. Since then, she has spent the past decade doing a mix of analytics, data science, and game economics in the gaming industry. She began her quest at S2 Games (Heroes of Newerth), then Carbine (Wildstar), Epic (Fortnite and Paragon), Blizzard, and now Dreamhaven. Her greatest achievement in that journey was creating and leading the incredible team of data scientists at Blizzard. Her favorite games include anything PC and PvP, with an unhealthy obsession for ARK: Survival Evolved. You can read more about it in her book, if she ever stops playing games long enough to finish it!

Vanessa Volz is an AI researcher at modl.ai (Copenhagen, Denmark), with focus in computational intelligence in games. She received her PhD in 2019 from TU Dortmund University, Germany, for her work on surrogate-assisted evolutionary algorithms applied to game optimisation. She holds B.Sc. degrees in Information Systems and in Computer Science from WWU Münster, Germany. She received an M.Sc. with distinction in Advanced Computing: Machine Learning, Data Mining and High Performance Computing from University of Bristol, UK, in 2014. Her current research focus is on employing surrogate-assisted evolutionary algorithms to obtain balance and robustness in systems with interacting human and artificial agents, especially in the context of games.

Experienced Principal Program Manager with a demonstrated history of working in the computer software industry. Strong program and project management professional with a Bachelor of Science (BS) focused on Computer Engineering from Rose-Hulman Institute of Technology.

For single player content in Hearthstone, scripting the AI for new content and adapting the AI for existing content as new cards and mechanics are released is a time-consuming task that has become increasingly difficult over time. Each new set of cards has the potential to impact the AI’s behavior in large swathes of previously released content. We present our work towards replacing the existing scripted AI with an agent generated from self-play reinforcement learning. This talk will give an overview of the algorithmic choices and infrastructure used to integrate reinforcement learning into the content development pipeline.

Wayne is a data scientist at Blizzard Entertainment working on incorporating machine learning throughout the game development process to accelerate processes through automation, enable new opportunities for game designers, and ultimately improve the player experience. His academic background is in mathematical statistics with an M.Sc. in statistics from the University of Washington and he is also interested in the software tools and infrastructure required for allowing machine learning to be useful in practical settings.

Welcome to day 1

Haiyan Zhang is a designer, engineer and maker of things.

Haiyan is Chief of Staff at Xbox, Microsoft’s Gaming division. She was formerly Senior Director of Innovation at Microsoft Research Cambridge and Technical Advisor to Lab Director, Christopher Bishop. She has also served as an inventor and TV host on the BBC series, Big Life Fix, inventing cutting-edge technology in support of people and communities in need.

An innovation and technology leader, Haiyan has spent the past 20 years working hands-on in software engineering, user experience, hardware R&D, service design, Cloud platforms, design thinking and blue-sky envisioning. She continues to push on the boundaries where technology can transform people’s lives for the better.

Previously roles include Innovation Director at Xbox Games Studio; Co-Founder, Design & Development Lead for the SaaS platform, OpenIDEO. With over 150,000 users worldwide solving challenges for social good and used as the enterprise innovation engine for organisations such as British Airways, DeutscheBank, Harvard Business School.

Working at the design consultancy, IDEO, Haiyan has created innovative tech experiences for community building, entertainment, financial services. She has worked with leading video game makers on research and ideas for new play experiences. Clients have included Mattel, Electronic Arts, HBO, Citibank, France Telecom, Alcatel, Cisco, and AT&T.

Haiyan’s work is informed by her previous profession as a software engineer & user interface designer creating applications for the biomedical & data-mining industries.

Haiyan has a Masters degree with Distinction in Interaction Design from the renowned Interaction Design Institute in Ivrea, and has a Bachelor of Computer Science (1st-Class Honours) from Monash University, Australia.

Haiyan is a Fellow of the Royal Society of Arts (FRSA) and a member of the prestigious British Academy of Film & Television Arts (BAFTA).

She has lived and worked in Australia, Toronto, China, Italy, San Francisco, London and now resides in Bellevue.