Research

An Architecture for Novelty Handling in a Multi-Agent Stochastic Environment: Case Study in Open-World Monopoly

March 2022

The ability of AI agents and architectures to detect and adapt to sudden changes in their environments remains an outstanding challenge. In the context of multi-agent games, the agent may face novel situations where the rules of the game, the available actions, the environment dynamics, the behavior of other agents, as well as the agent’s goals suddenly change. In this paper, we introduce an architecture that allows agents to detect novelties, characterize those novelties, and build an appropriate adaptive model to accommodate them. Our agent utilizes logic and reasoning (specifically, Answer Set Programming) to characterize novelties into different categories, as to enable the agent to adapt to the novelty while maintaining high performance in the game. We demonstrate the effectiveness of the proposed agent architecture in a multi-agent imperfect information board game, Monopoly. We measure the success of the architecture by comparing our method to heuristics, and vanilla Monte-Carlo Tree Search approaches. Our results indicate precise novelty detection, and significant improvements in the performance of agents utilizing the novelty handling architecture.

CLIMBING ROBOT PROJECT

October - December 2019

Our task was to create a robot capable of climbing a simulated “tree”. The vertical pillar would have variable geometry and surface properties by the addition of various materials to a 4”x4” pole. Our robot was to be controlled by a Raspberry Pi Zero and would be capable of autonomously climbing the pillar. We were tasked with integrating camera-based surface detection as a means of navigating the different portions of the pillar.

As required by the project, our robot would aim to be untethered, but a power cable would be acceptable. A goal mass of under 1 kg (max 1.5 kg) and size constraint of fitting in a 30.5 cm cube (40.5 cm cube when extended) was also imposed as conditions on the project.

Our design integrates MakerBeam extruded aluminum as the structural base of an otherwise 3D-printed PLA body and gripper. Each gripper uses a rack and pinion system in which the extruded aluminum slides through PLA linear bearing surfaces on either side of a pinion mounted to a servo. With this symmetrical and pseudo-modular design, we mirrored the first gripper and connected the two with an extendable central body, based on the same telescoping rack and pinion system. In order to maneuver over the complicated geometry of the course, we incorporated pitch servos between the grippers and either end of the body, giving five total degrees-of-freedom to our final design.

SMARTER LEARNING: FROM PAC-MAN TO PAC-MAN CTF VIA TRANSFER LEARNING

June - December 2020

Reinforcement Learning (RL) is one of the most popular machine learning paradigms applied to address problems that have only limited environmental feedback. With the advent of reinforcement learning algorithms and tree search algorithms, significant research has been conducted on its potential application to several research areas such as robotics, games, strategy planning, or data processing. However, most of the existing reinforcement learning agents have been made to solve a single task. The essential idea of transfer learning is to utilize experience gained in learning to perform a different task with similar characteristics. Transfer learning is used to mitigate the training time of an AI agent and also optimize the outcomes. This project aims to develop a transfer learning method which can be applied to pathfinding and object avoidance AI agent in different Pac-Man environments such as Pac-Man and Pac-Man Capture the Flag.

CAP - COOPERATIVE AGENTS IN PAC-MAN

October - December 2020

Varying forms of artificial intelligence (AI) have long been implemented in video games. However, in most games, not much research has been done on collaborative game AI. The purpose of this paper is to find an effective collaborative method for a multiagent game environment. In recent years, Pac-Man and Ms. Pac-Man have been used widely as an environment to test artificial intelligence agent’s behavior. This project aims to build a collaborative environment for two Pac-man agents. In Ms. Pac-Man, there are some important characteristics, such as pathfinding and object avoidance. Our goal is to find an efficient method in collaborative AI to approach and solve a more complex system using two Pac-Man agents’ knowledge. In order to improve the performance of the agent, the multi-agents concept is introduced. Multiagents is an artificial intelligence (AI) technique which utilized multi-agents to make the decision. This paper aims to develop an offensive sub-agent system in a Pac-Man capture the flag environment that can be used to improve the overall performance against AI Pac-Man teams.

WHAT? - WAR-GAME HEURISTIC AI TRAINING

January - August, 2019

With the advent of advanced search and machine learning algorithms, significant research has been conducted on its potential application to strategy board game AI’s. Many game AI’s, as a result, have begun to take advantage of these algorithms strong learning capabilities. More specifically, neural network-based game AI’s such as AlphaGo Zero have demonstrated their potential at defeating expert players. However, the application of these techniques has largely been explored for games of perfect information in which no knowledge is hidden from both players. Many board games exist where information about the opponent is unknown and must be revealed through piece interactions. This project explores the effectiveness of these algorithms when applied to Stratego: a strategy board game with elements of randomness and unknown information. The results of this experiment hope to reveal how effective these algorithms are at making rational decisions when applied to systems with incomplete information.