Intrinsically Motivated High-Level Planning for Agent Exploration

This paper proposes a new open-ended learning framework which aims at implementing an autonomous agent using intrinsic motivations (IM) at two different levels. At the first level, the IM paradigm is exploited by the agent to learn new operational skills, described in terms of sub-symbolic options. After discovering the options, the agent iteratively: (1) executes them to explore the world, collecting the necessary data and (2) automatically abstracts the collected data into a high-level representation of the domain, expressed in PPDDL language. At the second level, the IM paradigm is used to exploit the abstracted representation of the domain by identifying particular symbolic states deemed promising according to a specific criterium, which in the present work is the farthest distance covered by the agent (i.e., the most promising states are those that rest at the frontier of the visited space). Once these states are identified, they can be successively reached through an internally generated high-level plan and used as promising starting points for discovering new knowledge. The presented framework is tested in the so-called Treasure Game domain described in the recent literature. The tests we have performed show that the proposed idea of implementing intrinsic motivations at two different levels of abstraction facilitates the discovery of new knowledge, compared to a previous approach proposed in the literature.