rss_2.0Journal of Artificial General Intelligence FeedSciendo RSS Feed for Journal of Artificial General Intelligence of Artificial General Intelligence Feed Environments to Measure Self-reflection in Reinforcement Learning<abstract> <title style='display:none'>Abstract</title> <p>We consider an extended notion of reinforcement learning in which the environment can simulate the agent and base its outputs on the agent’s hypothetical behavior. Since good performance usually requires paying attention to whatever things the environment’s outputs are based on, we argue that for an agent to achieve on-average good performance across many such extended environments, it is necessary for the agent to self-reflect. Thus weighted-average performance over the space of all suitably well-behaved extended environments could be considered a way of measuring how self-reflective an agent is. We give examples of extended environments and introduce a simple transformation which experimentally seems to increase some standard RL agents’ performance in a certain type of extended environment.</p> </abstract>ARTICLEtrue Intelligence and Growth Rate: Variations on Hibbard’s Intelligence Measure<abstract> <title style='display:none'>Abstract</title> <p>In 2011, Hibbard suggested an intelligence measure for agents who compete in an adversarial sequence prediction game. We argue that Hibbard’s idea should actually be considered as two separate ideas: first, that the intelligence of such agents can be measured based on the growth rates of the runtimes of the competitors that they defeat; and second, one specific (somewhat arbitrary) method for measuring said growth rates. Whereas Hibbard’s intelligence measure is based on the latter growth-rate-measuring method, we survey other methods for measuring function growth rates, and exhibit the resulting Hibbard-like intelligence measures and taxonomies. Of particular interest, we obtain intelligence taxonomies based on Big-O and Big-Theta notation systems, which taxonomies are novel in that they challenge conventional notions of what an intelligence measure should look like. We discuss how intelligence measurement of sequence predictors can indirectly serve as intelligence measurement for agents with Artificial General Intelligence (AGIs).</p> </abstract>ARTICLEtrue Synthesis and Decoding of Meaning<abstract> <title style='display:none'>Abstract</title> <p>Thinking machines must be able to use language effectively in communication with humans. It requires from them the ability to generate meaning and transfer this meaning to a communicating partner. Machines must also be able to decode meaning communicated via language. This work is about meaning in the context of building an artificial general intelligent system. It starts with an analysis of the Turing test and some of the main approaches to explain meaning. It then considers the generation of meaning in the human mind and argues that meaning has a dual nature. The quantum component reflects the relationships between objects and the orthogonal quale component the value of these relationships to the self. Both components are necessary, simultaneously, for meaning to exist. This parallel existence permits the formulation of ‘meaning coordinates’ as ordered pairs of quantum and quale strengths. Meaning coordinates represent the contents of meaningful mental states. Spurred by a currently salient meaningful mental state in the speaker, language is used to induce a meaningful mental state in the hearer. Therefore, thinking machines must be able to produce and respond to meaningful mental states in ways similar to their functioning in humans. It is explained how quanta and qualia arise, how they generate meaningful mental states, how these states propagate to produce thought, how they are communicated and interpreted, and how they can be simulated to create thinking machines.</p> </abstract>ARTICLEtrue Reinforcement Learning: Part II. Structured MDPs<abstract> <title style='display:none'>Abstract</title> <p>The Feature Markov Decision Processes ( MDPs) model developed in Part I (Hutter, 2009b) is well-suited for learning agents in general environments. Nevertheless, unstructured (Φ)MDPs are limited to relatively simple environments. Structured MDPs like Dynamic Bayesian Networks (DBNs) are used for large-scale real-world problems. In this article I extend ΦMDP to ΦDBN. The primary contribution is to derive a cost criterion that allows to automatically extract the most relevant features from the environment, leading to the “best” DBN representation. I discuss all building blocks required for a complete general learning algorithm, and compare the novel ΦDBN model to the prevalent POMDP approach.</p> </abstract>ARTICLEtrue New Approach to Creation of an Artificial Intellect and Method of its Implementation<abstract> <title style='display:none'>Abstract</title> <p>On the basis of the author’s earlier works, the article proposes a new approach to creating an artificial intellect system in a model of a human being that is presented as the unification of an intellectual agent and a humanoid robot (ARb). In accordance with the proposed new approach, the development of an artificial intellect is achieved by teaching a natural language to an ARb, and by its utilization for communication with ARbs and humans, as well as for reflections. A method is proposed for the implementation of the approach. Within the framework of that method, a human model is “brought up” like a child, in a collective of automatons and children, whereupon an ARb must master a natural language and reflection, and possess self-awareness. Agent robots (ARbs) propagate and their population evolves; that is ARbs develop cognitively from generation to generation. ARbs must perform the tasks they were given, such as computing, whereupon they are then assigned time for “private life” for improving their education as well as for searching for partners for propagation. After having received an education, every agent robot may be viewed as a “person” who is capable of activities that contain elements of creativity. The development of ARbs thanks to the evolution of their population, education, and personal “life” experience, including “work” experience, which is mastered in a collective of humans and automatons.</p> </abstract>ARTICLEtrue and No Free Lunch in Formal Measures of Intelligence<abstract xml:lang="en"><title style='display:none'>Bias and No Free Lunch in Formal Measures of Intelligence</title><p>This paper shows that a constraint on universal Turing machines is necessary for Legg's and Hutter's formal measure of intelligence to be unbiased. Their measure, defined in terms of Turing machines, is adapted to finite state machines. A No Free Lunch result is proved for the finite version of the measure.</p></abstract>ARTICLEtrue of Evidence: A Comparative Study<abstract xml:lang="en"><title style='display:none'>Formalization of Evidence: A Comparative Study</title><p>This article analyzes and compares several approaches of formalizing the notion of evidence in the context of general-purpose reasoning system. In each of these approaches, the notion of evidence is defined, and the evidence-based degree of belief is represented by a binary value, a number (such as a probability), or two numbers (such as an interval). The binary approaches provide simple ways to represent conclusive evidence, but cannot properly handle inconclusive evidence. The one-number approaches naturally represent inconclusive evidence as a degree of belief, but lack the information needed to revise this degree. It is argued that for systems opening to new evidence, each belief should at least have two numbers attached to indicate its evidential support. A few such approaches are discussed, including the approach used in NARS, which is designed according to the considerations of general-purpose intelligent systems, and provides novel solutions to several traditional problems on evidence.</p></abstract>ARTICLEtrue Reinforcement Learning: Part I. Unstructured MDPs<abstract xml:lang="en"><title style='display:none'>Feature Reinforcement Learning: Part I. Unstructured MDPs</title><p>General-purpose, intelligent, learning agents cycle through sequences of observations, actions, and rewards that are complex, uncertain, unknown, and non-Markovian. On the other hand, reinforcement learning is well-developed for small finite state Markov decision processes (MDPs). Up to now, extracting the right state representations out of bare observations, that is, reducing the general agent setup to the MDP framework, is an art that involves significant effort by designers. The primary goal of this work is to automate the reduction process and thereby significantly expand the scope of many existing reinforcement learning algorithms and the agents that employ them. Before we can think of mechanizing this search for suitable MDPs, we need a formal objective criterion. The main contribution of this article is to develop such a criterion. I also integrate the various parts into one learning algorithm. Extensions to more realistic dynamic Bayesian networks are developed in Part II (Hutter, 2009c). The role of POMDPs is also considered there.</p></abstract>ARTICLEtrue What Makes JAGI Special<abstract xml:lang="en"><title style='display:none'>Editorial: What Makes JAGI Special</title></abstract>ARTICLEtrue Issue “On Defining Artificial Intelligence”—Commentaries and Author’s Response Utility Functions<abstract xml:lang="en"><title style='display:none'>Abstract</title><p>Orseau and Ring, as well as Dewey, have recently described problems, including self-delusion, with the behavior of agents using various definitions of utility functions. An agent's utility function is defined in terms of the agent's history of interactions with its environment. This paper argues, via two examples, that the behavior problems can be avoided by formulating the utility function in two steps: 1) inferring a model of the environment from interactions, and 2) computing utility as a function of the environment model. Basing a utility function on a model that the agent must learn implies that the utility function must initially be expressed in terms of specifications to be matched to structures in the learned model. These specifications constitute prior assumptions about the environment so this approach will not work with arbitrary environments. But the approach should work for agents designed by humans to act in the physical world. The paper also addresses the issue of self-modifying agents and shows that if provided with the possibility to modify their utility functions agents will not choose to do so, under some usual assumptions.</p></abstract>ARTICLEtrue Logic in the Mind or in the World? Why a Philosophical Question can Affect the Understanding of Intelligence<abstract xml:lang="en"><title style='display:none'>Abstract</title><p>Dreyfus' call ‘to make artificial intelligence (AI) more Heideggerian‘ echoes Heidegger's affirmation that pure calculations produce no ‘intelligence’ (Dreyfus, 2007). But what exactly is it that AI needs more than mathematics? The question in the title gives rise to a reexamination of the basic principles of cognition in Husserl's Phenomenology. Using Husserl's Phenomenological Method, a formalization of these principles is presented that provides the principal idea of cognition, and as a consequence, a ‘natural logic’. Only in a second step, mathematics is obtained from this natural logic by abstraction.</p><p>The limitations of pure reasoning are demonstrated for fundamental considerations (Hilbert's ‘finite Einstellung’) as well as for the task of solving practical problems. Principles will be presented for the design of general intelligent systems, which make use of a natural logic.</p></abstract>ARTICLEtrue Non-Dominated Parameter Sets for Computational Models from Multiple Experiments<abstract><title style='display:none'>Abstract</title><p>Creating robust, reproducible and optimal computational models is a key challenge for theorists in many sciences. Psychology and cognitive science face particular challenges as large amounts of data are collected and many models are not amenable to analytical techniques for calculating parameter sets. Particular problems are to locate the full range of acceptable model parameters for a given dataset, and to confirm the consistency of model parameters across different datasets. Resolving these problems will provide a better understanding of the behaviour of computational models, and so support the development of general and robust models. In this article, we address these problems using evolutionary algorithms to develop parameters for computational models against multiple sets of experimental data; in particular, we propose the ‘speciated non-dominated sorting genetic algorithm’ for evolving models in several theories. We discuss the problem of developing a model of categorisation using twenty-nine sets of data and models drawn from four different theories. We find that the evolutionary algorithms generate high quality models, adapted to provide a good fit to all available data.</p></abstract>ARTICLEtrue Measure of Real-Time Intelligence<abstract><title style='display:none'>Abstract</title><p>We propose a new measure of intelligence for general reinforcement learning agents, based on the notion that an agent’s environment can change at any step of execution of the agent. That is, an agent is considered to be interacting with its environment in real-time. In this sense, the resulting intelligence measure is more general than the universal intelligence measure (Legg and Hutter, 2007) and the anytime universal intelligence test (Hernández-Orallo and Dowe, 2010). A major advantage of the measure is that an agent’s computational complexity is factored into the measure in a natural manner. We show that there exist agents with intelligence arbitrarily close to the theoretical maximum, and that the intelligence of agents depends on their parallel processing capability. We thus believe that the measure can provide a better evaluation of agents and guidance for building practical agents with high intelligence.</p></abstract>ARTICLEtrue Sigma Cognitive Architecture and System: Towards Functionally Elegant Grand Unification<abstract><title style='display:none'>Abstract</title><p>Sigma (Σ) is a cognitive architecture and system whose development is driven by a combination of four desiderata: <italic>grand unification</italic>, <italic>generic cognition</italic>, <italic>functional elegance</italic>, and <italic>sufficient efficiency</italic>. Work towards these desiderata is guided by the <italic>graphical architecture hypothesis</italic>, that key to progress on them is combining what has been learned from over three decades’ worth of separate work on <italic>cognitive architectures</italic> and <italic>graphical models</italic>. In this article, these four desiderata are motivated and explained, and then combined with the graphical architecture hypothesis to yield a rationale for the development of Sigma. The current state of the cognitive architecture is then introduced in detail, along with the graphical architecture that sits below it and implements it. Progress in extending Sigma beyond these architectures and towards a full cognitive system is then detailed in terms of both a systematic set of higher level <italic>cognitive idioms</italic> that have been developed and several <italic>virtual humans</italic> that are built from combinations of these idioms. Sigma as a whole is then analyzed in terms of how well the progress to date satisfies the desiderata. This article thus provides the first full motivation, presentation and analysis of Sigma, along with a diversity of more specific results that have been generated during its development.</p></abstract>ARTICLEtrue Agent for General Environment<abstract><title style='display:none'>Abstract</title><p> One of the essential aspect in biological agents is dynamic stability. This aspect, called homeostasis, is widely discussed in ethology, neuroscience and during the early stages of artificial intelligence. Ashby’s homeostats are general-purpose learning machines for stabilizing essential variables of the agent in the face of general environments. However, despite their generality, the original homeostats couldn’t be scaled because they searched their parameters randomly. In this paper, first we re-define the objective of homeostats as the maximization of a multi-step survival probability from the view point of sequential decision theory and probabilistic theory. Then we show that this optimization problem can be treated by using reinforcement learning algorithms with special agent architectures and theoretically-derived intrinsic reward functions. Finally we empirically demonstrate that agents with our architecture automatically learn to survive in a given environment, including environments with visual stimuli. Our survival agents can learn to eat food, avoid poison and stabilize essential variables through theoretically-derived single intrinsic reward formulations.</p></abstract>ARTICLEtrue via ultrafilters: structural properties of some intelligence comparators of deterministic Legg-Hutter agents<abstract><title style='display:none'>Abstract</title><p>Legg and Hutter, as well as subsequent authors, considered intelligent agents through the lens of interaction with reward-giving environments, attempting to assign numeric intelligence measures to such agents, with the guiding principle that a more intelligent agent should gain higher rewards from environments in some aggregate sense. In this paper, we consider a related question: rather than measure numeric intelligence of one Legg-Hutter agent, how can we compare the relative intelligence of two Legg-Hutter agents? We propose an elegant answer based on the following insight: we can view Legg-Hutter agents as candidates in an election, whose voters are environments, letting each environment vote (via its rewards) which agent (if either) is more intelligent. This leads to an abstract family of comparators simple enough that we can prove some structural theorems about them. It is an open question whether these structural theorems apply to more practical intelligence measures.</p></abstract>ARTICLEtrue Variants of AIXI which are More Powerful than AIXI<abstract><title style='display:none'>Abstract</title><p>This paper presents Unlimited Computable AI, or UCAI, that is a family of computable variants of AIXI. UCAI is more powerful than AIXI<italic>tl</italic>, which is a conventional family of computable variants of AIXI, in the following ways: 1) UCAI supports models of terminating computation, including typed lambda calculi, while AIXI<italic>tl</italic> only supports Turing machine with timeout ˜<italic>t</italic>, which can be simulated by typed lambda calculi for any ˜<italic>t</italic>; 2) unlike UCAI, AIXI<italic>tl</italic> limits the program length to some ˜<italic>l</italic> .</p></abstract>ARTICLEtrue Action Execution Process Implemented in Different Cognitive Architectures: A Review<abstract><title style='display:none'>Abstract</title><p> An agent achieves its goals by interacting with its environment, cyclically choosing and executing suitable actions. An action execution process is a reasonable and critical part of an entire cognitive architecture, because the process of generating executable motor commands is not only driven by low-level environmental information, but is also initiated and affected by the agent’s high-level mental processes. This review focuses on cognitive models of action, or more specifically, of the action execution process, as implemented in a set of popular cognitive architectures. We examine the representations and procedures inside the action execution process, as well as the cooperation between action execution and other high-level cognitive modules. We finally conclude with some general observations regarding the nature of action execution.</p></abstract>ARTICLEtrue General Intelligence: Concept, State of the Art, and Future Prospects<abstract><title style='display:none'>Abstract</title><p> In recent years broad community of researchers has emerged, focusing on the original ambitious goals of the AI field - the creation and study of software or hardware systems with general intelligence comparable to, and ultimately perhaps greater than, that of human beings. This paper surveys this diverse community and its progress. Approaches to defining the concept of Artificial General Intelligence (AGI) are reviewed including mathematical formalisms, engineering, and biology inspired perspectives. The spectrum of designs for AGI systems includes systems with symbolic, emergentist, hybrid and universalist characteristics. Metrics for general intelligence are evaluated, with a conclusion that, although metrics for assessing the achievement of human-level AGI may be relatively straightforward (e.g. the Turing Test, or a robot that can graduate from elementary school or university), metrics for assessing partial progress remain more controversial and problematic.</p></abstract>ARTICLEtrue