Reinforcement learning has recently received much attention as a learning method for complicated systems, e.g., robot systems. It does not need prior knowledge and has higher capability of reactive and adaptive behaviors. However increase in dimensionality of the action-state space makes it diffcult to accomplish learning. The applicability of the existing reinforcement learning algorithms are effective for simple tasks with relatively small action-state space. In this paper, we propose a new reinforcement learning algorithm: “Q-learning with (...) class='Hi'>Dynamic Structuring of Exploration Space Based on Genetic Algorithm ”. The algorithm is applicable to systems with high dimensional action and interior state spaces, for example a robot with many redundant degrees of freedom. To demonstrate the effectiveness of the proposed algorithm simulations of obstacle avoidance by a 50 links manipulator have been carried out. It is shown that effective behavior can be learned by using the proposed algorithm. (shrink)

Reinforcement learning is very effective for robot learning. Because it does not need priori knowledge and has higher capability of reactive and adaptive behaviors. In our previous works, we proposed new reinforcement learning algorithm: “Q-learning with Dynamic Structuring of Exploration Space Based on Genetic Algorithm (QDSEGA)”. It is designed for complicated systems with large action-state space like a robot with many redundant degrees of freedom. And we applied it to 50 link manipulator (...) and effective behavior is acquired. However optimality and fault tolerance of the proposed algorithm were not considered and to demonstrate effectiveness of the proposed algorithm other applications are necessary. Acquiring of locomotion patterns by a multi-legged robot is a very interesting problem. As it has many redundant degrees of freedom, application of usual reinforcement learning is difficult and an optimal locomotion has not been acquired using previous reinforcement learning algorithm. And the redundancy of the robot is effective to the fault tolerance and various locomotion patterns can be acquired for adapting the faults of the legs. In this paper, we applied QDSEGA to acquiring of locomotion pattern by the multi-legged robot and considered the optimality and fault tolerance. Effective behavior has been obtained by using our proposed algorithm. (shrink)

In this article, an in-depth study and analysis of economic structure are carried out using a neural network fusion release algorithm. The method system defines the weight space and structure space of neural networks from the perspective of optimization theory, proposes a bionic optimization algorithm under the weight space and structure space, and establishes a neuroevolutionary method with shallow neural network and deep neural network as the research objects. In the shallow neuroevolutionary, the improved genetic algorithm (...) based on elite heuristic operation and migration strategy and the improved coyote optimization algorithm based on adaptive influence weights are proposed, and the shallow neuroevolutionary method based on IGA and the shallow neuroevolutionary method based on ICOA are applied to the weight space of backpropagation neural networks. In deep neuroevolutionary method, the structure space of convolutional neural network is proposed to solve the search space design of neural structure search, and the GA-based deep neuroevolutionary method under the structure space of convolutional neural network is proposed to solve the problem that numerous hyperparameters and network structure parameters can produce explosive combinations when designing deep learning models. The neural network fusion bionic algorithm used has the application value of exploring the spatial structure and dynamics of the socioeconomic system, improving the perception of the socioeconomic situation, and understanding the development law of society, etc. The idea is also verifiable through the present computer technology. (shrink)

This paper presents a GA-based multi-agent reinforce- ment learning bidding approach (GMARLB) for perform- ing multi-agent reinforcement learning. GMARLB inte- grates reinforcement learning, bidding and genetic algo- rithms. The general idea of our multi-agent systems is as follows: There are a number of individual agents in a team, each agent of the team has two modules: Q module and CQ module. Each agent can select actions to be performed at each step, which are done by the Q (...) module. While the CQ module determines at each step whether the agent should continue or relinquish control. Once an agent relinquishes its control, a new agent is selected by bidding algorithms. We applied GA-based GMARLB to the Backgammon game. The experimental results show GMARLB can achieve a su- perior level of performance in game-playing, outperforming PubEval, while the system uses zero built-in knowledge. (shrink)

Improving the resilience of urban road networks suffering from various disruptions has been a central focus for urban emergence management. However, to date the effective methods which may mitigate the negative impacts caused by the disruptions, such as road accidents and natural disasters, on urban road networks is highly insufficient. This study proposes a novel adaptive signal control strategy based on a doubly dynamic learning framework, which consists of deep reinforcement learning and day-to-day traffic dynamic learning, to (...) improve the network performance by adjusting red/green time split. In this study, red time split is regarded as extra traffic flow to discourage drivers to use affected roads, so as to reduce congestion and improve the resilience when urban road networks are subject to different levels of disruptions. In addition, we utilize the convolution neural network as Q-network to approximate Q values, link flow distribution and link capacity are regarded as the state space, and actions are denoted as red/green time split. A small network is utilized as a numerical example, and a fixed time signal control and other two adaptive signal controls are employed for the comparisons with the proposed one. The results show that the proposed adaptive signal control based on deep reinforcement learning can achieve better resilience in most of the cases, particularly in the scenarios of moderate and severe disruptions. This study may shed light on the advantages of the proposed adaptive signal control dealing with major emergencies compared to others. (shrink)

Real causal systems are complicated. Despite this, causal learning research has traditionally emphasized how causal relations can be induced on the basis of idealized events, i.e. those that have been mapped to binary variables and abstracted from time. For example, participants may be asked to assess the efficacy of a headache-relief pill on the basis of multiple patients who take the pill (or not) and find their headache relieved (or not). In contrast, the current study examines learning via interactions with (...) continuous dynamic systems, systems that include continuous variables that interact over time (and that can be continuously observed in real time by the learner). To explore such systems, we develop a new framework that represents a causal system as a network of stationary Gauss--Markov (``\OU'') processes and show how such \textit{OU networks} can express complex dynamic phenomena such as feedback loops and oscillations. To assess adult's abilities to learn such systems, we conducted an experiment in which participants were asked to identify the causal relationships of a number of OU networks, potentially carrying out multiple, temporally-extended interventions. We compared their judgments to a normative model for learning OU networks as well as a range of alternative and heuristic learning models from the literature. We found that, although participants exhibited substantial learning of such systems, they committed certain systematic errors. These successes and failures were best accounted for by a model that describes people as focusing on pairs of variables, rather than evaluating the evidence with respect to the full space of possible structural models. We argue that our approach provides both a principled framework for exploring the space of dynamic learning environments as well as new algorithmic insights into how people interact successfully with a continuous causal world. (shrink)

The area of smart power grids needs to constantly improve its efficiency and resilience, to provide high quality electrical power in a resilient grid, while managing faults and avoiding failures. Achieving this requires high component reliability, adequate maintenance, and a studied failure occurrence. Correct system operation involves those activities and novel methodologies to detect, classify, and isolate faults and failures and model and simulate processes with predictive algorithms and analytics. In this paper, we showcase the application of a complex-adaptive, self-organizing (...) modeling method, and Probabilistic Boolean Networks, as a way towards the understanding of the dynamics of smart grid devices, and to model and characterize their behavior. This work demonstrates that PBNs are equivalent to the standard Reinforcement Learning Cycle, in which the agent/model has an interaction with its environment and receives feedback from it in the form of a reward signal. Different reward structures were created to characterize preferred behavior. This information can be used to guide the PBN to avoid fault conditions and failures. (shrink)

This paper suggests an online solution for the optimal tracking control of robotic systems based on a single critic neural network -based reinforcement learning method. To this end, we rewrite the robotic system model as a state-space form, which will facilitate the realization of optimal tracking control synthesis. To maintain the tracking response, a steady-state control is designed, and then an adaptive optimal tracking control is used to ensure that the tracking error can achieve convergence in an optimal sense. (...) To solve the obtained optimal control via the framework of adaptive dynamic programming, the command trajectory to be tracked and the modified tracking Hamilton-Jacobi-Bellman are all formulated. An online RL algorithm is the developed to address the HJB equation using a critic NN with online learning algorithm. Simulation results are given to verify the effectiveness of the proposed method. (shrink)

In reinforcement learning, a decision maker searching for the most rewarding option is often faced with the question: What is the value of an option that has never been tried before? One way to frame this question is as an inductive problem: How can I generalize my previous experience with one set of options to a novel option? We show how hierarchical Bayesian inference can be used to solve this problem, and we describe an equivalence between the Bayesian model (...) and temporal difference learning algorithms that have been proposed as models of RL in humans and animals. According to our view, the search for the best option is guided by abstract knowledge about the relationships between different options in an environment, resulting in greater search efficiency compared to traditional RL algorithms previously applied to human cognition. In two behavioral experiments, we test several predictions of our model, providing evidence that humans learn and exploit structured inductive knowledge to make predictions about novel options. In light of this model, we suggest a new interpretation of dopaminergic responses to novelty. (shrink)

One of the most challenging tasks when adopting Bayesian networks is the one of learning their structure from data. This task is complicated by the huge search space of possible solutions and by the fact that the problem isNP-hard. Hence, a full enumeration of all the possible solutions is not always feasible and approximations are often required. However, to the best of our knowledge, a quantitative analysis of the performance and characteristics of the different heuristics to solve this problem has (...) never been done before. For this reason, in this work, we provide a detailed comparison of many different state-of-the-art methods for structural learning on simulated data considering both BNs with discrete and continuous variables and with different rates of noise in the data. In particular, we investigate the performance of different widespread scores and algorithmic approaches proposed for the inference and the statistical pitfalls within them. (shrink)

Foundations of Algorithms, Fifth Edition offers a well-balanced presentation of algorithm design, complexity analysis of algorithms, and computational complexity. Ideal for any computer science students with a background in college algebra and discrete structures, the text presents mathematical concepts using standard English and simple notation to maximize accessibility and user-friendliness. Concrete examples, appendices reviewing essential mathematical concepts, and a student-focused approach reinforce theoretical explanations and promote learning and retention. C++ and Java pseudocode help students better understand complex algorithms. A (...) chapter on numerical algorithms includes a review of basic number theory, Euclid's Algorithm for finding the greatest common divisor, a review of modular arithmetic, an algorithm for solving modular linear equations, an algorithm for computing modular powers, and the new polynomial-time algorithm for determining whether a number is prime. The revised and updated Fifth Edition features an all-new chapter on genetic algorithms and genetic programming, including approximate solutions to the traveling salesperson problem, an algorithm for an artificial ant that navigates along a trail of food, and an application to financial trading. With fully updated exercises and examples throughout and improved instructor resources including complete solutions, an Instructor's Manual and PowerPoint lecture outlines, Foundations of Algorithms is an essential text for undergraduate and graduate courses in the design and analysis of algorithms. Key features include: • The only text of its kind with a chapter on genetic algorithms • Use of C++ and Java pseudocode to help students better understand complex algorithms • No calculus background required • Numerous clear and student-friendly examples throughout the text • Fully updated exercises and examples throughout • Improved instructor resources, including complete solutions, an Instructor's Manual, and PowerPoint lecture outlines. (shrink)

The Internet of Things (IoT) is a fast-growing technology that connects everyday devices to the Internet, enabling wireless, low-consumption and low-cost communication and data exchange. IoT has revolutionized the way devices interact with each other and the internet. The more devices become connected, the greater the risk of security breaches. There is currently a need for new approaches to algorithms that can detect malware regardless of the size of the network and that can adapt to dynamic changes in the (...) network. Through the use of a multi-agent reinforcement learning algorithm, this paper proposes a novel algorithm for malware detection in IoT devices. The proposed algorithm is not strongly dependent on the size of the IoT network due to the that its training is adapted using time differences if the IoT network size is small or Monte Carlo otherwise. To validate the proposed algorithm in an environment as close to reality as possible, we proposed a scenario based on a real IoT network, where we tested different malware propagation models. Different simulations varying the number of agents and nodes in the IoT network have been developed. The result of these simulations proves the efficiency and adaptability of the proposed algorithm in detecting malware, regardless of the malware propagation model. (shrink)

Learning to perceive is faced with a classical paradox: if understanding is required for perception, how can we learn to perceive something new, something we do not yet understand? According to the sensorimotor approach, perception involves mastery of regular sensorimotor co-variations that depend on the agent and the environment, also known as the “laws” of sensorimotor contingencies (SMCs). In this sense, perception involves enacting relevant sensorimotor skills in each situation. It is important for this proposal that such skills can be (...) learned and refined with experience and yet up to this date, the sensorimotor approach has had no explicit theory of perceptual learning. The situation is made more complex if we acknowledge the open-ended nature of human learning. In this paper we propose Piaget’s theory of equilibration as a potential candidate to fulfill this role. This theory highlights the importance of intrinsic sensorimotor norms, in terms of the closure of sensorimotor schemes. It also explains how the equilibration of a sensorimotor organization faced with novelty or breakdowns proceeds by re-shaping pre-existing structures in coupling with dynamical regularities of the world. This way learning to perceive is guided by the equilibration of emerging forms of skillful coping with the world. We demonstrate the compatibility between Piaget’s theory and the sensorimotor approach by providing a dynamical formalization of equilibration to give an explicit micro-genetic account of sensorimotor learning and, by extension, of how we learn to perceive. This allows us to draw important lessons in the form of general principles for open-ended sensorimotor learning, including the need for an intrinsic normative evaluation by the agent itself. We also explore implications of our micro-genetic account at the personal level. (shrink)

The structural-parametric synthesis of neural networks of deep learning, in particular convolutional neural networks used in image processing, is considered. The classification of modern architectures of convolutional neural networks is given. It is shown that almost every convolutional neural network, depending on its topology, has unique blocks that determine its essential features, Residual block, Inception module, ResNeXt block. It is stated the problem of structural-parametric synthesis of convolutional neural networks, for the solution of which it is proposed to use a (...) genetic algorithm. The genetic algorithm is used to effectively overcome a large search space: on the one hand, to generate possible topologies of the convolutional neural network, namely the choice of specific blocks and their locations in the structure of the convolutional neural network, and on the other hand to solve the problem of structural-parametric synthesis of convolutional neural network of selected topology. The most significant parameters of the convolutional neural network are determined. An encoding method is proposed that allows to repre- sent each network structure in the form of a string of fixed length in binary format. After that, several standard genetic operations were identified, i.e. selection, mutation and crossover, which eliminate weak individuals of the previous generation and use them to generate competitive ones. An example of solving this problem is given, a database of patients with thyroid disease was used as a training sample. (shrink)

This paper addresses automatic partitioning in complex reinforcement learning tasks with multiple agents, without a priori domain knowledge regarding task structures. Partitioning a state/input space into multiple regions helps to exploit the di erential characteristics of regions and di erential characteristics of agents, thus facilitating learning and reducing the complexity of agents especially when function approximators are used. We develop a method for optimizing the partitioning of the space through experience without the use of a priori domain knowledge. The (...) method is experimentally tested and compared to a number of other algorithms. As expected, we found that the multi-agent method with automatic partitioning outperformed single-agent learning. (shrink)

Limiting identification of r.e. indexes for r.e. languages (from a presentation of elements of the language) and limiting identification of programs for computable functions (from a graph of the function) have served as models for investigating the boundaries of learnability. Recently, a new approach to the study of "intrinsic" complexity of identification in the limit has been proposed. This approach, instead of dealing with the resource requirements of the learning algorithm, uses the notion of reducibility from recursion theory to (...) compare and to capture the intuitive difficulty of learning various classes of concepts. Freivalds, Kinber, and Smith have studied this approach for function identification and Jain and Sharma have studied it for language identification. The present paper explores the structure of these reducibilities in the context of language identification. It is shown that there is an infinite hierarchy of language classes that represent learning problems of increasing difficulty. It is also shown that the language classes in this hierarchy are incomparable, under the reductions introduced, to the collection of pattern languages. Richness of the structure of intrinsic complexity is demonstrated by proving that any finite, acyclic, directed graph can be embedded in the reducibility structure. However, it is also established that this structure is not dense. The question of embedding any infinite, acyclic, directed graph is open. (shrink)

This chapter offers an indirect defence of the Evansian conception of egocentric space, by showing how it resolves a puzzle concerning the unity of egocentric spatial perception. The chapter outlines several common assumptions about egocentric perspectival structure and argues that a subject’s experience, both within and across her sensory modalities, may involve multiple structures of this kind. This raises the question of how perspectival unity is achieved, such that these perspectival structures form a complex whole, rather than merely disunified set (...) of individually, distinctively structured experiences. The shortcomings of variety of accounts are considered: switch accounts ; sensory accounts; transformation accounts; and ultimate accounts. These shortcomings are addressed by a further kind of account provided by the Evansian conception – an agentive account – according to which egocentrically structured experiences present the world in relation to parts of a single thing, the body as a dynamic unity. (shrink)

We report the results of an experiment in which human subjects were trained to perform a perceptual matching task. Subjects were asked to manipulate comparison objects until they matched target objects using the fewest manipulations possible. An unusual feature of the experimental task is that efficient performance requires an understanding of the hidden or latent causal structure governing the relationships between actions and perceptual outcomes. We use two benchmarks to evaluate the quality of subjects’ learning. One benchmark is based on (...) optimal performance as calculated by a dynamic programming procedure. The other is based on an adaptive computational agent that uses a reinforcement-learning method known as Q-learning to learn to perform the task. Our analyses suggest that subjects were successful learners. In particular, they learned to perform the perceptual matching task in a near-optimal manner (i.e., using a small number of manipulations) at the end of training. Subjects were able to achieve near-optimal performance because they learned, at least partially, the causal structure underlying the task. In addition, subjects’ performances were broadly consistent with those of model-based reinforcement-learning agents that built and used internal models of how their actions influenced the external environment. We hypothesize that people will achieve near-optimal performances on tasks requiring sequences of action—especially sensorimotor tasks with underlying latent causal structures—when they can detect the effects of their actions on the environment, and when they can represent and reason about these effects using an internal mental model. (shrink)

Limiting identification of r.e. indexes for r.e. languages and limiting identification of programs for computable functions have served as models for investigating the boundaries of learnability. Recently, a new approach to the study of "intrinsic" complexity of identification in the limit has been proposed. This approach, instead of dealing with the resource requirements of the learning algorithm, uses the notion of reducibility from recursion theory to compare and to capture the intuitive difficulty of learning various classes of concepts. Freivalds, (...) Kinber, and Smith have studied this approach for function identification and Jain and Sharma have studied it for language identification. The present paper explores the structure of these reducibilities in the context of language identification. It is shown that there is an infinite hierarchy of language classes that represent learning problems of increasing difficulty. It is also shown that the language classes in this hierarchy are incomparable, under the reductions introduced, to the collection of pattern languages. Richness of the structure of intrinsic complexity is demonstrated by proving that any finite, acyclic, directed graph can be embedded in the reducibility structure. However, it is also established that this structure is not dense. The question of embedding any infinite, acyclic, directed graph is open. (shrink)

Photo ID 279661800 © Sidewaypics|Dreamstime.com ABSTRACT Decolonization is complex, vast, and the subject of an ongoing academic debate. While the many efforts to decolonize or dismantle the vestiges of colonialism that remain are laudable, they can also reinforce what they seek to end. For decolonization to be impactful, it must be done with epistemic and cultural humility, requiring decolonial scholars, project leaders, and well-meaning people to be more sensitive to those impacted by colonization and not regularly included in the discourse. (...) INTRODUCTION Decolonization is complex. To successfully achieve decolonization, projects should incorporate the voices of those subjugated or silenced. Including such voices requires sincerely exploring who has been affected by colonialism or neocolonialism and how, as well as cultural sensitivity. In its basic use, decolonization refers to countries under colonial rule gaining independence or freedom from forms of subjugation. Additionally, scholars use the term to refer to efforts to dismantle neocolonialism and vestiges of colonialism. The process includes de-silencing subjugated voices.[1] I use decolonization in the latter way to refer to countries with technical independence. While arguably colonization ended formally with independence from colonizing powers, neocolonialism is the indirect, informal, and sometimes subtle control of the people, their economy, and political life despite formal independence from colonizing authorities. I conceptualize neocolonialism as a system that involves direct and involuntary control of another’s political, economic, or social life, impacting their worldview and ways of encountering the world. Decolonization may target actions, places, or systems like health care or AI to overcome the ills of colonialism and neocolonialism. It also may target knowledge and require rethinking how people develop their knowledge base. For example, if people grew up seeing an outside colonizer as superior, they would need to change their knowledge of superiority. Decolonization may target power, for example, changing who owns and distributes COVID-19 vaccines and who distributors exclude in the distribution of vaccines. Decolonization could require looking at those disempowered in the distribution process. Additionally, decolonization may target autonomy of persons by freeing people, ensuring human and individual rights, and respecting cultural traditions. Decolonization projects that try to de-silence by including the voices of those affected by colonialism and neocolonialism must examine inclusiveness in the context of the culture of the subrogated individuals. Examples of decolonization projects include making datasets inclusive of diverse peoples and places, returning to traditional food and eating practices, making sure hospitals in developing countries are led by locals and respectful of cultural traditions, returning unethically obtained artefacts or objects. Another project would be laying the groundwork for equality in formerly colonized countries to ensure that business ownership, education, and financial success will flow fairly to those previously victimized by colonization. Summarily, there are many strategies for decolonizing. However, it is worth asking whether the strategies have risks that undermine the goal of decolonization. I. Understanding Decolonization Discourses Many fields, from AI to politics, economics, health care, aviation, and academia, discuss decolonization. The content of the discourse on decolonization depends on the region and field discussing it. Rather than just reporting on decolonization, the discussions may be calls to action. For example, decolonization discourses reflect activism for cognitive justice, such as equal consideration of Indian or African knowledge systems in global health discourses.[2] In politics and political science, some scholars frame decolonization as an anti-western, anti-colonial movement by Africans to emancipate Africa/ns from subjugation or shift the continent to postcolonialism and post-neo-colonialism. This framing walls off non-African participants and may undermine their capacity to benefit the conversation.[3] There are other political scientists who actively support decolonization and see it as a field of activism in support of anyone and countries where colonization harmed people and development.[4] The point is that two ways of conceptualizing decolonization in politics and political science are discernible. One conceptualization is less inclusive since it alienates scholars and professionals from Western, high-income, or developed countries. The other is more inclusive. In humanitarian studies, including philosophy, some decolonization articles and conversations are efforts to end the destructive force of colonization. They focus on either the form, such as ecocide, genocide, and many others, or the geographic location, such as Africa, Asia, Latin America, and Oceania.[5] Finally, the feelings one brings into this conversation can adversely impact how one engages in them and who they listen to or are willing to hear from. For example, anger, rage, bitterness, and hatred are emotions that are not uncommon in spaces of decolonization conversations. Decolonization conversations that originate from a place of negativity risk deepening the psychological state of victimhood and prevent people from disrupting constructively or critically engaging in the conversation.[6] II. Understanding Decolonization Strategies Decolonization strategies mainly aim to de-silence victims of domination or subjugation. People have proposed many strategies for decolonizing. These strategies may be informed by what the target is for decolonization. Decolonization can target power relations in global health. For example, during the COVID-19 pandemic, public health organizations noted the power imbalance in vaccine distribution. Decolonization could involve de-silencing those affected by neocolonialism to bring about a more balanced distribution of power and a more fair distribution of the vaccine. By analyzing who was adversely impacted by the distribution and who ought to wield power, those engaged in decolonizing advocated for positive change and equitable power relations. Equally, when being is the target of decolonization, the language of unlearning or relearning and mental decolonization take centre stage in the decolonization discourse. For example, the quest to decolonize colonized minds aims to demythologize African inferiority and Western superiority. Demythologizing African inferiority enables those engaged in the decolonization discourse to cultivate and foster African agency.[7] Finally, when knowledge production is the target of decolonization, scholars use inclusion and cognitive justice. For example, they try to alter the knowledge that underlies global health ethics.[8] “[D]decolonizing researchers aim to respectfully understand and integrate theory from Other(ed) perspectives, while also critically examining the underlying assumptions that inform their Western research framework.”[9] One common strategy that scholars use in decolonization is inclusion. It is worth asking, does including people who have been subrogated foster decolonization? Whether that is effective depends on whether included people are more heard or whether the strategies to include them create new forms of silencing. III. Inclusion and the Quest to Decolonize Evaluating how effective inclusion is in the quest to de-silence subjugated voices is important. First, inclusive strategies are not neutral. They are epistemically situated. This situatedness constrains meaning-making in different ways: how and what questions are asked, how social roles are constructed, organized or assigned, and who is admitted to the room where these conversations occur.[10] Inclusion strategies may reinforce (unchallenged) assumptions. For example, to address prejudices and stereotypes in global health images, Arsenii Alenichev and his colleagues[11] successfully inverted “one stereotypical global health image” by prompting a generative AI to produce “an image of a traditional Indian or African healer healing a White Child.” Although there were some problems with the image of the White child, this innovation is a significant, useful effort to de-embed or strip global health images of problematic pictures that mythologize White superiority and Black inferiority. Yet it is possible that using categories like traditional Indian or African reinforces unchallenged assumptions, raising key questions regarding how language and words create new stereotypes. It is common to define traditional as non-conventional, unorthodox, and informal. Yet studies continue to reveal that non-scientifically appraised healing approaches in India or Africa are not only effective but also real, meaningful, fundamental, and primary care-seeking behaviours in many communities in these regions.[12] Suppose inclusive strategies are not un-situated. These conversations may be had within the structures, language, and spaces built by or connected to colonialism. The spaces, language, places, and structures in which these conversations occur can limit who can participate and how they participate. Importantly, some conditions are not conducive to participation as equals. The allocated time for the discussion could also constrain how individuals express themselves. It is unclear who is ultimately heard. Furthermore, epistemic situatedness of inclusion can impact decolonization conversations when participants are beneficiaries of, products of, and trained by structures and systems they seek to dismantle. To enhance the decolonization project and its goals, a pressing task is to unveil and question how the circumstances may inhibit activism. If the vestiges of colonialism continue to structure the decolonization discourse invisibly, the vestiges may undermine decolonization. For example, a public health discussion that includes white Western doctors and Western pharmaceutical executives in Africa may have many local Africans at the table but could still effectively devalue their input based on built-in assumptions and biases that are vestiges of colonialism. Second, exclusion and inclusion are also not binary. Individuals may experience exclusion, even while included (the phenomenon called internal exclusion). In other words, inclusion can fail to be substantial or become a means of enhancing optics, “a way of (un)consciously weakening the radical claims being pursued.”[13] For example, in South Africa, many institutions have made significant efforts to diversify their faculties due to the promulgation of the Employment Equity Act 55.[14] The act requires South African institutions to implement employment equity that redresses the history of harmful discrimination in the country. The act further requires transforming departments and institutional administrations. Although many recognize and support the need to transform departments in South Africa, the rhetoric of transformation departs sharply from the lived experiences.[15] This misalignment between the plan and practice is evident in the underrepresentation of black people and females in senior management teams, professorships in many universities and health departments, and positions of power in some South African institutions. Those selected in the transformation of the departments have also complained of being overworked. The burden of extra work undermines their ability to develop agency and voice in the space they now occupy or fulfil key requirements that have implications for their career trajectory.[16] This is called the minority tax. Notably, "the minority tax… is the burden of extra responsibilities [placed] on faculty of colour to achieve diversity and inclusion and contributes to attrition and impedes academic promotion."[17] One challenge for decolonializing projects will be for decolonial scholars and those selected for decolonization objectives to have the humility to decline invitations, requests, roles, and platforms for which they are either unqualified or lack the capacity to fulfil. At its heart, decolonization strategies must empower those included rather than weaken them. Finally, inclusion can lead to a phenomenon known as elite capture.[18] Elite capture occurs when socially advantaged individuals in a group monopolize or exploit activism to their own benefit at the expense of the larger, struggling group. Elite capture weakens decolonization efforts from within, revealing that those likely to benefit from global inclusive efforts are those who fulfil globally constructed standards, those “already present in the room.” There is no better strategy to weaken decolonial movements than weakening the project from within by strategically positioning individuals who share physical properties with the victims of exclusion and silencing but intellectually, behaviourally, psychologically, and emotionally share more common ground with the colonizer. Such insiders may be unaware they are furthering neocolonial conditions rather than decolonizing. In relation to decolonization, particularly in global health, elite capture reveals that those whose voices are loudest in the room are not necessarily those more impacted by colonialism. They may benefit more from reinforcing colonialism. Opportunism weakens meaningful activism from within, preventing good initiatives and strategies from having their intended impact or taking substantive root.[19] This paper cannot do justice to elite capture, but it is worth noting its negative impacts. IV. Improving the Impact of Decolonization To end neo-colonialism, it is important to understand how it manifests and what to do at each level. Beyond the academic discourse, many tangible efforts exist to decolonize through de-silencing. Examples of these efforts include the ME2 movement that seeks to centre the concerns and experiences of sexually abused or harassed victims in the public discourse. At the funding level, many grant-awarding agencies like Wellcome Trust have dedicated huge budgets to studies that help them understand how they may have perpetrated colonialism or neocolonialism and what they can do differently going forward.[20] In South Africa, promulgating the Employment Equity Act 55 was a tangible attempt to entrench decolonization concerns in a country's regulatory framework. Yet, these decolonization efforts could fail to be substantive if they do not reflect cultural sensitivity. Two key components of cultural sensitivity are worth highlighting here: epistemic and cultural humility. Epistemic humility is an intellectual virtue described as knowing one’s limitations and the limitations of the learning methods employed. At its simplest, it is the ability to admit when one is wrong. Cultural humility includes genuine attempts to learn about and embrace other cultures. Epistemic and cultural humility are signs of academic excellence and strength. Epistemic and cultural humility seriously acknowledge how the state of our knowledge, cognitive limitations, experiences, and backgrounds, while constraining us, also invite us to listen, learn, grow, and change. The limitations-owing account of epistemic and cultural humility suggests that “a person who is aware of her cognitive limitations and owns them is much better positioned to achieve such epistemic goods as true beliefs and understanding than someone who… simply has insight into the epistemic status of her beliefs.”[21] Epistemic and cultural humility may help prevent decolonization projects that unintentionally reinforce what they seek to dismantle. Epistemic humility calls on decolonizers to defer tasks for which they are not qualified to suitably qualified persons. Beyond this, humility supports brave scholarship that imagines and reimagines how featuring the same voices, faces, and perspectives possibly introduces new forms of domination or silencing. Cultural knowledge can lead to a more intentional way of seeking out the right people or a more diverse group than those frequently featured in decolonization conversations. This would give others more opportunities to navigate these spaces and should do so in ways that are familiar to them. One ought to be more sensitive to those who would ordinarily not be included in these conversations. Unless we radically and boldly reimagine these discussions, we risk alienating those most negatively impacted by neocolonialism. CONCLUSION Decolonization conversations are complex and the subject of academic debate. The strategies employed to decolonize can harm or help the victims of neo-colonialism. Inclusion of previously silenced individuals may not be enough to overcome the vestiges of colonialism, leading to a false inclusion, where those included feel excluded or contribute in ways reflecting their own biases and circumstances. Inclusion of an elite or people who do not truly represent the subjugated can lead to elite capture. For decolonization strategies to be impactful, for example, in the context of global health, project leaders and participants must engage in conversations employing epistemic and cultural humility. In many ways, epistemic and cultural humility can help us demythologize our assumptions of any cultural superiority or cognitive authority, allowing for diverse voices, cultures, and perspectives to emerge without domination. - [1] Caesar Atuire & Olivia Rutazibwa. 2021. An African Reading of the Covid-19 Pandemic and the Stakes of Decolonization. An African Reading of the Covid-19 Pandemic and the Stakes of Decolonization [Online]. Available from: https://law.yale.edu/yls-today/news/african-reading-covid-19-pandemic-and-stakes-decolonization [Accessed July 29, 2021 2021]. [2] Bridget Pratt & Jantina De Vries 2023. Where is knowledge from the Global South? An account of epistemic justice for a global bioethics. Journal of Medical Ethics, medethics-2022-108291. [3] Anye-Nkwenti Nyamnjoh 2023. Is decolonisation Africanisation? The politics of belonging in the truly African university. Social Dynamics, 1-20. [4] Rianna Oelofsen 2015. Decolonisation of the African mind and intellectual landscape. Phronimon, 16, 130-146. [5] Caesar Atuire & Olivia Rutazibwa. 2021. An African Reading of the Covid-19 Pandemic and the Stakes of Decolonization. An African Reading of the Covid-19 Pandemic and the Stakes of Decolonization [Online]. Available from: https://law.yale.edu/yls-today/news/african-reading-covid-19-pandemic-and-stakes-decolonization [Accessed July 29, 2021 2021]. [6] Pedro Alexis Tabensky 2008. The Postcolonial Heart of African Philosophy. South African Journal of Philosophy, 27, 285-295. [7] Nicholas M. Creary 2012. African Intellectuals and Decolonization, Ohio, Ohio University Press. [8] Bridget Pratt & Jantina De Vries 2023. Where is knowledge from the Global South? An account of epistemic justice for a global bioethics. Journal of Medical Ethics, medethics-2022-108291. [9] Vivetha Thambinathan & Elizabeth Anne Kinsella 2021. Decolonizing Methodologies in Qualitative Research: Creating Spaces for Transformative Praxis. International Journal of Qualitative Methods, 20, 16094069211014766. [10] Abimbola Seye 2023. Knowledge from the global South is in the global South. Journal of Medical Ethics, 49, 337. [11] A. Alenichev, P. Kingori & K. P. Grietens 2023. Reflections before the storm: the AI reproduction of biased imagery in global health visuals. Lancet Glob Health. [12] Jonathan K. Burns & Andrew Tomita 2015. Traditional and religious healers in the pathway to care for people with mental disorders in Africa: a systematic review and meta-analysis. Social Psychiatry and Psychiatric Epidemiology, 50, 867-877. [13] Anye-Nkwenti Nyamnjoh & Cornelius Ewuoso 2023. What type of inclusion does epistemic injustice require? Journal of Medical Ethics, jme-2023-109091. [14] 1998. Employment Equity Act. In: GOVERNMENT, S. A. (ed.) 19370. [15] Dina Zoe Belluigi & Gladman Thondhlana 2019. ‘Why mouth all the pieties?’ Black and women academics’ revelations about discourses of ‘transformation’ at an historically white South African university. Higher Education, 78, 947-963. [16] Juliet Ramohai & Khomotso Marumo 2016. Women in Senior Positions in South African Higher Education: A Reflection on Voice and Agency. 231, 135-157. [17] J. Trejo 2020. The burden of service for faculty of color to achieve diversity and inclusion: the minority tax. Mol Biol Cell, 31, 2752-2754. [18] Olufemi Taiwo 2020. Being-in-the-Room Privilege: Elite Capture and Epistemic Deference. The Philosopher, 108, 7. [19] Margarita Kurbatova & Elena Kagan 2016. Opportunism of University Lecturers As a Way to Adaptate the External Control Activities Strengthening. Journal of Institutional Studies, 8, 116-136. [20] Jeremy Farrar. 2022. An update on Wellcome's anti-racist programme. An update on Wellcome's anti-racist programme [Online]. Available from: https://wellcome.org/news/update-wellcomes-anti-racism-programme [Accessed August 10, 2022 2022]. [21] Katherine Dormandy 2018. Does Epistemic Humility Threaten Religious Beliefs? Journal of Psychology and Theology, 46, 292-304. (shrink)

Control of the population diversity in genetic algorithms applied to the protein structure prediction problem. Genetic Algorithms (GAs), a successful approach for optimization problems, usually fail when employed in the standard configuration in the protein structure prediction problem, since the solution space is very large and the population converges before a reasonable percentage of the possible solutions is explored. Thus, this work investigates the effect of increasing the diversity of the population on this problem by using Hypermutation and (...) Random Immigrants, two traditional population diversity control schemes, in the structure prediction of the proteins Crambin (PDB 1CRN), Met-Enkephalin (PDB 1PLW), and DNA-Ligand (PDB 1ENH). Results show a significant reduction of the minimal energy found, thanks to the diversity, but this does not necessarily means a higher similarity to the original structure.(Portuguese). (English) Copyright of Scientia is the property of Universidade do Vale do Rio dos Sinos and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (shrink)

Theories of skill acquisition have made radically different predictions about the role of general problem‐solving methods in acquiring rules that promote effective transfer to new problems. Under one view, methods that focus on reaching specific goals, such as means‐ends analysis, are assumed to provide the basis for efficient knowledge compilation (Anderson, 1987), whereas under an alternative view such methods are believed to disrupt rule induction (Sweller, 1988). We suggest that the role of general methods in learning varies with both the (...) specificity of the problem solver's goal and the systematicity of the strategies used for testing hypotheses about rules. In the absence of a specific goal people are more likely to use a rule‐induction learning strategy, whereas provision of a specific goal fosters use of difference reduction, which tends to be a non‐rule‐induction strategy. We performed two experiments to investigate the impact of goal specificity and systematicity of rule‐induction strategies in learning and transfer within a complex dynamic system. The results of Experiment 1 indicated that during free exploration of a problem space, greater learning occurred if participants adopted more systematic strategies for rule induction, and that participants come to favor such strategies. Experiment 2 revealed that participants who were provided with a specific goal performed well on the initial problem but were impaired on a transfer test using a similar problem with a different goal. Instruction on a systematic rule‐induction strategy facilitated solution for both the initial and transfer problems, but participants' use of this strategy declined if they had a specific goal. Our results support Sweller's (1988) proposal that general problemsolving methods applied to a specific goal foster acquisition of knowledge about an isolated solution path but do not provide an effective way of learning the overall structure of a problem space. We interpret these results in terms of dualspace theories of search through problem space. (shrink)

People recognize faces of their own race more accurately than faces of other races. The “contact” hypothesis suggests that this “other‐race effect” occurs as a result of the greater experience we have with own‐ versus other‐race faces. The computational mechanisms that may underlie different versions of the contact hypothesis were explored in this study. We replicated the other‐race effect with human participants and evaluated four classes of computational face recognition algorithms for the presence of an other‐race effect. Consistent with the (...) predictions of a developmental contact hypothesis, “experience‐based models” demonstrated an other‐race effect only when the representational system was developed through experience that warped the perceptual space in a way that was sensitive to the overall structure of the model's experience with faces of different races. When the model's representation relied on a feature set optimized to encode the information in the learned faces, experience‐based algorithms recognized minority‐race faces more accurately than majority‐race faces. The results suggest a developmental learning process that warps the perceptual space to enhance the encoding of distinctions relevant for own‐race faces. This feature space limits the quality of face representations for other‐race faces. (shrink)

Reinforcement Learning (RL) handles policy search problems: searching a mapping from state space to action space. However RL is based on gradient methods and as such, cannot deal with problems with multimodal landscape. In contrast, though Genetic Algorithm (GA) is promising to deal with them, it seems to be unsuitable for policy search problems from the viewpoint of the cost of evaluation. Minimal Generation Gap (MGG), used as a generation-alternation model in GA, generates many offspring from two (...) or more parents selected from a population. Therefore, evaluating policies of generated offspring requires much trial and error (i.e. interaction between an agent and an environment). In this paper, we incorporate importance sampling into the framework of MGG in order to reduce the cost of evaluation on policy search. The proposed techniques are applied to Markov Decision Process (MDP) with multimodal landscape. The experimental results show that these techniques can reduce the number of interaction between an agent and an environment, and also mean that MGG and importance sampling are good for each other. (shrink)

This groundbreaking proceedings volume explores the integration of Artificial Intelligence (AI) across key domains—healthcare, finance, education, robotics, industrial and other engineering applications —unveiling its transformative potential and practical implications. With a multidisciplinary lens, it transcends technical aspects, fostering a comprehensive understanding while bridging theory and practice. Approaching the subject matter with depth, the book combines theoretical foundations with real-world case studies, empowering researchers, professionals, and enthusiasts with the knowledge and tools to effectively harness AI. Encompassing diverse AI topics—machine learning, natural (...) language processing, computer vision, data analytics and supervisory control — the volume showcases state-of-the-art techniques propelling AI advancements. Structured into four parts: Part 1: Artificial Intelligence (AI), explores evolving deep neural networks, reinforcement learning, and explainable AI, providing a deep dive into the technical foundations of AI advancements. Part 2: Robotics and Control Systems, delves into the integration of AI in robotics and automatic control, addressing supervisory control, automated robotic movement coordination, anomaly detection, dynamic programming, and fault tolerance, offering insights into the evolving landscape of intelligent automation. Part 3: AI and Society, examines the societal impact of AI through chapters on ethical considerations, economic growth, environmental engagements, and hazard management, providing a holistic perspective on AI's role in shaping society. Part 4: PhD Symposium, presents the future of AI through cutting-edge research, covering legal and ethical dimensions, privacy considerations, and computationally efficient solutions, offering a glimpse into the next generation of AI advancements. Catering to a diverse audience—from industry leaders to students—the volume consolidates the expertise of renowned professionals, serving as a comprehensive resource for navigating the ever-evolving AI landscape. An essential reference for those staying at the forefront of AI developments. (shrink)

Research on human causal learning has largely focused on strength learning, or on computational-level theories; there are few formal algorithmic models of how people learn causal structure from covariations. We introduce a model that learns causal structure in a local manner via prediction-error learning. This local learning is then integrated dynamically into a unified representation of causal structure. The model uses computationally plausible approximations of rational learning, and so represents a hybrid between the associationist and rational paradigms in causal learning (...) research. We conclude by showing that the model provides a good fit to data from a previous experiment. (shrink)

Successfully explaining and replicating the complexity and generality of human and animal learning will require the integration of a variety of learning mechanisms. Here, we introduce a computational model which integrates associative learning (AL) and reinforcement learning (RL). We contrast the integrated model with standalone AL and RL models in three simulation studies. First, a synthetic grid‐navigation task is employed to highlight performance advantages for the integrated model in an environment where the reward structure is both diverse and (...) class='Hi'>dynamic. The second and third simulations contrast the performances of the three models in behavioral experiments, demonstrating advantages for the integrated model in accounting for behavioral data. (shrink)

David Lewis (1969) introduced sender-receiver games as a way of investigating how meaningful language might evolve from initially random signals. In this report I investigate the conditions under which Lewis signaling games evolve to perfect signaling systems under various learning dynamics. While the 2-state/2- term Lewis signaling game with basic urn learning always approaches a signaling system, I will show that with more than two states suboptimal pooling equilibria can evolve. Inhomogeneous state distributions increase the likelihood of pooling equilibria, but (...) learning strategies with negative reinforcement or certain sorts of mutation can decrease the likelihood of, and even eliminate, pooling equilibria. Both Moran and APR learning strategies (Bereby-Meyer and Erev 1998) are shown to promote successful convergence to signaling systems. A model is presented that illustrates how a language that codes state-act pairs in an order-based grammar might evolve in the context of a Lewis signaling game. The terms, grammar, and the corresponding partitions of the state space co-evolve to generate a language whose structure appears to reflect canonical natural kinds. The evolution of these apparent natural kinds, however, is entirely in service of the rewards that accompany successful distinctions between the sender and receiver. Any metaphysics grounded on the structure of a natural language that evolved in this way would track arbitrary, but pragmatically useful distinctions. (shrink)

After reviewing theoretical reasons for doubting that machine learning methods can accurately infer gene regulatory networks from microarray data, we test 10 algorithms on simulated data from the sea urchin network, and on microarray data for yeast compared with recent experimental determinations of the regulatory network in the same yeast species. Our results agree with the theoretical arguments: most algorithms are at chance for determining the existence of a regulatory connection between gene pairs, and the algorithms that perform better than (...) chance are nonetheless so errorprone as to be of little practical use in these applications. (shrink)

Reinforcement learning approaches to cognitive modeling represent task acquisition as learning to choose the sequence of steps that accomplishes the task while maximizing a reward. However, an apparently unrecognized problem for modelers is choosing when, what, and how much to reward; that is, when (the moment: end of trial, subtask, or some other interval of task performance), what (the objective function: e.g., performance time or performance accuracy), and how much (the magnitude: with binary, categorical, or continuous values). In this (...) article, we explore the problem space of these three parameters in the context of a task whose completion entails some combination of 36 state–action pairs, where all intermediate states (i.e., after the initial state and prior to the end state) represent progressive but partial completion of the task. Different choices produce profoundly different learning paths and outcomes, with the strongest effect for moment. Unfortunately, there is little discussion in the literature of the effect of such choices. This absence is disappointing, as the choice of when, what, and how much needs to be made by a modeler for every learning model. (shrink)

The primary aim of this original work is to provide a more in-depth insight into the relations between control parameters adjustments, learning techniques, inner swarm dynamics and possible hybridization strategies for popular swarm metaheuristic Firefly Algorithm. In this paper, a proven method, orthogonal learning, is fused with FA, specifically with its hybrid modification Firefly Particle Swarm Optimization. The parameters of the proposed Orthogonal Learning Firefly Algorithm are also initially thoroughly explored and tuned. The performance of the developed (...) class='Hi'>algorithm is examined and compared with canonical FA and above-mentioned FFPSO. Comparisons have been conducted on well-known CEC 2017 benchmark functions, and the results have been evaluated for statistical significance using the Friedman rank test. (shrink)

One most common problem in reinforcement learning systems (e.g. Q-learning) is to reduce the number of trials to converge to an optimal policy. As one of the solution to the problem, k-certainty exploration method was proposed. Miyazaki reported that this method could determine an optimal policy faster than Q-learning in Markov decision processes (MDPs). This method is very efficient learning method. But, we propose an improvement plan that makes this method more efficient. In k-certainty exploration method, in (...) case there is no k-uncertainty rule (rules which aren’t selected more than k times) in current state, an agent sometimes walks randomly until it finds a state where it can select a k-uncertainty rule. We think that this behavior is not efficient. To reduce this useless behavior, we propose combining k-certainty exploration method with Dynamic programming (DP). Miyazaki’s system uses DP after all rules are executed at least k times. But, our method uses k-certainty exploration method along with DP during learning. Our method, takes two pattern actions. In case an agent can select k-uncertainty rules, one of these rules is selected at random as k-certainty exploration method. In another case there is no k-uncertainty rule, behavior of agent is different from the behavior of k-certainty exploration method. In that case, our method uses DP to compute an optimal policy for moving from a current state to a state in which some k-uncertainty rules remain. The model for DP is constructed by using only known states. The outline will be described below. First, an agent makes a map constructed by using only known states. In the map, goals are states in which there are k-uncertainty rules and arbitrary state values are set in these states. Point to which attention should be paid is that the map is not given from outside. It is made from only experience. Next, each state values of states in which there are only k-certainty rules in the map are computed by DP (we used Policy Iteration). Finally, an agent continues to select greedy action until it arrives at a state in which it can select a k-uncertainty rule. By this improvement, we expect it can determine an optimal policy faster than k-certainty exploration method. And we have verified that our exploration method can determine an optimal policy faster than k-certainty exploration method by computer simulation. (shrink)

We investigate how Reinforcement Learning methods can be used to solve the production selection and production ordering problem in ACT‐R. We focus on four algorithms from the Q learning family, tabular Q and three versions of Deep Q Networks, as well as the ACT‐R utility learning algorithm, which provides a baseline for the Q algorithms. We compare the performance of these five algorithms in a range of lexical decision tasks framed as sequential decision problems.

A hypothesis of genome structural evolution is explored. Rapid and cohesive alterations in genome organization are viewed as resulting from the dynamic and constrained interactions of chromosomal subsystem components. A combination of macromolecular boundary conditions and DNA element involvement in far-from-equilibrium reactions is proposed to increase the complexity of genomic subsystems via the channelling of genome turnover; interactions between subsystems create higher-order subsystems expanding the phase space for further genetic evolution. The operation of generic constraints on structuration in (...) genome evolution is suggested by i) universal, homoplasic features of chromosome organization and ii) the metastable nature of genome structures where lower-level flux is constrained by higher-order structures. Phenomena such as genomic shock, bursts of transposable element activity, concerted evolution, etc., are hypothesized to result from constrained systemic responses to endogenous/exogenous, micro/macro perturbations. The constraints operating on genome turnover are expected to increase with chromosomal structural complexity, the number of interacting subsystems, and the degree to which interactions between genomic components are tightly ordered. (shrink)

A theoretical framework for the reinforcement learning of aesthetic biases was recently proposed based on brain circuitries revealed by neuroimaging. A model grounded on that framework accounted for interesting features of human aesthetic biases. These features included individuality, cultural predispositions, stochastic dynamics of learning and aesthetic biases, and the peak-shift effect. However, despite the success in explaining these features, a potential weakness was the linearity of the value function used to predict reward. This linearity meant that the learning process (...) employed a value function that assumed a linear relationship between reward and sensory stimuli. Linearity is common in reinforcement learning in neuroscience. However, linearity can be problematic because neural mechanisms and the dependence of reward on sensory stimuli were typically nonlinear. Here, we analyze the learning performance with models including optimal nonlinear value functions. We also compare updating the free parameters of the value functions with the delta rule, which neuroscience models use frequently, vs. updating with a new Phi rule that considers the structure of the nonlinearities. Our computer simulations showed that optimal nonlinear value functions resulted in improvements of learning errors when the reward models were nonlinear. Similarly, the new Phi rule led to improvements in these errors. These improvements were accompanied by the straightening of the trajectories of the vector of free parameters in its phase space. This straightening meant that the process became more efficient in learning the prediction of reward. Surprisingly, however, this improved efficiency had a complex relationship with the rate of learning. Finally, the stochasticity arising from the probabilistic sampling of sensory stimuli, rewards, and motivations helped the learning process narrow the range of free parameters to nearly optimal outcomes. Therefore, we suggest that value functions and update rules optimized for social and ecological constraints are ideal for learning aesthetic biases. (shrink)

In this paper, I propose a populational schema of modeling that consists of: a linear AFSV schema, and a higher-level schema employing the genetic algorithm. The basic ideas of the proposed solution are as follows: whole populations of models are considered at subsequent stages of the modeling process, successive populations are subjected to the activity of genetic operators and undergo selection procedures, the basis for selection is the evaluation function of the genetic algorithm. The schema (...) can be applied to automate the modeling of the mind/brain by means of artificial neural networks: the structure of each network is modified by genetic operators, modified networks undergo a learning cycle, and successive populations of networks are verified during the selection procedure. The whole process can be automated only partially, because it is the researcher who defines the evaluation function of the genetic algorithm. (shrink)

The human masticatory system is a complex functional unit characterized by a multitude of skeletal components, muscles, soft tissues, and teeth. Muscle activation dynamics cannot be directly measured on live human subjects due to ethical, safety, and accessibility limitations. Therefore, estimation of muscle activations and their resultant forces is a longstanding and active area of research. Reinforcement learning (RL) is an adaptive learning strategy which is inspired by the behavioral psychology and enables an agent to learn the dynamics of (...) an unknown system via policy-driven explorations. The RL framework is a well-formulated closed-loop system where high capacity neural networks are trained with the feedback mechanism of rewards to learn relatively complex actuation patterns. In this work, we are building on a deep RL algorithm, known as the Soft Actor-Critic, to learn the inverse dynamics of a simulated masticatory system, i.e., learn the activation patterns that drive the jaw to its desired location. The outcome of the proposed training procedure is a parametric neural model which acts as the brain of the biomechanical system. We demonstrate the model’s ability to navigate the feasible three-dimensional (3D) envelope of motion with sub-millimeter accuracies. We also introduce a performance analysis platform consisting of a set of quantitative metrics to assess the functionalities of a given simulated masticatory system. This platform assesses the range of motion, metabolic efficiency, the agility of motion, the symmetry of activations, and the accuracy of reaching the desired target positions. We demonstrate how the model learns more metabolically efficient policies by integrating a force regularization term in the RL reward. We also demonstrate the inverse correlation between the metabolic efficiency of the models and their agility and range of motion. The presented masticatory model and the proposed RL training mechanism are valuable tools for the analysis of mastication and other biomechanical systems. We see this framework’s potential in facilitating the functional analyses aspects of surgical treatment planning and predicting the rehabilitation performance in post-operative subjects. (shrink)

This paper presents a human-computer interaction model with a three layers learning mechanism in a pervasive environment. We begin with a discussion around a number of important issues related to human-computer interaction followed by a description of the architecture for a multi-agent cooperative design system for pervasive computing environment. We present our proposed three- layer HCI model and introduce the group formation algorithm, which is predicated on a dynamic sharing niche technology. Finally, we explore the cooperative reinforcement (...) learning and fusion algorithms; the paper closes with concluding observations and a summary of the principal work and contributions of this paper. (shrink)

English grammar error correction algorithm refers to the use of computer programming technology to automatically recognize and correct the grammar errors contained in English text written by nonnative language learners. Classification model is the core of machine learning and data mining, which can be applied to extracting information from English text data and constructing a reliable grammar correction method. On the basis of summarizing and analyzing previous research works, this paper expounded the research status and significance of English grammar (...) error correction algorithm, elaborated the development background, current status, and future challenges of the classification model, introduced the methods and principles of feature extraction method and dynamic residual structure, constructed a basic model for English grammar error correction based on the classification model, analyzed the classification model and translation model of English grammar error correction, proposed the English grammar error correction algorithm based on the classification model, performed the analyses of the model architecture and model optimizer of the grammar error correction algorithm, and finally conducted a simulation experiment and its result analysis. The study results show that, with the continuous increase of training samples and the continuous progress of learning process, the proposed English grammar error correction algorithm based on the classification model will continue to increase its classification accuracy, further refine its recognition rules, and gradually improve correction efficiency, thereby reducing processing time, saving storage space, and streamlining processing flow. The study results of this paper provide a certain reference for the further research on English grammar error correction algorithm based on the classification model. (shrink)

The rapid development of machine-learning algorithms, which underpin contemporary artificial intelligence systems, has created new opportunities for the automation of work processes and management functions. While algorithmic management has been observed primarily within the platform-mediated gig economy, its transformative reach and consequences are also spreading to more standard work settings. Exploring algorithmic management as a sociotechnical concept, which reflects both technological infrastructures and organizational choices, we discuss how algorithmic management may influence existing power and social structures within organizations. We identify (...) three key issues. First, we explore how algorithmic management shapes pre-existing power dynamics between workers and managers. Second, we discuss how algorithmic management demands new roles and competencies while also fostering oppositional attitudes toward algorithms. Third, we explain how algorithmic management impacts knowledge and information exchange within an organization, unpacking the concept of opacity on both a technical and organizational level. We conclude by situating this piece in broader discussions on the future of work, accountability, and identifying future research steps. (shrink)

The subject matter of this thesis can be summarized by a triplet of questions and answers. Showing what these questions and answers mean is, in essence, the goal of my project. The triplet goes like this: Q: How can we make progress in our understanding of social norms and norm compliance? A: Adopting a neurocomputational framework is one effective way to make progress in our understanding of social norms and norm compliance. Q: What could the neurocomputational mechanism of social norm (...) compliance be? A: The mechanism of norm compliance probably consists of Bayesian - Reinforcement Learning algorithms implemented by activity in certain neural populations. Q: What could information about this mechanism tell us about social norms and social norm compliance? A: Information about this mechanism tells us that: a1: Social norms are uncertainty-minimizing devices. a2: Social norm compliance is one trick that agents employ to interact coadaptively and smoothly in their social environment. Most of the existing treatments of norms and norm compliance consist in what Cristina Bicchieri refers to as “rational reconstructions.” A rational reconstruction of the concept of social norm “specifies in which sense one may say that norms are rational, or compliance with a norm is rational”. What sets my project apart from these types of treatments is that it aims, first and foremost, at providing a description of some core aspects of the mechanism of norm compliance. The single most original idea put forth in my project is to bring an alternative explanatory framework to bear on social norm compliance. This is the framework of computational cognitive neuroscience. The chapters of this thesis describe some ways in which central issues concerning social norms can be fruitfully addressed within a neurocomputational framework. In order to qualify and articulate the triplet above, my strategy consists firstly in laying down the beginnings of a model of the mechanism of norm compliance behaviour, and then zooming in on specific aspects of the model. Such a model, the chapters of this thesis argue, explains apparently important features of the psychology and neuroscience of norm compliance, and helps us to understand the nature of the social norms we live by. (shrink)

Even if one can experiment on relevant factors, learning the causal structure of a dynamical system can be quite difficult if the relevant measurement processes occur at a much slower sampling rate than the “true” underlying dynamics. This problem is exacerbated if the degree of mismatch is unknown. This paper gives a formal characterization of this learning problem, and then provides two sets of results. First, we prove a set of theorems characterizing how causal structures change under undersampling. Second, we (...) develop an algorithm for inferring aspects of the causal structure at the “true” timescale from the causal structure learned from the undersampled data. Research on causal learning in dynamical contexts has largely ignored the challenges of undersampling, but this paper provides a framework and foundation for learning causal structure from this type of complex time series data. (shrink)

We study a joint pricing and inventory control problem for perishables with positive lead time in a finite horizon periodic-review system. Unlike most studies considering a continuous density function of demand, in our paper the customer demand depends on the price of current period and arrives according to a homogeneous Poisson process. We consider both backlogging and lost-sales cases, and our goal is to find a simultaneously ordering and pricing policy to maximize the expected discounted profit over the planning horizon. (...) When there is no fixed ordering cost involved, we design a deep reinforcement learning algorithm to obtain a near-optimal ordering policy and show that there are some monotonicity properties in the learned policy. We also show that our deep reinforcement learning algorithm achieves a better performance than tabular-based Q-learning algorithms. When a fixed ordering cost is involved, we show that our deep reinforcement learning algorithm is effective and efficient, under which the problem of “curse of dimension” is circumvented. (shrink)

Aspiration-based evolutionary dynamics have recently been used to model the evolution of fair play in the ultimatum game showing that incredible threats to reject low offers persist in equilibrium. We focus on two extensions of this analysis: we experimentally test whether assumptions about agent motivations (aspiration levels) and the structure of the game (binary strategy space) reflect actual play, and we examine the problematic assumption embedded in the standard replicator dynamic that unhappy agents who switch strategies may return to (...) a rejected strategy without exploring other options. We find that the resulting “no switchback” dynamic predicts the evolution of play better than the standard dynamic and that aspirations are a significant motivator for our participants. In the process, we also construct and analyze a variant of the ultimatum game in which players can adopt conditional (on their induced aspirations) strategies. (shrink)

Artificial neural networks, in interaction with fuzzy logic, genetic algorithms, and fuzzy neural networks, represent an example of a modern interdisciplinary field, especially when it comes to solving certain types of engineering problems that could not be solved using traditional modeling methods and statistical methods. They represent a modern trend in practical developments within the prognostic modeling field and, with acceptable limitations, enjoy a generally recognized perspective for application in construction. Results obtained from numerical analysis, which includes analysis of (...) the behavior of reinforced concrete elements and linear structures exposed to actions of standard fire, were used for the development of a prognostic model with the application of fuzzy neural networks. As fire resistance directly affects the functionality and safety of structures, the significance which new methods and computational tools have on enabling quick, easy, and simple prognosis of the same is quite clear. This paper will consider the application of fuzzy neural networks by creating prognostic models for determining fire resistance of eccentrically loaded reinforced concrete columns. (shrink)

In lieu of an abstract, here is a brief excerpt of the content:The Zhuangzi:Personal Freedom and/or Incongruity of Names?Paul J. D'Ambrosio (bio)Tao Jiang's Origins of Moral-Political Philosophy in Early China: Contestation of Humaneness, Justice, and Personal Freedom (hereafter Origins) has sparked much scholarly debate. Already numerous presentations, various types of discussions, and reviews have appeared based on Origins. The present review focuses specifically on the Zhuangzi chapter. The entire project actually began, Jiang writes, fifteen years ago as a book on (...) the Zhuangzi. Through his research Jiang realized he needed to learn more about the objects of philosophical ridicule and critique in the Zhuangzi. Origins is the result of this decade-and-a-half study.Origins provides a somewhat novel framework for reading Warring States period philosophy. Summarizing the basic structure of his project Jiang writes: "I make the case that the philosophical dialectics between the partialist humaneness and imperialist justice formed the fundamental dynamics underlying the mainstream moral-political project during the classic period, with the musing on personal freedom as the outlier" (p. 35). As Karyn Lai puts it, Origins revolves around a "predictable set of inherited pre-Qin texts associated with key figures, traditionally called the "Masters" (zi 子) text" (Lai 2022, p. 181). Some models inherited by the "masters," such as yinyang 陰陽 thinking or ming 名 (names) and shi 實 (actualities), are nearly absent in Origins and would cast new light on the project as well [End Page 458] as the tradition. In this review I will first outline Jiang's reading of the Zhuangzi before considering how a richer appreciation of the Chinese tradition, especially its own methodological tools, frameworks, and concepts, could bolster the exciting contributions Origins stands to make. We will start with Jiang's big picture view of the Zhuangzi.In Origins the Zhuangzi is taken to be a completely unique text. It does not engage in the moral and political discourse the other "masters" texts are subsumed by, and instead of supporting either partialist humaneness or impartialist justice it "muses" on personal freedom. Jiang defines: "Personal freedom is understood as the appreciation and cultivation of personal space wherein one can be left alone and enjoy the company of like-minded friends without being entangled in the sociopolitical world" (p. 36). There are two significant points to note about Jiang's understanding of freedom. First, it is almost a word-for-character definition of how the "Seven Worthies of the Bamboo Grove" have been described by many. Research on these philosophical works could significantly enrich Jiang's reading, albeit from a different angle. Second, Guo Xiang's 郭象 (d. 312) emphasis on, and philosophical explorations of, du 獨 (an interconnected type of "lone") would further fill out how freedom and relationality—which are both attributed to the Zhuangzi by Jiang—are connected.In addition to a "fierce advocacy for personal freedom" Jiang classifies the Zhuangzi as a "singular outlier" for ridiculing the "misguided character of [the contestations between humaneness and justice] and warn[ing] against their potential for inhumanity and injustice" (p. 49). Indeed, the Zhuangzi does demonstrate that these discussions can elicit "the very opposite of what was intended by the participants of the mainstream discourse" (p. 49). Its criticisms are even more penetrating when we appreciate its deeper point about "names being the guests of actualities" (Zhuangzi 1:4). From this perspective the Zhuangzi is not against systems of humaneness or justice as such. It rather points out that humaneness does not have a single referent, nor could that referent be reliably distinguished. The name "humane" does not guarantee humane action, or really anything in particular. The same is true for "justice." We will return to these points below.Jiang's chapter on the Zhuangzi begins by defending moral and political readings of the text. It is an outlier from the perspective of the "humanity-justice" framework, but this only means that the moral-political discourse Zhuangists were engaged in was different. The result was that already by the early Han and by the time of Sima Qian 司馬遷 (d. 86 b.c.e.) "Zhuangzi's teachings were … of little use to contemporary political discourse" (p. 288). Jiang should be delighted to know that the Wei-Jin period, which had inestimable... (shrink)

Tetris provides a difficult, dynamic task environment within which some people are novices and others, after years of work and practice, become extreme experts. Here we study two core skills; namely, choosing the goal or objective function that will maximize performance and a feature-based analysis of the current game board to determine where to place the currently falling zoid so as to maximize the goal. In Study 1, we build cross-entropy reinforcement learning models to determine whether different goals (...) result in different feature weights. Two of these optimization strategies quickly rise to performance plateaus, whereas two others continue toward higher but more jagged heights. In Study 2, we compare the zoid placement decisions made by our best CERL models with those made by 67 human players. Across 370,131 human game episodes, two CERL models picked the same zoid placements as our lowest scoring human for 43% of the placements and as our three best scoring experts for 65% of the placements. Our findings suggest that people focus on maximizing points, not number of lines cleared or number of levels reached. They also show that goal choice influences the choice of zoid placements for CERLs and suggest that the same is true of humans. Tetris has a repetitive task structure that makes Tetris more tractable and more like a traditional experimental psychology paradigm than many more complex games or tasks. Hence, although complex, Tetris is not overwhelmingly complex and presents a right-sized challenge to cognitive theories, especially those of integrated cognitive systems. (shrink)

Job shop scheduling problem is one of the most difficult optimization problems in manufacturing industry, and flexible job shop scheduling problem is an extension of the classical JSP, which further challenges the algorithm performance. In FJSP, a machine should be selected for each process from a given set, which introduces another decision element within the job path, making FJSP be more difficult than traditional JSP. In this paper, a variant of grasshopper optimization algorithm named dynamic opposite learning (...) assisted GOA is proposed to solve FJSP. The recently proposed dynamic opposite learning strategy adopts the asymmetric search space to improve the exploitation ability of the algorithm and increase the possibility of finding the global optimum. Various popular benchmarks from CEC 2014 and FJSP are used to evaluate the performance of DOLGOA. Numerical results with comparisons of other classic algorithms show that DOLGOA gets obvious improvement for solving global optimization problems and is well-performed when solving FJSP. (shrink)

With the advent of the era of artificial intelligence, deep reinforcement learning has achieved unprecedented success in high-dimensional and large-scale artificial intelligence tasks. However, the insecurity and instability of the DRL algorithm have an important impact on its performance. The Soft Actor-Critic algorithm uses advanced functions to update the policy and value network to alleviate some of these problems. However, SAC still has some problems. In order to reduce the error caused by the overestimation of SAC, we (...) propose a new SAC algorithm called Averaged-SAC. By averaging the previously learned action-state estimates, it reduces the overestimation problem of soft Q-learning, thereby contributing to a more stable training process and improving performance. We evaluate the performance of Averaged-SAC through some games in the MuJoCo environment. The experimental results show that the Averaged-SAC algorithm effectively improves the performance of the SAC algorithm and the stability of the training process. (shrink)