Emergent Minds, Inc.

Loops, Metaloops, and Adaptive Lansdscapes

Abstract
A new model of neural learning is presented in which multifurcating and self-reinforcing feedback loops within a network of connected neural elements are the functional repositories of memory. Metaloops - coordinate interactions among these loops - represent the assimilation and integration of data and concepts within the network, and form stable peaks on an adaptive landscape. Competition between metaloops causes the network to make its own heuristic search through the universe of possible connection weights to find local optima. The network is trainable at any time in the event of an incorrect output by simply lowering the neuronal thresholds, thereby increasing neuronal activity, destabilizing extant metaloops, and initiating a forced search for new adaptive peaks. Additionally, the network is capable of autonomous adaptive exploration through the landscape by selective focusing on subloops within the network. The emergent ability to autonomously search for higher optima is a quantifiable measure of machine intelligence.

Introduction
Understanding how intelligent beings can successfully assimilate data and concepts into an integrated, functional world-model is one of the fundamental research goals of cognitive science. An extension of this objective is the development of the technological means to endow machines with these same capabilities. Although it might seem that the realization of the former must precede progress in the latter, this hindrance has not inhibited computer scientists from trying to build computerized minds (Franklin, 1995). Towards that end, artificial neural networks (ANNs) based loosely on the inner workings of human brains have been developed (Hopfield, 1982; Dayhoff, 1996), and have seen limited successes in various applications (Sánchez-Sinencio and Lau, 1992).

ANNs are networks of connected automata, implemented as analogues of neural cells and the synaptic connections between them. But the functional resemblance of these automata to real neurons is only superficial; most conventional models are manifestly unbrainlike in their architecture and operation, being essentially correlation algorithms that rely on elaborate mathematical computations in their learning and data processing (Zurada, 1995).

While conventional ANNs are able to recognize and classify data inputs, and are very robust with respect to sensitivity to noisy data, most are unable to manipulate or draw inferences or deductions from the data they classify (Marcus, 1998). Furthermore, they are incapable of autonomous experiential learning: when the training phase of network implementation is complete, learning ends, and the processing capability of a network is frozen at its current level of competence.

Many capabilities usually associated with human intelligence - integration of data and concepts, generalization, continuous learning - have remained frustratingly elusive to ANN technology. Indeed, many cognitive scientists have proposed that human brains must have additional pre-coded constructs that make knowledge representation and symbol manipulation possible, and that ANNs alone will never be sufficient to produce intelligence (e.g., Hofstadter, 1995; Hummel and Holyoak, 1997; Pinker and Prince, 1988; Fodor and Pylyshyn, 1988).

In response to these challenges, we present a new neural system. The new network is trainable at any time during the course of its operation, generally without the loss of previously learned faculties, and is capable of integrating new data into previously stored memories and concepts, continuously evaluating and updating the dynamic and adaptive connections between related concepts, and synthesizing these concepts into new broader generalizations or metaconcepts, all without resorting to complex mathematical formulae, random number generators, or predefined operational rules for high level symbol manipulation.

Network architecture and functioning
The neural learning system presented here differs in several respects from conventional ANN implementations. Following is list of critical structural and operational features of the network, generally modeled after what is known about the functioning of biological neurons (Dowling, 1992).

Connections between neurons

Conventional ANNs consist of neurons arranged in two or more layers, usually comprising an input layer, possibly one or more hidden layers, and an output layer. Adjacent layers are fully interconnected, and signal processing proceeds feedforward; that is, each neuron within a given layer receives input signals from every neuron in the layer behind it, and transmits its output to every neuron in the layer ahead of it. Simple feedback connections may be incorporated into the feedforward architecture such that the output signals of subsequent layers are incorporated into the input signals of preceding layers.

In the cerebral cortex of human brains, as well as in artificial neural systems that closely model them (e.g., Rochester, 1956), and in the learning system presented here, neurons may send connections to any other nearby neurons - unlike conventional ANN models in which connections occur only between specified layers and only in specified directions. Thus, in the present neural learning system there are no pre-defined macroneuronal structural elements like the layers (input, hidden, and output) of conventional feedforward ANNs, and any neuron within the network may be designated as a data input or data output neuron.

The pattern of neuronal connections of the present model, in which output signals radiate from each active neuron and reticulate throughout the network instead of being serially processed by successive layers, gives rise to a critical emergent feature within the network: loops and metaloops, described below.

Connection weights

The weight (w) of a connection between two neurons is a measure of the impact the signal from the transmitting neuron will have on the behavior of the receiving neuron. The weight may be positive in the case of an excitatory connection, or negative in the case of an inhibitory connection. In conventional ANNs, connection weights are adjusted only during the training phase; during the implementation phase all network parameters are fixed. Training such a network to produce the correct outputs is done by gradually modifying the weights either by applying mathematical equations that force the network output to converge upon the desired values or by introducing and evaluating random changes in network parameters, but never by autonomous information processing and never in response to new data or experience encountered after training.

Connection weights in human brains as well as in the present network model are subject to modifications continuously throughout the entire operational life of the network. Instead of a weight being modified according to its distal effect on a remote output neuron, a weight is updated according to the firing activity of the two connected neurons. Whenever the transmitting neuron fires and the receiving neuron also fires, either simultaneously or very shortly thereafter, the weight between them will be strengthened (Hebb, 1949; Tang, et al., 1999). Whenever either of the two neurons remains inactive the connection weight will decay.

The particular weight update rules implemented in the present model are illustrated in the figure below. Since weights may be positive or negative the weight update functions apply to their absolute values. There are two separate functions that define the value of the updated weight, |w_updated|, as a function of its erstwhile value, |w|. The weight strengthening curve lies above the diagonal line |w_updated|=|w|; the weight decay curve lies below the diagonal.

At higher weights the strengthening curve has a higher response, drawing them more rapidly toward |w_max|. At these values the decay curve asymptotically converges to the diagonal line, so that once a weight is at or near |w_max| it will tend to remain there despite prolonged inactivity. This ensures the retention of long term memories: repeated neuronal firings will increase the absolute values of the weights connecting them so that they may become virtually fixed.

Since |w_min| and |w_max| are attractors, the dynamic fluctuation of weights throughout the operation of the network results in a U-shaped distribution, with most of the weights clustered around their terminal values, and the remaining in some state of transit between them. Thus, the weights of established neuronal pathways and interacting circuits remain stable at their fixed points, while only the weights of circuits that are actively engaged in adaptive or predictive firing will be found at intermediate values.

Conventional ANNs require their internal weights to remain fixed with high precision at very specific values that are almost always intermediate within the range of minimum and maximum possible strengths. Since the values of their weights are not located on any kind of attractors, they will be vulnerable to noise and degeneration. Maintaining weights at arbitrary values is not difficult for virtual or digital hardware implementations, but it does present a challenge for analog circuits (Vittoz, et al., 1991), and most likely would for biological systems as well.

The attractor-based dynamics of the neural weights in the model presented here provides the network with the standard advantages associated with digital hardware - robust insensitivity to noise and degeneration. An established pathway will consist of the neural weight equivalents of 0s and 1s. Connection weights only need to be - and are automatically - their stable endpoints, and this can be accomplished easily in any type of implementation. Intermediate values are transitory fluctuations between the stable endpoints, and do not require great precision or stability. It is possible that the dynamics of biological neurons may be similar.

Continually updating the connection weights in response to local activity perpetuates their adaptive strengthening and decay, enabling the autonomous generation, maintenance, and modulation of integrated patterns of neuronal firings within the loops and metaloops (described below) that will be central to the adaptive integration of data and concepts within the network.

Neuron firing

In conventional ANN models, the weighted inputs (x's) from all neurons firing into a receiving neuron at any given moment are added together. The output signal (y) of this neuron is then a function of the summed inputs, and may be either binary (1 for firing; 0 or -1 for not firing) or some continuous mathematical function of the summed inputs. Symbolically, the output signal of a neuron at any given instant is y=f(Σxw). Regardless of the neuron's output, the summation of the weighted inputs is recalculated from zero at each time interval. Any input that does not cause the neuron to fire, or any portion of the input in excess of the neuron's firing threshold, will be unused and discarded.

Biological neurons, as well as those of ANNs that closely model them (e.g., Bugmann, 1992; Shigematsu, et al., 1992) and the neurons of the learning system presented here, continuously accumulate the values of the weighted inputs until the total is greater than the neuronal threshold, at which point the neuron fires; only then is the accumulated input reset to its basal level. The accumulated input summations may undergo a gradual attenuation process, and immediately after firing the neuron may have a brief refractory period during which it will be unresponsive to any further input, but generally the input signals have far less opportunity for inutility than they do in conventional ANNs.

When a neuron fires, the weight values of each of its separate output connections are transmitted to the corresponding receiving neurons. The continual accumulation of input signals by each neuron allows low-weight connections that might otherwise never have any influence exert their effect by repeated signaling that can accumulate and eventually cause the receiving neuron to fire. According to the weight update rules described above, a connection weight will strengthen when the connected neurons both fire. In this manner new neural circuits and active pathways comprised of linked firing neurons and strong connections between them may be generated. The development of active pathways will permit the process of adaptive exploration, described below.

After repeated or sustained use the connections along an active pathway may be strengthened until they become virtually fixed, achieving a state analogous to long-term potentiation in biological neurons. Alternatively, if the connections are not reinforced and the connection weights do not build up sufficient strength because the pathway is of low or only temporary utility, the weights will gradually decay. The network thus has the capacity for short and long term memory.

Loops and metaloops
Critical to a cognitive system is the ability to integrate new data into previously stored memories, classifications, and generalizations, forming relationships among them that may be adaptively strengthened or weakened. This is accomplished in the present model by means of an emergent property of the adaptive connections and network architecture: self-reinforcing, mutually interactive loops of firing neurons.

Structurally, loops are closed circuits of linked neurons connected in the same circular direction. Much like biological networks (Sholl, 1956), the present model is replete with them. If a loop consists of only excitatory connections, then once initiated, the neurons will fire in cycles around the loop, continuously strengthening their connections and forming a stable circuit. With inhibitory as well as excitatory connections more complicated circuits may be formed, depending on the strengths of the connections and the firing patterns of the neurons. Since individual connections can participate in a large number of loops, the functional commitment of each is quite substantial. Loops are thus highly interactive, and can be either mutually reinforcing or mutually competitive.

Loops are the fundamental units of stored memory. Higher circuit dimensions - metaloops - made up of hierarchical conflations of interacting loops, represent higher levels of integration and coordination of stored memories. Furthermore, the self-reinforcing nature of these circuits implemented by the weight update rules described above allows them to retain their activity, or at least their potential, even in the absence of data input into the network. These circuits may fire and interact in complete autonomy, forming new circuits and new pathways, creating new and higher levels of integration, independently processing stored data and concepts.

In the present model, loops are the primary operational unit, pervasive throughout the network, endowing the network with its emergent adaptive and integrative capabilities. Loops and metaloops serve the dual role of being both the repositories of stored information as well as the acting agents that form adaptive relationships between memories. While loops and metaloops are essentially the memory units of the network, the interactions among them can be viewed as the network's intelligence. Because environmental inputs occur sequentially, there exists an important temporal dimension to the associative connections formed between resulting loops. The order in which neurons, loops, and metaloops fire into one another provides a logical chain which can be viewed as the backbone of predictive intelligence. When the neuronal firing of one metaloop triggers the subsequent firing of another previously established metaloop, a chain-reaction of firing occurs, following out sequential, experienced-based paths of related memories. The network is exhibiting its emergent capability to draw analogies, extrapolate, predict consequences, or otherwise relate current activity to prior experience. In short, it is thinking.

These interactions among loops and metaloops make up the roadmap by which the network navigates its adaptive landscape, described next.

Metaloops and adaptive landscapes
Functional systems such as computer programs that must perform in complex environments are usually implemented as predetermined sequences of coded instructions that specify responses to data inputs that fall within defined boundaries. Preprogrammed routines have the advantages of being precise and knowable to the user. However, the performance potential is limited by the foresight and skill of the programmer, and such programs are often not very robust with respect to the ability to adapt to novel or changing environments.

An alternative approach is to let performance and adaptation occur autonomously, by repeated cycles of evaluating many small stochastic changes in functioning parameters, and selecting those most advantageous (Holland, 1975). Performance enhancements are found by meandering through multidimensional parameter space searching for slight improvements, just as in nature a species will gradually evolve changes in gene structures and frequencies in the search for genetic combinations better adapted to its environment (Wright, 1932).

An adaptive landscape is a metaphor for the fitness values of the universe of parameter combinations that an autonomous agent may blindly explore (Kauffman, 1993, Ch. 2). Peaks represent local optima in which performance is higher than that of any of its close neighbors; valleys represent local minima, where virtually any change would lead to improved function. As the agent explores neighboring regions in parameter space, it identifies those points along its periphery having higher fitness; as it moves toward them, it repeats this process of peak climbing until a local optimum is reached, at which point further improvement may not be possible. Since this process only permits direct and immediate improvements in performance, achieving a local optimum is almost certain. On the other hand, reaching the global optimum may be very unlikely without external stimuli or very deep searches through parameter space. The adaptive landscape itself may not be stable. Environmental fluctuations and even the mere act of traversing parameter space may cause peaks and valleys to shift or even disappear. As fitness surfaces become more complex and dynamic, attaining and maintaining a global maximum becomes more difficult.

An adaptive agent that is blindly but freely allowed to explore the fitness landscape will often achieve very high performance levels. Conventional ANN implementations generally suffer from the lack of adaptability in a complex and evolving environment because their connection weights are frozen at the completion of training, even though their weight values may have been trained according to optimization functions that arrive - and permanently remain - at local peaks (e.g., Hopfield, 1982; Hoppensteadt, 1986). Even ANNs that are trained using genetic algorithms (van Rooij et al., 1996) will be left static once training is complete, immobilized on the adaptive surface.

Optimum performance by the present neural learning system depends on the ability to successfully integrate data from the environment into previously stored classifications and generalizations - and the ability to generate new classifications and generalizations and integrate those with successively more inclusive hierarchical levels of metaclassifications. Fitness peaks on the adaptive landscape correspond to the successful integration of data and concepts; the higher the peak the more complete the integration. Integration of information is accomplished by forming coherent relationships among loops and metaloops. The more stable the interaction of active loops and active pathways are within the network, the higher the peak; the more discordant and unstable they are, the deeper the valley.

The parameter space through which a neural network must explore is made up of the universe of possible connection weights between every pair of connected neurons. As environmental data is perceived by the network, active pathways are initiated and interact with the loops and metaloops established in the network, and neuronal connections are accordingly adaptively strengthened or weakened. If the newly formed pathways and loops are in concert with previously established metaloops, then they will be mutually reinforcing and stable, and an adaptive peak will be reached. Continual re-affirmation of previously stored concepts and memories is critical to building a relatively stable foundation of firing patterns, which in a sense serve as the identity of the network. On the other hand, if newly formed pathways are in conflict with extant metaloops, the resulting instability will lead to desultory firings and fluctuating connection weights, causing the network to wander about the fitness surface until a point of relative stability is found, and a new peak ascended. This experiential invalidation of previously stored concepts and memories is equally valuable, insofar as it stimulates dynamic adaptation to environmental changes.

While the network can easily identify, create and climb adaptive peaks, most such peaks are only local optima. Higher peaks, representing even greater coherence and stability among interacting loops and pathways, may exist nearby, and yet remain inaccessible as long as they are separated by valleys. Since stable loops will not disrupt themselves, crossing an adaptive valley to reach a distant peak will often require the initiation of a more extensive search through the circumjacent landscape. Such a search is carried out by adaptive exploration, described in the next section.

Adaptive landscapes and adaptive exploration
The integrated network is a repository of stored memories and concepts, intertwined and interconnected, adapted to make sense of and function in its own environment. As data inputs are processed, active pathways and adaptive peaks are created and conjoined, giving birth to new loops and obliterating old ones. But extant metaloops may at times be functionally inadequate, improperly or incompletely classifying and generalizing information, occasionally producing a spurious output. Whenever the need arises, the architecture and functioning of the network permit adaptive exploration through the fitness landscape in search of other peaks.

To dislodge an active pathway off a peak, all that is required is an increase in network activity. If neurons within an active pathway increase their rate of firing then they will send more frequent signals to non-firing neurons (outside the current pathways), increasing the latter's probability of firing as they accumulate input signals. New pathways will branch off of existing active pathways and generate new loops or link up to and activate previously established but currently dormant loops and pathways. Since new pathways will contain inhibitory as well as excitatory connections that intersect with other pathways, currently active pathways and loops may become disrupted or dismantled as new adaptive peaks are created and ascended.

Increasing neuronal activity may accomplished by several ways, but most conveniently by temporarily lowering the thresholds of inactive neurons. As these neurons receive more frequent input signals from the active pathways, their probability of firing will be materially increased, enabling exploration activity around neighboring regions of the adaptive landscape. Only active pathways (and surrounding neurons) will be impacted, since threshold lowering will have no affect on neurons that are not receiving input. This training mechanism therefore has the distinct advantage of only impacting relevant connections; previously established and unrelated metaloops and pathways distributed throughout the network will generally be left intact.

The network can be trained in this manner. Adaptive exploration is initiated in response to an external signal provided by the user, somewhat analogous to a pain signal in biological systems. When a desired network output has been produced the signal is discontinued, causing the thresholds to rebound. Whichever newly-created pathways that actively participated in generating the correct output at the time the thresholds rebound will be maintained since every connection along that pathway will be strengthened. Therefore, nascent pathways and loops can quickly achieve coordinated stability with others in the network, having established themselves on new adaptive peaks. This procedure will be repeated as required throughout the life of the network.

Typical operation of the network will proceed as follows. Input data presented to the network will trigger a diffusion of neuronal firings. If the data are familiar to the network then the inputs may activate a previously established pathway. The activated pathways will then assimilate the data into extant loops and metaloops, eventually settling on an adaptive peak, and as a result of this processing, network output neurons may be activated. If incorrect or inadequate outputs are produced, either because the network has been incompletely trained or because the data are unfamiliar, then adaptive exploration may be initiated by the user by temporarily lowering neuronal thresholds. Input signals will hence be deflected and rerouted, and new peaks will be explored. New pathways and new loops may be formed, but they will be ephemeral if they fail to produce the desired output, as the process of adaptive exploration (and the resultant updating of connection weights) will conduce alternative pathways, causing weak and unreinforced pathways and loops to dematerialize as they are either abandoned or overrun. The weight update rules will also prevent previously fixed loops from becoming substantially affected by the chance encounter with exploratory pathways, no matter how useless or even deleterious the resulting output, except possibly after prolonged exploratory activity. As soon as a desired output has been produced the user will discontinue the threshold lowering signal, the thresholds will rebound to their normal levels, adaptive exploration will cease, and the network will have completed another round of training.

While lowering neuronal thresholds is a practical way to trigger adaptive exploration, this neural learning system is also capable of exploring its adaptive landscape autonomously, in the absence of external stimuli and threshold modulation, as the network strives on its own to reach higher fitness peaks. This process is described in the next two sections.

Autonomous Adaptive Exploration, Part I
The phenomenon of exploratory firing has important implications regarding the functions of sleeping and dreaming in biological neural networks, and their potentially critical role in artificial learning systems as well. On the face of it, sleep seems to be a rather peculiar candidate for evolutionary survival, yet no creature with a complex brain can live without it. Given that sleep is invariably accompanied by decreased sensory awareness, increased vulnerability to attack, and continued metabolic activity in the absence of resource consumption, one might expect that sleep must offer powerful adaptive benefits to compensate for the cost.

The authors conjecture that firing thresholds within biological networks are reduced during sleep, and in general, that firing thresholds within a network may vary in proportion to the amount of input signal the network receives. Thus, during periods of relative environmental stasis, firing thresholds within the network drop, and as a result, adaptive exploration is induced, offering recently acquired data and concepts the opportunity to crystallize and integrate into the network. New loops and metaloops begin making and breaking connections autonomously, both with each other and with pre-existing memories whose adaptive strength had allowed them to survive. Organizing and reorganizing themselves into higher fitness peaks, memories cooperate and compete for survival in a virtual microcosm of natural selection. Dreaming could simply be described as the subjective experience of the network passively bearing witness to the process of reorganization and exploration. Because loops and metaloops represent discreet circuits of sequential firing patterns, the process of shuffling them about in the absence of external sensory anchors avoids the spatial and temporal restrictions of association, without compromising the integrity of the individual memories being integrated. In other words, dreams often consist of memories of events or objects that were originally widely separated by space or time firing into one another directly, creating richly multidimensional conceptual associations.

Autonomous adaptive exploration of this sort may well be critical for the successful integration of data and concepts into the network's world model, but entails only the passive assimilation of newly acquired knowledge. The fountain head of genuine intelligence emerges from a more purposive adaptive exploration through the network's entire inner universe of sentient experience, described next.

Autonomous adaptive exploration, Part II
As the network forms loops and pathways that interact with each other by virtue of multiple shared connections, metaloops emerge consisting of the integration of many interrelated loops. Metaloops themselves become interrelated in higher hierarchical dimensions of metaloops. Ultimately, in a network of sufficient complexity both in terms of the number of neurons and connections and the number of interrelated loops and pathways, every loop will become incorporated into a single unified cohesive metaloop, embodying the collective repertoire of memories, classifications, and generalizations gleaned throughout its existence.

In such a single unified metaloop each subloop is mutually accessible to every other, and potentially able to form bonds of association with any other. Localized active pathways will circulate throughout the metaloop, advancing in directions determined by the patterns of connections and their weights, and guided by external data input. The active regions of the metaloop constitute the network's focus. Even in the absence of data inputs to the network, active pathways with enough internal momentum will circulate in autonomy and generate varying degrees of focusing.

With active pathways focused on a particular region of the metaloop, the activity of the neurons in those subloops correspondingly increases, and as activity increases, adaptive exploration is automatically initiated. In this manner, by the emergent and autonomous activity of the integrated network, and without any external signaling or parameter modulation, adaptive exploration may occur, the adaptive landscape may be traversed, new peaks ascended and old ones abandoned.

The network will not always be in a state of focusing and resultant exploration. Transitory pathways such as those connecting a familiar input to a previously established corresponding output may flicker only briefly, with insufficient activity to initiate adaptive exploration. And without prolonged periods of intense neural activity it is usually neither desirable nor possible to alter pathways that have been previously fixed. Nevertheless, the focusing mechanism will often enable the network to selectively and adaptively reconfigure its own internal pattern of connection weights in search of more fruitful integrations of its assimilated stores of data and concepts.

Manifesting the ability to autonomously abandon one adaptive peak in favor of another higher would constitute a demonstration of intelligent behavior. Indeed, testing for such an ability would be a more generally applicable test of machine intelligence than the famous Turing test (Turing, 1950). The peak jumping test may be applied by presenting to any artificial device a problem with several possible solutions of varying degrees of completeness that are not simple extensions of one another. To pass the test, the network must soon find one possible solution, and after some period of autonomous activity and in the absence of additional data input it must arrive at a superior solution.

As a benchmark for intelligent behavior, the ability to find higher adaptive peaks is a quantifiable measure of a network's inherent capabilities. The network parameters can be optimized according to the maximization of this capacity. Such parameters to be optimized include the number of connections per neuron, the neuronal thresholds, the coefficients that characterize the weight update curves, and the number of loops in which each connection participates. It is likely that certain optimum combinations of these parameters will produce networks that are more proficient than others, exhibiting greater intelligent behavior as manifested by the efficiency of autonomous adaptive exploration.

Acknowledgments
The authors thank Dr. Joon Yun for discussions and inspiration.

References

Bugmann, G. (1992). Multiplying with neurons: Compensation for irregular input spike trains by using time-dependent synaptic efficiencies. Biol. Cybern. 68: 87-92.

Dayhoff, J. (1996). Neural Network Architectures. Boston: International Thomson Computer Press.

Dowling, J. E. (1992). Neurons and Networks: An Introduction to Neuroscience. Cambridge, MA: Harvard University Press.

Fodor, J. A., & Pylyshyn, Z. W. (1988). Connectionism and cognitive architecture: A critical analysis. Cognition 28: 3-71.

Franklin, S. (1995). Artificial Minds. Cambridge: MIT Press.

Hebb, D. O. (1949). The Organization of Behavior: A Neuropsychological Theory. New York: Wiley.

Hofstadter, D. (1995). Fluid Concepts and Creative Analogies. New York: Basic Books.

Holland, J. H. (1975). Adaptation in Natural and Artificial Systems. Ann Arbor: University of Michigan Press.

Hopfield, J. (1982). Neural networks and physical systems with emergent collective computational abilities. PNAS 79, 2554-2558.

Hoppensteadt, F. C. (1986). An Introduction to the Mathematics of Neurons. Cambridge: Cambridge University Press.

Hummel, J. E., & Holyoak, K. J. (1997). Distributed representations of structure: A theory of analogical access and mapping. Psych. Rev. 104: 427-466.

Kauffman, S. A. (1993). The Origins of Order: Self-Organization and Selection in Evolution. Oxford: Oxford University Press.

Marcus, G. F. (1998). Rethinking eliminative connectionism. Cognit. Psych. 37: 243-282.

Pinker, S., & Prince, A. (1988). On language and connectionism: Analysis of a parallel distributed processing model of language acquisition. Cognition 28: 73-193.

Rochester, N., Holland, J. H., Haibt, L. H., & Duda, W. L. (1956). Tests on a cell assembly theory of the action of the brain, using a large digital computer. IRE Transactions of Information Theory, IT-2: 80-93.

Sánchez-Sinencio, E., & Lau, C. (Eds.) (1992). Artificial Neural Networks: Paradigms, Applications, and Hardware Implementations. New York: IEEE Press.

Shigematsu, Y., Matsumoto, G., & Akiyama, S. (1992). A dynamic neural cell model driven by pulse train, In: International Symposium on Neural Information Processing, Jul. 12-15, 1992 (pp. 130-133).

Sholl, D. A. (1956). The Organization of the Cerebral Cortex. London: Methuen.

Tang, Y.-P., Shimizu, E., Dube, G. R., Rampon, C., Kerchner, G. A., Zhuo, M., Liu, G., & Tsien, J. Z. (1999). Genetic enhancement of learning and memory in mice. Nature 401: 63-69.

Turing, A. M. (1950). Computing machinery and intelligence. Mind 59: 434-460.

van Rooij, A. J. F., Jain, L. C., & Johnson, R. P. (1996). Neural Network Training Using Genetic Algorithms, Series in Machine Perception and Artificial Intelligence, Vol. 26, H. Bunke & P. S. P. Wang (Eds.). Singapore: World Scientific Publishing Co.

Vittoz, E., H. Ougney, M. A. Maher, O. Nys, E. Dijkstra, and M. Chevroulet. (1991). Analog storage of adjustable weights, In: VLSI Design of Neural Networks, U. Ramacher, U. Rueckert (Eds.). Boston: Kluwer Academic Publishers.

Wright, S. (1932). The roles of mutation, inbreeding, crossbreeding, and selection in evolution. Proc. XI Internat. Congr. Genetics 1:356-366.

Zurada, J. M, (1995). Introduction to Artificial Neural Systems. Boston: PWS Publishing Company.