A Model of Human Cognition: From Observation to Theory

This article proposes a model of human cognition.

Experimental evidence suggests that humans are born with an innate capacity to both discover patterns through observation and imagination, and to construct progressively higher levels of representation from those observations. Learning and eventual understanding are the construction of progressively higher-order representational objects from observed patterns. Thinking is the recursive manipulation of those objects as they are tested, combined, and refined to create new representational objects that help explain future observations.

The model proposed here suggests that learning, understanding and thinking proceed through a hierarchy of typed higher-order operations:

At each level, many lower-level objects are processed into higher-level objects that in turn can be processed further.

At each level, a new type of object is introduced that defines how it can be processed by the next level. Thus, the hierarchy proceeds not by patterns of patterns but by semantically richer objects that support operations at a level of the hierarchy.

Observations, patterns, categories and relations

At the top of the hierarchy, an observation is made and added to all other observations, the output eventually being a pattern to which the observation reasonably matches.

Two operations take place. One is add(observation). The human has a number of proto-patterns and the new observation is added to the closest matching proto-pattern. To do this requires the second operation: match(observation).

Over time, a third operation occurs which turns a proto-pattern into a pattern. This new pattern has shown itself to be a coherent enough collection of related observations that help explain future observations. The pattern has utility for the human and is therefore worth expending energy to retain and use.

As time passes, output from higher-up the hierarchy is processed into high-level objects further down.

Patterns are processed into categories. The operation type is process([pattern]) -> category.

Categories are linked together to form relations (married_to). A person category can be linked together with another person category into the married_to relation. The operation type here is link_to([category]) -> relation.

An instance of using this type is link_to(person category, person category) -> married_to(A, B), where A and B are instances of a person category.

Thus, as we move down the hierarchy, semantic value is added through operations that have different types. They have differing types as the inputs and outputs at each level are different, and the value added depends on the type of object being processed at each level.

For example:

• John is married to Mary
• Sarah is married to David
• Alice is married to Ben

becomes married_to(A,B), which may later become SymmetricRelation, which may become part of TheoryOfRelations.

Hierarchy or network

The hierarchy above implies that to get to a higher-order object, the learner must have first completed all lower levels. However, human cognition may look more like a graph as higher-order objects may influence more fundamental objects: theories influence observations; relations redefine categories.

Rather than a top-down hierarchy where we pour in observations and out come a stream of higher-order objects over time, humans may use a learning network characterised by feedback loops.

In the above hierarchy, everything below relations is more speculative although there is some supporting experimental evidence. Human performance of observation, pattern and category operations is more widely accepted and supported by experimental evidence as shown in the next section.

Evidence for observations, patterns and categories

Humans moving from observation to patterns to categories is an idea established in cognitive science by research outcomes.

An observation may be something noticed by a person in the world around them or the observation may stem from personal imagination. Albert Einstein used thought experiments to explore complex physical concepts that led to breakthroughs in relativity and quantum theory.

The work of Saffran, Aslin and Newport in Science has shown strong evidence that infants extract patterns from raw observations. In their work, they presented 8-month-olds with a stream of nonsense syllables, that had no pauses or word boundaries, but certain syllables tended to occur together more often than others. After only a few minutes, infants distinguished the co-located syllables from the otherwise random stream.

Infants can be shown many different examples of a novel object:

• Object A
• Object B
• Object C
• Object D

which vary in appearance but share a common type.

Later, when shown a new object from the same group, they will treat it as familiar. An object from a different group will be treated as novel. This is evidence that the infant is performing categorization, rather than simply memorizing the separate objects.

Multiple studies show that giving the same label to several objects encourages infants to group them into a common category. Consistent naming helps infants form object categories more successfully than random labels or no labels.

For example, when presented with multiple instances of an object (e.g., a tulip), the results of LaTourrette and Waxman have shown that infants are detecting commonality (matching patterns), and are constructing a category that was not present in any individual observation. Only tulips were presented to the infant, not TulipCategory which has been independently formed by the infant.

Discussion

Given the above, we can be confident that infants form categories for objects based first on observation, then grouping for pattern similarity, and then category formation.

Understanding is not merely pattern recognition. Understanding occurs when the learner constructs representational objects that are:

1. not explicitly present in the observations
2. capable of explaining observations (is a member of, is not a member of, ...)
3. capable of predicting future observations
4. useful for generating further abstractions

An example of number three is correlated actions. Given a parent picking up a baby bottle, the infant can be confident that milk feeding is likely to happen soon after.

For four, a box with a toy inside leads to learning that one object may contain another which is useful for further abstractions such as set membership, inclusion, hierarchy, and ownership.

Relations

Observation to pattern and then to category is supported by the above experimental evidence.

Evidence also shows that humans reason about relations between categories.

There are multiple explanations of how relations come to be. Some consider relations to be fundamental (Relational Frame Theory) or that relations have their own representational status (Structural Mapping Theory). Some models suggest that categories and relations are learned simultaneously.

Categories provide additional descriptions of objects. "Dog" is an example of a category. The features of an animal and its behaviour help determine if that particular animal is a member of the "Dog" category.

But categories do not exist in isolation. Dogs chew bones. Both "Dog" and "bone" are examples of categories. "chew" is an example of a relation that applies to categories. "Humans chew bones" applies just as well. The relation is the same, but the inputs (the categories that the relation acts upon) differ.

Dedre Gentner has shown that people recognize the relation (the act), even though the categories are completely different. For example:

Bird : Nest :: Beaver : Dam

People recognize the relation "animal creates habitat".

It is not only that birds live in nests and that beavers live in dams but that birds and beavers both build their habitat, so live_in and build_habitat become new relations.

Gentner's experimental evidence shows that humans often privilege relational similarity over superficial feature similarity. This suggests that cognition contains representations that are more abstract than categories alone.

Categories group similar objects but relations help explain how those objects are used.

Deriving married_to(A, B) from many instances of married

When presented with the data:

John is married to Mary Sarah is married to David Alice is married to Ben

there are a vast number of possible relations that might be inferred:

• adult_pairing
• vowel_name_association
• married_to

The question becomes what mechanism leads to married_to(A, B) and not the others?

The presented data on its own does not determine the relation. Something must bias infants and humans to a particular relation outcome.

There are a number of possible explanations:

• Humans possess or acquire a repertoire of candidate relation types
• Comparison highlights which relation best aligns multiple examples
• Relations embedded in larger inferential systems are favoured
• Those relations are then generalized to new instances

A repertoire of candidate relation types

In seeing data examples: John is married; Sarah is married; Alice is married, a possible explanation is that children do not search all possible relations - having to consider adult_pairing and vowel_name_association - but that they process the data with a restricted hypothesis language.

It is possible that children are born knowing a set of fundamental relations:

• same
• different
• inside
• contains
• owns
• causes

or that they acquire them very early in their development.

And then when seeing:

John is married to Mary Sarah is married to David Alice is married to Ben

they ask:

which of my available relations best explains these examples?

This dramatically reduces the search space so that all possible relations do not have to be considered.

This does beg three questions: what is the initial set of relations? If we are born with none, what is a sufficiently useful set, and when is it acquired?

Relation comparison

Suppose a learner experiences:

• bird in nest
• horse in barn
• dog in kennel

As before, the objects differ enormously. Birds are different to horses which are both different to dogs. Nests are different to barns which are both different to kennels.

The characteristics of the objects themselves (bird, nest, horse, barn, dog, kennel) do not help the learner settle on a relation candidate.

However, when the examples are compared side by side for object usage, the common pattern becomes clear:

home_for(X, Y): A bird lives in a nest, a horse lives in a barn, a dog lives in a kennel

The learner has noticed that the same relation between certain object pairs applies.

Larger systems of inference are favoured

Many relations are possible (married_to, adult_pairing, same_number_of_vowels) but each of these differs enormously in what they connect to.

For example: married_to(A, B)

Links to concepts such as:

• spouse
• family
• children
• wedding
• cohabitation
• inheritance
• divorce
• kinship

A whole network of inference becomes available, with married_to informing the learner about concepts such as family, spouse, children and wedding.

Whereas same_number_of_vowels connects to almost nothing.

A preferred relation may be one that exists within a rich interconnected system. That is, there is a bias to relations that are in some way related to other concepts, possibly giving the learner more mental tools to use in the future to help explain new observations. A new relation that does not connect to other concepts or only connects to a few may be a candidate for being discarded.

Perhaps intelligence is partly the retention of weakly-connected relations that explain observations. If such a relation was discarded, the human could not explain future observations using it.

The model presented here suggests that humans generate relations having seen categories first, e.g., deriving relations such as married_to from observation of many married couples. This might be the case, but it is not as clearly shown from the experimental evidence.

Generating further abstractions.

married_to(A, B) can be applied in different settings.

For example, if the learner observes that Jane married Tom, the learner can predict that Jane and Tom are spouses.

married_to(A, B) adds detail through analogy, e.g., a king and queen are married, just as a husband and wife are.

As someone observes that John married Mary and that Jane married Tom, the category of "married couples" is formed where two people have entered into a union.

This is likely to be formed when taken with additional observation, such as:

1. adult pair
2. live together
3. raise children together
4. referred to each other as husband or wife
5. share resources
6. appear together repeatedly

Therefore, it is possible that a relation becomes a valuable abstraction because it has, over time, shown itself to enable many further inferences.

married_to(A, B)

Once a learner has derived married_to(A, B) from observation and the above processes, additional learning through observation is also possible.

married_to(A, B) is a symmetric relation as:

married_to(A, B) implies married_to(B, A)

This symmetry is neither directly observed nor explicitly taught. It is constructed by the learner through observation of examples like the six above, and relation and abstraction construction.

This helps explain the world to the learner. Being told Jane and Tom are married informs the learner that Tom and Jane are married, which tells the learner that Tom is in a relationship with Jane, and that if they do not already have children, that children are a possibility.

Thinking

Thinking is the manipulation of representational objects according to operations appropriate to their type, and the generation and manipulation of new objects.

Thinking is the recursive creation and manipulation of representations, and representations survive according to two criteria:

• explanatory success
• utility

Does the abstraction help explain and help predict future observations and does the abstraction support useful reasoning, decision-making, and formation of further abstractions?

The mind is therefore behaving like a continual generator and tester of candidate representations. Those candidates that provide low explanatory success or low utility are discarded.

Innateness

In this model, the innate component is not specific concepts or theories. This model does not require humans to be born with:

• marriage
• symmetry
• causality
• grammar

Instead humans are born with the capacity to construct such representations from experience and the driver that leads to representations appears biased. Humans can:

• detect patterns
• form abstractions and test them
• retain useful abstractions
• build higher-order abstractions
• reason about abstractions

This model acknowledges that patterns underlie all cognition. But patterns are turned into higher-order objects (categories, relations) that are used in different ways than patterns. The operation for patterns is to add a new observation into a matching pattern or patterns. And to then take a number of patterns and create a category that can be used through new processing when presented with a new observation (member-of).

This operation is unique to the category type and is not a way of processing patterns.

In the same way that biology is implemented by atoms, the whole of biology is not usefully explained only in terms of atoms. In humans, the autonomic nervous system is a higher order system that defines its own operations that are not modelled in terms of atoms, such as respond-to-threat or recover-from-stress. Atoms are certainly a building block or an implementation detail of the autonomic nervous system, but they are too low-level to explain the operation of the autonomic nervous system.

Words are implemented using letters but how we process words is different to how we process letters. Words are formed into sentences, and we reason about them in a different way to a single word.

"This statement is false." conveys a message. This is a different type of operation to how a word is used within a sentence or how a letter may be combined with others to create a word.

A word on its own conveys information about that object: best, worst, times.

But combining those objects into a sentence:

"It was the best of times, it was the worst of times"

This conveys meaning beyond the individual words and letters. The words are subject to higher-order sentence functions that govern how they are correctly and effectively used.

The higher-order operations are defined by how they are used, not how they are implemented:

• observations can be compared
• relations can be instantiated
• symmetries can generate inferences
• theories can explain and predict

Understanding occurs when a representation can participate in new forms of reasoning unavailable at the level below.

Conclusion

Experimental evidence suggests that humans are born with an innate capacity to both discover patterns through observation and imagination, and to construct progressively higher levels of representation from those observations.

Moving toward higher levels, whether through a hierarchy or a network, enables humans to build models that help explain their world through observations and the testing and discarding of higher-level operations.

Read next: The International AI Race May Not Be Won by the Smartest AI
The approach to AI in the US and China is in stark contrast. But the long-term competition may not be between their models, but who can turn AI into the most useful product, at the lowest cost, and at the greatest scale. And China is uniquely placed to deliver this.

Related Articles

• LLMs struggle when interpreting context. This article discusses the LLM interpretation problem across four disciplines
• When Code Is Cheap, Judgement Matters More
• What Tech Executives Need to Know About Working With LLMs
• The Missing Structure Agile Cannot Fix
• When Urgency is High but Progress is Slow