In our recent paper published in Scientific Reports, we uncover the surprising ability of untrained artificial neural networks (ANNs) to solve abstract visual reasoning tasks. This challenges conventional beliefs about the necessity of extensive prior training for abstract reasoning, highlighting that neural networks can exhibit reasoning capabilities independent of memorized information.

The Unexpected Capabilities of Untrained Networks

One of the enduring puzzles in artificial intelligence and cognitive science is whether abstract reasoning—our ability to identify and utilize patterns and relationships—is fundamentally about learned knowledge or if it represents an innate computational skill. Traditional wisdom suggests that extensive training is crucial for neural networks to solve abstract reasoning tasks. Indeed, recent advancements with large language models and deep neural architectures show impressive problem-solving abilities, but often these models have undergone massive-scale training, leading critics to question whether they genuinely reason or merely recall similar problems encountered in training data.

In our research, we challenged this assumption directly. We asked a provocative question: can artificial neural networks (ANNs) solve abstract reasoning tests without prior training? By doing so, we sought to test the idea that abstract reasoning might not necessarily require extensive prior exposure to similar problems.

The Unexpected Capabilities of Untrained Networks

We designed a series of visual reasoning problems, akin to classic intelligence tests such as Raven’s Progressive Matrices. Each problem consisted of a sequence of images that changed along a specific feature (such as size, number, or color). The network was tasked with identifying the pattern from the sequence alone and selecting the correct subsequent image.

Remarkably, our results demonstrated that randomly-initialized networks, optimized only with the data of the current problem (and without any prior training or exposure), successfully solved these abstract visual reasoning tasks substantially better than chance. This indicates an innate computational potential for abstract reasoning within neural network architectures, independent of extensive memorization or previous learning.

“All these results were obtained using networks that were randomly initialized before each problem, thus demonstrating that ANNs can perform abstract reasoning that does not depend on memorization.” — From our paper

Our analysis revealed that the success of these naive neural networks arises from two key factors:

  1. Random Convolutional Layers: These initial layers spontaneously extract useful features from raw visual inputs, serving as general-purpose feature extractors.

  2. Optimization at Test-Time: By allowing the network to adapt its parameters during the solution process itself, it successfully highlighted the relevant features of each individual problem.

These insights challenge the conventional belief that neural networks’ abstract reasoning capabilities arise predominantly from massive training on extensive datasets.

Implications for Cognitive Science and AI

Our findings have intriguing implications for the fields of cognitive science and artificial intelligence. The fact that untrained networks can exhibit abstract reasoning abilities parallels discussions in cognitive science about human fluid intelligence and its underlying computational mechanisms.

This research aligns with findings from neuroscience, suggesting that some human abstract reasoning tasks could be explained by rapid adaptation mechanisms in the higher cortical regions, similar to our network’s optimization process.

From an AI perspective, our results emphasize that abstract reasoning doesn’t necessarily require immense datasets and extensive pre-training, opening up possibilities for developing more efficient neural models capable of performing complex reasoning tasks.

Looking Forward

Our findings open numerous directions for future research:

  • Generalization and Transfer: Can optimization at test-time similarly enable ANNs to generalize their abstract reasoning capabilities across different task structures and problem contexts?
  • Complex Relations: Our current model addresses monotonic patterns effectively but struggles with more complex non-monotonic relationships. Future work could explore architectures capable of capturing more complex relational dynamics.
  • Integration of Cognitive Mechanisms: Incorporating cognitive elements such as working memory or explicit analogical mappings could further align neural networks’ performance with human reasoning.

If you’re interested in learning more, our paper is available in Scientific Reports, and the code for replicating our experiments is available on GitHub.