Technical Summary: Unsupervised Cognition

Problem Statement

Traditional unsupervised learning methods primarily rely on mathematical clustering to group data based on proximity in an encoding space (e.g., K-Means, Invariant Information Clustering). The authors argue that these approaches suffer from a fundamental limitation: they optimize for mathematical relationships rather than "cognitive" relationships. For instance, in the MNIST dataset, the cognitive relationship between samples is the numerical value they represent, whereas mathematical clustering often prioritizes relationships between pixel encodings that do not align with human cognition. Furthermore, existing methods typically divide an input space into subspaces (optimization perspective) rather than constructing abstractions of the data (constructive perspective). This often leads to models that cannot handle small or incomplete datasets effectively, lack explainability, and fail to recognize when an input does not fit any known pattern.

Methodology: The Nodule Algorithm

The paper proposes Nodule, a primitive-based, unsupervised learning algorithm inspired by the Synthetic Cognition framework and the Self-Projecting Persistence Principle (SPPP). Unlike traditional clustering, Nodule models the input space constructively by building a distributed hierarchical structure of representations. The architecture consists of three main components:

1. The Embodiment

The Embodiment acts as an interface that transforms any input (tabular, image, etc.) into Sparse Distributed Representations (SDRs). This ensures the algorithm is input-agnostic.

• Encoding: Inputs are converted into SDRs. For images, pixels are normalized; for numerical data, values are binned with a specific overlap (e.g., 10%) to allow similarity detection; for categorical data, one-hot encoding is used.
• Decoding: A corresponding decoder allows the internal SDRs to be translated back into human-understandable outputs (labels).

2. The Nodule (Core Processing Unit)

The Nodule processes SDRs using a hierarchical, tree-like structure composed of Footprints and Cells.

• Footprint: The basic primitive. It stores a data SDR and a metadata SDR (e.g., a label). It possesses an updating function (computing a running average of SDRs) and an activation function that generates an Archetype. The Archetype is an abstracted, constructive representation of the input, derived from the SPPP.
• Cell: A container that groups multiple Footprints to avoid redundancy. When an input arrives, the Cell selects the most similar Footprint (based on a similarity threshold) to process it. If no Footprint is sufficiently similar, a new one is created.
• Hierarchy (Nodule): Cells are organized in a tree structure. A Seed Cell sits at the root. When a Footprint in a Cell receives an input that is too dissimilar to its existing Footprints, a new child Cell is created to subdivide that specific subset of the input space. This creates a distributed hierarchy where leaf Cells represent specific inputs and higher-level Cells represent broader abstractions.

Processing Flow:

1. Filtering Phase: The input SDR is compared against all Footprints. The most similar Footprint (exceeding the Cell's threshold) is selected. If no Footprint matches, a new one is created in the Seed Cell.
2. Abstracting Phase: The selected Footprint is activated, updating its internal representation and generating an Archetype. This Archetype is passed up to the parent Cell, recursively updating parent Footprints. This aggregation allows the system to generalize and build constructive representations rather than mere copies.

3. Spatial Attention Modulator

To avoid arbitrary threshold settings, the algorithm employs a Spatial Attention Modulator. This component dynamically calculates the similarity threshold for each Cell based on the average mean similarity between the Footprints within that Cell. This allows the system to auto-regulate spatial discriminability, with child Cells naturally developing higher thresholds (finer discrimination) as they represent more specific subdomains.

Key Contributions

1. Constructive Representation: The paper introduces a shift from partitioning input spaces to building constructive representations (Archetypes) that approximate latent information.
2. Input-Agnostic Architecture: By utilizing SDRs, the algorithm handles diverse data types (tabular, grayscale, color) within a unified framework.
3. "I Do Not Know" Capability: The algorithm can explicitly state when an input does not match any learned pattern (if similarity falls below all thresholds), preventing hallucination and false classification.
4. Explainability: Due to the hierarchical structure of SDRs, the decision path can be traced from the Seed Cell down to the activated Footprint, making the classification process transparent.
5. Primitive-Based Processing: The system relies on a single primitive (Footprint) scaled through a hierarchy, avoiding the complex optimization landscapes of deep neural networks.

Experimental Results

The authors evaluated Nodule against state-of-the-art unsupervised methods (K-Means, IIC) and supervised baselines (K-NN) across four experimental categories:

1. Unsupervised Classification:

   • Datasets: Tabular (Sonar, Ionosphere, etc.) and Image (MNIST, CIFAR10).
   • Results: Nodule performed on par with or better than K-Means and IIC. Notably, it achieved high accuracy with significantly fewer training samples (e.g., <200 samples for MNIST vs. ~1,700 for IIC) and required only a single pass through the data (one epoch), whereas IIC required hundreds of epochs.

2. Small and Incomplete Datasets:

   • Setup: Simulated missing data (up to 50%) in six datasets.
   • Results: Nodule outperformed Random Forest and SANDA (a state-of-the-art method for incomplete data), demonstrating robustness when data is sparse or missing.

3. Medical Classification (Cancer Types):

   • Setup: Classification of cancer subtypes using the Cancer Type dataset.
   • Results: Nodule achieved an accuracy of 0.746, surpassing Graph Convolutional Networks (GCN), Graph Attention Networks (GAT), and other state-of-the-art models (which ranged from 0.712 to 0.735), despite being an unsupervised approach.

4. Cognition-Like Capabilities (Noise Robustness):

   • Setup: MNIST digits with random pixels removed (0% to 100% noise).
   • Results: Nodule demonstrated superior robustness, maintaining high accuracy even when up to 80% of pixels were removed. It achieved an Area Under the Curve (AUC) of 92.60 (training) and 76.23 (test), significantly outperforming K-Means, IIC, and K-NN. The authors attribute this to the hierarchical, constructive nature of the representations, which allows the system to match noisy inputs to intermediate abstractions rather than literal pixel patterns.

Significance and Claims

The paper claims that Nodule represents a disruptive state-of-the-art in unsupervised learning for decision-making. Its significance lies in:

• Cognitive Alignment: It exhibits behavior more akin to human cognition (recognizing patterns despite noise, understanding "unknown" inputs) compared to traditional optimization-based algorithms.
• Data Efficiency: It achieves high performance with minimal data and single-pass learning, making it suitable for scenarios where data is scarce or expensive to label.
• Transparency: It offers a level of explainability and "I do not know" capability that is rare in modern AI, addressing the "black box" problem.

The authors acknowledge limitations, primarily high memory costs due to the storage of numerous SDRs and the lack of an explicit optimization objective (which may limit peak classification accuracy compared to fine-tuned supervised models). However, they argue that the trade-off is justified by the algorithm's ability to build meaningful, constructive representations of the input space. Future work is proposed to address memory scalability through growth inhibition mechanisms and to extend the architecture to other modalities (e.g., sound) and temporal sequences.