Part 1: Learn four objects with three sensors

This visualization shows:

• 3 cortical columns
• A "world" that contains 4 objects
• An animal in the world. It has a body and 3 sensors.

This visualization shows a body, 3 sensors, and 4 objects.

In the simulation below, the animal moves its sensors over 4 objects. It does 1-shot learning. (1-shot learning is not necessary, it's just good for this demo.)

To learn, the animal activates a random "body relative to specific object", then proceeds to move its sensors over the object.

import htmresearchviz0.IPython_support as viz
viz.init_notebook_mode()
with open("logs/100-cells-learn.log", "r") as fileIn:
    viz.printMultiColumnInferenceRecording(fileIn.read())

For every sensation, a feature-location pair is predicted after 1 timestep.

These feature-location pairs are associated with 4 active cells per module. Because these discrete neurons are handling continuous space, the "sensor relative to specific object" is always at least 4 cells, because the "sensor relative to body" is somewhat ambiguous.

Part 2: Infer the objects

Now the animal has moved elsewhere in the world. It's going to experience this objects at an egocentric location that it has never experienced these objects at before.

It touches every object twice.

In this visualization, every timestep is divided into two parts: "?" and "!", which you can pronounce as "predict" and "sense".

with open("logs/100-cells-infer.log", "r") as fileIn:
    viz.printMultiColumnInferenceRecording(fileIn.read())

It works!

These unions of locations are often huge. This is because:

• Every feature-location pair activates 4 "sensor relative to specific object" cells
• Hence, every feature-location pair activates 9 "body relative to specific object".
  • (Because, again, this egocentric location is represented by discrete neurons so it has some ambiguity)

It would be nice if we had a way to associate feature-location pairs with 1 cell per module, rather than 4. That would dramatically reduce the union size.