hierarchical temporal memory

November 26th, 2008

As is often said, we humans (if you are not one of us you can join on the website, membership fees are high but not impossible) are pattern seeking animals. This implies that it is difficult for us to understand a completely "new kind of thing", we tend to seek something else that we can compare it to. Psychology got a minor win when computers emerged, because it finally had a model for the brain. Psychology professors could point to the computer and say "the brain, it's somewhat like that". It behooves psychology that the computer we know has a distinct memory, a processing/reasoning unit, and input channels that receive transmissions of "sensory perception".

The computer as we know it is the so called Von Neumann architecture, every computer we've ever had has been designed in those basic components. This design is simple enough (and, in fact, dumb enough) to handle just about anything at all, it is the general purpose computer (a way of saying that it doesn't have any purpose).

Now a bunch of neuroscientists have figured out that the memory in our computers is too dumb to do certain things well. Our linear memory, where a memory cell has no relationship to the neighboring cells, is abstract and general enough for anyone's pleasure, but it's not the way human memory works. Our memory is hierarchical, that is to say it's made up of levels where the bottom levels remember very simple things, like shapes and sounds in time. As you ascend the hierarchy, the levels above that do not remember "discrete" things, they remember unifications over the simple things. That is the way in which you understand that a leg is both a discrete thing as well as it can be part of a human body, one part in something larger.

Now, if you think about it, this is a crude first model for learning, you are being fed a lot of facts in the hope that you will be able to unify them and see "meaning" to them as a whole. This, unfortunately, is necessary, because we don't know how to transmit the meaning itself, we think the only way is to send the facts and then the mind will infer the meaning by itself. (It's quite an optimistic strategy, isn't it?) Interestingly, there is a trade off at this point. Apparently, you cannot both remember all the discrete facts *and* be able to unify them. So that could explain how some people have a propensity for lots of facts without seeing the bigger picture, while others can't hold on to all the little pieces. In a way it makes sense, doesn't it? Like doing research. Once you've stated your thesis, you don't need all those little notes anymore, they are subsumed in the larger unifying rationale.

But now back to technology. A bunch of people have built this model of memory in software, calling it a "hierarchical temporal memory". It's an absolutely fascinating premise.

:: random entries in this category ::