The Firmware Analogy



Watching for Artifacts

By the time data from our senses reach our consciousness, they have been selected and formatted to correspond to what we would see in a photograph of the same location. But along the path from sense organs to awareness, there are many steps where the same data are not directly recognizable. We can’t directly "see" the information traveling along our nerve pathways, any more than we can see the latent chemical image in exposed but undeveloped photographic film.

Normally when we attend to the resulting photograph or conscious awareness, we try to pick out the objects and qualities that are important to our consciousness, while disregarding any flaws and artifacts of the sensation and transmission process. Even in a poor-quality photograph, we can recognize familiar faces and the emotions they were showing at the captured moment. But the more one knows about the process of photography, the harder it becomes to avoid focusing on the flaws, and thinking about which steps in the pathway from original scene to reproduction could have been performed differently.

The psychoros is a part of the pathway from our sense organs to our consciousness. We typically disregard any artifacts that might demonstrate its existence, by focusing on our sensory content rather than its flaws. To identify and study the psychoros we must reverse this preference, seeking out situations and illusions that trigger artifacts, and focusing on such indirect evidence of how our sensory pathways work.

Chemical photography is too simple to provide a fertile analogy for the psychoros. The latent image may be invisible, but it is bound into the coating on the relatively rigid flat film, and even in the case of a color reversal process, development will reveal a readily recognizable image. The anamorphic lenses used to squeeze a widescreen movie into the narrower rectangle of physical film stock take a small step closer to the kinds of effects the psychoros can mediate, but even that concept is readily understandable - we are no longer disturbed when a television picture is squeezed to half the screen width to make space for credits or advertisements.

Electronic photography, and the use of digital computers to process the resulting images, provide a richer source of possible analogs to the kinds of effects that reveal the psychoros. Even there, however, the psychoros manipulates and extrapolates from three-dimensional space while our current computer technologies are just beginning to explore 3-D displays. The firmware analogy I discuss here is concerned not so much with the representational content of our sense experiences, as with mapping their relationship to the underlying sensory organs and the space around our bodies.



Semi-Permanent

“Firmware” signifies a layer of programming which is more permanent than software, because it is typically stored in a programmable integrated circuit rather than in more volatile memory, and because its functions are intimately linked to the hardware they support. The purpose of firmware is usually to create a generalized abstraction by which higher-level software may refer to the functions of a specific collection of hardware chips, sensors and circuits. For instance, when you press the ‘\’ key, regardless of which of the several possible locations it may occupy on your particular computer’s keyboard, the keyboard firmware reports “scancode 5D”. Higher level software knows 5D equals backslash, and does not have to care about the specific layout of your physical keyboard.

In some very early computers, firmware was loaded into working memory at power-up, just like software; only its need to match the underlying hardware kept you from changing it at will. When sufficiently large one-time programmable “PROM” or electrically programmable “EPROM” chips became common, they became the favorite home for firmware. To change the low-level functionality of your computer you unplugged a chip and plugged in a different one containing different firmware. Now that electrically erasable “EEPROM” chips and FLASH memory chips are common, changing firmware is almost as fast and easy as installing new software - except your opportunities for change are still limited by your hardware, and the consequences of a failure to reload, or loading the wrong firmware, are disastrous.

Human “firmware” is characterized by this same semi-permanence and necessary relationship to physical resources. There is, for instance, a firmware-level routine which synchronizes your eyes focusing on closer objects as they converge inward. This happens whether you are conscious of it or not, and learning to override it (to see those 3-D shapes hidden in posters of repeating patterns, for instance) can be maddeningly tricky. Adjusting to new glasses which change the relationship of your focus and convergence can be painful as well as disorienting. While you may have moments of success with resolving the hidden object but then lose it again, or find relief from the pain of new lenses in certain situations, it generally takes days or weeks of experience and practice to make the new firmware pathway readily available.



Re-Programmable

Some human responses, like the classic hammer-just-below-the-kneecap reflex, appear to be hardwired into the nervous system, requiring no learning or programming. The relationship between convergence and focus is sometimes called a reflex, but I believe our capability to learn voluntary control over it and adjust to externally imposed changes in the “hardware” prove it is a learned response. I will argue in other parts of this site that learning how our sensory systems work by experimenting on our own and by copying people around us and absorbing cultural norms is as much a part of human development as learning to walk, run, and dance.

Just like our patterns of walking and running tend to persist unchanged unless we are injured and required to adapt, or until age reduces the effectiveness of our bodies, our perceptual habits rarely change, and only gradually deteriorate. Yet at the end of the Concept page I talked about reconfiguration of the psychoros, which can take place suddenly and spontaneously. In my experience my psychoros can change instantaneously - but the adjustments required for other perceptual habits involving muscles and posture to cope with the psychoros change take far longer and much struggle. Quite similarly to the body’ms response to a sudden injury...



An Inner Map

I mentioned that one of the functions of a computer’ms firmware is mapping its “sense organs”, the keyboard switches, to standard scan codes. To be more precise, that bit of firmware typically lives inside the keyboard itself. Scan codes travel up the wire (or wireless connection) to a bit of hardware the main processor can read. It uses "driver" software, which is functionally much like firmware but can be installed via normal user-accessible methods, to turn scan codes into the letters and numbers you typed. Those are passed to the “operating system” (Windows, Mac, *N*X, the consciously accessible part of your computer which is always present and active) and a user application (the part you can easily select for performing a specific task). Each of those layers of abstraction shapes and limits our awareness of what is really happening, with the goal of hiding all the details between key press and screen character.

The psychoros is where human consciousness maps its incoming sensory experiences to their internal representations and the patterns of muscular activity required to find them again. Just as the firmware and software abstractions hide the details and artifacts of the actual computer hardware, the psychoros tries to hide the complexity of perception, restricting our conscious experience to images that follow photo and cinema rules, and concepts for which we have familiar names.

Confronted with input which simply cannot be fit into photorealistic space, like the kind of experience portrayed in the Escher “Print Gallery” image, the psychoros may well reconfigure that section of its internal map to hide the “impossible” parts of the experience from consciousness.

If the previous paragraph didn’t make much sense, remember that we’mre talking about an analog to computer firmware. Just as you have no awareness of the scan codes traveling from your keyboard to your operating system, you probably have no awareness of the inner spatial map your psychoros uses to make sense of your spatial perceptions. So it is not surprising that talk of artifacts and distortions within that unconscious realm doesn’mt immediately make sense.

While I’m attempting to sneak toward the psychoros by noticing artifacts in consciousness, I’m excited to learn how neuroscience is approaching from the other direction. The regular triangular pattern of grid cells found in the dorsomedial entorhinal cortex in 2005 by Edvard Moser, May-Britt Moser and their students “appear to encode a type of abstract spatial structure that is constructed inside the brain and imposed on the environment by the brain with no regard for the sensory features of the environment”.*[Wikipedia] 
*[The Wikipedia page on grid cells provides a brief introduction with illustrations of the grid and links to original sources.]

Place cells, found in the hippocampus by O’Keefe and Dostrovsky in 1971, act as a cognitive representation of a specific location in space. “Place cells mainly rely on set distal cues rather than cues in the immediate proximal environment. Movement can also be an important spatial cue. The ability of place cells to incorporate new movement information is called path integration, and it is important for keeping track of self-location during movement. Path integration is largely aided by grid cells, which establish a grid representation of a location, so that during movement place cells can fire according to their new location while orienting according to the reference grid of their external environment”.*[Wikipedia] 
*[The Wikipedia page on place cells provides a good introduction with many links to original sources.]

I’ll definitely be attempting to integrate these clues about the underlying brain hardware into my explorations of the experiential artifacts they mediate.



 

Table of Contents


Revised 19 January 2014