4.5 Orientation encoding

The previous two sections showed how the afferent and lateral connections self-organize into a highly structured map with very specific internal connections, as found in the cortex. As one might expect from the properties of the individual neurons in the map, the response of the network to an input varies systematically depending upon the orientation and position of that input. This section will show examples of the network's actual response to different orientations, and will examine possible methods for determining what orientation is perceived by the cortex for that activity pattern. Calculating the perceived orientation is an essential prerequisite to measuring tilt aftereffects and illusions, since those are manifested as differences in the perceived orientation in different circumstances.

At any point in time the visual system is processing an image that has only a small number of oriented features in any local area, so only a few portions of the orientation map will be active at a given time. Figure 4.6 shows the sparse activity that results for Gaussian inputs of various orientations at the center of the retina.

**Figure 4.6:** **Cortical responses to various oriented inputs.**
The first column (a) shows sample oriented Gaussians, at +60°, 0°, -30°, and -90° from vertical (top to bottom, respectively.) The second column (b) shows the initial response of the trained map to that input, based on the afferent weights only (before the lateral interactions are allowed to settle). The third column (c) shows the settled activity for that input. For (b) and (c), colors indicate the orientation preference of each activated neuron, as in figure 4.3. Each input activates neurons which prefer orientations like it, within the cortical region corresponding to the active area of the retina. The initial response is wide and diffuse, like the input pattern, but the settled response is focused and sharp.
$\begin{figure} \begin{minipage} {\textwidth} \centering \begin{minipage} {\thr... ...\makebox[0pt][c]{ \usebox {\orkey} } \end{minipage} \end{minipage} \end{figure}$

However, it is unknown precisely how the higher visual areas extract the encoding of orientation from figure 4.6 and arrive at the perception of an individual oriented line. It has been suggested that the perceived orientation is either the orientation preference of the unit with the highest activation (Carpenter and Blakemore, 1973), or a weighted average of the preferences of all active units (Coltheart, 1971). An intermediate method could also be used, computing an average of all the units having activity greater than some arbitrary activity level. In order to determine if a choice between these methods is crucial, the two extreme options were tested for the RF-LISSOM model.

The weighted average of orientation preferences must be computed as a vector sum, since angles repeat every 180°. Two nearly horizontal lines (e.g. -85° and +85°) should average to represent a horizontal line (±90°). However, the arithmetic average of -85° and +85° is 0°, which is a vertical line and is clearly incorrect as an estimated perception. Instead, each neuron is represented by a vector. The vector must represent adjacent orientations as adjacent vector angles. Thus each of the 180° possible orientations must be scaled by two to get the angle of the vector, which ranges over 360°. Since each neuron is to contribute only to the extent that it is active, the magnitude of the vector is taken to be the activation level of the neuron. Once the neurons have been represented in this fashion, the average orientation can be computed from the orientation of the vector sum. Figure 4.7 illustrates these calculations.

**Figure 4.7:** **Computing the weighted average orientation.**
The activations and orientations of three neurons are shown as vectors (solid lines). The angle of each vector is twice the orientation preference of that neuron, in order to map the 180° of orientation preferences onto the full 360°, making the vector orientation for 0° and 180° identical. The magnitude of each vector is the activation level, to ensure that the encoding primarily reflects those neurons that are most active. The sum of these vectors is shown as a dashed line. The average orientation of the three neurons shown is half of the angle of the dashed vector.
$\begin{figure} \centering \putpictype{\psline} estimating-perceived (2.0in) \end{figure}$

Using the vector sum method of computing the average, the perceived orientation was calculated for the trained network using both the average value and the maximum value methods. For each each angle, an oriented Gaussian (of the same shape as in training) was presented at the center of the retina, and the estimated orientation was computed using each method. The results are presented in figure 4.8.

**Figure 4.8:** **Estimates of perceived orientation.**
For each possible orientation of a Gaussian at the center of the retina, the heavy line shows the orientation that would be perceived if perception were veridical (i.e., we would perceive the actual orientation of the Gaussian.) The other lines show the perceived orientation of a Gaussian at the center of the RF-LISSOM retina estimated using two different algorithms. The dotted line shows the estimate obtained by computing an average of the orientation of each activated neuron weighted by the activation level of that neuron. The dashed line shows the estimate obtained by performing a similar computation on only those units that have reached maximum activation. Both methods are reasonably accurate estimates, although using all activated units (dotted line) is slightly more accurate.
$\begin{figure} \centering \hputpictype{\psline} 970101_ae_090d_1212.setl_perceivedangles (\xmgrplotwidth) \end{figure}$

Both methods appear to be reasonably accurate representations of orientation. Small deviations from the true angle are present, however, because the activated cortical area is comparable to the size of an orientation column. The number of neurons with RFs receptive to the peak of the input Gaussian is relatively small, and thus the distribution of orientation preferences will not necessarily be uniform within that population. Thus, at different locations on the retina, the estimated orientation will differ slightly. If much larger inputs are used, activating a large cortical area, these effects should cancel out, allowing arbitrarily accurate orientation encoding. However, this would be computationally prohibitive to verify at present since much larger cortex and retina sizes would need to be simulated.

As one might expect, utilizing all active units in the calculation of the perceived orientation is slightly more accurate than only using those at maximum activation, but the difference is not significant because many neurons are at maximum and thus most of the neurons averaged in each case are identical. For simplicity, I have chosen in later experiments to use only the method of averaging all units, but the results should be the same for either method, or for any similar method. Note that there is insufficient biological evidence to support a particular method, and I do not claim that orientation perception actually need be occurring in precisely this way. In any case, to ensure that the biases shown in figure 4.8 do not distort the results, all perceived orientation measurements in this thesis are stated in terms of differences in perceived angles, rather than in terms of the actual orientation on the retina.