Abstract
Recent development of neural prosthetics for assisting paralyzed patients has focused on decoding intended hand trajectories from motor cortical neurons and using this signal to control external devices. In this study, higher level signals related to the goals of movements were decoded from three monkeys and used to position cursors on a computer screen without the animals emitting any behavior. Their performance in this task improved over a period of weeks. Expected value signals related to fluid preference, the expected magnitude, or probability of reward were decoded simultaneously with the intended goal. For neural prosthetic applications, the goal signals can be used to operate computers, robots, and vehicles, whereas the expected value signals can be used to continuously monitor a paralyzed patient's preferences and motivation.
Neural prosthetics are being designed to record brain activity related to intended movements from the sensorimotor pathway of paralyzed patients and to use these signals to control external devices. It would be valuable to determine what parameters can be decoded and used for prosthetic applications. Previous research has concentrated on extracting the online (real-time) intended trajectories of the hand by recording signals primarily, but not exclusively, from the motor cortex (1–5). This study explores whether a higher level signal of the goal of a movement can be decoded for prosthetic control. For example, a goal signal indicates the intention to reach for an apple, whereas a trajectory signal would indicate the intended direction of the hand movement during the reach. Another high-level signal of interest is expected value, which is used for making decisions. For instance, if an individual has two potential reach goals, an apple and an orange, and the subject prefers apples over oranges, there are signals in his or her brain that indicate this preference and will influence the decision to reach for the apple instead of the orange. We refer to this approach of extracting high-level signals as cognitive based; intended trajectories can also be considered among this group of signals, although at a lower level.
Recordings were made at points along a major pathway for visually guided movement which begins in the extrastriate visual cortex (6) and passes through the parietal reach region (PRR) and area 5 to the dorsal premotor cortex (PMd) and then to the primary motor cortex (7, 8). Although PRR is specialized for reaching movements (9, 10), it represents the goals of the reach in visual coordinates (11). This visual representation indicates that the planned movement is at an abstract level and codes the intended goal rather than how to move the hand. Further emphasizing its cognitive nature, this goal signal is present when animals plan but withhold movements in the dark; that is, the planning activity exists apart from any visual or movement-related signals. Area 5 has also been shown to encode movement intention in both visual and limb coordinates (12). Although less is known about the reference frame of PMd, it appears that at least a subset of cells have similar properties to PRR (13–15).
In three monkeys, 64 and 32 electrode arrays were implanted in the medial intraparietal area (MIP) (a component of PRR) and area 5, respectively (16). Only cells from MIP were used for decoding in two monkeys (monkeys S and C), whereas a small minority of area 5 neurons were included for monkey O. Monkey S also had 64 electrodes implanted in PMd in a separate surgery.
Experiments were initiated 2 weeks after array implantation and each daily experimental session consisted of 250 to 1100 trials (median number of trials was 819, 726, and 361 for monkeys S, C, and O, respectively). Each session was divided into a reach segment for collecting a database and a brain control segment to decode the position of a cursor on a computer screen. Each session started with the reach segment, during which monkeys performed 30 (or 20 for PMd) memory guided reaches per direction. This task required them to reach to a flashed cue after a delay of 1.2 to 1.8 s (memory period, Fig. 1A). The go signal was the offset of a central green target (17). Neural data recorded during successful reaches were added to a database to be used in the brain control segment. The brain control trials began similarly to the reach trails, with a cue flashed in the periphery. However, during the memory period the movement intention was decoded with 900 ms of neural data beginning 200 ms after the cue offset. When the correct goal was decoded, a cursor was placed at the intended reach location and the subjects were rewarded. Trials were aborted if a hand movement occurred during the memory period. If the wrong target was decoded, then monkeys were instructed by the offset of the central green target to reach to the cued location (17). Trials in both the reach and brain control segments were aborted whenever any eye movements occurred that placed the gaze outside of a 5° centrally located window.
(A) Task for reach and brain control trials (17). At 500 ms after the monkeys touched a central green cue and looked at a central fixation point (red), a peripheral cue (green) was flashed for 300 ms. For reach trials, the monkeys were rewarded if they reached to the target at the end of a memory period of 1500 ± 300 ms. During brain control trials, data from 200 to 1100 ms of the memory period was used to decode the intended reach location. Monkeys were rewarded if the correct target was decoded. (B) Comparison of neural activity during reach (red) and brain control (black). Each row is a single trial aligned to the beginning of the memory period. Thickness of the poststimulus-time histogram (PSTH) represents the standard error calculated with the bootstrap method. M, start of memory period; Sp, spikes. (C) Cumulative decode performance of monkey S during brain control trials for four targets and six targets on the basis of eight neurons from the parietal cortex. Dashed line indicates chance performance. (D) (Left) Cumulative performance of a brain control session with 16 neurons recorded from the dorsal premotor cortex of monkey S. (Right) Offline decode with the same data, showing the effect of the number of cells on decode performance. Notably, the number of neurons that can achieve a high success rate remained relatively low. (E) Mean success rate across all sessions for three monkeys. Values are the percentage of successfully decoded brain control trials. Number in parentheses is the standard deviation of the distribution of success rates. NS, number of sessions; *, recordings from dorsal premotor cortex; all other recordings are from parietal cortex.
In different sessions, the database was either not updated after the end of the reach segment (frozen database), or updated after successfully decoded brain control trials (adaptive database). The adaptive and the frozen database yielded similar decode results (fig. S2), indicating that the database update was unnecessary. Notably, the adaptive database eventually contained only brain control trials without leading to loss in performance. This indicates that training sets for cognitive prosthetics can be obtained in paralyzed patients who do not have the ability to reach.
The use of memory period activity ensured that only the monkeys' intentions were being decoded and not signals related to motor or visual events. An example of the absence of motor-related signals during brain control trials can be seen in Fig. 1B. The memory activity of this cell is present for both reach and brain control trials, but the motor burst, which occurs about 1.4 s after the onset of the memory period in reach trials (red), is absent during successfully decoded brain control trials (black).
Three representative performances of online cursor control with intentional signals from three separate sessions from monkey S are shown in Fig. 1, C and D. On the basis of only memory period activity from eight PRR neurons, four targets were correctly decoded with 64.4% accuracy with 250 brain control trials, and six targets were decoded with 63.6% accuracy with 275 brain control trials in separate sessions (Fig. 1C). On the basis of recordings of 16 cells from PMd, eight targets were decoded with 67.5% accuracy with 310 brain control trials (Fig. 1D). Figure 1E describes the overall performance of the three monkeys across all sessions for brain control trials.
The performance can likely be improved by increasing the number of cells. Although many neurons were tuned during the visual and/or motor period of the task, they were not used during brain control trials unless they showed significant tuning during the memory period as assessed by an analysis of variance (17). The right panel of Fig. 1D shows the effect of the number of PMd neurons on the performance of the decode offline. Neurons were randomly chosen without replacement from the pool of 16 used in the brain control trials. The improved performance offline, compared with the left panel of Fig. 1D, is due to the larger training sets used (17). Increasing the number of neurons improves performance. However, the total number of neurons still remained relatively low. A similar result was found for PRR neurons with an offline decode of cells that were recorded one at a time (18).
Shorter decode intervals than 900 ms can also be used for the online decoding (17) (fig. S6). Offline analysis on the subset of sessions that yielded decode rates greater than 60% on 900-ms intervals showed that a 100-ms interval (200 to 300 ms of the memory period) decreased the performance by a mean (±SD) of 14.3 ± 6.1%. Thus, it is likely that increasing the number of cells will result in very fast and accurate online decodes.
Significant learning resulted in improved performance of the brain control task for PRR recordings over a period of weeks. The percentage of trials successfully decoded from the parietal cortex in monkeys S and C for all sessions with four targets (250 to 1100 brain control trials per session) is shown in Fig. 2A. Not enough sessions were available for monkey O and the PMd recordings from monkey S to permit a similar analysis. For monkeys S and C, our ability to decode their intentions was initially poor, hovering just above chance level. However, it continuously improved over the course of a number of weeks. Regression coefficients of the performance as a function of session number for monkey S and monkey C were 0.5 and 0.08 percentage points per session, respectively. Both of these positive regression coefficients are significant (P < 0.01 for monkey S and P < 0.02 for monkey C).
(A) Overall success rates for decoding movement intention from four possible target positions. Black circles, monkey S; blue squares, monkey C. The number of brain control trials varied from 250 to 1100 trials. (B) The mean mutual information of the cells from monkey S, whose activity was used to build the database (red) and perform the brain control task (black), is depicted for all 68 sessions. For each session, a selection of cells was chosen on the basis of significant tuning. These cells were then used in the brain control trials. The mutual information of these cells was calculated for the 120 reach trials and the subsequent 120 brain control trials.
Over the course of all the sessions, the amount of information carried by neurons in the brain control task increased more than the amount of information during the reach segment of the task. We calculated the mean of the mutual information of the neurons in the memory period during the reach segment and during the brain control segment for each of the first 68 sessions for monkeys S and C (17). The mutual information measure quantifies the degree of spatial tuning and can be used as a metric that can describe any change in the degree of tuning. Data from monkey S are shown in Fig. 2B. This analysis yielded two points per session: the mean information during the reach segment while constructing the database (120 reaches, red points, Fig. 2B) and the information during the initial 120 brain control trials immediately following the reach segment (black). The information carried by cells recorded during the same session increased more during the brain control segment than during the reach segment over the course of 68 sessions (P < 0.01). A similar result was seen in monkey C (17).
Another series of experiments examined whether expected value could also be decoded from PRR activity. We ran a variant of the memory reach task in which cue size indicated the amount, probability, or type of reward that monkeys would receive upon completion of a successful trial. Only one aspect of the reward (amount, probability, or type) was varied in a single session. Cue size was randomly varied trial by trial and the interpretation of cue size was varied across sessions so that a large cue represented the more desirable and less desirable rewards on different days.
PRR neurons increased their spatial tuning during brain control and reach trials when the preferred type of reward (orange juice versus water) was indicated (Fig. 3, A and B) (19). The prior knowledge of a high probability of receiving a reward or the impending delivery of a large volume of reward also increased the tuning of these cells (Fig. 3, C and D). The latter two effects were observed for all three animals. The increased activity is unlikely to be due to attention, given that no increase to the expected delivery of the nonpreferred reward was recorded when it was aversive [0.076 M NaCl (20)]; the response to the saline was similar to the response of a neutral (water) reward. In addition, the increased activity during the memory period for preferred rewards was not related to an associated increase in muscle activity (17) (fig. S3).
(A) Response of a neuron during brain control trials in which reward type was varied; orange juice (black) versus water (red) and (B) its tuning curve. Rasters are aligned to the onset of the memory period. The direction of the intended reach that elicited the responses is written on the figure. Vertical lines superimposed on the PSTH enclose the 900-ms memory segment used to calculate the tuning curves and the duration of the neural activity used to decode reach intention during brain control trials. Volume of juice and water was the same (0.12 ml). (C and D) Tuning curve calculated from the firing rates of two additional cells while the probability (C) and magnitude (D) of reward was varied.
For the brain control trials, those ending with the delivery of the preferred reward carried more information than trials ending in nonpreferred rewards (nonpreferred reward: median, 0.062; 95% confidence interval, 0.0571 to 0.0671; preferred reward: median, 0.091; 95% confidence interval, 0.077 to 0.097) (Fig. 4A). Accordingly, the increased information encoded during the preferred reward condition should improve the success with which movement intentions could be decoded. To test this assertion, two independent decodes were run online and in parallel during the brain control task: one for the preferred reward and one for the nonpreferred reward. Within a given experimental session, a single aspect of the reward (size, probability, or type) was varied. The preferred and nonpreferred rewards were randomly interleaved on a trial-by-trial basis (17). An example of the performance of these dual decodes, which used six neurons and varied reward size, is shown in Fig. 4B. The expectation of a high volume of reward improved the overall decode success rate during the brain control segment by 12.2%. Over all the sessions, the increase in the expected value for larger reward volume increased the successful online decode of goals by up to 21% (Fig. 4C) with a median of 8.1% (n = 32). The increase in decode performance also occurred when probability (median = 10%, n = 4) and reward type (median = 9%, n = 8) were varied. Taken together, these results show that cells were better tuned during the preferred reward trials, providing greater information about the target location and thereby improving the decode.
(A) Mutual information for all the cells used to decode reach intentions for nonpreferred (left) and preferred (right) rewards during brain control trials for all sessions. Cells for each session are not unique. Yellow vertical lines running through the histogram represent the median of the distribution. (B) The performance of monkey S from one session during preferred (black) and nonpreferred (red) reward conditions. Dashed line represents chance. Decode performance for the two reward conditions is indicated on the plot. (C) Improvement in decode between preferred and nonpreferred reward. Black, variable magnitude (high volume, 0.12 ml; low volume, 0.05 ml); red, variable type (juice versus water, volume = 0.12 ml); green, variable probability (high probability = 80%, low probability = 40%). Total number of sessions is 44 (32 reward magnitude, 4 reward probability, and 8 reward type). (D) Offline simultaneous decode of four directions and expected value (dashed line shows chance). Error bars show mean ± SD and were obtained by cross-validation (leaving 30 trials out per iteration).
The expected value could also be decoded on a trial-by-trial basis from brain control trials. Offline decodes similar to those used for the goal, produced a mean accuracy of 84.7 ± 8.5% (17) (fig. S5). Even more importantly, expected value (preferred versus nonpreferred for type, magnitude, or probability) and reach goals were simultaneously read out with a mean accuracy of 51.2 ± 8.0% (mean ± SD; chance = 12.5%) (Fig. 4D).
The results of this study show that the goal signal can be used as a source of prosthetic control. At first, it would appear that the retinotopic coding of the plan could be problematic for prosthetic applications when subjects are free to move their eyes. However, the activity within the map of space in PRR is updated with each eye movement to maintain activity for the same locations in extrapersonal space (9), and the patterns of eye and hand movements are highly coordinated and stereotyped (21). In addition, one brain control session from monkey O conducted with free viewing yielded a four-target decode performance of 80.5%, indicating that intended reaches of animals who are allowed free viewing during reach tasks can be read out with PRR activity.
Recently, the human equivalent of PRR has been identified by means of functional magnetic resonance imaging (22). One advantage of using PRR as a target for a cortical prosthetic is the visual nature of the area. Somatosensory feedback regarding the outcome of a movement is often lost with paralysis, but vision generally remains intact. Thus, PRR can receive a very direct visual “error signal” for learning to operate a neural prosthetic in the face of paralysis. PRR is also more anatomically remote from the somatosensory and motor pathways that are damaged in paralysis (23). It is possible that PRR will show less degeneration than is seen in other cortical areas that are more closely connected to these pathways.
Consistent with work on cortical plasticity (24), the animals learned to improve their performance with time. This plasticity is important for subjects to learn to operate a neural prosthetic. The time course of the plasticity in PRR is in the range of 1 or 2 months, similar to that seen in motor areas for trajectory decoding tasks (2, 4).
Short-term improvements in performance were achieved by manipulating the expected value of reward [see also (25)]. The expected value of the probability of reward, the size of the reward, and the type of reward were decoded from the activity in the brain control experiments. These signals in PRR have not been previously observed, and parallel a similar finding of expected value in the nearby lateral intraparietal area (26) as well as other cortical and subcortical areas (20, 27, 28). This activity does not appear to be linked to attention, given that PRR is active selectively for reach plans independent of attention (10), and also did not show an enhancement of activity when aversive outcomes were expected.
The correlation of increased activity with increased expected reward is substantiated by behavioral data that showed a decrease in reaction times for the preferred rewards (fig. S4). Expected value is a necessary component of the neural system that mediates decision-making (26, 29). On the other hand, it is possible that we are seeing motivational effects that are a direct consequence of expected value (30). Further experiments will be required to distinguish between these two explanations.
The decoding of intended goals is an important feature for a cognitive-based prosthetic. Once these goals are decoded, then smart external devices can perform the lower level computations necessary to obtain the goals. For instance, a smart robot can take the desired action and can then compute the trajectory. This cognitive approach is very versatile because the same cognitive, abstract commands can be used to operate a number of devices. The decoding of expected value also has a number of practical applications, particularly for patients that are locked in and cannot speak or move. These signals can directly indicate, online and in parallel with their goals, the preferences of the subject and their motivational level and mood (Fig. 4D). Thus, they could be used to assess the general demeanor of the patient without constantly querying the individual (similar to assessing body language). These signals could also be rapidly manipulated to expedite the learning that patients must undergo to use an external device. Moreover, this research suggests that all kinds of cognitive signals can be decoded from patients. For instance, recording thoughts from speech areas could alleviate the use of more cumbersome letter boards and time-consuming spelling programs, or recordings from emotion centers could provide an online indication of a patient's emotional state.
The cognitive-based prosthetic concept is not restricted for use to a particular brain area, as can be seen by the finding that PRR and PMd activity could both provide goal information. However, some areas will no doubt be better than others depending on the cognitive control signals that are required. Future applications of cognitive-based prosthetics will likely record from multiple cortical areas to derive a number of variables. Moreover, online trajectory information can also be considered as a cognitive variable that can be decoded along with other cognitive variables.
Supporting Online Material
www.sciencemag.org/cgi/content/full/305/5681/258/DC1
Materials and Methods
Figs. S1 to S6
References
