Action Prevents Error

Predictive Processing without Active Inference

Jona Vance

According to predictive processing, minds relentlessly aim at a single goal: prediction error minimization. Prediction error minimization is said to explain everything the mind does, from perception to cognition to action. Here I focus on action. ‘Active inference’ is the standard approach to action in predictive processing. According to active inference, as it has been developed by Friston and collaborators, action ensues when proprioceptive predictions generate prediction error at the motor periphery, and classical reflex arcs engage to quash the error. In this paper, I raise a series of problems for active inference. I then offer an alternative approach on which action prevents error, rather than quash it. I argue that the action prevents error approach solves all the problems raised for active inference. In addition, I show how the alternative approach can be independently motivated by further commitments of predictive processing and that it is compatible with other prominent approaches to sensorimotor psychology, such as optimal feedback control.

Keywords

Action | Active inference | Deafferentation | Optimal control | Proprioception

Acknowledgments

Thanks to two anonymous referees for feedback on an earlier draft. Thanks also to Michael Anderson, Maria Brincker, Christopher Burr, Karl Friston, Jakob Hohwy, Bryce Huebner, Max Jones, and Alex Kiefer for discussions of this material. Special thanks to Thomas Metzinger and Wanja Wiese for very helpful comments and for their patience.

1Introduction

In predictive processing (PP) frameworks, prediction error minimization explains everything the mind does, from perception to cognition to action. The mind constructs models containing information about objects and properties of all kinds. To test its models, the mind predicts the sensory inputs it’s likely to receive if the models are accurate. Then it compares the predicted inputs with the inputs it actually receives. If there’s a match between the predicted and actual inputs, the models are confirmed. If there’s a mismatch, prediction error occurs.

To minimize prediction error the mind has two options. It can either revise its models to better conform to the actual signal, or it can act on the world to change its sensory inputs, bringing them into conformity with predictions. Call the former, model-altering form of prediction error minimization revision-PEM. Revision-PEM provides the central recipe for perception. Revision-PEM plays a key role in allowing organisms to modify their models to reliably track changing features of the world. Call the latter input-altering form of prediction error minimization action-PEM. It provides the central recipe for action (conceived liberally to include saccades and other involuntary behavior). This paper addresses the standard approach to action in PP and offers an alternative.1

The paper proceeds as follows. Section 2 introduces the standard approach to action in PP and contrasts it with an alternative approach that is prominent in motor control theory outside PP. Sections 3-5 raise problems for active inference. Section 6 describes an alternative to active inference that is compatible with the core commitments of PP and argues for its superiority.

2Active Inference and an Alternative

The standard approach to action in predictive processing is active inference (Adams et al. 2013; Brown et al. 2013; Clark 2013; Clark 2016; Friston 2009; Friston 2011; Friston et al. 2010; Hohwy 2013; Shipp et al. 2013). In active inference, motor control is carried out in a probabilistic (Bayesian) hierarchy of numerous layers. Each layer, except the lowest, sends predictions to the layer below. These predictions are carried by top-down connections between the layers. The connections encode the system’s generative model, specifying probabilistic relationships between the features represented at adjacent layers. In addition, each layer sends bottom-up error signals to the layer above. Error signals are generated by comparing predicted activity at a layer with actual activity at that layer and encoding the discrepancy. Finally, each layer also makes top-down precision estimates of the reliability of the error signal it receives from below. In these respects, active inference entails that the motor hierarchy has deep structural similarity with the perceptual hierarchy responsible for exteroceptive processes such as visual and auditory processing.2

In active inference, motor control and proprioception are deeply functionally and anatomically unified. Suppose an agent deliberately reaches for a cup. In active inference, the process is characterized as follows. The motor system encodes a hierarchically distributed set of prior probabilities over possible trajectories. The intention to reach for the cup is realized by activation at high levels of the motor hierarchy. These high level activations initiate further activations at lower layers of the motor hierarchy. The downward process carries predictions guided by the brain’s somatomotor generative model of the body and world. The downward predictions unpack the subject’s high level intention into expected proprioceptive sensory inputs; here, the inputs that would be received if the body moved so as to carry out the intended action of reaching for the cup. When the intention is initially formed, the body is not currently moving along the desired trajectory: the hand is not yet moving toward the cup. As a result, proprioceptive and other sensory inputs associated with the expected trajectory are not received. The mismatch between the expected sensory inputs and the actual inputs results in prediction error. According to active inference, this prediction error is encoded by alpha motoneuron activity (Adams et al. 2013), which compares downward predictions from primary motor cortex with upward Ia afferent inputs. Having encoded the discrepancy between these top-down and bottom-up signals, alpha motoneurons send the resulting error signal outward to enervate relevant muscle fibers via classical reflex arcs, which quash the prediction error and entrain movement along the expected trajectory. These are the basic details of active inference.

The term ‘active inference’ arguably has multiple meanings in the literature. Friston et al. 2010 (p. 232) characterize active inference as “the interplay of perception and action”. Similarly, Brown et al. 2013 characterize active inference as the combination of perceptual inference and action. They write, “The brain can minimize prediction error in one of two ways. It can either change its predictions to better cohere with sensory input, or change the sampling of the environment such that sensory samples conform to predictions. The former process corresponds to perceptual inference...the latter to action: together they constitute ‘active inference’” (p. 614). By contrast, Hohwy 2013 characterizes active inference more like Brown et al.’s notion of action (pp. 89-90). On his characterization, active inference contrasts with perceptual inference, rather than including it (Cf. p. 83). Similarly, Anil Seth writes that active inference is “Classically conceived of as the minimization of prediction error by performing actions that confirm sensory predictions” (Seth 2015, p. 2). Seth then argues that active inference should also be characterized as including the performance of actions to disconfirm predictions and the performance of actions to disambiguate among competing hypotheses. Finally, for Rescorla 2016, ‘active inference’ denotes the specific theory of motor control developed by Friston and colleagues, as outlined in this section and in the citations therein. It is in this final way that I use the term ‘active inference’ in this paper: to name the specific approach to motor control advanced by Friston and colleagues; including its integration with perceptual processing; its probabilistic, hierarchical modeling; and its commitment to the role of outward error signals from alpha motoneurons and reflex arcs at the motor periphery.

Although active inference is the standard approach to action in predictive processing, it is not the standard approach to action in sensorimotor psychology more generally.3 One dominant approach more generally is optimal feedback control (Todorov 2004; Todorov 2009; Todorov and Jordan 2002). Like active inference, optimal feedback control adopts a hierarchical Bayesian approach to motor control. Active inference and optimal feedback control are also both probabilistic, hierarchical approaches to action. And both can be used to model motor control as a probabilistic inference problem (Friston et al. 2010; Todorov 2009). However, the two approaches also have significant differences (Adams et al. 2013; Rescorla 2016). Several differences are worth highlighting for our purposes.

First, the two approaches differ in how they characterize the functional architecture of the motor control and proprioception. In optimal control theory, motor pathways and proprioceptive pathways interact, but they are distinct. By contrast, in active inference motor pathways double as proprioceptive pathways. According to active inference, motor control and proprioception are implemented using one and the same generative model. The generative model is encoded in the weights between nodes of a belief net. Nodes are realized by populations of neurons. Since, in active inference, the network connections that carry efferent motor signals are one and the same as the network connections that carry proprioceptive predictions, motor command pathways and proprioceptive pathways are one and the same. Thus, in active inference — but not optimal control — the neural channels that implement motor control are, to a large extent, the very same channels that implement proprioception.

Second, the two approaches differ in how they characterize downward motor signals. In active inference, downward motor signals are sensory predictions that are unpacked using the somatomotor generative model mentioned above. Thus, in active inference there are no motor commands per se, only proprioceptive predictions that serve as (or implement) a kind of implicit motor command. By contrast, in optimal control theory, downward motor signals are modeled as commands carried by an inverse model (or controller). The controller assigns the motor command required to achieve the desired outcome.

Third, motor efference copy plays an important role in optimal control theory, whereas active inference does without efference copy. In optimal control theory, each time a motor command is given, the system sends a copy of the command back to the Bayesian estimator, which allows the system to estimate the likely effect of the command on the environmental state using a forward model. Engaging the estimator through efference copy allows the system to compare expected sensory inputs (associated with the intended action) with actual inputs, make online corrections as needed, and update the model through learning. In active inference, there are no motor commands, so there’s nothing to copy. Instead, there are only sensory predictions carried downward by the generative model. Proprioceptive predictions terminating at alpha motor neurons implement (or play the role of) ‘implicit’ motor commands (Clark 2016, pp. 127-128). Sensory predictions terminating at other parts of the periphery constitute “corollary discharge”, without efference copy (Clark 2016, pp. 125-127).

Fourth, the two approaches differ in how they characterize the outward signal from alpha motoneurons to muscle fibers. In active inference, alpha motoneurons generate the outward signal to muscle fibers by comparing Ia afferent feedback and downward predictions from motor cortex, encoding the mismatch, and sending the resulting error as an outward message (Adams et al. 2013). By contrast, in optimal control theory, downward signals from motor cortex engage alpha motoneurons, which further engage muscle fibers directly, without first comparing the downward signal with Ia afferent feedback. That is, active inference differs from most other approaches (including optimal control) by characterizing outward signals from alpha motoneurons to muscle fibers as error signals.

In the following sections, I raise a series of problems for active inference. In subsequent sections, I show how PP can be combined with optimal control theory to solve the problems.

3The Action/Revision Problem

In active inference, action-PEM and revision-PEM both occur via proprioceptive predictive pathways. What explains why proprioceptive predictions at a given location in the hierarchy sometimes entrain action-PEM and at other times drive revision-PEM? Call this the action/revision problem for active inference. There are various statements of the action/revision problem in the literature. For example, Hohwy puts the problem as follows:

[T]his overall account [active inference] creates a puzzle about how action is triggered, that is how the agent shifts from perceptual to active inference. This is because there will be competition between assessment of the actual proprioceptive input and the counterfactual proprioceptive input. Rather than changing the world to fit with the counterfactual predicted input, the system could just adjust its proprioceptive prediction in light of the actual input — it could realize that it is not actually in that state. This would prevent action from arising. A mechanism is thus needed to ensure agency (Hohwy 2013, p. 83).

Hohwy’s formulation implies that the problem is to explain how and why the system shifts from, in my terminology, revision-PEM to action-PEM. The implication is that revision-PEM is the primary or default form of prediction error minimization, and that the system must be moved out of that default mode to engage in action. However, one need not assume that revision-PEM is the default form of PEM. And, indeed, some prominent recent developments of the PP framework eschew that assumption (Clark 2016; Seth 2015). In setting up the action/revision problem for PP, I do not assume that one form of PEM is primary or default.4

It’s useful to contrast the action/revision problem with the so-called “dark room problem” for PP. The dark room problem asks: if the system aims at minimizing prediction error why doesn’t the subject just go into a dark room where there is little sensory information thereby reducing the chance of error? If there is little sensory information in the first place, then, one might think, avoiding prediction error will be easier. A promising solution to the dark room problem does not solve the action/revision problem. The proposed solution entails that some predictions about endogenous states are unrevisable, such as predictions about blood sugar levels (Shea 2013, p. 229). When blood sugar levels drop below predictions, the systems cannot revise predictions; increasing blood sugar is the only way to minimize prediction error. The proposal works well when applied to subsystems that monitor blood sugar levels and inflexibly predict target levels. But that is not how most action works. Most goals are highly flexible and require constant revision. Action-oriented processing will have to involve plenty of revision along the way. Any solution to the action/revision puzzle must allow for revision during action.

Proponents of active inference hold that, in my terminology, both action-PEM and revision-PEM occur simultaneously (Clark 2016, p. 124). This is possible at different places in the network. However, note that engaging in action and revision simultaneously at the same location in the network would not minimize prediction error. For example, if one simultaneously engaged peripheral proprioceptive layers in action and revision, action-PEM would throw the body into motion in the predicted way while the revised predictions anticipate that the body was not moving in the predicted way. This would create new prediction error because there would be a new mismatch of predicted and actual state. The problem for PP is to explain when, how, and why the system engages in action-PEM or revision-PEM at a given time and location in the network.

I now consider and criticize two proposed solutions to the action/revision problem. The first proposal appeals to counterfactual or subjunctive content to distinguish predictions that entrain action-PEM from those that do not. Here is Hohwy:

Now consider how action comes about...The representations of predicted sensory input are counterfactual in the sense that the say how the sensory input would change if the system were to act in a certain way (Hohwy 2013; p. 82, italics original).5

The suggestion is that action-entraining predictions are distinct from others by involving counterfactual content. For example, to raise one’s arm, one predicts something in the vicinity of: if I were to raise my arm, I would receive inputs Y. Now, generative models do encode counterfactuals by encoding information about the relationship between sensory inputs and features of the body or environment. And these counterfactuals are utilized in generating action according to active inference. But action-entraining proprioceptive predictions cannot themselves be conditionals. They are not in the subjunctive form: if X were the case, Y would be the case. Whenever a counterfactual conditional is false, it’s because, at the nearest world where the antecedent is true, the consequent is false. So, for example, the conditional “if I were to raise my arm, I would receive inputs Y” is false if and only if at the nearest world where my arm is raised, I don’t receive inputs Y. But the falsity of this conditional does not generate the prediction error needed for movement. Such a conditional is encoded in the generative model; if it’s falsified, the model must be revised to encode which inputs are really connected with a raised arm. The prediction that can incite action is one such that raising one’s arm fulfills the prediction. Raising one’s arm would not make true the counterfactual conditional “if I were to raise my arm, I’d receive inputs Y”.

According to Clark 2016, the predictions that entrain action refer to states of affairs that are non-actual. This strikes me as the right way to put things. In active inference, when the somatomotor system predicts some non-actual state of affairs, action can ensue to make that state of affairs actual, thereby quashing prediction error. For example, if I want to raise my arm, my somatomotor system can make a prediction that my arm is raised, even though it is not. This prediction generates further predictions down the somatomotor hierarchy yielding more specific proprioceptive predictions of non-actual sensory inputs. When reflex arcs are engaged to move the arm in the appropriate way, the predictions are fulfilled and the prediction errors are quashed. But predicting non-actual states of affairs is not unique to action, nor can it explain why action ensues. Predicting non-actual states of affairs is typically what generates prediction error in every modality, from vision to audition and so on. When the predicted state of affairs and the actual state of affairs differ, prediction error occurs. The upshot is that there is nothing about whether the content is counterfactual or non-actual that can provide a satisfactory response to the action/revision puzzle.

I now turn to the most prominent solution to the action/revision problem. It appeals to precision weighting along proprioceptive pathways. Recall that in active inference proprioceptive predictions in the very same neural pathways sometimes entrain action-PEM and at other times drive revision-PEM. On the precision balance view, the shift between these two forms of PEM results from shifts in the relative weighting of proprioceptive precision expectations. On Brown et al.’s characterization, the balance is between precision expectations for proprioceptive error signals and the precision of proprioceptive predictions.6 On the precision balance view, action ensues when the precision of proprioceptive predictions is higher than the precision estimate for the proprioceptive error signal.7

The precision balance approach to active inference initially appears to solve the action/revision probem. However, it faces an extension of the problem in accounting for proprioceptive revision during action. The precision balance view concedes that when the precision balance is set to favor action-PEM at the proprioceptive periphery, proprioceptive revision-PEM at the periphery cannot occur. This is because, in active inference, motor predictions and proprioceptive predictions occur in the very same neural pathways, and only one form of PEM can obtain at a given time and layer. Now, during an action, the motor system sends multiple messages to the periphery. Sending multiple motor messages controls the degree and duration of muscle activation (Knierim 1997, Ch. 3). This means that, on the precision balance view, at various times during a single movement, the precision balance will have to be set to favor action-PEM and proprioceptive predictions of expected trajectories will have to be sent down the motor hierarchy. However, in addition, there is significant evidence that proprioceptive processing engages in revision-PEM regularly during action in order to deliver feedback to the motor system so that it can make online adjustments at various points during movement. Such evidence comes from neurological studies of the activation of feedback pathways during action (Azim et al. 2014) as well as behavioral studies of prescribed adjustment during action (Liu and Todorov 2007). As a result, the precision balance approach to active inference must explain how action-PEM and revision-PEM can both occur during a single action.

On the precision balance view, the most natural suggestion requires that the precision balance shifts multiple times during a single action, so that action-PEM and revision-PEM can both occur. But the view requires more than repeated shifts to the precision balance in proprioceptive pathways during action. Recall that the action/revision problem is pressing in part because on active inference approaches, the very same generative model (and neural pathways) are responsible both for action-PEM through motor control and proprioceptive revision-PEM. Proprioceptive predictions that entrain action-PEM will have significantly different content from proprioceptive predictions that allow for useful feedback about the actual state of the body. As a result, the shift proposal requires that the system make desired-trajectory predictions to initiate action; then, during action, the same system must adopt a different set of predictions aimed at correctly predicting the actual-current-state of the body, and so on, each time a new motor prediction is issued. Since the alternation is supposed to be realized in the same proprioceptive pathways, activity at the relevant nodes will have to shift rapidly between predicting the desired and actual sensory inputs, without being able to predict both simultaneously. As such, it’s not clear how the system could keep track of the desired trajectory while engaged in revision-PEM aimed at anticipating the body’s actual current state, since the very same nodes in the proprioceptive hierarchy that predict the desired trajectory will have to be recruited to predict the body’s actual state (and vice versa).

4Deafferentation and Proprioceptive Experience

In this section, I raise a second problem for active inference. The problem appeals to patients who have large fiber neuropathy in their limbs and entirely lack proprioceptive experience of the affected limbs. Despite their lack of proprioceptive experience, these deafferented patients can move their affected limbs (Forget and Lamarre 1987; Ghez and Sainburg 1995; Messier et al. 2003; Rothwell et al. 1982). Deafferented patients tend to exhibit significant motor deficits when compared to normal subjects on prescribed tasks. Yet, motor performance significantly improves when these patients can utilize exteroceptive feedback (e.g. from vision) to help guide action (Sainburg et al. 1993; Sainburg et al. 1995). The challenge for proponents of active inference is to explain how the proprioceptive channels allegedly responsible for action-PEM and revision-PEM remain functional while such patients entirely lack proprioceptive experience of the affected limbs.

Here we must distinguish lack of proprioceptive experience from lack of proprioceptive reliability. Proprioceptive experience concerns phenomenology. Reliability concerns how accurately the relevant processing responds to inputs at the proprioceptive periphery over a range of circumstances. The reliability of proprioceptive processing in deafferented subjects is greatly reduced compared to controls. But reduction of reliability implies nothing about whether such processing generates phenomenology or not. In principle, a perfectly reliable process could be entirely lacking in phenomenology, and a phenomenally rich experience could be entirely unreliable, as would be the case in some forms of hallucination. Deafferented patients under discussion lack both proprioceptive phenomenology and reliability with respect to affected areas. My focus in this section concerns their lack of proprioceptive phenomenology.

The central neuropathic difference between the deafferented patients in question and normal controls is that deafferented patients lack Ia afferent feedback at the proprioceptive periphery, while Ia feedback remains intact for controls. Active inference could use this difference to explain the radical difference in proprioceptive reliability between deafferented patients and controls. However, the presence or absence of Ia afferent feedback cannot on its own explain deafferented patients’ total lack of proprioceptive phenomenology in affected areas on the active inference approach. In PP, phenomenology partly supervenes on properties of top-down and lateral neural activation—i.e. features of predictions and precision estimates. Input signals at the sensory and proprioceptive periphery, help drive and shape the system’s processing. But these bottom-up signals are not part of the supervenience base of perceptual or proprioceptive phenomenology. So the presence or absence of Ia afferent feedback cannot on its own explain the lack of proprioceptive phenomenology in deafferented patients.

Besides the lack of Ia feedback, there are some other differences between deafferented and normal subjects. But none of these differences can explain the the total lack of proprioceptive phenomenology with respect to deafferented limbs, given that deafferented patients can nevertheless move the affected limbs and can improve their movement with exteroceptive feedback. For example, proprioceptive predictions in deafferented subjects may be quite different in content from control subjects. A difference in the content of proprioceptive predictions during action can explain some difference in the phenomenal character of proprioceptive experience during action in deafferented subjects. But it cannot explain the total lack of proprioceptive experience for such subjects. After all, in order to explain how voluntary movement is possible in the affected limbs of deafferented patients, proponents of active inference must accept that proprioceptive predictions occur with respect to the affected limbs for deafferented patients.

Proponents of active inference could respond to my argument so far as follows. During movement in normal subjects, it is well known that sensory attenuation occurs: that is, normal subjects are less reliable and feel proprioceptive experience less robustly in their active limbs. On the precision balance approach, this is because action ensues when the precision balance favors proprioceptive predictions rather than proprioceptive prediction error. The precision balance is used to explain proprioceptive attenuation in normal subjects during action: by attenuating the precision expectation for error signals from the relevant limbs, proprioceptive sensitivity and the robustness of phenomenology in those limbs is reduced. If deafferented patients lack relevant proprioceptive prediction error, then the balance might be such as to fully attenuate proprioceptive experience in deafferented patients; that is, lack of relevant prediction error during action in deafferented limbs might explain why deafferented patients wholly lack proprioceptive phenomenology.

I reply as follows: contrary to the above response, deafferented subjects do not entirely lack the relevant proprioceptive error signal in all cases. As Adams et al. 2013 note, parts of the somatomotor hierarchy dealing with affected limbs of deafferented patients are not entirely without relevant feedback in all cases. When such patients can gain visual information about their body and the environment, their accuracy on prescribed tasks significantly improves (Sainburg et al. 1993; Sainburg et al. 1995). In active inference, this means that the somatomotor system in deafferented patients can utilize visual and other exteroceptive information to help generate relevant prediction errors in the somatomotor hierarchy, which can help improve the accuracy of somatomotor predictions during movement. When visual inputs play a role at low levels of the proprioceptive hierarchy, active inference entails that deafferented patients can approximate proprioceptive processes of normal subjects. Visual feedback, say, can provide an approximate substitute for Ia afferent feedback. In such cases, if active inference were true (including its claim that implicit motor commands are realized in proprioceptive pathways), we would expect deafferented patients to have some proprioceptive phenomenology with respect to the affected areas. For, there is considerable proprioceptive phenomenology in normal subjects under such conditions: both during movement where the phenomenology is present but less robust and while the limbs in question are at rest. Deafferented patients’ proprioceptive systems engage in the relevant processes of top-down predictions, precision estimates (with respect to error signals in the somatomotor hierarchy that are generated not by Ia feedback but indirectly through visual input), and error correction (again with indirect visual origin). And proponents of active inference accept that all these processes occur to explain why deafferented subjects can engage in prescribed movement and improve accuracy of movement in part through revision-PEM during movement. Yet once this concession is made, there remains no relevant difference to explain why some of these deafferented subjects entirely lack proprioceptive experience during visually guided movement or at rest while visually attending to affected limbs. The only difference in these cases between deafferented patients and controls is that, for deafferented patients, the relevant proprioceptive prediction error comes entirely via visual input, whereas in controls the relevant error comes both from Ia feedback and from vision. This difference can account for some qualitative difference in the proprioceptive phenomenology between deafferented and normal subjects, but it cannot explain the total lack of proprioceptive phenomenology in deafferented subjects in affected areas.

5Deafferenation and Movement

In this section, I raise a final problem for active inference. Like the previous section, my objection in this section appeals to deafferented patients with large fiber neuropathy. Unlike the previous section, here I am concerned with accuracy of movement rather than proprioceptive phenomenology. In some deafferented patients, there is a complete lack of Ia afferent feedback from muscle spindles. Yet these patients are still able to move their affected limbs, sometimes with surprising accuracy. My objection in this section is that active inference cannot account for the accuracy of some deafferented patients’ movements.

To be clear, my objection is not that active inference lacks the resources to explain how deafferented patients can move their affected limbs at all. Friston and colleagues are aware of that objection. They put it as follows:

One important question for active inference is: if movement depends on spinal reflex arcs, then why can neuropathic patients—who lack Ia afferent feedback from muscle spindles—still move? Surely, in the absence of anything to predict there can be no prediction error and no movement (Adams et al. 2013, p. 636).

And they respond as follows:

In fact, the absence of primary afferents does not mean there is no prediction error—top-down predictions can still elicit alpha motor neuron activity. Under active inference, a forward model in the brain converts visuospatial predictions in extrinsic coordinates (low dimensional extrapersonal space) to proprioceptive predictions in intrinsic coordinates (high dimensional proprioceptive space). These predictions then leave the brain and are converted to motor commands by a simple inverse mapping in the spinal cord (see ‘‘Discussion’’). This spinal inverse mapping is effectively driven by proprioceptive prediction errors and corresponds to the classical reflex arc.

A loss of proprioceptive feedback, therefore, will severely impact upon the spinal inverse mapping, while the cortical forward model can compensate using visual feedback (Bernier et al. 2006). Descending proprioceptive predictions should still be able to activate motor neurons, but they can no longer be compared with precise proprioceptive information and cannot be modified by proprioceptive feedback. (p. 636)

According to Friston and colleagues, even without Ia feedback, the motor systems of deafferented patients can generate the prediction error encoded by alpha motor neurons used to engage reflex arcs. Recall that the relevant error arises when alpha motoneurons compare downward predictions from motor cortex and upward Ia signals. They encode the difference as error and send that error signal outward to enervate the muscles. This error message encodes information for how the muscle should contract. When Ia feedback is lacking, there is still a mismatch between the efferent prediction received by the alpha motoneuron and the (nonexistent) afferent signal. In such a case, the mismatch just is the prediction: when a prediction is compared to no bottom-up signal, the entire prediction is mismatched. Hence, in such a case, alpha motoneurons send the entire prediction as an error signal outward to enervate the muscle. Movement ensues. The worry that active inference entails deafferented subjects cannot move at all is misguided. It is not the worry I raise here. My worry grants that active inference can account for the fact that deafferented patients can move at all.

My objection is that active inference cannot account for the accuracy of these movements in some cases and inaccuracy in others. In active inference, the precise control of movement depends critically on the details of the error signal encoded by alpha motoneurons. A large error signal engages the muscles differently than a small one does. The generative model cannot send just any prediction to the alpha motoneurons and expect to get the desired action trajectory. The predictions must be calibrated such that, when combined with Ia afferent signals, the resulting error signal encodes the right instructions sent to the muscles.

However, in deafferented patients, the error signals encoded by relevant alpha motoneurons will be radically different from the error signals encoded by normal subjects given the same proprioceptive predictions. Given some downward predictions in the motor hierarchy with content P, the error signal for normal subjects will encode the mismatch between the prediction of P and Ia afferent feedback. But given a prediction of P in deafferented patients, the error signal generated by affected alpha motoneurons will be very different, since there’s no afferent feedback to compare the prediction with. Since, according to active inference, the specifics of the muscle activation depend critically on the information encoded in the outward error signal generated by alpha motoneurons, active inference predicts that, given the same proprioceptive predictions delivered to the alpha motoneurons of deafferented patients and controls, the movements by deafferented patients will be radically different from controls — at least when no supplementary feedback is available, e.g. from exteroception.

Many movements by deafferented subjects are indeed radically different from (and less accurate than) movements by normal subjects with the same intentions (Gordon et al. 1995). However, a number of studies show that deafferented patients without Ia feedback can perform some movements without motor control deficit compared to normal subjects. Some such movements are single-joint. For example, Rothwell et al. 1982 found of a deafferented man that “Although he was grossly disabled, it was remarkable to find that he could execute a large repertoire of learned manual motor tasks with both speed and accuracy, despite lacking any useful feedback from his hands” (Rothwell et al. 1982, p. 516; Cf. Forget and Lamarre 1987). Importantly, some patients in these studies could engage in movements with accuracy comparable to controls even when performing prescribed tasks without feedback from exteroceptive modalities such as vision.

In another set of studies, Messier et al. 2003 found that a deafferented subject could achieve accuracy equal to controls when engaging in reaching movements that required multi-joint coordination, even when performing the task with eyes closed, so that there was no perceptual feedback to help guide the movement. Interestingly, Messier et al. found that the speed of action was a significant factor. They write, “Surprisingly, however, he [the deafferented patient] made much larger errors than control subjects at slow and natural speeds, but not at fast speed” (Messier et al. 2003, p. 399). That is, when deafferented patients performed actions at natural and slower-than-natural speeds, they exhibited significant motor deficits compared to normal subjects performing the same tasks at the same speeds. By contrast, deafferented subjects performed as well as normal subjects when they performed the tasks rapidly.

These results are difficult to explain on the active inference approach. Assume for the moment that the alpha motoneuron error signals in areas relevant to movement are very different for deafferented and normal subjects. If that’s the case, it’s not clear how active inference can explain any of the cases where deafferented and normal subjects perform equally well, since radical differences in the error signals enervating the muscles should result in radical differences in muscle activation. Moreover, it’s not clear why speed of movement should matter so much on the active inference approach. Concerning Messier et al.’s results, active inference can appeal to a significant difference in alpha motoneuron error signals in affected areas to explain why deafferented patients perform poorly when performing tasks slowly. But if the error signals are significantly different for deafferented patients (when compared with normal subjects) during slow movements, they will also be significantly different (compared with controls) for fast movements. Thus, active inference predicts that these deafferented patients will exhibit motor deficits compared to controls on both fast and slow movements. Since deafferented and normal subjects perform roughly equally well on fast movements, active inference fails to account for the finding.

Proponents of active inference could offer a number of replies. I now consider several. First, active inference proponents could argue that deafferented patients’ motor systems learn a new mapping from alpha motoneuron activation to muscle activation. This proposal is a non-starter because the mapping from alpha motoneuron activity to muscle activation cannot be revised in the PP framework. It engages a classical reflex arc and does not benefit from error correction. A second reply could be that deafferented patients use visual feedback to help guide activation of alpha motor neurons, without Ia afferent proprioceptive feedback. However, as noted above, in some of the relevant cases, deafferented patients’ moved without deficit compared to controls, even without exteroceptive feedback. Third, perhaps deafferented patients use visual imagery to help guide activation of alpha motor neurons, without Ia afferent proprioceptive feedback. Unfortunately, visual imagery could not explain improved accuracy of movement. Imagination does not provide relevant feedback correction.

One further reply is worth considering in more detail: perhaps in deafferented patients who exhibit accurate prescribed movement, the proprioceptive generative model has been revised to accommodate deafferentation. For example, suppose that in the deafferented patients in question, the generative model encoded in the motor hierarchy has been changed so that a high-level action intention (e.g. reaching for a cup) is unpacked into different predictions sent to alpha motoneurons. By altering the predictions sent to alpha motoneurons, the motor hierarchy could generate the same error signal from alpha motoneurons in deafferented patients as in controls, using different downward predictions, even without online proprioceptive or exteroceptive feedback.

The last reply is the most promising. However, it too fails to account for the full range of empirical findings. For one thing, the suggestion entails that there will be a time immediately after deafferentation during which the motor system must relearn the relevant hierarchical relationships. The proposal predicts that at onset of neuropathy the hierarchy will still encode the pre-pathology top-down generative model and will thus issue incorrect motor commands. However, as far as I know, there is no evidence that patients experience any such relearning period. In addition, the proposal fails to account for deafferented patients’ retention of some motor abilities while losing others. For example, one deafferented patient, G.O., retained the ability to accurately draw figure 8s in the air but lost the ability to grasp a pen (Rothwell et al. 1982). If the generative model could be revised to allow the subject to draw figure 8s, one would expect it could be revised to allow for grasping a pen. Finally, the proposal fails to account for deafferented patients’ ability to retain some skills while being unable to acquire very similar skills after deafferentation. For example, Rothwell et al. (Rothwell et al. 1982, p. 252) write that G.O. “was able to continue driving his old car [used prior to deafferentation] even at night, but found it impossible to learn to drive his new car”. If the correct explanation for G.O.’s continued ability to drive the old car were that the generative model in G.O.’s motor hierarchy was radically revised, he should have been able to learn to drive a new car as well. For the revision required to adapt to the new car could not have been sufficiently different from the revision required to retain his old driving abilities. That he could not learn to drive the new car counts strongly against the proposal that the generative model is radically revised to accommodate deafferentation.

6Action Prevents Error

Active inference is not the only way to model action-PEM in a PP framework. On active inference, proprioceptive predictions lead to prediction error, which is then quashed when classical reflex arcs are engaged. However, to account for prediction error minimization through action, one need not assume that error arises first, only to be quashed. Instead, it might be the case that action ensues to prevent prediction error. Call this the action prevents error thesis.

The action prevents error (APE) thesis is consistent with widely-accepted approaches to motor control, such as optimal feedback control theory. Recall that in optimal control theory, motor commands project downward to the motor periphery driven by a hierarchical inverse model (or controller), and that each time a motor command is issued, an efference copy is sent to the Bayesian estimator. The estimator then predicts the likely effects of the motor command on the environmental state, which allows the system to predict the likely sensory consequences of the intended action. In a PP framework, the Bayesian estimator can work via downward predictions in a hierarchical generative model, just as downward predictions operate in exteroceptive hierarchies. As a result, adopting an optimal control approach in a broadly PP framework allows us to model efference copy (corollary discharge) using the familiar hierarchical message passing process.

On APE, as in active inference, the motor system initiates downward proprioceptive and exteroceptive predictions of sensory inputs associated with an expected trajectory. Unlike active inference, in APE, downward predictions associated with expected trajectories do not serve as implicit motor commands. There is a separate channel driven by the controller for that. In APE, as in optimal control, downward predictions resulting from efference copy help the motor system make online adjustments to the motor command sequence by providing feedback. However, unlike optimal control theory, we can emphasize in APE that the predictions carried in efference copy also provide a crucial resource for prediction error minimization in action. Because the predictions are of the sensory inputs associated with intended action, the occurrence of the intended action minimizes error with respect to these predictions. If the intended action did not occur, prediction error would be more pronounced. In APE, action prevents the prediction error that would occur as the result of corollary discharge predictions if the action did not occur. As a result, the APE thesis shows how one could develop an optimal control approach that differs from active inference, but is consistent with the core claim of PP frameworks; i.e. that everything the mind does aims at minimizing prediction error.8

The action prevents error approach solves all the problems raised against active inference in the previous sections. It solves the action/revision problem. It can account for the complete lack of proprioceptive experience in mobile deafferented patients. And it can account for deafferented patients’ ability to sometimes move in prescribed ways with surprising accuracy without exteroceptive feedback. I now explain each point in turn.

APE solves the action/revision problem. In APE, proprioceptive predictions and motor commands do not travel along the same neural pathways. They are realized by distinct but interactive subsystems. The explanation for why action-PEM or revision-PEM occurs appeals to this architecture. Action-PEM ensues when motor commands initiate action, thereby minimizing prediction error that would arise from the predictions that are made as part of corollary discharge. On APE, there is no competition between action-PEM and revision-PEM in motor pathways. Relatedly, APE explains how extensive proprioceptive revision can occur during actions requiring online adjustment during movement. APE entails that proprioception occurs via distinct pathways from motor control. So proprioceptive predictions can be made constantly during an action, simultaneously with the occurrence of motor commands. The result is that according to APE — but not active inference — proprioception and motor control can proceed simultaneously and interactively, rather than alternating in competition for use of the same pathways.

APE also allows for the complete lack of proprioceptive experience in deafferented patients who can move their limbs. On APE, the motor commands that entrain limb movement are realized by neural pathways distinct from those that are crucial for proprioceptive experience. APE accounts for limb movement in deafferented patients by appeal to the fact that, in such patients, motor pathways remain intact. Moreover, in deafferented patients, the proprioceptive predictions (distinct from motor commands) that would give rise to proprioceptive experience cease to occur. As a result of deafferentation, the afferent signal in the affected areas is eliminated. With no incoming signal, the relevant proprioceptive pathways have nothing to predict, so predictions cease, and with their cessation proprioceptive experience is eliminated with respect to the affected areas.

In addition, APE accounts for the cases in which deafferented patients retain considerable accuracy in prescribed actions, even without feedback from exteroceptive modalities. On active inference, alpha motoneurons encode error as the mismatch between Ia afferent signals and downward predictions from motor cortex. In deafferented patients, there is no Ia afferent signal. As Friston and colleagues note, in such patients, the error encoded by alpha motoneurons is equivalent to the downward prediction, since there’s no Ia afferent signal at all. But, as I argued above, this means that the error signal used to engage reflex arcs are very different for deafferented patients compared with normal subjects. It is implausible that the somatomotor hierarchy compensates for such a radical change concerning which error signals occur at the periphery given the same downward predictions, especially given other limitations experienced by deafferented patients in modifying motor routines. The APE approach faces none of these difficulties. On APE, downward motor commands engage muscle fibers directly, without prediction error arising first. So, on APE, the motor commands sent to affected limbs in deafferented patients will be similar to the commands sent in normal subjects. The main difference will be in the lack of proprioceptive feedback in deafferented patients. The role of feedback on APE does well to explain the results surveyed above. For example, it explains why some deafferented patients perform relatively poorly compared to controls when reaching at slow speeds with their eyes closed. In such cases, the motor systems of normal subjects utilize significant proprioceptive feedback that is not available in deafferented systems. However, at high speeds, feedback plays less of a role in normal subjects. As a result, normal subjects’ performance is closer to that of deafferented subjects at high speeds.

In addition to solving the above problems for active inference, APE gains additional support from core commitments in PP frameworks. A familiar challenge for PP is to explain why organisms engage in playful and explorative activity. For example, why do humans climb mountains, explore caves, or attend parties with strangers? Such activity involves lots of novel stimuli and often increases short-term prediction error compared with less playful and explorative alternatives. A plausible and promising reply to this worry is that organisms aim to minimize ‘global’ prediction error (Clark 2016; Lupyan 2015) or ‘long-term’ prediction error (Hohwy 2013) under a counterfactually rich set of scenarios the organism might encounter (Seth 2015). That is, predictive minds accept—and even seek out—prediction error in the short term or in some restricted domain in the service of minimizing prediction error over the longer term and over a wider range of scenarios. Giving a central explanatory role to the minimization of long-term and wide ranging prediction error makes sense only if we characterize at least some prediction error minimization in terms of preventing prediction error proactively rather than waiting for it to arise and then quashing it. So a version of APE seems required in PP. According to the version of APE I have developed here, action is thoroughly proactive in its prevention of proprioceptive prediction error: preventing prediction error during action is the primary way in which action-PEM occurs. But it is worth remembering that APE fits well with the general emphasis in PP on proactive global and long-term prediction error minimization.

7Conclusion

In this paper, I have argued for the action prevents error thesis as an alternative to active inference, as developed by Friston and collaborators. I raised three problems for active inference, and I argued that APE solves all three problems. In addition, since APE is compatible with optimal control theories, it can claim any further advantages that such theories might have. Finally, APE fits well with the need to account for long-term and indirect prediction error minimization in PP.

Although I have criticized active inference, I have not argued against the predictive processing framework more broadly. On the contrary, my proposals are fully consistent with the core commitment of PP to the centrality of prediction error minimization in everything the mind does. APE fits well with what Clark 2013; Clark 2016 call ‘action-oriented’ predictive processing in a broad sense. Despite its recent rise in prominence, predictive processing remains highly controversial in many quarters. One of my goals in this paper has been to show how the core claims of PP are consistent with highly successful approaches to sensorimotor psychology, such as optimal feedback control. Distinguishing the core commitment to prediction error minimization in perception and action from a commitment to active inference as Friston and colleagues have developed it is useful in an intellectual climate where PP remains controversial.

References

Adams, R. A., Shipp, S. & Friston, K. J. (2013). Predictions not commands: Active inference in the motor system. Brain Structure and Function, 218 (3), 611–643.

Azim, E., Fink, A. J. & Jessell, T. M. (2014). Internal and external feedback circuits for skilled forelimb movement. Cold Spring Harbor Symposia on Quantitative Biology, 79, 81–92. https://dx.doi.org/10.1101/sqb.2014.79.024786.

Brown, H., Adams, R. A., Parees, I., Edwards, M. & Friston, K. (2013). Active inference, sensory attenuation and illusions. Cognitive Processing, 14 (4), 411–427.

Clark, A. (2013). Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behavioral and Brain Sciences, 181–204.

——— (2016). Surfing uncertainty: Prediction, action, and the embodied mind. New York: Oxford University Press.

Forget, R. & Lamarre, Y. (1987). Rapid elbow flexion in the absence of proprioceptive and cutaneous feedback. Hum Neurobiol, 6 (1), 27–37.

Friston, K. (2009). The free-energy principle: A rough guide to the brain? Trends in Cognitive Sciences, 13 (7), 293–301.

——— (2011). What is optimal about motor control? Neuron, 72, 488-98.

Friston, K., Daunizeau, J., Kilner, J. & Kiebel, S. J. (2010). Action and behavior: A free-energy formulation. Biol Cybern, 102(3), 227-260.

Ghez, C. & Sainburg, R. (1995). Proprioceptive control of interjoint coordination. Canadian Journal of Physiology and Pharmacology, 73 (2), 273–284.

Gordon, J., Ghilardi, M. F. & Ghez, C. (1995). Impairments of reaching movements in patients without proprioception. I. Spatial Errors. Journal of Neurophysiology, 73 (1), 347–360.

Haruno, M., Wolpert, D. M. & Kawato, M. (2003). Hierarchical MOSAIC for movement generation. International Congress Series, 1250, 575–590. https://dx.doi.org/10.1016/S0531-5131(03)00190-0.

Hohwy, J. (2013). The predictive mind. Oxford University Press.

Knierim, J. (1997). Motor systems. In J. Byrne (Ed.) Neuroscience online. http://neuroscience.uth.tmc.edu/s3/index.htm.

Liu, D. & Todorov, E. (2007). Evidence for the flexible sensorimotor strategies predicted by optimal feedback control. Journal of Neuroscience, 27 (35), 9354–9368.

Lupyan, G. (2015). Cognitive penetrability of perception in the age of prediction: Predictive systems are penetrable systems. Review of Philosophy and Psychology, 6 (4), 547–569.

Messier, J., Adamovich, S., Berkinblit, M., Tunik, E. & Poizner, H. (2003). Influence of movement speed on accuracy and coordination of reaching movements to memorized targets in three-dimensional space in a deafferented subject. Experimental Brain Research, 150 (4), 399–416.

Metzinger, T. (2017). The problem of mental action. Predictive control without sensory sheets. In T. Metzinger & W. Wiese (Eds.) Philosophy and predictive processing. Frankfurt am Main: MIND Group.

Rescorla, M. (2016). Bayesian sensorimotor psychology. Mind & Language, 31 (1), 3–36.

Rothwell, J. C., Traub, M. M., Day, B. L., Obeso, J. A., Thomas, P. K. & Marsden, C. D. (1982). Manual motor performance in a deafferented man. Brain, 105 (3), 515–542.

Sainburg, R. L., Poizner, H. & Ghez, C. (1993). Loss of proprioception produces deficits in interjoint coordination. Journal of Neurophysiology, 70 (5), 2136–2147.

Sainburg, R. L., Ghilardi, M. F., Poizner, H. & Ghez, C. (1995). Control of limb dynamics in normal subjects and patients without proprioception. Journal of Neurophysiology, 73 (2), 820–835.

Seth, A. K. (2015). The cybernetic Bayesian brain. In T. K. Metzinger & J. M. Windt (Eds.) Open MIND. Frankfurt am Main: MIND Group. https://dx.doi.org/10.15502/9783958570108.

Shea, N. (2013). Perception versus action: The computations may be the same but the direction of fit differs. Behavioral and Brain Sciences, 228–229.

Shipp, S., Adams, R. A. & Friston, K. J. (2013). Reflections on agranular architecture: Predictive coding in the motor cortex. Trends in Neurosciences, 36 (12), 706–716.

Todorov, E. (2004). Optimality principles in sensorimotor control. Nature Neuroscience, 7 (9), 907–915.

——— (2009). Parallels between sensory and motor information processing. In M. S. Gazzaniga (Ed.) The Cognitive Neurosciences, 613–623. Cambridge, MA / London, UK: MIT Press.

Todorov, E. & Jordan, M. I. (2002). Optimal feedback control as a theory of motor coordination. Nature Neuroscience, 5 (11), 1226–1235.

Vance, J. & Stokes, D. (2017). Noise, uncertainty, and interest: Predictive coding and cognitive penetration. Consciousness and Cognition, 47, 86–98. http://dx.doi.org/10.1016/j.concog.2016.06.007.

Wolpert, D. M. & Kawato, M. (1998). Multiple paired forward and inverse models for motor control. Neural Networks, 11 (7), 1317–1329.