Communication is the fundamental basis of social life. In schizophrenia, it is particularly affected: Two of the DSM-5 core symptoms of schizophrenia include speech disorders (disorganized speech and alogia—one of the negative symptoms). Patients suffer from speech and language aberrations on all linguistic levels. In acute phases, communication is extremely limited—some studies report parallels of schizophrenic speech and aphasia (4). Therefore, referring to the term “aphasia”, researchers call the productive speech of patients “schizophasia” (5). Further linguistic impairments are for instance an incoherent discourse, less referential cohesion and loosening of associations. These symptoms of disordered speech are evidence of formal thought disorder (6), which can also be a symptom of other psychiatric diseases, such as depression.

Not only the production of speech is impaired in schizophrenia, but also the general understanding and interpretation of the meaning of speech and language information is seriously affected. For instance, patients show difficulties in the understanding of figurative speech or abstract concepts being conveyed through metaphors, proverbs, humorous or ironic expressions, which they tend to misinterpret in a concrete way (7–11). Hence, this phenomenon is clinically termed “concretism” (12). In everyday life, metaphors are very frequently used to refer to abstract concepts such as feelings or events (e.g., “out of the blue”). Dysfunctional interpretation of metaphors thus have a huge impact on successful social interactions (13). By contrast, the understanding of concrete speech seems relatively unaffected in schizophrenia (14–16).

As one integral feature of embodiment (17), gesture “serves as an outward manifestation of several interacting fundamental processes” (18) including speech perception (19, 20), memory (14, 21), and social functioning (22, 23). Therefore, dysfunction in the integration and interpretation of gesture information represents a core feature of disordered communication processes. Aberrations in gesture processing are found across all stages of schizophrenia, in production as well as in perception and interpretation. Patients use less and incoherent, mismatching gestures (24, 25). Furthermore, they tend to misinterpret gestures in a negative way (22, 26). Some authors therefore claim a general disruption of the integration of two modalities (auditory = speech and visual = gesture) in this disease (22, 26–29).

We can classify gestures—according to speech—in concrete gestures and figurative or abstract gestures: Concrete gestures, for example, are iconic gestures (e.g., literal gestures such as forming the shape of a dog's mouth with a hand while discussing a dog), abstract gestures are metaphorical hand and arm movements that accompany metaphors in speech (e.g., forming a cup with a hand while discussing a concept such as love). Moreover, similar to patients suffering from disordered speech, patients with schizophrenia also have trouble interpreting abstract meaning in gestures (30, 31). Strikingly, integration skills interact with symptomatology: Nagels and colleagues found worse integration skills, reflected in the evaluation of the semantic relationship between speech and gesture, in patients with severe symptoms compared to patients with mild symptom severity in formal thought disorders (32).

As mentioned above, decoding of abstract meaning is particularly challenging for patients with schizophrenia—on the level of auditive (speech) as well as in the visual (gesture) modality. Furthermore, the binding of information from multiple modalities (such as gesture and speech) is a complex unification process (33) which seems to be impaired in patients: Several studies have investigated the neural activity during the interpretation and integration of abstract, especially metaphoric, speech and gesture. According to these studies, neural activity correlates with the processing of abstract speech and the integration of gesture. Compared to healthy controls, patients with schizophrenia show abnormal activation, mostly in fronto-temporal regions, during the interpretation of metaphorical speech (10). During the integration of metaphoric gestures, patients with schizophrenia also show abnormal activation of and connectivity between frontal regions, especially in the inferior frontal gyrus (IFG) and middle and superior temporal areas (30, 31). Strikingly, aberrant neural activation, as well as reduced behavioral performance in communication tasks, has been found in medicated patients with chronic schizophrenia in most of the studies and hence seems to be stable despite of medication.

As outlined above, the exclusive medical treatment of schizophrenia is not effective for social-communicative functioning. Studies in the last 20 to 30 years have proved psychotherapeutic intervention to be an effective treatment of schizophrenia and at least support the medical treatment (3). Nowadays, several therapeutic programs exist which can efficiently complement the therapy of patients with schizophrenia. Along with psychotherapy (e.g., cognitive behavioral therapy), there exist occupational therapy (34, 35), physical therapy (36)/dance and movement therapy (37), and diverse art therapies (38) [e.g., art therapy—with inconsistent outcomes, however (39)—and music therapy (40–43)] for patients with schizophrenia.

So far, there have been few studies dealing with speech or communication therapy for patients with schizophrenia, although patients show problems in production and perception of speech and gesture integration (44). In a systematic review on speech language therapy in schizophrenia, Joyal and colleagues complain about speech language therapy not yet being a systematical part of a comprehensive intervention. They claim that speech and language deficits “might not always be the most preoccupying symptoms, in comparison with other symptoms such as hallucinations ” (45). Nevertheless, first studies are investigating the outcome of communication therapy in schizophrenia. There already exist some single subject studies showing positive effects of language and speech therapy on speech production in patients with symptoms such as alogia or delusional speech (46–49). Further studies have found positive effects for discourse production (score of intelligibility, appropriateness and elaboration of responses) (50), verbal fluency (51–53) and naming (54). On the other hand, some studies could neither find a benefit of speech language therapy for sentence understanding, repetition and naming (55), nor for verbal fluency (52, 56) and pragmatic non-verbal skills (57). However, except one of these studies, all previous studies focused on speech production, not on understanding language [for more information see the systematic review of Joyal and colleagues, mentioned above (45)]. According to the classic Wernicke-Lichtheim-Geschwind model, speech production is secondary to perception (58). Hence, in speech language therapy, for example in patients with aphasia after stroke where both modalities are affected, speech perception is treated first or at least parallel to the production of speech. Due to the fact that in schizophrenia understanding language is frequently affected, a training of speech perception and interpretation of meaning is appropriate.

Because some of the main communication problems of patients with schizophrenia happen on the pragmatic level, it is important to not only take into account isolated words or phrases, but whole sentences which create a context. It can be assumed that one of the communication problems in schizophrenia results from a lack of integrating words into larger units (59). The integration of words in context is strongly associated with working memory capacities, as Kintsch and van Dijk claim in their model of text comprehension and production (60). Hence, working memory and language processing seem to be thoroughly connected. This is also true for working memory and the integration of gesture (14, 21, 61). For that reason, not only sentence level but also specifically working memory for speech and gesture should be taken into account. For working memory (but not in the context of speech-gesture integration) there already exist some training approaches using n-back tasks (62–66).

Social communication is not only based on speech itself but also on the integration of nonverbal information such as gesture (67). Thus, a potential social communicative gesture training might help to develop, reactivate or promote communication resources in patients with schizophrenia. Nonverbal trainings are rare in the field of psychiatry. One study executed no training, but could show positive effects of transcranial direct current stimulation (tDCS) on gesture integration in a group of patients with schizophrenia spectrum disorder (68). Another study could show a benefit from single session transcranial magnetic stimulation (TMS) also for gesture production in patients with schizophrenia (69). An effect, which could be potentially increased and prolonged in combination with an adequate gesture training program. Despite the lack of previous studies about (perceptual) speech and gesture training for psychiatric patients, there is some evidence about effects of gesture training in other cognitive impairments such as aphasia (which is close to the speech symptoms in schizophrenia as one can see in the term “schizophasia” for speech and language problems in schizophrenia). In a systematic review from 2013, Rose and colleagues state that an isolated gesture training has no impact, but 50% of patients with aphasia are able to benefit from a combined speech-gesture-training. Some of the evaluated studies even showed generalization effects (70). Gesture cues (perceptual modality) also seem to have a positive outcome (71). The majority of the currently existing gesture trainings for patients with aphasia are based on the one word level and were shown to specifically improve naming performance. The influence of speech-gesture-training on the sentence level has scarcely been examined so far, but given the previous findings, it seems to be promising to implement such a speech-gesture-training for patients with schizophrenia.

Considering the potential impact of communication on social life and therefore life quality in general, it is particularly important to focus on the possibility of a holistic communication intervention.

To solve the open questions regarding the efficiency of a specific MSG training in patients with schizophrenia, considering 1) natural communication—productive and perceptive, 2) working memory functions on sentence level and 3) integration of nonverbal communication, we developed a specific MSG training program, which will be evaluated with neural, behavioral and social outcome measures. Due to the possible impact dysfunctional communication has on life quality, we also evaluate transfer in everyday life social functioning.

For the first time, a specific MSG training has been tailored to the requirements of patients with schizophrenia. It is therefore important to assess whether the training is delivered as described in practice and whether the training is accepted by the patients. Given the potential difficulties in motivation and regularly attending sessions, we proposed offering eight sessions of high frequent intensive single speech-gesture-training to the participants. A high frequency of training sessions is also suggested by Joyal et al. for speech therapy in patients with schizophrenia (45). This number of sessions appears to be the minimum of sessions where participants can improve in such a novel training program (72).

The training was developed with focus on the main communication problems of persons diagnosed with schizophrenia. These include “concretism” (the inability to interpret abstract meanings considering context) and problems in nonverbal communication, especially in gesture accompanying abstract speech.

The setup of the MSG training sessions follows best practice in therapy intervention. Exercises are executed with increasing complexity (first perceptual, then productive and free productive tasks) and accompanied by an examiner. Items with concrete as well as abstract speech content are included in the exercises. To motivate the participants and to establish a trusting and respectful relationship between participant and examiner, the examiner introduces the training sessions with small talk and offers information about (nonverbal) communication. For reasons of transfer, the MSG training program includes homework which the participant is invited to do with a person he/she is in close contact.

We expect to achieve a range of positive behavioural (73), neural (74, 75) and social outcomes (76).

We measure behavioral outcomes through the perception of videos with speech and gesture during fMRI measurements. We further expect that initial difficulties in memorizing gesture and speech information can be reduced due to the MSG training so that behavioral performance in a speech gesture working memory task (see below) is more similar to healthy controls after training. In addition to the outcome measures, we explore improvement over time during the eight training sessions regarding speech-gesture matching performance [reaction times (RT) and percent correct], gesture-speech working memory performance (RTs and percent correct), SG fluency performance (number of correct items) and gesture imitation (experimenter evaluation of accuracy) in dependency from speech content (concrete/abstract).

We measure neural patterns through fMRI during the perception of videos with speech and gesture information using specific experiments with iconic (concrete) and metaphoric (abstract) gesture materials (15) as well as an n-back working-memory task. Furthermore, we correlate the behavioral results with neural activation patterns using fMRI techniques before and after the training. Similar to the behavioral outcomes, we expect the neural activity of the patients in relevant brain regions [specifically the left posterior temporal lobe and the left inferior frontal gyrus, cf. (15, 30, 75, 77)] to converge to the neural activity of healthy controls.

As communication is the basis for social interaction and hence for quality of life, transfer effects will be investigated through standardized psychological questionnaires and a specifically outlined questionnaire about nonverbal communication and social life.

However, we do not yet know which exercises and measurements might be most appropriate in terms of acceptability of completion and variability of outcome. We are going to provide descriptive statistics.

This study aims to investigate the behavioral and neural effects of a new speech-gesture-training program for patients with schizophrenia.

The objectives are to:

Assess the acceptability of a MSG training in patients.

To reduce dropout rates, we decided to undertake one familiarization session before we start with the actual training program and measurements. In this session, patients and control subjects are familiarized with the MRI environment and specific procedures of the MSG training procedure (specifically, with perceptive tasks and gesture imitation). For this reason, participants undergo two MRI measurements before (pre-fMRI) and after (post-fMRI) a short speech-gesture training of approximately 30 min. After the post-fMRI measurement, participants are asked to solve a memory task.

Before we start the measurement, the participants are informed about the setting of the study. A member of the research team will go through the study information and answer any questions. This familiarization procedure should reduce dropout rates due to problems with the scanning environment or other training related procedures (e.g., compliance regarding gesture production tasks).

Our interest is the behavioral outcome of an MSG training on communication, social skills and quality of life as well as its neural correlates in brain regions.

Proposed behavioral outcomes are:

In addition to that, we are planning to correlate the neural and behavioral outcomes as well as neural and social and behavioral and social outcomes.

For a summary/overview of the study see Figure 2.

The study started first fMRI measurements (assessment A) on 21^st December 2017 and after treatment as usual and the second fMRI measurement (assessment B), the training started on 20^th January 2018 with participants of the wait-training group. So far (end of March 2019), 18 patients and three healthy controls have been measured and completed the MSG training. With a sample of this size we cannot calculate any group differences yet. This will be done as soon as the samples are complete. After the first participants had completed measurements and training, we evaluated our design and the training make-up. By majority, participants were satisfied with the training and so far, we had only one dropout. Satisfaction can be shown through data from our standardized post training questionnaire, where 13 participants answered that the training was very satisfying, five answered that it was satisfying, no one of the patients who finished the training was unsatisfied with the training. Concerning transfer into everyday life social functioning, the training was rated as very relevant from five patients and as relevant from another five patients. Four patients where not sure, three patients rated the training as less relevant and one patient rated the training as not relevant at all. Regarding the ratings from the patients and considering the high rate of dropouts in other training studies involving patients with schizophrenia (93), we evaluated our design and training as feasible for this group of patients.

As one aspect of embodied communication, (17) gesture is an integral feature of interpersonaldirect nonverbal communication that serves as an outward manifestation of several interacting fundamental processes. The binding of information from gesture and information from speech is a complex unification process (33) which seems to be impaired in schizophrenia. Especially metaphorical gesture processing seems to be disordered (30). Given the fact that in everyday life, metaphors are very frequently used to refer to abstract concepts, (13) a dysfunctional processing of abstract gestures affects integral elements of social skills which may lead to social isolation and therewith to a reduced life quality. These social-communicative deficits are accompanied by aberrations in neural activation (12, 30). Especially hypofrontality (96) and fronto-temporal disconnection (97) are reported in schizophrenia.

Unfortunately, these deficits in interpersonal communication cannot be significantly improved by medication, which might lead to persistent social isolation. Despite this fact, until now, scarcely any therapies can be found which consider specifically communicative and pragmatic skills for those who suffer from schizophrenia (98). Here we provide a first framework, training-procedural description and information about the current study design to solve the open questions regarding the efficiency of a specific MSG training in patients with schizophrenia. In our program, we consider 1) natural communication – productive and perceptive, 2) working memory functions on sentence level and 3) integration of nonverbal communication as well as the important transfer into everyday life social functioning. Furthermore, our newly developed MSG training program will be evaluated with neural, behavioral and social outcome measures, to directly relate dysfunctional neural mechanisms to the potential improvement expected in the intervention groups.

The present study offers an intensive MSG training to patients with schizophrenia and a healthy control group and correlates the behavioral outcomes from this training with neural activation before and after the training as well as with social skills and quality of life. The randomized controlled trial with waiting list and follow-up measurement enables us to demonstrate not only the effect of the gesture training on behavior, self-reports and neural correlates but also to explore long-term effects and potential predictors of treatment outcome (44, 98). While it remains to be seen whether changes can be detected within our proposed measures, the intervention seems at least to be profitable to the patients, who have completed the measurements and trainings so far (see section 2.6 Trial Status).

Previous evidence on training of communicative and pragmatic skills in schizophrenia suggest that it may be an acceptable and tolerable intervention to conduct an intensive MSG training (45). For example, some single subjects studies have shown positive effects of language and speech therapy on speech production in patients with symptoms such as alogia or delusional speech (46–49). Further studies have found positive effects for discourse production (50), verbal fluency (51–53) and naming (54) However, these studies focused on speech production, not on understanding language. According to the Wernicke-Lichtheim-Geschwind model, speech production is secondary to perception (58). Due to the fact that in schizophrenia understanding language is frequently affected, a training of speech perception and interpretation of meaning is appropriate. A multimodal communication training which also addresses gesture and speech perception may offer the opportunity to complement the currently recommended treatments and enable the patients to broaden their experiences. However, there is no evidence on the feasibility, effectiveness, behavioral, social or neural outcomes of such a training.

Since there is rarely evidence of how to design an effective multimodal communication training intervention for patients with schizophrenia, the design of our novel MSG training is mainly based on findings from aphasia treatment and speech therapy in the context of other cognitive deficits (e.g., on findings from the gesture training in aphasia review from Rose and colleagues (79).

As previously shown, interventions such as psychotherapy can change functional neural processes in clinical populations (99) including psychosis (100) For example, it has been shown that cognitive behaviour therapy for psychosis decreased activation of the inferior frontal, insula, thalamus, putamen and occipital areas to fearful and angry expressions at treatment follow-up compared with baseline. In line with this study, fMRI will be used to demonstrate training effects in two paradigms.

Highlighting the advantages of our novel MSG training intervention, in contrast to regular interventions, coupling of social-communicative skills along with motoric features are directly trained in our program. Beyond that, extra-linguistic and pragmatic information found in meaningful hand and arm movement interpretation is addressed. The playful character of the exercises simultaneously trains neurophysiological capacities and may result in a generalisation of possible training effects. Concerning the data that we collect, one of the current study's strengths is the brain imaging techniques that allow us to investigate the neural plasticity effects of the MSG training on social-cognitive abilities in patients with schizophrenia. Furthermore, we focus on cognitive aspects of gesture to facilitate social-cognitive functioning in everyday life. Due to the possible impact dysfunctional communication has on life quality, transfer in everyday life social functioning is one of our main objectives.

Limitations might be that we cannot eliminate effects of medication (changes in antipsychotic treatment) or possible (co-)influence of other therapeutic interventions which may influence the study results. Furthermore, some of the behavioral data that we collect are based on subjective self-given information from the patients and/or their relatives. In addition, the study is rather comprehensive and long-lasting, asking patients not only to train but also to participate in neural imaging assessments. This complex study design might result in a high number of dropouts from patients with severe symptoms. So far, at least from the patients who were willing to participate in our study, only one out of 18 patients stopped to participate prematurely.

Approaches using noninvasive brain stimulation techniques such as transcranial direct current stimulation (tDCS) (68) and transcranial magnetic stimulation (TMS) (66), already showed positive effects on gesture integration and gesture production in patients with schizophrenia, respectively. This is an effect, which could be potentially increased and prolonged in combination with an adequate multimodal speech-gesture training intervention.

With the help of functional Magnetic Resonance Imaging (fMRI) and behavioral measures, further implications for future therapy could be investigated, in order to improve treatment methods for patients with schizophrenia.