Written informed consent was obtained from each participant to procedures approved by the cantonal ethics committee. Fourteen adults (7 women and 7 men; age range 18–64 years, mean ± SD: 40 ± 12.6 years) were tested. Each was compensated 50 Swiss francs for their participation. Ten of the participants were completely blind, and 4 retained some low vision. Most participants were right-handed, only one was left-handed. Regarding Braille literacy, six of them reported good literacy, six of them reported no literacy, and two of them reported little literacy. Most of our participants had an acquired visual impairment or blindness (N = 10), and four of them were congenitally blind. Diagnostic visual acuity measurements were transformed into logMAR (Bonavolonta et al., 2010; Patel et al., 2017). Characteristics about patients' demographics and diagnoses are presented in Table 1. No participant reported tactile deficits or had a history of or a current neurological or psychiatric illness.

The apparatus was identical to that used in Tivadar et al. (2019). It entailed a 7″ tablet with a 1,024 × 600 pixel touchscreen. The tablet's operating system is Linux-based (Raspbian) and the tablet itself is equipped with a software tool to render and control the presentation of haptic textures. Briefly, the software recodes images in jpeg format into a haptic format using a kit written in C++. Full details of the technique and the haptic rendering are provided in Vezzoli et al. (2016, 2017) and Rekik et al. (2017).

Stimuli were identical to those employed in Tivadar et al. (2019). These were 4 upper-case letters—L, P, F, and G—drawn in Paint (see e.g., Carpenter and Eisenberg, 1978; see also Figure 1). We also used Matlab to generate images wherein each letter was rotated 0°, 90°, 180°, or 270° from upright and mirrored. In terms of the conversion from image to haptic rendering, the stimuli appeared centrally and on a white (i.e., blank in terms of haptic rendering) background. All pixels of the letter stimuli were coded with the same haptic texture, based on the Hap2u pre-installed Texture Editor software. The shape of the ultrasonic vibration was a square wave of roughly 2 μm amplitude (see e.g., Sednaoui et al., 2017). This has been shown to produce the most intense and quick reduction of the friction of the screen under the finger; i.e., a “pointy” sensation. Additionally, a “coarse” texture was produced by using a 3,500 μm period of the square wave (see Hollins and Risner, 2000).

Likewise, the procedure and task are identical to what is described in Tivadar et al. (2019). All participants completed the experiment within a sound-attenuated, darkened room (WhisperRoom MDL 102126E). All of our participants wore a blindfold as well as noise-canceling headphones (Bose model QuietComfort 2). This was done so as to block any potential residual light and to mask any sounds from the tablet (see also Murray et al., 2005). This may likely minimize any cross-modal facilitation (Lacey et al., 2007b), impeding a recognition of the presence/absence or location of the letters by the sound the tablet made when explored. We would note that while all participants were familiar with letters, no participant was familiar with the haptic tablet or had prior exposure to the haptic stimuli. We instructed participants to explore the tablet with a finger from their dominant hand and to respond with their non-dominant hand. For each letter, the participant was asked to indicate via a computer mouse if it was presented in normal or mirror-reversed manner (left and right buttons, respectively). We explicitly instructed participants to explore the haptic rendering, to identify the letter, and to imagine rotating the letter to upright (0°) so as to judge if the letter was normal vs. mirror-reversed. Stimulus duration was 30 s followed by a 20 second response window. These timings were determined during pilot experiments, where it was evident that participants needed some time to recognize when a stimulus appeared on or disappeared off the screen of the tablet. The ensuing trial was initiated after the response and with the further inclusion of an inter-trial interval ranging 500–1,000 ms. To train the participants with the haptic tablet, we had each of them complete three training blocks, each comprised of 16 trials (2 per condition). During training, a participant was exposed to only 2 of the 4 letters (either L and P or alternatively only F and G). We counterbalanced across individuals the letters to which they were exposed. The pairing of letters was based on their perceptual proximity, which allowed a progressive learning procedure. We trained participants with pairs of letters in this way in order to assess whether effects generalized to untrained stimuli. Prior to exposure to the letter stimuli, we first trained participants to explore the tablet screen via lateral sweeps (Stilla and Sathian, 2008). While we did allow participants to change the finger used for exploring the tablet (to minimize any adaptation or habituation effects), we did not allow for changing hands. We then taught participants to differentiate vertical from horizontal lines. Afterwards, participants completed the abovementioned training blocks. The experimenter (R.I.T.) provided participants with verbal instructions as well as feedback regarding general performance and the accuracy of responses throughout the training session. The testing phase comprised 3 blocks of 32 trials, making 96 trials in total per participant [i.e., six trials per each of the 16 combinations of 4 angles (0°, 90°, 180°, and 270°) × 2 conditions (normal/mirror) × 2 training (trained/untrained)]. During the experiment, participants were ecnouraged to take regular breaks between blocks of trials to maintain high concentration and prevent fatigue. The total experiment duration was no longer than 3 h 30 min. Stimulus delivery and behavioral response collection were controlled by Psychopy software (Peirce, 2007).

We used Matlab to pre-process the data and R (R Core Team, 2018) for analyses. First, we excluded all trials that were classed as missed trials, i.e., trials with RTs over 20 s (15.1% of trials). Any remaining outlier trials were then excluded on a single-subject basis (i.e., for each subject and condition), applying a criterion of the mean ±2 standard deviations around their RTs (2.9% of all trials, see Ratcliff, 1993; Field, 2012). Accuracy means were calculated from the remaining trials. Missing means in certain conditions were replaced with the mean of all subjects for the specific condition (14 cases in total, 8.75% of total data). Upon inspection of Accuracy scores, we found 4 of the 14 patients had a global performance (i.e., across all angles of presentation) for letters in their normal form that was equal to or lower than chance, i.e., ≤50% (Figure 2A). We excluded these participants. Our final group comprised 10 participants (aged 27–64 years; mean ± SD: 40.7 ± 12.1 years). We compared Accuracy with a 2 × 2 × 4 repeated measures ANOVA with factors Training (trained/untrained), Condition (normal/mirror) and Angle (0°, 90°, 180°, 270°), after not having found a significant deviation from the normal distribution and from homoscedasticity (assumptions tested with Shapiro and Levene tests). Greenhouse-Geiser corrections were applied whenever sphericity was violated. Partial eta-squared is reported as a measure of effect size. As in Tivadar et al. (2019), RTs were not further analyzed as they represented somewhat cued responses, and were not deemed informative as such. More specifically and as described above in the Methods section, participants had to explore the tablet for 30 s before being able to give responses on any given trial. This was done out of practical reasons, in order to ensure correct interaction with the tablet. Therefore, RTs are not representative of the time it took participants to correctly identify the form of a given letter, as even if participants had correctly identified the letter before the 30 s passed, they were not able to respond.

Group-averaged (N = 10) accuracy rates are displayed in Figure 2B. The 2 × 2 × 4 repeated measures ANOVAs (Training × Condition × Angle) exhibited a significant 3-way interaction (Training × Condition × Angle) [F_(3,27) = 3.28, p = 0.04, η_p² = 0.27], and a significant 2-way interaction (Condition × Angle) [F_(3,27) = 3.01, p = 0.05, η_p² = 0.25]. There was also a main effect of Condition [F_(1,9) = 7.6, p = 0.02, η_p² = 0.46], with generally higher accuracy on Normal than on Mirrored letters. We therefore next conducted two separate 2 × 4 repeated measures ANOVAs (Trained × Angle) for Normal and Mirrored letters. For Normal letters, we observed a significant Trained × Angle interaction [F_(3,27) = 3.81, p = 0.02, η_p² = 0.30]. Despite this interaction, the post-hoc contrasts comparing accuracy rates for each angle revealed no significant differences between Trained and Untrained conditions (all p′s > 0.08; without correction for multiple comparisons). Crucially, however, the within-subjects contrasts revealed a linear main effect of Angle [F_(1,9) = 10.00, p = 0.01, η_p² = 0.53]. Participants thus demonstrated typical mental rotation effect (Shepard and Metzler, 1971), i.e., decreasing performance accuracy with increasing angular disparity from upright, for letters presented in their normal form. No significant interactions or main effects were observed for Mirrored letters (F′s < 0.9).

We have previously provided a proof-of-concept for sighted subjects with this same protocol as well (Tivadar et al., 2019). We display these data here in Figure 2C. In this prior work, we observed a mental rotation effect for trained letters, but not for untrained letters, presented in their normal format. Given that identical paradigms and equipment were used in both studies, we a posteriori compared performance across sighted and visually-impaired for the normal format condition, as these were those exhibiting a mental rotation in each group when studied separately. Specifically, we performed a 2 × 2 × 4 mixed model repeated measures ANOVA with group as the between-subjects factor and Trained and Angle as the within-subjects factors, as described above. The within-subjects contrasts revealed a significant 3-way interaction [F_(1,22) = 6.71, p = 0.017, ηp2 = 0.23]. To better understand the basis for this interaction, additional 2-way mixed model ANOVAs were run for trained and untrained conditions separately. For trained letters, there was a linear main effect of angle [F_(1,22) = 6.89, p = 0.015, ηp2 = 0.24], as expected, but no significant interaction with group (p > 0.45). By contrast, for untrained letters, there was both a linear main effect of angle [F_(1,22) = 9.99, p = 0.005, ηp2 = 0.31], as expected, as well as a significant interaction with group [F_(1,22) = 6.93, p = 0.015, ηp2 = 0.24]. This overall pattern indicates that visually impaired individuals exhibit greater generalization to untrained letters than their sighted counterparts (Figures 2B,C).

The fact that visually-impaired and blind participants exhibited mental rotation effects for both trained and untrained normal stimuli might be indicative of an item-independent encoding system operating in these individuals, which allows for facilitated transfer between new and learnt stimuli and faster generalization of encoding rules. It was previously proposed that this might be a result of changes in stimulus processing and encoding that are driven by neuroplasticity (Collignon et al., 2006, 2009, 2015). Specifically, it was suggested that the congenitally blind may show different learning strategies from the sighted as a result of allocating more attention to sensory information processing (Pring, 1988; Collignon et al., 2006). In fact, to test the hypothesis that the lack of visual input results in data-driven rather than conceptual encoding strategies, Röder and Rösler (2003) tested memory for environmental sounds in sighted, congenitally-blind and late-blind subjects. They had participants encode sound either in a semantic (“deep”) or in a physical (“shallow”) fashion, and found that while all three groups profited most from semantic encoding, congenitally blind individuals outperformed the sighted ones on conceptually similar items after encoding (Röder and Rösler, 2003), and age-matched late-blind performed as well as congenitally-blind participants. Such results refute previous beliefs that the blind are less able to use conceptually based encoding strategies (for a discussion, see Thinus-Blanc and Gaunet, 1997), and instead support the hypothesis that the visually-impaired are able to make elevated use of perceptual encoding to aid recognition (Röder and Rösler, 2003; Rokem and Ahissar, 2009). Our results are also consistent with these hypotheses. The lack of differences in performance of our participants between trained and untrained stimuli suggests that once a general schema of stimulus encoding is created, visually-impaired and blind individuals easily transfer the learned concepts to unfamiliar stimuli. The visually-impaired and blind might thus rely more on conceptual item-independent processing (Röder et al., 1997; Collignon et al., 2006; Rokem and Ahissar, 2009) that may help compensate for visual loss.

The multisensory or “supra-modal” nature of object and spatial representations has important implications for rehabilitation applications using sensory substitution in individuals where input from one sensory modality, for example vision, is either impaired or absent. In such cases, both multisensory and cross-modal processes are primary drivers of neuroplasticity in visual areas (Kirkwood et al., 1996; Amedi et al., 2004; Merabet et al., 2005; Pascual-Leone et al., 2005; Murray et al., 2015), which may promote a task-selective and modality-independent re-specialization of these cortical structures (Murray et al., 2016; Amedi et al., 2017). In addition, spatial feature processing does not generally seem to rely on information from a specific modality (Pribram, 1971; Amedi et al., 2001; Pietrini et al., 2004; Struiksma et al., 2009). Lacey et al. (2007b) also demonstrated that such “supra-modal” representations of spatial characteristics are viewpoint-independent, and thus unaffected by object constancy issues (see also Lacey et al., 2009). One implication is that it may prove easier to achieve an “abstract” object representation (Pietrini et al., 2004). It has been repeatedly demonstrated that tactile information specifically can support spatial functions in blind, visually-impaired, and sighted subjects (Marmor and Zaback, 1976; Carpenter and Eisenberg, 1978; Grant et al., 2000; Ptito et al., 2005; Sathian, 2005; Chebat et al., 2007; Wan et al., 2010; Rovira et al., 2011; Vinter et al., 2012).

However, classical devices for conveying tactile information, such as the Braille alphabet, the white cane, and tactile maps, remain limited in the breadth of information they can convey, their accessibility, and their ergonomics (among other considerations) (reviewed in Gori et al., 2016). There are several potential advantages of digital haptics using the finger/hand over technologies such as the BrainPort or TDU delivered to the mouth. For one, digital haptics are completely non-invasive. The abovementioned technologies are not only ergonomically invasive (i.e., the mouth is full), but also use electrical stimulation of the tongue. Second, participants using the TDU needed 9 h of training on a single letter from Snellen's E test (i.e., the letter E) to recognize the letter presented in sizes ranging from 5 to 1.3 cm with 100% accuracy (see Figure 2 in Sampaio et al., 2001). By comparison, our participants needed 45 min to recognize trained and untrained letters in their normal form at a 0° and 90° angle (i.e., 4 letters, size on screen height × length: 4 × 5 cm) with ~70–75% accuracy. Thus, our method allows faster learning of a wider vocabulary of stimuli. In addition, the renderings via digital haptics allow for rapid and even dynamically changing simulation of a wide variety stimuli (from letters to full spatial maps) that need only to be digitally translated (i.e., coded) into a haptic form. This can be based on pre-programmed libraries or alternatively on real-time image-to-haptics conversation algorithms. Moreover, haptics allow the user a greater degree of control; the device can be explored at the discretion of the individual when they actively explore the tablet with their hand, leaving their mouth (and voice) unobstructed. These collective features may also augment the accessibility of such devices to both children and the elderly alike. However, we would also note that 4 of our 14 patients (i.e., 29%) did not meet our inclusion criterion of greater than chance levels with upright stimuli, independent of their angle of rotation. We can only speculate as to the potential causes and contributing factors. However, it is in our view unlikely that experience with Braille or the etiology of visual impairment has a direct link to performance with the haptic tablet. Both groups included individuals with either congenital or acquired impairments as well as individuals literate and illiterate in Braille. It will be important for further applications of this technology to establish if and how tactile sensitivity and discrimination abilities may underscore abilities to quickly learn to use this device. These points notwithstanding there are a number of promising domains for the application of this technology. One is the transmission of the concept of size-invariance (and perhaps also perspective invariance that can help promote both egocentric and allocentric representations) of haptic stimuli. This is an aspect that requires further exploration.

To date, devices including BrainPort and TDU have shown to be effective for functions ranging from object recognition, including measures of “visual” acuity (Chebat et al., 2007; Arnoldussen and Fletcher, 2012), to tasks requiring actions coordinated with mental representations of the perceived tactile objects (reviewed in Bach-y-Rita and Kercel, 2003). The present results similarly provide evidence that participants were able to understand the shape and even the form that the letters are haptically presented in (i.e., normal vs. mirror-reversed), speaks in favor of the application of digital haptics for simulation of spatial features. This is important, as spatial features rely on internal representations, and are directly related to spatial behaviors (Thinus-Blanc and Gaunet, 1997), such as object manipulation, localization, and spatial mapping. Ongoing work from our laboratory focuses on the applicability of this haptic technology in simulating trajectories modeled on a realistic indoor environment (i.e., an apartment's layout and corridors). Thus, the spatial functions that this technology has the potential to support can be directly extended to independent mobility.

The main innovation from our study is the successful application of a digital method. This is important, as digital rehabilitation methods promise to alleviate the medical field by reducing the necessary resources. Digital haptic stimulation finds applications not only in restoring spatial functions in the blind and visually-impaired, but also for rehabilitation of such functions in participants after sight restoration, for example in sight-restored cataract patients (McKyton et al., 2015). Specifically (McKyton et al., 2015), cataract operated children and young adults demonstrated immense deficits in mid-level visual processing (such as 3D object shapes) after cataract removal, despite intact low-level visual abilities. Using digital haptics, such patients may retrain their deficient spatial skills, in a safe, easy, and cost-effective way. As these representations have a “supra-modal” nature, digital tactile stimulation could aid existing therapy techniques to help patients encode a more abstract object representation. Thus, it is evident that the applicability of digital haptics is very promising for the medical domain.

CC is the CEO and Founder of Hap2U. CC thus has commercial, proprietary, and financial interest in Hap2U, which provided the haptic tablet device instrument related to this article. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.