Front. Hum. Neurosci.Frontiers in Human NeuroscienceFront. Hum. Neurosci.1662-5161Frontiers Media S.A.10.3389/fnhum.2013.00516NeuroscienceOriginal Research ArticleTemporal hemodynamic classification of two hands tapping using functional near—infrared spectroscopyThanh HaiNguyen^{1}CuongNgo Q.^{2}Dang KhoaTruong Q.^{1}^{*}Van ToiVo^{1}^{1}Biomedical Engineering Department, International University of Vietnam National Universities in Ho Chi Minh CityHo Chi Minh City, Vietnam^{2}Department of Electronics and Telecommunications, Faculty of Electrical and Electronics Engineering, University of Technical Education HCMCHo Chi Minh City, Vietnam
Edited by: Nobuo Masataka, Kyoto University, Japan
Reviewed by: Carlo Cattani, University of Salerno, Italy; Thang M. Hoang, University of Canberra, Australia
*Correspondence: Truong Q. Dang Khoa, Biomedical Engineering Department, International University of Vietnam National Universities in Ho Chi Minh City, Quarter 6, Linh Trung Ward, Thu Duc District, Ho Chi Minh City, Vietnam e-mail: khoa@ieee.org
This article was submitted to the journal Frontiers in Human Neuroscience.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
In recent decades, a lot of achievements have been obtained in imaging and cognitive neuroscience of human brain. Brain's activities can be shown by a number of different kinds of non-invasive technologies, such as: Near-Infrared Spectroscopy (NIRS), Magnetic Resonance Imaging (MRI), and ElectroEncephaloGraphy (EEG; Wolpaw et al., 2002; Weiskopf et al., 2004; Blankertz et al., 2006). NIRS has become the convenient technology for experimental brain purposes. The change of oxygenation changes (oxy-Hb) along task period depending on location of channel on the cortex has been studied: sustained activation in the motor cortex, transient activation during the initial segments in the somatosensory cortex, and accumulating activation in the frontal lobe (Gentili et al., 2010). Oxy-Hb concentration at the aforementioned sites in the brain can also be used as a predictive factor allows prediction of subject's investigation behavior with a considerable degree of precision (Shimokawa et al., 2009). In this paper, a study of recognition algorithm will be described for recognition whether one taps the left hand (LH) or the right hand (RH). Data with noises and artifacts collected from a multi-channel system will be pre-processed using a Savitzky–Golay filter for getting more smoothly data. Characteristics of the filtered signals during LH and RH tapping process will be extracted using a polynomial regression (PR) algorithm. Coefficients of the polynomial, which correspond to Oxygen-Hemoglobin (Oxy-Hb) concentration, will be applied for the recognition models of hand tapping. Support Vector Machines (SVM) will be applied to validate the obtained coefficient data for hand tapping recognition. In addition, for the objective of comparison, Artificial Neural Networks (ANNs) was also applied to recognize hand tapping side with the same principle. Experimental results have been done many trials on three subjects to illustrate the effectiveness of the proposed method.
Human brain has a complex structure with around 100 billion neurons, so it is a big challenge for all scientists in biological computing (Wolpaw et al., 2002). These neurons can communicate from one to another with or without external excitations to make typical decisions (pattern recognition, cognition, motion, and others; Critchley, 2009). Moreover, in prefrontal cortex of human brain plays an important role in social activity for both adults and children. Tobias Grossmann represented a review related to the role of prefrontal cortex of human brain, in which specific areas in the adult human brain as social brain could process the social world (Aydore et al., 2010; Grossmann, 2013) and also Tila Tabea Brink et al. investigated about orbitofrontal cortex in children with 4− to 8-year-old through processing empathy stories (Brink et al., 2011). The result is that children could passively follow these stories presenting social situations. Regarding prefrontal cortex, EEG electrodes were mounted on frontal positions of human brain for wheelchair control (Ahmed, 2011). In particular, user could move eyes to drive the electrical wheelchair to reach the desired target.
In recent decades, a lot of achievements have been obtained in imaging and cognitive neuroscience of human brain. Brain's activities can be explored using different kinds of non-invasive technologies, such as: Magnetic Resonance Imaging (MRI), Near-Infrared Spectroscopy (NIRS), and ElectroEncephaloGraphy (EEG; Wolpaw et al., 2002; Weiskopf et al., 2004; Blankertz et al., 2006; Ince et al., 2009). Many researchers have been attracted by these technologies with many approaches to find out problems related to human brain for rehabilitation and treatment. For the rehabilitation problem, information obtained from human brain using EEG technique could be employed to perform shared control of motion wheelchairs (Tanaka et al., 2005). A brain simulator can lead to improve or to recover the cognitive/motor functions of tetraplegic patients with degenerative nerve diseases spinal cord injuries (Kauhanen et al., 2006). In these non-invasive technologies, the NIRS technology is often applied to measure Oxygen Hemoglobin (Oxy-Hb), deOxy-Hb, and Total-Hb concentration changes. These changes allow us predict brain activations related to body behaviors.
fNIRS has become the convenient technology for experimental brain purposes. This non-invasive technique emits near infrared light into the brain to measure cerebral hemodynamics as well as to detect localized blood volume and oxygenation changes (Tsunashima and Yanagisawa, 2009). The change of oxy-Hb along task period depending on the location of channels the cortex has been studied: sustained activation in the motor cortex, transient activation during the initial segments in the somatosensory cortex, and accumulating activation in the frontal lobe (Gentili et al., 2010). Oxy-Hb concentration at the aforementioned sites in the brain can also be used as a predictive factor allows prediction of subjects' investigation behavior with a considerable degree of precision (Shimokawa et al., 2009).
fNIRS technique is a non-invasive technique which is applied to monitor human body for diagnosis and treatment (Bozkurt et al., 2005; Macnab et al., 2011; Reher et al., 2011). Hiroshi Taniguchi et al. investigated six subjects with unilateral spatial neglect (USN)-positive (+) and 6 others with USN-negative (Taniguchi et al., 2012). In this research, brain activity was simulated by prism adaptation tasks using fNIRS. The result showed that there was a typically great reduction in Oxy-Hb of the USN (+). For monitoring carotid endarterectomy, one was applied the NIRS technique to evaluate its reliability in the detection of clamping ischemia (Pedrini et al., 2012). The result found that there were three patients who represented transient ischemic deficits at awakening and no case of perioperative stroke or death.
In addition, fNIRS technique has been appeared as an alternative brain-based experimental technique (Lloyd-Fox et al., 2010) to measure human thoughts and activities for rehabilitation. For evaluating behaviors related to human brain during experiments, subjects feel free for performing his or her brain activities. In particular, this technique has been successfully used to study brain functions such as assessment of motor task from everyday living, athletic performance, recovery from neurological illness (Hu et al., 2010), assessment of verbal fluency (Schecklmann et al., 2010), and quantification of brain function during finger tapping (Sato et al., 2007). However, to the best of our knowledge, there have been a few applications of the fNIRS technique to quantify the motor control signals leading to brain simulator for rehabilitation (Chunguang et al., 2010; Gentili et al., 2010).
Neural networks can be used for cognition brain tasks as a classification module, in which wavelet decomposition can be used as feature extractions (Khoa and Nakagawa, 2008); wavelet can be used to remove artifacts (Molavi and Dumont, 2010). Base on the slope of straight line, hand side tapping can be distinguished (Ngo et al., 2012). Oxy-Hb and Deoxy-Hb can also be used directly with SVM algorithm for the recognition of hand tapping (Sitaram et al., 2007).
Savitzky–Golay (SG) filters have been used to smooth signals and images with noises as well as artifacts in recent years. In the SG filters, the coefficients of the local least-square polynomial fit are pre-computed to preserve higher movements and then the output of the filter is taken at the center of the window (Savitzky, 1964; Steinier et al., 1972; Gorry, 1990). In this paper, the SG filter was applied to reduce spike noises of Oxy-Hb signals. The Oxy-Hb signals after filtering allow us be easier in recognizing left (LH) or right hand (RH) tapping status. Moreover, a Polynomial Regression (PR) approach has been applied for estimation of signals and images with noise (Cui and Alwan, 2005; Cai et al., 2007; Zhang et al., 2009; Khan et al., 2011). In our research, in order to estimate Oxy-Hb signals, the PR algorithm was used to produce polynomial curves with their features. Based on these features, one can classify tapping hand tasks.
Support Vector Machine (SVM) algorithms have been applied for classification problems in the machine learning community in recent years. In this case, the SVM was employed to classify hypothyroid disease based on UCI machine learning dataset (Chamasemani and Singh, 2011). Another application related to medical images is that the SVM was utilized to recognize the leaf spectral reflectance with different damaged degrees in the image processing and spectral analysis technology (Dake and Chengwei, 2006). In this project, the SVM algorithm (Sitaram et al., 2007) was applied to recognize hand tapping tasks using fNIRS technology. Oxy-Hb signals after reducing noise will be extracted features using a PR algorithm. Based on coefficients obtained from the PR, the SVM algorithm will be applied for the recognition of the LH and RH tapping tasks.
Another algorithm for classification is that a recursive training algorithm for EEG signals using Artificial Neural Networks (ANNs) to generate recognition patterns from EEG signals was proposed to control electric wheelchair (Tanaka et al., 2005; Singla et al., 2011). Mental tasks were classified for wheelchair control using prefrontal EEG (Rifai Chai, 2012). The relevant mental tasks used in this paper are mental arithmetic, ringtone imagery, finger tapping, and words composition with additional tasks which are baseline and eyes closed. The feature extraction is based on the Hilbert Huang Transform (HHT) energy method and then the ANNs with the Genetic Algorithm (GA) optimization (Subasi et al., 2005) was applied for classification. The result is that the accuracy of the proposed classification algorithm with the five subjects participated was between about 76 and 85%.
In this paper, we proposed the recognition algorithm for developing a brain computer interface using fNIRS. First of all, Savitzky–Golay filter is used to reduce noises as well as artifacts. Coefficients, which are features of Oxy-Hb signals, are found by using a PR algorithm. For the recognition of tapping hands related to the left and right brain activation, ANN and SVM algorithms were used. These two methods will be compared to find out the best one. The results and discussion about tapping hand activity will be shown to illustrate the effectiveness of the proposed approaches. This process is shown in Figure 1.
Recognition algorithm block diagram. First of all, Savitzky–Golay filter is used to reduce noises as well as artifacts. After that, feature of Oxy-Hb is found by a polynomial regression based on its coefficients. Finally, Artificial Neural Network or Support Vector Machines is used to determine whether left hand or right hand is tapped.
Materials and methodsSubjects and the experimental setup
A multichannel fNIRS instrument, FOIRE-3000 (SHIMAZU Co. LTD, Japan), is used to acquire brain Oxy-Hb. This machine was located at Lab-104 of Biomedical Engineering Department, International University, VNU, Vietnam. The FOIRE 3000 system with the eight pairs of the probes, consisting of the illuminator and detector optodes, produces 24 channels as shown in Figure 2A. These probes were placed on the scalp to collect fNIRS data, in which the detectors were installed at a 3 cm distance from the illuminators. The optodes were arranged to install at the left hemisphere on the head of the subject.
(A) fNIRS FOIRE-3000 system. This system operates at three different wavelengths of 780, 805, and 830 nm. (B) Subject' head with installed probes. The distance between pairs of emitter and detector probes was set at 3 cm and all probes were attached with holders.
Oxy-Hb concentration changes in motor control area of human brain was captured from a set of the holder with 24 channels for both hemispheres using the fNIRS technique as shown in Figure 2B. In particular, when the subject performs one typical activity, brain signals will be obtained from the fNIRS system and then calculated to produce three types of signals [Oxy-Hb (red), Total-Hb (green) and Deoxy-Hb (blue)] corresponding to three wavelengths (780, 805, and 830 nm), in which [Total-Hb] = [Oxy-Hb] + [Deoxy-Hb]. Moreover, the distance between pairs of emitter and detector probes was set at 3 cm and all probes were attached with holders arranged on different sides of human brain hemispheres depending on users. Concentration changes of three signals produce time points in an output. In this research, Oxy-Hb changes are calculated in the following formula (Shimadzu Corporation, 2010):
Oxy=−3.6132∗Abs[780nm]+1.1397∗Abs[805nm]+3.0153∗Abs[830nm]
in which Abs: Absorbance.
Three subjects (male, average: 25 years old, 60 kg weights, right-handed) were participated into this study. All participants were healthy and showed no musculoskeletal or neurological restrictions or diseases. Before participating into the experiments, each subject was asked to fill out a questionnaire consisting of patient's identification, age and gender, which was kept confidential. The tenets of the Declaration of Helsinki were followed; the local Institutional Review Board approved the study. These subjects informed consent agreement after reading and understanding of the experiment protocol and the fNIRS technique.
After reading and understanding the experiment protocol and the fNIRS technique, he will start doing hand tapping. The subject was required to perform hand tapping motions, both left and right sides as motor activities. In these hand tapping motions, a protocol includes 20 s (Rest)—20 s (Task)—20 s (Rest), it means that the subject relaxed in 20 s, tapped his hand up/down about 10 times in 20 s, and then rested 20 s, as shown in Figure 3.
Setting of experiment protocol. The subject relaxed in 20 s, tapped his hand up/down about 10 times in 20 s, and then rested 20 s.
Oxy-Hb data were collected on 20 channels, in which 10 channels are of the left brain side and that of the opposite side will be obtained for hand tapping recognition. However, we just chose 4 channels of each side which focus on hand and leg motion area to analyze and to estimate features. In particular, the left brain channels are 2, 5, 6, 9, and the 12, 15, 16, 19 channels are of the right brain side as in Figures 4A,B. In this research, Oxy-Hb data obtained from these channels will be processed to recognize hand tapping tasks. Without loss of generality, the natural architecture is different from person to person. The probes are allocated on the holder, in which the transmitter probes and receiver probes are predicted to cover as much as area of brain based on the physical structure of each subject. The authors (Aihara et al., 2012) combined EEG and NIRS for estimation of cortical current source. The probes position using stylus marker to allow co-registration of EEG and NIRS results. In this paper, we also used marker to find out the average positions of motor area of human brain cortex. To achieve more accuracy, the NIRS activity was mapped onto cerebral cortex using fusion software (Shimadzu Corporation, 2010). From this evidence, we proposed the selection of channels 2, 5, 6, 9 and 12, 15, 16, 19 for hand tapping recognition with the 20-channel NIRS system configured above.
Probes location and channels on two hemispheres. (A) Probes location (red—emitter, blue—detector) and channels on the motor control area of the left hemisphere. (B) Probes location and channels (yellow) on the motor control area of the right hemisphere.
Data pre-processing
Brain data of a subject acquired from the channels have noise and artifacts. In order to obtain more smoothly brain data, a filter as the Savitzky–Golay filter was applied in this paper. The Savitzky–Golay filters (Orfanidis, 2010) are also known as polynomial smoothing. It means that the idea of the polynomial smoothing is replacing samples of signal by the values that lie on the smoothing curve. In moving an average FIR filter, the output is a simply average version of its inputs, in which this filter has the response of the low-pass filter. In practice, NIRS signals fluctuate along time corresponding to excitations and have the unknown specific frequency. Therefore, it could not be the average of inputs with the arbitrary FIR filter length. In this research, to track the acquired signal, the Savitzky–Golay filter as the FIR filter can be used.
In general, we can evaluate a polynomial with the order of d to smooth the length-N data x with the condition N ≥ d + 1. Assume that, the data x is the type of a vector
x=[x−M,…,x−1,x0,x1,…,xM]T
in which N samples of x are replaced by the polynomial with the order of d as follow:
x^m=c0+c1m+⋯+cdmd,−M≤m≤M
where c_{0}, c_{1}, …, c_{d} denote polynomial coefficients. M is the number of points on either side of x_{0}
In this case, there are d + 1 based on the vector s_{i}, i = 0, 1, …, d as follows:
si(m)=mi,−M≤m≤M
Thus, we can write the vector S as follows:
S=[s0,s1,…,sd]
in which s_{0}, s_{1}, …, s_{d} are the polynomial basic vectors.
The smooth values in (3) can be re-written in the following equation:
x^=∑i=0dcisi=[s0,s1,…,sd][c0c1⋮cd]=Sc
Coefficients of the desired filters are obtained as follows:
B=SGT=GST=SF−1ST=[b−M,…,b0,…bM]
in which, b_{−M}, …, b_{0}, …, b_{M} are the column filters of the Savitzky–Golay filter set.
{F=STSG=SF−1
Finally, the values to create more smoothly signals are estimated in the following equation:
x^m=bmTx,m=−M,…,0…,M
in which, b^{T}_{m} are the transpose version of b_{m}.
In this paper, the Savitzky–Golay filter will be utilized to smooth spikes of brain Oxy-Hb signals for identifying hand tapping tasks. The filtered Oxy-Hb signals allow us extract features with reliable information.
Feature extraction
In general, the first step in classification work is to find the features of data samples. For this purpose, there are many methods such as Principle Component Analysis (PCA), Independent Component Analysis (ICA) and etc. However, hemodynamic response of human brain changes in time domain. Moreover, we want to evaluate the Oxy-Hb concentration corresponding to hand tapping tasks based on analyzing numeric as well as having a look in graphical figures.
PR algorithm (Montgomery and Runger, 2003) presents the relationship between amplitude and time of a signal. In this paper, the PR algorithm was applied to analyze brain Oxy-Hb data in blood flow corresponding to hand tapping tasks. From the processed data, one can distinguish the difference between the LH and RH tapping times corresponding to the difference of the Oxy-Hb concentration changes.
Assumed that we have the set of two-dimensional data, (x_{1}, y_{1}), …, (x_{n}, y_{n}), where each of x and y has no information about the other. Our problem is fitting a polynomial curve generated by a typical data. Thus, the relationship between x and y can be found out. Based on the coefficients of the regression curve with the order of 5, one can estimate the hand tapping. In particular, the PR equation between independent variable x and y fitted can be expressed as:
y^=h^0+h^1x+h^2x2+⋯+h^mxm
in which, h^0,h^1,h^2,⋯,h^m are estimated values of h_{0}, h_{1}, h_{2}, …, h_{m}. There are m regressors and n observations, (x_{i1}, x_{i2}, …, x_{im}, y_{i}), i = 1, 2, …, n corresponding to (x_{i}, x^{2}_{i}, …, x^{m}_{i}, y_{i}). In this equation, the powers of x play the role of different independent variables.
The PR model can be re-written as a system of linear equations
y=Xh+ε
where: ε = [ε_{1}, ε_{2}, …, ε_{n}]^{T} is a vector of error.
The ordinary least square h^ of h given by the arguments that minimize the residual sum of squares and the distributive law is employed. One obtains the equation,
RSS(h)=y′y+h′(X′X)h−2y′Xh
Equation 12 is minimized by taking ∂RSS∂h and set the result to zero. This leads to
X′Xh=X′y
The ordinary least square in the case of the inverse of X′X exists is given by
h^=(X′X)−1X′y
From these coefficients, one can determine problems of the LH tapping or RH tapping tasks with the measured brain data using the fNIRS technology. Figure 5 represents the regressed signal of the channel-2 corresponding to Equation 15. Similarly, the regression signals of channels 5, 6, 9, 12, 15, 16, and 19 can be shown.
yC2=−0.0001x5+0.0023x4−0.0114x3+0.0182x2+0.0043x−0.0329
The regression signal of filtered channel 2. Sudden changes had been removed with the window size of 11.
In this figure, the blue Oxy-Hb signal after the Savitzky–Golay filter was calculated to produce the red regressive curve. Each hand tapping creates the regressive Oxy-Hb curves which contain information or its feature coefficients. For recognition of hand tapping types, these coefficients will be given the input of the identification system or called the identification algorithms.
The regressed polynomial must represent the original signal with the best fit. The smaller error between the origin (here is the filtered NIRS signal) and the regressed signal is higher than the order of the polynomial is. It means that one should choose the order not only to fit the origin but also to show the general trend and the characteristic of NIRS signal. In practice, the NIRS signal can not change immediately at the moment of tapping hand. For example, one hand moving up or down will make an excitation to both hemispheres. Therefore, in 20 s of tapping hand, one person could take 10 times of moving hand up and down. In this case, Oxy-Hb level, which will flow from the lowest to highest level in short time, is not the “trend” of overall signal. This is the reason for choosing the polynomial with the order of 5.
Artificial neural network
ANNs are the very powerful tools for the problems of classification and pattern recognition. We can use the estimated coefficients as the features from the PR algorithm by connecting with a multilayer feed forward network for recognition. The architecture of this network used here consists of an input layer, one hidden layer, and the output layer as shown in Figure 6. In particular, input samples are the features from channel coefficients corresponding to Oxy-Hb concentration changes. The number of hidden nodes is carefully chosen for this case to obtain higher performance. Therefore, it can be chosen as an average of number of the input nodes and the output nodes. With the hidden layer, we used the double sigmoid function and this sigmoid function was also used for the output layer.
Architecture of classification network. This net has 48 nodes input, 100 nodes at hidden layer and 2 nodes at output.
In general, standard back propagation is used for training the network with three layers. It is a gradient descent algorithm, in which the network weights are moved along the negative of the gradient of the performance function. With this argument, the training is based on the minimization of the following error function:
E=∑n=1N(on−dn)2,
where N is number of samples, o is network output and d is desired output.
Suppose that the network has I nodes of the input layer, J nodes of the hidden layer and the output layer is K nodes. Call w^{(1,0)}_{j, i} is weight from the ith node of the input layer to the jth node of the hidden layer and w^{(2,1)}_{k, j} is weight from the jth node of the hidden layer to the kth node of the output layer. The backpropagation learning of the 3-layers network is shown in Table 1. The application is that with the LH tapping, the output is desired to get the value of [1; 0] and [0; 1] is the desirable value of the right tapping. The ANN is one of the approaches which is often used for recognition. In this research, the SVM is also applied to identify hand tapping tasks through Oxy-Hb flowing in brain blood.
The three-layers network with backpropagation learning.
Random initial weights
While the Mean Square Error (MSE) is unsatisfied or the number of epochs is not exceed,
For each input x_{p}, 1 ≤ p ≤ P, (^{*})
Compute the inputs of hidden layer net^{(1)}_{p, j};
Compute the outputs of hidden layer x^{(1)}_{p, j};
Compute the inputs of ouput layer net^{(2)}_{p, k};
Where: S() is the active function, η is the learning rate.
Support vector machines
In order to estimate hand tapping tasks, after determining coefficients of hand tapping times using the PR algorithm, we also used the linear SVM algorithm (Shawe-Taylor, 2000) to validate the coefficient data. In the linear SVM algorithm, assume that the training data are {x_{i}, y_{i}}, i = 1, …, l, y_{i} ∈ {−1, 1},x_{i} ∈ R^{d}. The points x which lie on the hyperplane satisfy w.x + b = 0, in which | b |/∥ w ∥ is the distance from the hyperplane to the origin (where ∥ w ∥ is the Euclidean norm of w). Let d_{+} (d_{−}) be the shortest distance from the seperation hyperplane to the closest positive (negative) samples corresponding to the coefficients of LH tapping and RH tapping, respectively. This is showed in Figure 7.
Linear seperation hyperplane for right hand tapping feature and left hand tapping.
Margin of the hyperplane is d_{+} + d_{−}. In the linear case, the support vector looks for the separating hyperplane with the largest margin using the primal Lagrangian. Suppose that all training data satisfy the following contraints:
xi·w+b≥+1, for yi=+1xi·w+b≤−1, for yi=−1
The optimization problem is considered to transform Equations 17 and 18 using the primal Lagrangian as follows:
Lp(w,b,α)=12||w||2−∑i=1lαiyi(xi·w+b)+∑i=1lαi
where α_{i} ≥ 0 are the Lagrange multipliers.
Differentiating L_{p} with respect to w and b and then getting the results to zeros, we have the following equation:
∂Lp(w,b,α)∂w=w−∑i=1lyiαixi=0∂Lp(w,b,α)∂b=∑i=1lyiαi=0
Equations can be re-written to calculate the support vector as follows:
w=∑i=1lyiαixi
The regressed data will trained using the SVM method, in which the hyperplane is a linear function and divided into two planes: D_{+} contains the coefficients and y = +1 is of the left tapping; similarly D_{−} has the coefficients and y = −1 is of the right tapping.
Results and discussion
Oxy-Hb raw signals (blue) were collected from the fNIRS system using the proposed protocol (see Figure 5) which plays an important role during measure tasks. In particular, each subject tapped his hand up or down 10 times in 20 s. Therefore, we could separate this task into 10 parts, in which each part has 1 s up and 1 s down as shown in Figure 9. Before analyzing Oxy-Hb signals, the Savitzky–Golay (SG) filter was chosen to produce the smooth Oxy-Hb signals (red) as shown in the Figures 8A,B. In this filter, if the size of the window is too small, noises still affect upon the Oxy-Hb signals. Otherwise, if the large window size is chosen, the useful information may be lost. As mentioned before, Oxy-Hb signals using fNIRS technique are the concentration of Oxy-Hb in blood flow of human brain related to excitations or activities of human body. Therefore, choosing the window size as well as the order of the filter is very important and also depends on each typical case. For this reason, the SG with the window size of 11 and the order of 3 was applied (see Figures 8A,B).
Raw data and its smoothed version. (A) Raw and smoothed NIRS data of channel 2, 5, 6, 9 of the left hemisphere. (B) Raw and smoothed NIRS data of channel 12, 15, 16, 19 of the right hemisphere.
After smoothing Oxy-Hb signal by using the SG filter, the features of Oxy-Hb signal corresponding to hand tapping are extracted using a PR algorithm. In this case, the PR algorithm with the order-5 polynomial produces six coefficients and its equation is represented as follows:
y=h5x5+h4x4+h3x3+h2x2+h1x+h0
where x represents time from 0 to 7 s with the resolution of 0.07.
The fact is that choosing the window size as well as the order of the polynomial plays an important role due to avoidance of loosing information of signals. In Figures 8, 9, the red Oxy-Hb signals are the smoothed signals, in which the window size and the order were carefully calculated and chosen so that the peaks of the signals removed after filtering do not affect consequences on the analysis.
Smoothed signal analysis. Black dash line shows tapping time period while the green solid curve shows the changes in theory for channel 2.
This equation was applied to determine the regressed Oxy-Hb signals of the channels 2, 5, 6, 9 (left hemisphere) and the channels of the right hemisphere, 12, 15, 16, 19. Thus, the obtained results of the RH tapping and the LH tapping as showed in Figures 10A,B are compared together. However, these features of the Oxy-Hb signals obtained at two hemispheres are very hard to distinguish between are the right tapping and the left tapping. For this reason, training data, which are coefficients of the regressed polynomials as shown in Table 1, were applied to identify hand tapping tasks. In particular, in each time of hand tapping, Oxy-Hb concentration changes of two hemispheres allow us obtain the regressed coefficients using the PR algorithm. Moreover, six coefficients of each channel as shown in Table 2 are arranged to be a vector. For classification of hand tapping tasks, the vector was employed to the algorithms such as the ANN or SVM for training data.
Regressed polynomial corresponding to hand tapping. (A) Regressed polynomial of 8 channels of the left hand tapping. Blue curves are the signals on the left brain side and red curves are of the right brain side. (B) Regressed polynomial of 8 channels of right hand tapping. Blue curves are the signals on the left brain side and red curves are of the right brain side.
The arrangement of the regressed coefficients of hand tapping tasks for obtaining the input of the recognition networks.
Left hand tapping coefficients
Right hand tapping coefficients
Ch-2
Ch-5
Ch-6
Ch-9
Ch-12
Ch-15
Ch-16
Ch-19
h_{21} … h_{26}
h_{51} … h_{56}
h_{61} … h_{66}
h_{91} … h_{96}
h_{121} … _{126}
h_{151} … _{156}
h_{161} … _{166}
h_{191} … h_{196}
Assume that v_{r} is the vector of the RH tapping and the vector of the LH tapping is v_{l}. In one run of experiment, the subject performed a hand tapping task 20 times, in which 10 times for the LH tapping and 10 times for the RH one. Therefore, a set of the LH tapping coefficients S_{l} includes 10 vectors (from v_{l1} to v_{l10}) and that of the RH tapping coefficients S_{r} is 10 vectors (from v_{r1} to v_{r10}). With 80 sample vectors obtained from subjects, the recognition algorithm was worked out by splitting the sample vectors to be 4 runs of 20-fold cross recognition. For identifying the LH tapping, one used 9 vectors of the S_{l} set combined with the 10 vectors of the S_{r} set and the remaining vector of the S_{l} set is used to be a sample vector for identification. In the case of identifying the RH tapping, 9 vectors of the S_{r} set combined with the 10 vectors of the S_{l} and the remaining one is used to be the sample vector for identifying. As known, Oxy-Hb signals obtained from human brain have many noises and artifacts. Therefore, identifying hand tapping tasks corresponding to Oxy-Hb concentration changes is not easy. For this reason, the identification algorithms such the ANN and SVM are reliable in this research. In the SVM method, the linear hyperplane was chosen. In each training process, the values α (having 15 values of α) are produced, also there are 15 support vectors w (each vector w is 48 elements) and b is 0.068. In similarity, the ANN algorithm with the hidden layer of 100 nodes was applied to obtain the training result, in which the goal of training is set up of 0.001 and the number of epochs is 5000.
From the data sets of the hand tapping tasks, the SVM algorithm was applied for learning to analyze data and recognize patterns. In understanding this SVM training algorithm, data vectors from the hand tapping tasks are given the input of the classifier with a hyperplane which forms two possible classes of the output. In this method, experimental results to the LH tapping of Subject-1 and Subject-3 are the same and Subject-2 showed the lower performance with just 72.5% of the accuracy compared with 82.5% of Subject-1 and Subject-3 as shown in Table 3. While basically its results are the same to that of tapping the RH side.
Experiment result of 3 subjects with SVM.
Run
Hand tapping
Accuracy—subject 1 (%)
Accuracy—subject 2 (%)
Accuracy—subject 3 (%)
1
Right
70
80
80
Left
80
80
70
2
Right
90
80
80
Left
100
70
90
3
Right
90
60
90
Left
80
70
100
4
Right
70
80
80
Left
70
70
70
Average
Right
80
75.0
82.5
Left
82.5
72.5
82.5
The ANN algorithm used for identifying hand tapping tasks here consists of one input layer, one hidden layer of 100 nodes and the output layer with two nodes. In addition, the second method in this paper is one of recognition methods which have been applied in recent years. Although this algorithm is used very popular for recognition problems, it still uses here due to giving the good performances and also being a reliable method. The result is that classification using the ANN method gave the around 83% performance of tapping the RH side is higher than the performance of around 73% for the RH tapping as shown in Table 4.
Experiment result of 3 subjects with ANN.
Run
Hand tapping
Accuracy—subject 1 (%)
Accuracy—subject 2 (%)
Accuracy—subject 3 (%)
1
Right
80
80
70
Left
70
80
60
2
Right
90
90
80
Left
80
70
80
3
Right
90
70
100
Left
70
70
80
4
Right
80
90
80
Left
70
80
70
Average
Right
85
82.5
82.5
Left
72.5
75
72.5
All the results of hand tapping tasks obtained here have the accuracy of more than 70%. In this research, two methods were applied to find the best one. The first method is that the SVM algorithm is used for recognition on three subjects and produce different performances. In particular, Subject-3 with tapping the RH has the best result with over 80% of the accuracy, while the accuracy of Subject-2 is only 75% for the case of the RH tapping and 72.5% for that of the LH tapping. While the second method using the ANN is that the accuracy in the case of the RH tapping is equal to or greater than 82.5%. In particular, Subject-1 has the best accuracy of the right tapping, while the accuracy of the left tapping just stops at 72.5% for both Subject-1 and Subject-3. Moreover, the result is that Subject-2 has the best accuracy in the case of the LH tapping. It is clear that two methods used in this research give a little bit different performance. In general, the SVM method is better in this case. We also observed that Subject-2 produced the best accuracy in the case of the LH tapping. The right tapping accuracy is greater than the left tapping of all three subjects in the case of the SVM is performed. Each classification network has a different response to the same inputs. It can give the good accuracy in some cases of the right tapping, but it can show the poor in others. Because of this selective problem, one should more carefully choose the classification network type to obtain the higher performance.
In recent years, researchers have proposed different algorithms in exploring body activities related to human brain. The poor spatial resolution of NIRS made it difficult to distinguish two closely located cortical areas from each other. A combination of the multi-channel NIRS and a Center of Gravity (CoG) approach widely accepted in the field of Transcranial Magnetic Stimulation (TMS) could be used to discriminate between closely located cortical areas activated during hand and foot movements of the subject (Koenraadt et al., 2012). Hemodynamic responses were measured using a NIRS system of 8 channels. For estimating adapt of Oxyhemoglobin (OHb) and Deoxyhemoglobin (HHb), a CoG algorithm was determined for each condition using the mean hemodynamic responses and the coordinates of the channels. Therefore, significant hemodynamic responses were found for hand and foot movements. This is the interesting methods which can be applied to develop for identifying hand tapping. Based on this information, the proposed algorithms in our research can be improved with some thresholds to find out which channel gives the valuable information. The order of the filter we had chosen here belongs to the pulses time of moving hand up and down. Thus, the method to quantitatively estimate the start and end timing of the hand movement using the neural network was proposed.
In (Muroga et al., 2006), the authors measured regional cerebral blood flow during tapping movement of the RH using NIRS technique. The following tendencies of total-Hb were observed, in which Hb increased within 10 s from the movement start time, decreased within 10 s from the movement end time. The direction of arm force from hemoglobin concentration changes measured by using NIRS technique was discriminated. A Self-Organizing Map (SOM) was used to classify the force direction information obtained from the NIRS signals. The results confirmed that the direction of the arm force is discriminable through the NIRS signal. In the simple classification approach, the average discrimination rate gave the performance of 87.5% for two directions. The experimental results showed that the NIRS signal from arm force contained information related to the force directions (Sato et al., 2009). This research is from our research about the proposed methods and experimental tasks. While the SOM method, possibly called the ANN, evaluated the arm force directions with the 87.5% large performance is a little bit higher than that compared with the SVM and the ANN for the LH and RH tapping tasks. This is one of methods which we need to apply for our experimental tasks to determine the best one.
In using NIRS technology, the local distribution of fingers (right thumb and ring finger, respectively) was distinguished to hemodynamic responses on Somatosensory cortex by the electrical stimuli intensity (SI), whose results showed in good accordance with the anatomical arrangement of hand area (Xu et al., 2007). Another application is that in NIRS-based brain activation mapping, a novel real-time NIRS signal analysis framework based on the General Linear Model (GLM) and the Kalman estimator was proposed (Ge et al., 2010). A set of simulated data was processed using the proposed framework. The results obtained suggested that the method can effectively locate brain activation areas in real-time, thereby demonstrating its potential for real-time NIRS-based brain imaging applications. Both these researches, the authors were proposed the same experiment with finger movements using different methods. It is clear that the NIRS technology is not only used to distinguish hand tapping tasks in this paper, but also applied for finger movement tasks.
From the previous researches, we have realized that the proposed algorithm can be accompanied with other algorithms for finding more accuracy. The NIRS technology has been used to obtain Oxy-Hb signals in recent years. However, these Oxy-Hb signals always exist noises and artifacts due to subject movements, noisy environments, human biological changes and others. Proposing a good method for estimating Oxy-Hb concentration changes related to brain activities is always necessary to researchers. In particular, the poor resolution in spatial domain needs to be overcome and also applications in real time are an interesting field for research developments using the NIRS technology.
Conclusion
In this paper, original brain signals of hand tapping tasks were filtered by the Savitzky–Golay filter to produce the smooth signals. Moreover, the smoothed signals of the LH and RH tapping tasks corresponding to Oxy-Hb concentration changes in human brain were analyzed using the PR algorithm. Based on different coefficients of the curves obtained from the PR algorithm, the ANN and SVM algorithms were employed to validate Oxy-Hb data for the recognition of the hand tapping times. Experimental results with hand tapping times showed that one could distinguish the LH or RH tapping tasks of the subject. In addition, from the obtained results of two methods, it was realized that the SVM algorithm is faster than the ANN one in term of time recognition. Based on the proposed algorithms, future work is that experiments will be developed on many subjects to investigate more accuracy and to apply for treatment and rehabilitation.
Conflict of interest statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
We would like to thank Vietnam National Foundation for Science and Technology Development-NAFOSTED for supporting research grant No. 106.99-2010.11. Furthermore research was partly supported by a research fund from the Vietnam National University in Ho Chi Minh City. Finally, an honorable mention goes to our volunteers, families and friends for their supports on us in completing this project.
ReferencesAhmedK. S. (2011). Wheelchair movement control VIA human eye blinks. AiharaT.TakedaY.TakedaK.YasudaW.SatoT.OtakaY.. (2012). Cortical current source estimation from electroencephalography in combination with near-infrared spectroscopy as a hierarchical prior. AydoreS.MıhcakM. K.CiftciK.AkınA. (2010). On temporal connectivity of PFC via Gauss–Markov modeling of fNIRS signals. BlankertzB.DornhegeG.KrauledatM.MüllerK.-R.KunzmannV.LoschF.. (2006). The Berlin brain-computer interface:EEG-based communication without subject training. BozkurtA.RosenA.RosenH.OnaralB. (2005). A portable near infrared spectroscopy system for bedside monitoring of newborn brain. BrinkT. T.UrtonK.HeldD.KirilinaE.HofmannM. J.Klann-DeliusG.. (2011). The role of orbitofrontal cortex in processing empathy stories in 4- to 8-year-old children. CaiX.ChenX.YinS.ChenC. (2007). Wavelet image fusion based on the high order polynomial regression, in ChamasemaniF. F.SinghY. P. (2011). Multi-class Support Vector Machine (SVM) classifiers–an application in hypothyroid detection and classification, in ChunguangL.TaoL.YoshioI.KyokoS. (2010). Evaluation of a bimanual-coordinated upper-limbs training system based on the near infrared spectroscopic signals on brain, in CritchleyH. D. (2009). Psychophysiology of neural, cognitive and affective integration: fMRI and autonomic indicants. CuiX.AlwanA. (2005). Noise robust speech recognition using feature compensation based on polynomial regression of utterance SNR. DakeW.ChengweiM. (2006). The Support Vector Machine (SVM) based near-infrared spectrum recognition of leaves infected by the leafminers, in GeS. S.HuX.-S.HongK.-S. (2010). NIRS based brain activation mapping: a real-time application, in GentiliR. J.AyazH.HadaviC.ShewokisP. A.Contreras-VidalJ. L. (2010). Hemodynamic correlates of visuomotor motor adaptation by functional near infrared spectroscopy, in GorryP. A. (1990). General least-squares smoothing and differentiation by the convolution (Savitzky-Golay) method. GrossmannT. (2013). The role of medial prefrontal cortex in early social cognition. HuX.-S.HongK.-S.GeS. S.JeongM.-Y. (2010). Kalman estimator- and general linear model-based on-line brain activation mapping by near-infrared spectroscopy. InceN.GoksuF.TewfikA.AricaS. (2009). Adapting subject specific motor imagery EEG patterns in space-time-frequency for a brain computer interface. KauhanenL.NykoppT.LehtonenJ.JylänkiP.HeikkonenJ.RantanenP.. (2006). EEG and MEG brain-computer interface for tetraplegic patients. KhanM. A. H.MuntasiT.LayekM. A. (2011). Multiple polynomial regression for modeling a MOSFET in saturation to validate the early voltage, in KhoaT. Q. D.NakagawaM. (2008). Functional near infrared spectroscope for cognition brain tasks by wavelets analysis and neural networks. KoenraadtK. L. M.DuysensJ.SmeenkM.KeijsersN. L. W. (2012). Multi-channel NIRS of the primary motor cortex to discriminate hand from foot activity. Lloyd-FoxS.BlasiA.ElwellceC. E. (2010). Illuminating the developing brain: the past, present and future of functional near infrared spectroscopy. MacnabA.ShadganB.AfsharK.StothersL. (2011). Near-infrared spectroscopy of the bladder: new parameters for evaluating voiding dysfunction. MolaviB.DumontG. (2010). Wavelet based motion artifact removal for functional near infrared spectroscopy, in MontgomeryD. C.RungerG. C. (2003). Applied statistics and probability for engineers, in MurogaT.TsuboneT.WadaY. (2006). Estimation algorithm of tapping movement by NIRS, in NgoC. Q.NguyenT. H.VoT. V. (2012). Linear regression algorithm for hand tapping recognition using functional near infrared spectroscopy, in OrfanidisS. J. (2010). Signal processing applications, in PedriniL.MagnoniF.SensiL.PisanoE.BallestrazziM. S.CirelliM. R.. (2012). Is near-infrared spectroscopy a reliablemethod to evaluate clamping ischemia during carotid surgery?ReherP.ChrcanovicB. R.SpringettR.HarrisM. (2011). Near infrared spectroscopy: a diagnostic tool to evaluate effects of radiotherapy in the mandible?Rifai ChaiG. P. H. (2012). Mental task classifications using prefrontal cortex electroencephalograph signals, in SatoT.ItoM.SutoT.KameyamaM.SudaM.Yamagi-ShY.. (2007). Time courses of brain activation and their implications for function: a multichannel near-infrared spectroscopy study during finger tapping. SatoT.TsuboneT.WadaY. (2009). Estimation of the direction of arm force by using NIRS signals, in SavitzkyA. (1964). Smoothing and differentiation of data by simplified least squares procedures. SchecklmannM.EhlisA.PlichtaM.FallgatterA. (2010). Influence of muscle activity on brain oxygenation during verbal fluency assessed with functional near-infrared spec-troscopy. Shawe-TaylorN. C. A. J. (2000). Support vector machines, in Shimadzu Corporation. (2010). ShimokawaT.MisawaK. S. T.MiyagawaK. (2009). Predictability of investment behavior from brain information measured by functional near-infrared spectroscopy: a bayesian neural network model. SinglaR.ChambayilB.KhoslaA.SantoshJ. (2011). Comparison of SVM and ANN for classification of eye events in EEG. SitaramR.ZhangH.GuanC.ThulasidasM.HoshiY.IshikawaA.. (2007). Temporal classification of multichannel near-infrared spectroscopy signals of motor imagery for developing a brain–computer interface. SteinierJ.TermoniaY.DeltourJ. (1972). Comments on smoothing and differentiation of data by simplified least square procedure. SubasiA.KiymikM. K.AlkanA.KoklukayaE. (2005). Neural network classification of EEG signals by using AR with MLE preprocessing for epileptic seizure detection. TanakaK.MatsunagaK.WangH. O. (2005). Electroencephalogram-based controlof an electric wheelchair. TaniguchiH.HiyamizuM.TominagaT.MoriokaS. (2012). Brain activity stimulated by prismadaptation tasks utilized for the treatment of unilateral spatial neglect: a study with fNIRS. TsunashimaH.YanagisawaK. (2009). Measurement of brain function of car driver using Functional Near-Infrared Spectroscopy (fNIRS). WeiskopfN.ScharnowskiF.VeitR.GoebelR.BirbaumerN.MathiakK. (2004). Self-regulation of local brain activity using real-time functional magnetic resonance imaging (fMRI). WolpawJ.BirbaumerN.McFarlandD.PfurtschellereG.VaughanT. (2002). Brain-computer interfaces for communication and control. XuM.TakataH.ShengGe.HayamiT.YamasakiT.TobimatsuS.. (2007). NIRS measurement of hemodynamic evoked responses in the primary somatosensory cortex by finger stimulation, in ZhangZ. G.ChanS. C.ZhangX.LamE. Y.WuE. X.HuY. (2009). High-resolution reconstruction of human brain MRI image based on local polynomial regression, in