1.

Introduction

Functional near-infrared spectroscopy (fNIRS) is a new noninvasive neuroimaging technique that has attracted increasing attention in recent years. It shines near-infrared light into the outer cerebral cortex and the light absorption rate varies with the change of hemoglobin intensity.¹ Thus, a certain degree of inference of the neural and psychological activities that lead to a change in hemoglobin intensity can be deduced based on the measurement of the change in light intensity. FNIRS has an acceptable spatial and temporal resolution for current neuroimaging studies. Furthermore, its good portability and small bondage of body movements make it possible to carry out experiments in near-natural circumstances that enhance the ecological validity of studies, especially those exploring affective and social issues. The ability to use “friendly” experimental environments also provides more convenience for special populations when performing neuroimaging studies, such as infants,^2,3 children,^4,5 pregnant women,⁶ patients with psychiatric disorders,^7–9 and animals.^10,11

Hoshi and Chen⁴ used fNIRS to observe regional cerebral blood flow changes in children with different emotions. This study introduced fNIRS to the field of affective neuroscience for the first time. Since then, there have been considerable numbers of emotion studies conducted using the fNIRS technique. For example, some studies^12–15 observed cerebral responses to emotional stimuli, such as affective pictures, emotional faces, and affective sounds. Some studies^16–18 examined the association between mood states and cognition. In studies on emotion regulation, activation of the prefrontal cortex was reported.^5,19 Studies of interpersonal affective interactions^2,20,21 have also benefited from the near-natural data acquisition environment of fNIRS. This technique was also used in studies on emotion decoding^22–24 and enhanced the future practicability of these kinds of studies.

Taken together, the literature indicates that fNIRS may be a promising tool in the field of affective and social neuroscience. However, although there has been increasing interest and application of fNIRS in the affective and soical scope, there has not been a report of fNIRS’s reliability in emotion studies. Because it is a new technique, its reliability is the foundation of further extensive applications. When researchers evaluate the advantages and disadvantages of a measurement tool, its reliability is always an essential aspect. Some of the emotion studies using fNIRS have reported similar conclusions in some research topics. However, it is hard to evaluate the fNIRS reliability strictly and directly based on these similarities, because these studies may have great differences with regards to the participants, stimulation materials, experimental paradigms, data acquisition and analysis methods, and experimental equipment. There have been several reports of fNIRS reliability when used to measure psychological functions, such as visual,²⁵ motor,^26,27 and executive^28,29 functions, and the network features of the human brain.^30,31 In these reports, both the fNIRS device and the data acquisition and analysis methods had acceptable test–retest reliability. But these results cannot be directly extrapolated to the field of emotion research because emotion studies have significant differences to the previously mentioned studies in terms of the research questions, experimental tasks, and brain areas studied. In addition, emotion phenomena are very subtle and complex, which has the salient properties of a black box. In this situation, it is necessary to conduct a pointed study to investigate the reliability of fNIRS in emotion research. FNIRS, similar to functional magnetic resonance imaging (fMRI), is a blood oxygenation level dependent (BOLD) neuroimaging technique. Currently, there have been a few reliability reports of fMRI used in measurements of emotion-related BOLD signals. For example, some studies^32–34 indicated relatively stable amygdala BOLD responses to emotional faces at the group level but lower reliability at the subjective-specific level. There was also a report of robust and reliable emotion-dependent amygdala habituation by fMRI measurement.³⁵ Good reliability was also reported for the node degree of limbic nodes by graph theory analyses in an emotional face-matching task.³⁶ In these studies, the brain area typically concerned is the amygdala. However, current fNIRS can only measure the cortical surface and cannot detect the subcortical areas. In addition, there is a difference in the imaging principle between fNIRS and fMRI. Thus, it is hard to deduce fNIRS reliability in emotion studies based on previous fMRI reports.

Currently, fNIRS has been used in the research field of affective and social neuroscience. Its reproducibility is one of the prerequisites for further applications, such as the search for emotional neuromarkers and the quantitative monitoring of mental status. Thus, the present study aimed to test the reliability of fNIRS when applied to an emotion task. In emotional research, the neural response to emotional stimuli is an important issue in the field of affective neuroscience. Currently, this issue has also been investigated by a large number of emotion studies using fNIRS, among which the function of the prefrontal cortex is most frequently concerned.^{8,12,14,15,37–40} Therefore, the current study focused on evaluating the reliability of prefrontal activation in response to affective pictures,⁴¹ which are widely used to elicit emotions in laboratories. Previous studies have paid more attention to negative (i.e., unpleasant) emotions; therefore, the current study followed this tradition and evaluated the reliability of the prefrontal responses elicited by unpleasant pictures with a neutral condition as a contrast. The fNIRS reproducibility of emotion-related prefrontal activation was evaluated by a test–retest reliability frequently used in reliability assessments. The time interval between test and retest reported in previous studies varied from a few minutes,³⁰ to a few weeks^31,33 and a few months,^26,42 and even over 1 year.⁴³ The current study set three weeks, a temporal length in the middle of previously used timepoints as the test–retest interval. For a comprehensive assessment, the current study evaluated the fNIRS reproducibility from global to localized scales at both the group and individual levels.

2.

Materials and Methods

2.3.

Procedure

In the first experimental session, on attending the laboratory, the experimenter briefly introduced participants to the experiment. Then participants were asked to complete the PANAS before beginning the main part of the study, during which fNIRS scanning was conducted. Then the experimenter explained the main task to participants in detail. The meaning of valence and arousal, and how to use SAM to make self-reports of subjective feelings were explained. The formal experiment began when participants understood the task and familiarized themselves with the reporting operation through practice. The scanning was carried out in a quiet room with dim light. Participants were seated comfortably in front of a 19-in. monitor. As shown in Fig. 1, their task began with a 30-s resting-state fNIRS scan with the mind relaxed and eyes open. Then a “+” fixation point was presented for 10 s to remind participants to pay attention to the upcoming pictures. Afterwards, eight blocks (four unpleasant blocks and four neutral blocks) of pictures were presented. The unpleasant block and the neutral block were presented alternatively and the same category of block would not appear in two consecutive times. Each block consisted of four pictures in the same valence category. Each picture was presented for 6 s thus, each block lasted 24 s. Participants were required to view the pictures carefully and try to immerse themselves in the scenes presented. After each block, they were asked to report their emotional experiences during the picture-viewing segment using the nine-point SAM scales. The SAM picture was being presented until participants had clicked the mouse to make a choice. After the self-reporting, there was a 24-s resting period with a “+” presented on the screen. Participants could take a short break after finishing the eight blocks (the first half of an experimental session). In the second half of the experiment, as for the first half, participants underwent a 30-s resting period and a 10-s fixation period, then viewed the affective pictures and reported their feelings for a further eight blocks. The pictures were presented, and self-report ratings were collected, via E-Prime 2.0. Each affective picture would be presented only once. Participants underwent the same experimental procedures in the second session after a three-week interval.

Fig. 1

A schematic diagram of the experimental processes. One experimental session consisted of the same two processes. One half of an experimental session consisted of eight blocks (four unpleasant and four neutral blocks) of picture viewing and self-reporting. Here, a neutral block was used as an example.

2.4.

fNIRS Measurements

The fNIRS measurements were conducted with a 57-channel fNIRS system (LABNIRS, Shimadzu Co., Japan) with 18 emitters and 18 detectors ($\text{interoptode distance}=30\mathrm{mm}$) arrayed as illustrated in Fig. 2(b). A holder cap was used to place the optodes upon the frontal cortex, based on the international 10–20 system. The optode between channels 47 and 48 was placed at position FPz, and channels 32 and 42 were placed near positions T3 and T4, respectively. To maintain the locations of optodes and channels consistently through the two experimental sessions, the holder positioning for both sessions was completed by the same well-trained experimenter. To estimate the cortical locations for the corresponding channels, the scalp positions of each optode and each channel were recorded using a 3-D magnetic digitizer (PATRIOT, Polhemus Inc.) on one of the participants. Then a probabilistic registration process⁴⁸ was performed to estimate the cortical sites using NIRS_SPM 4.0.⁴⁹

Fig. 2

(a) The activation maps at the group level. (b) Scheme of the fNIRS optodes and channels used in the current study. The center optode in the lowest row was placed at FPz of the International 10–20 system.

The absorption of near-infrared light at three wavelengths (780, 805, and 830 nm) was measured with a sampling rate of 17.54 Hz. Changes in the concentrations of oxygenated hemoglobin (HbO) and deoxygenated hemoglobin (HbR) were computed using the modified Beer–Lambert law. The total hemoglobin (HbT) signal was the sum of the HbO and HbR signals.

3.

Results

3.3.

Test–Retest Reliability

3.3.1.

Mapwise

Figure 3 shows the correlations of the group-level activation maps, with Pearson’s $r$ values annotated. Each data point in the scatterplots indicates a pair of $t$ values for the test and the retest occasions at the same channel. The mapwise intersession reliabilities at the group level were good to excellent ($r=0.65$ to 0.88) for all the three signals except that the HbO reliability (Inter-1) for the comparison between Sess-1-H1 and Sess-2-H1 had a poor grade ($r=0.11$). The mapwise intrasession reliability varied in the test and the retest sessions. In the test session (Intra-1), the HbT signal had an excellent ($r=0.77$) intrasession reliability, but the reliability for HbO and HbR was poor (up to 0.28). The intrasession reliability in the retest session (Intra-2) was better than that in the test session, with an excellent grade (up to 0.85) for HbO and HbT, and fair grade ($r=0.46$) for HbR.

Fig. 3

The scatter plots of map-wise reliability at the group level for HbO, HbR, and HbT signals. Each dot in the scatterplots represents the magnitude of group-level frontal activation (i.e., $t$ values) at the same channel. Inter-1 is the intersession reliability for the first halves of two sessions; inter-2 is the intersession reliability for the second halves of two sessions; intra-1 is the intrasession reliability for the first session; intra-2 is the intrasession reliability for the second session.

The mapwise reliability at the individual level is presented in Table 2. The averaged $r$ values indicated a poor reproducibility of activation maps derived from the three types of hemoglobin signals, regardless of the intersession or intrasession assessment. The reliability for HbT seemed slightly higher than that for HbO and HbR. As shown in Table 2, there were great interpersonal differences in the mapwise reliability assessment.

Table 2

Mapwise r values and clusterwise Roverlap values with cluster sizes of the top 40% of the entire channels for each participant. The averaged Pearson’s r values and Roverlap values across participants and the standard deviations (in the parentheses) are also shown.

Participant	Inter-1		Inter-2		Intra-1		Intra-2
	r	Roverlap (40%)	r	Roverlap (40%)	r	Roverlap (40%)	r	Roverlap (40%)
HbO
1	$-0.02$	0.39	0.07	0.39	0.10	0.39	0.55	0.61
2	0.33	0.65	0.78	0.74	$-0.61$	0.17	$-0.17$	0.26
3	$-0.01$	0.26	-0.17	0.30	0.66	0.74	0.38	0.65
4	$-0.14$	0.39	0.03	0.39	$-0.03$	0.30	0.39	0.52
5	$-0.10$	0.30	0.12	0.39	$-0.08$	0.35	0.55	0.57
6	0.27	0.43	0.44	0.57	0.64	0.70	0.52	0.61
7	0.70	0.78	0.36	0.48	0.77	0.74	0.51	0.70
8	0.27	0.52	0.14	0.43	0.15	0.48	$-0.11$	0.26
9	0.36	0.61	0.88	0.87	0.67	0.65	0.74	0.74
10	0.55	0.48	-0.07	0.30	$-0.17$	0.35	$-0.21$	0.35
11	0.68	0.70	0.67	0.74	0.51	0.61	0.85	0.78
12	0.77	0.78	$-0.09$	0.39	0.77	0.87	$-0.15$	0.30
13	$-0.39$	0.22	0.58	0.65	$-0.58$	0.26	0.48	0.65
14	0.79	0.78	0.17	0.43	0.52	0.65	0.45	0.57
15	$-0.17$	0.30	0.69	0.70	0.00	0.39	0.77	0.78
16	0.15	0.35	0.65	0.74	$-0.01$	0.35	0.84	0.91
17	$-0.11$	0.39	0.72	0.70	0.36	0.57	0.62	0.65
18	$-0.26$	0.35	-0.02	0.43	0.12	0.39	0.35	0.65
19	$-0.21$	0.30	0.15	0.43	0.23	0.48	$-0.03$	0.43
20	0.38	0.57	$-0.24$	0.22	0.49	0.61	0.04	0.39
21	$-0.13$	0.35	0.21	0.48	$-0.09$	0.39	$-0.06$	0.43
22	$-0.62$	0.13	0.67	0.83	$-0.36$	0.22	0.82	0.83
23	$-0.49$	0.13	0.50	0.70	$-0.56$	0.09	0.68	0.74
24	0.08	0.61	$-0.13$	0.22	0.19	0.57	0.56	0.78
25	$-0.07$	0.43	$-0.51$	0.26	$-0.19$	0.30	$-0.57$	0.26
26	0.12	0.57	0.09	0.48	$-0.02$	0.48	0.28	0.52
Mean (Std)	0.10 (0.39)	0.45 (0.19)	0.26 (0.37)	0.51 (0.19)	0.13 (0.41)	0.47 (0.19)	0.35 (0.39)	0.57 (0.19)
HbR
1	$-0.32$	0.30	$-0.47$	0.13	0.42	0.61	0.56	0.61
2	0.25	0.48	0.66	0.65	$-0.57$	0.17	$-0.38$	0.30
3	$-0.14$	0.39	$-0.23$	0.26	0.29	0.52	0.27	0.48
4	0.01	0.43	0.48	0.65	0.13	0.43	$-0.51$	0.22
5	0.18	0.48	0.12	0.52	0.41	0.52	0.64	0.74
6	0.29	0.61	$-0.04$	0.30	0.00	0.43	0.46	0.70
7	$-0.22$	0.26	0.02	0.52	0.25	0.48	0.83	0.87
8	$-0.15$	0.39	0.21	0.48	0.43	0.65	$-0.28$	0.17
9	0.11	0.43	0.20	0.61	0.47	0.61	0.09	0.43
10	0.10	0.57	0.19	0.43	$-0.04$	0.43	0.48	0.61
11	0.68	0.74	0.44	0.48	0.82	0.61	0.80	0.65
12	0.48	0.57	0.28	0.57	0.38	0.65	0.10	0.43
13	0.11	0.43	0.07	0.43	$-0.08$	0.30	0.25	0.57
14	0.23	0.52	0.23	0.57	$-0.12$	0.35	0.13	0.48
15	0.49	0.61	0.38	0.52	0.20	0.52	0.65	0.61
16	0.40	0.57	0.55	0.70	0.46	0.65	0.51	0.57
17	0.06	0.43	0.47	0.61	0.19	0.48	0.53	0.65
18	$-0.13$	0.35	0.13	0.48	$-0.22$	0.35	0.67	0.61
19	0.31	0.48	$-0.53$	0.26	0.12	0.52	0.54	0.43
20	0.05	0.35	0.47	0.61	0.36	0.65	0.26	0.43
21	$-0.08$	0.39	$-0.01$	0.30	0.34	0.52	0.30	0.57
22	$-0.21$	0.22	0.70	0.78	$-0.37$	0.22	$-0.05$	0.48
23	0.29	0.48	0.43	0.57	0.66	0.70	0.47	0.57
24	0.62	0.61	$-0.24$	0.22	0.68	0.65	0.05	0.39
25	$-0.47$	0.09	0.21	0.57	0.04	0.43	$-0.18$	0.30
26	$-0.12$	0.43	$-0.20$	0.43	0.21	0.43	0.08	0.35
Mean (Std)	0.11 (0.29)	0.45 (0.14)	0.17 (0.32)	0.49 (0.16)	0.21 (0.32)	0.50 (0.14)	0.28 (0.36)	0.51 (0.16)
HbT
1	0.40	0.65	0.39	0.70	0.33	0.57	0.50	0.57
2	0.18	0.52	0.43	0.57	$-0.22$	0.35	0.07	0.48
3	$-0.20$	0.22	$-0.04$	0.35	0.60	0.70	0.49	0.65
4	$-0.37$	0.22	0.02	0.35	0.14	0.43	0.47	0.57
5	$-0.13$	0.30	0.25	0.57	0.04	0.30	0.64	0.78
6	0.50	0.74	0.39	0.52	0.33	0.48	0.60	0.70
7	0.58	0.65	$-0.09$	0.39	0.83	0.87	0.33	0.52
8	0.39	0.65	0.08	0.43	0.11	0.48	$-0.02$	0.35
9	0.21	0.57	0.80	0.78	0.72	0.65	0.61	0.74
10	0.36	0.48	$-0.19$	0.30	$-0.06$	0.43	0.17	0.43
11	0.73	0.70	0.68	0.78	0.77	0.70	0.92	0.83
12	0.61	0.70	$-0.15$	0.35	0.67	0.74	$-0.23$	0.17
13	0.10	0.39	0.23	0.48	$-0.13$	0.26	0.33	0.52
14	0.70	0.70	0.04	0.43	0.10	0.39	0.06	0.48
15	0.26	0.43	0.61	0.70	0.35	0.57	0.78	0.70
16	$-0.05$	0.26	0.65	0.70	0.05	0.48	0.85	0.91
17	0.15	0.43	0.76	0.65	0.58	0.61	0.63	0.70
18	0.09	0.39	0.01	0.52	$-0.15$	0.35	0.61	0.70
19	0.11	0.26	$-0.04$	0.35	0.23	0.48	$-0.07$	0.30
20	0.30	0.48	0.23	0.43	0.55	0.57	0.07	0.39
21	$-0.13$	0.35	0.09	0.52	0.04	0.39	0.14	0.48
22	$-0.15$	0.22	0.00	0.35	0.28	0.43	0.81	0.87
23	0.09	0.52	0.70	0.78	0.20	0.52	0.82	0.83
24	0.66	0.83	0.60	0.70	0.66	0.65	0.65	0.83
25	$-0.02$	0.26	$-0.59$	0.13	$-0.16$	0.26	$-0.49$	0.22
26	0.02	0.48	0.47	0.74	0.06	0.48	0.28	0.52
Mean (Std)	0.21 (0.30)	0.48 (0.18)	0.24 (0.35)	0.52 (0.18)	0.27 (0.32)	0.51 (0.15)	0.39 (0.37)	0.59 (0.20)

3.3.2.

Clusterwise (${\mathsf{R}}_{\mathsf{\text{overlap}}}$)

The clusterwise reliability based on $R_{\text{overlap}}$ values is summarized in Table 3 (group level) and Table 4 (individual level). At the group level, the clusterwise reliability was generally acceptable (up to 0.91, 75% of $R_{\text{overlap}}$ values were higher than 0.4), especially for the HbT signal. The reliability had a trend of increase as the cluster size became larger.

Table 3

Clusterwise reliability indicated by Roverlap values at the group level. The Roverlap values with cluster sizes from the top 10% to top 40% are shown.

		Inter-1	Inter-2	Intra-1	Intra-2
HbO	Top 10%	0.00	0.33	0.17	0.83
	Top 20%	0.00	0.50	0.42	0.67
	Top 30%	0.33	0.56	0.50	0.83
	Top 40%	0.48	0.70	0.57	0.87
HbR	Top 10%	0.50	0.50	0.17	0.50
	Top 20%	0.58	0.83	0.25	0.58
	Top 30%	0.67	0.67	0.22	0.39
	Top 40%	0.78	0.65	0.30	0.43
HbT	Top 10%	0.17	0.17	0.50	0.50
	Top 20%	0.50	0.42	0.42	0.75
	Top 30%	0.61	0.72	0.67	0.83
	Top 40%	0.70	0.70	0.70	0.91

Table 4

Clusterwise reliability at the individual level. The averaged Roverlap values across participants and the standard deviations (in the parentheses) are shown. The cluster sizes were from the top 10% to top 40%.

		Inter-1	Inter-2	Intra-1	Intra-2
HbO	Top 10%	0.19 (0.20)	0.21 (0.16)	0.23(0.22)	0.18 (0.16)
	Top 20%	0.31 (0.22)	0.34 (0.18)	0.32 (0.19)	0.38 (0.22)
	Top 30%	0.40 (0.22)	0.42 (0.20)	0.40 (0.21)	0.50 (0.20)
	Top 40%	0.45 (0.19)	0.51 (0.19)	0.47 (0.19)	0.57 (0.19)
HbR	Top 10%	0.13 (0.19)	0.25 (0.19)	0.25 (0.22)	0.29 (0.25)
	Top 20%	0.23 (0.18)	0.32 (0.17)	0.31 (0.15)	0.36 (0.21)
	Top 30%	0.36 (0.16)	0.42 (0.18)	0.43 (0.15)	0.45 (0.17)
	Top 40%	0.45 (0.14)	0.49 (0.16)	0.50 (0.14)	0.51 (0.16)
HbT	Top 10%	0.20 (0.23)	0.19 (0.17)	0.23 (0.23)	0.22 (0.17)
	Top 20%	0.33 (0.21)	0.33 (0.17)	0.34 (0.19)	0.41 (0.23)
	Top 30%	0.41 (0.19)	0.44 (0.18)	0.41 (0.17)	0.52 (0.24)
	Top 40%	0.48 (0.18)	0.52 (0.18)	0.51 (0.15)	0.59 (0.20)

Similar to the situation in the mapwise assessment, the individual-level clusterwise reliability was markedly reduced compared with the group level. It varied from “poor” to “fair” as indicated by the averaged $R_{\text{overlap}}$ values. Only 50% of averaged $R_{\text{overlap}}$ values reached the fair grade. The individual-level reliability also increased as the cluster size was enlarged. Differences across the three hemoglobin signals, or across the intersession and intrasession, were not obvious. Clusterwise reliability with cluster sizes of the top 40% for each participant is shown in Table 2, where large individual differences are present.

3.3.3.

${\mathsf{ICC}}_{\mathsf{\text{cluster}}}$ and ${\mathsf{ICC}}_{\mathsf{\text{channel}}}$

The clusterwise reliability evaluated by ${\mathrm{ICC}}_{\text{cluster}}$, with cluster sizes from the top 10% to the top 40%, is listed in Table 5. Table 6 presents the mean and the standard deviation of the channelwise ICCs (${\mathrm{ICC}}_{\text{channel}}$, based on $t$ values of each channel), which were calculated across all the 57 channels in the optode holder and across the activated channels with different sizes (top 10% to top 40% of channels with highest $t$ values). The percentage of channels with $\mathrm{ICC}\ge 0.4$ (fair reliability) among the total interested channels is shown in Table 6.

Table 5

Clusterwise reliability indicated by intraclass correlation coefficient. The ICCcluster values with cluster sizes from the top 10% to top 40% are shown.

		ICCsingle				ICCaverage
		Inter-1	Inter-2	Intra-1	Intra-2	Inter-1	Inter-2	Intra-1	Intra-2
HbO	Top 10%	0.00	0.20	0.34	0.18	0.00	0.34	0.51	0.30
	Top 20%	0.00	0.15	0.26	0.24	0.00	0.26	0.41	0.38
	Top 30%	0.00	0.12	0.29	0.23	0.00	0.22	0.45	0.38
	Top 40%	0.00	0.10	0.29	0.23	0.00	0.18	0.45	0.37
HbR	Top 10%	0.21	0.17	0.25	0.19	0.34	0.29	0.40	0.32
	Top 20%	0.08	0.17	0.40	0.31	0.14	0.28	0.57	0.47
	Top 30%	0.15	0.07	0.40	0.30	0.26	0.14	0.57	0.46
	Top 40%	0.12	0.18	0.35	0.31	0.21	0.31	0.51	0.48
HbT	Top 10%	0.23	0.40	0.53	0.51	0.37	0.57	0.69	0.68
	Top 20%	0.23	0.30	0.45	0.47	0.37	0.46	0.62	0.63
	Top 30%	0.25	0.27	0.45	0.45	0.39	0.42	0.62	0.62
	Top 40%	0.26	0.25	0.44	0.43	0.42	0.39	0.61	0.60

Table 6

Channelwise reliability indicated by intraclass correlation coefficient. The ICCchannel values were calculated across all the 57 channels (whole) and across the activated channels with different sizes (top 10% to top 40%). The means (the values outside the parentheses) and the standard deviations (the first value in parentheses) across the interested channels are shown. The percentages of channels with ICC≥0.4 among the total interested channels (the second value in parentheses) are also reported.

		Inter-1	Inter-2	Intra-1	Intra-2
${\mathrm{ICC}}_{\text{single}}$
HbO	Whole	0.07 (0.11, 2%)	0.22 (0.15, 11%)	0.22 (0.14, 12%)	0.37 (0.18, 44%)
	Top 10%	0.10 (0.12, 0%)	0.22 (0.17, 17%)	0.37 (0.20, 50%)	0.20 (0.06, 8%)
	Top 20%	0.06 (0.09, 0%)	0.19 (0.14, 8%)	0.30 (0.16, 25%)	0.22 (0.09, 12%)
	Top 30%	0.09 (0.11, 0%)	0.17 (0.13, 6%)	0.27 (0.17, 22%)	0.23 (0.10, 13%)
	Top 40%	0.08 (0.12, 0%)	0.16 (0.14, 8%)	0.25 (0.16, 17%)	0.24 (0.13, 9%)
HbR	Whole	0.13 (0.14, 3%)	0.20 (0.14, 10%)	0.30 (0.17, 25%)	0.29 (0.15, 25%)
	Top 10%	0.19 (0.23, 17%)	0.26 (0.14, 17%)	0.27 (0.22, 17%)	0.29 (0.20, 50%)
	Top 20%	0.14 (0.19, 8%)	0.21 (0.13, 8%)	0.33 (0.19, 25%)	0.33 (0.19, 50%)
	Top 30%	0.17 (0.18, 11%)	0.21 (0.11, 11%)	0.33 (0.17, 28%)	0.32 (0.17, 44%)
	Top 40%	0.16 (0.16, 9%)	0.21 (0.12, 9%)	0.31 (0.19, 26%)	0.33 (0.16, 43%)
HbT	Whole	0.19 (0.15, 8%)	0.29 (0.16, 24%)	0.32 (0.15, 25%)	0.46 (0.15, 68%)
	Top 10%	0.28 (0.17, 17%)	0.42 (0.09, 50%)	0.50 (0.15, 67%)	0.49 (0.09, 83%)
	Top 20%	0.30 (0.14, 17%)	0.32 (0.16, 33%)	0.44 (0.18, 50%)	0.46 (0.09, 67%)
	Top 30%	0.24 (0.16, 11%)	0.29 (0.17, 28%)	0.40 (0.17, 39%)	0.43 (0.10, 56%)
	Top 40%	0.25 (0.18, 17%)	0.27 (0.16, 22%)	0.38 (0.17, 39%)	0.41 (0.12, 52%)
${\mathrm{ICC}}_{\text{average}}$
HbO	Whole	0.11 (0.17, 10%)	0.34 (0.20, 43%)	0.34(0.19, 39%)	0.52(0.20, 67%)
	Top 10%	0.17 (0.19, 17%)	0.34 (0.22, 33%)	0.51(0.24, 83%)	0.33(0.08, 17%)
	Top 20%	0.10 (0.15, 8%)	0.30 (0.18, 22%)	0.44(0.20, 67%)	0.35(0.12, 33%)
	Top 30%	0.14 (0.18, 17%)	0.27 (0.18, 22%)	0.40(0.22, 61%)	0.36(0.14, 39%)
	Top 40%	0.13 (0.18, 17%)	0.25 (0.20, 22%)	0.37(0.21, 52%)	0.37(0.16, 39%)
HbR	Whole	0.20 (0.21, 19%)	0.31 (0.19, 29%)	0.44(0.20, 61%)	0.43(0.19, 56%)
	Top 10%	0.27 (0.30, 33%)	0.40 (0.16, 33%)	0.39(0.25, 50%)	0.42(0.26, 50%)
	Top 20%	0.20 (0.26, 25%)	0.33 (0.16, 25%)	0.47(0.21, 75%)	0.47(0.23, 58%)
	Top 30%	0.25 (0.25, 28%)	0.34 (0.14, 28%)	0.48(0.19, 72%)	0.46(0.22, 61%)
	Top 40%	0.24 (0.23, 22%)	0.33 (0.16, 30%)	0.44(0.22, 65%)	0.47(0.20, 65%)
HbT	Whole	0.30 (0.20, 30%)	0.43 (0.20, 52%)	0.47(0.18, 68%)	0.62(0.15, 95%)
	Top 10%	0.41 (0.23, 67%)	0.59 (0.08, 100%)	0.66(0.13, 100%)	0.65(0.09, 100%)
	Top 20%	0.44 (0.18, 67%)	0.46 (0.20, 67%)	0.59(0.18, 83%)	0.63(0.08, 100%)
	Top 30%	0.37 (0.24, 57%)	0.42 (0.21, 56%)	0.55(0.17, 78%)	0.59(0.10, 100%)
	Top 40%	0.36 (0.24, 57%)	0.40 (0.21, 52%)	0.53(0.18, 74%)	0.57(0.13, 96%)

${\mathrm{ICC}}_{\text{cluster}}$ and ${\mathrm{ICC}}_{\text{channel}}$ by average measures (ICC (C, k)) were higher than by a single measure (ICC (C, 1)). The clusterwise ICCs were slightly higher than the channelwise ICCs. At the two spatial scales, the intrasession reliability was better than the intersession reliability for all the three types of hemoglobin signals. Generally speaking, ${\mathrm{ICC}}_{\text{cluster}}$ and ${\mathrm{ICC}}_{\text{channel}}$ derived from HbT signals were higher than those from HbO and HbR signals. Sixty-six percent of HbT ${\mathrm{ICC}}_{\text{cluster}}$ values were higher than 0.4, while the proportion was only 13% for HbO, and 28% for HbR. Similarly, 63% of HbT ${\mathrm{ICC}}_{\text{channel}}$ values were higher than 0.4, while the proportion was 10% for HbO, and 25% for HbR.

During the calculation of HbO-derived ICCs, several negative ICC values were yielded, which is theoretically impossible.⁵⁴ The reason for negative ICCs remains unclear.⁵⁵ Therefore, the negative ICCs were set to zero (i.e., least reliable) in the current study, as suggested in previous studies.^31,56

4.

Discussion

The current study evaluated the fNIRS test–retest reliability of prefrontal activation patterns elicited by unpleasant pictures. The self-reported affective status showed no significant change across the test and retest sessions, and across the first and the second halves. Both the intersession and intrasession reliabilities of emotion-related prefrontal activation were generally acceptable, especially at the group level. The highest mapwise reliability at the group level (0.88) appeared in the intersession comparison between second half of each session for HbO signal. And in the clusterwise assessment, the highest group-level reliability (0.91) was the $R_{\text{overlap}}$ (with a top 40% cluster size) of the intrasession comparison within the retest session for HbT signal. The reliability varied with the assessment scale used. Better stabilities were observed on bigger spatial scales. For different hemoglobin signals, the fNIRS reproducibility also showed some differences. The HbT signal had the best stability among the three types of hemoglobin signal tested.

It may be supposed that the intersession reliability would be worse than the intrasession reliability because the stabilities of equipment and participants’ mental status are more likely to show changes over relatively longer time intervals. Furthermore, optode displacement errors between the test and retest sessions may also cause a decrease in reliability. Indeed, there was a report³¹ that the fNIRS resting-state functional connectivity in the sensorimotor areas was more stable for intrasession assessment relative to intersession assessment. However, in the current study, the intersession versus intrasession contrasts were not fully consistent across the assessment levels, spatial scales, signal types, or assessment indices. Generally, the intersession and intrasession reliabilities in this study did not show obvious differences when compared. Therefore, emotion-related fNIRS patterns might maintain a similar stability over short and long intervals as set in the current study. However, reliability may be influenced by a variety of factors, some of which are hard to measure and control. For example, in the current case, the first and the second half of the same session used different pictures to elicit emotions. Although the self-reports indicated that the two halves of pictures did not have significant differences in the valence and the arousal dimensions, the stimuli differences due to other unknown or uncontrolled aspects could still potentially decrease the intrasession reliability. Thus, the current study did not find a supposed lower reliability in the intersession assessment relative to the intrasession situation. The influence of temporal interval on fNIRS stability and its underlying factors have rarely been addressed in previous studies. More empirical evidence is still needed to understand this issue better.

The results of the current study showed that the spatial activation patterns were generally more stable at the group level than at the individual level, which is consistent with previous reports^{25,28,31,57,58} of fNIRS reliability on other psychological functions. Group analysis, to a certain extent, reduced the influence of optode displacement error and other occasional factors on the test–retest reliability. Therefore, in terms of reliability, group-level analysis is more worthy of attention in future studies.

In line with previous findings,^28,31,58 the spatial scale obviously influenced the reproducibility of the fNIRS results. In the group analyses, the reliabilities were largely located in the “fair” to “excellent” range for the mapwise scale and the clusterwise scales, while for the channelwise scale, the reliabilities were from “poor” to “fair.” Therefore, cluster and larger spatial scales can guarantee an acceptable reliability, whereas this is not the case for the channelwise scale. Zhang et al.³¹ suggested that the reasons for the low reliability of the channel scale may be the optode placement variability between different sessions, individual differences in skull and brain anatomy, and the low spatial resolution of fNIRS. Analyses using relatively larger spatial scales, such as the map scale and the cluster scale, can help to decrease the optode displacement error and to cancel out individual differences. They suggested that a cluster of fNIRS channels should be treated as a minimal analytical unit when interpreting and comparing the results of fNIRS-based resting-state functional connectivity. The current study also revealed that, for the sake of reliability, the cluster can be used as a minimal spatial unit when studying the fNIRS-based cortical responses to emotional stimuli.

In the current study, the HbT signal was most stable among the three types of hemoglobin signal. At the group level, HbT usually had “fair” to “excellent” reliabilities. It is noteworthy that, although the fNIRS-based neurocognitive studies aimed to measure the functional hemodynamic responses at the outer cerebral cortex, the fNIRS results are inevitably contaminated by hemodynamic fluctuations occurring in the structures (such as the scalp and the pial) above the cortex. These physiological noises will affect the stability of fNIRS results to a certain extent. The tolerance of different types of hemoglobin signal to shallow layer interference is different. Kirilina et al.⁵⁹ measured the frontal lobe activation during a continuous performance task and reported that task-evoked scalp vessel artifacts were mainly observed in the HbO concentration changes. Gagnon et al.⁶⁰ estimated the cortical contribution to the fNIRS signals in a finger-tapping task. Their results suggested that, relative to HbO and HbR, the HbT signal change was far less sensitive to pial vein influence. Perhaps partly because HbT has a better antinoise ability, we found that it had the best stability in the current study. However, regarding the reliability of different kinds of near-infrared signals, no specific rule has been reported in the previous literature. For example, in a visual task²⁵ and a verbal fluency task,²⁸ the researchers did not find constantly higher reliability for one particular fNIRS signal. Although Zhang et al.³¹ reported that the rest–retest reliability was generally in the order of HbO > HbR (HbT reliability was between that of HbO and HbR) in their resting-state functional connectivity analyses, another study³⁰ on the reliability of graph metrics in functional brain networks found the reliability was concordant across HbO, HbR, and HbT. To answer why the reliabilities of different near-infrared signals are different, and what kind of signal is the most reliable, further research is required.

Although the current study found that the fNIRS-based prefrontal responses to emotional stimuli had acceptable stability at the group level and on relatively big spatial scales, it is worth noting that the prefrontal activation magnitudes for the unpleasant-neutral contrast did not reach a statistical significance. In previous studies similar to the current one, some also reported that the unpleasant condition did not differ from the neutral condition significantly for the HbO signal.^12,15,61 And some studies reported a decrease of HbO concentration when the unpleasant condition was compared to the resting condition in some participants.^24,62 In these studies and the current one, participants were induced into the desired emotion evidenced by the behavioral data. It is unclear why we, and others, failed to find an expected difference for the unpleasant-neutral or unpleasant-resting contrast in the brain data. However, we offer some considerations in the following text. First, we suggest that the phenomenon of a decrease in HbO may be understood from the neural inhibition hypothesis.^63–65 Although neuroimaging studies have traditionally focused more on task-induced neural activation, there are also studies using fNIRS^66,67 and fMRI⁶⁸ that have found a task-related signal decrease at the prefrontal area. The neural inhibition may be related to attentional modulation processes,⁶⁶ which work to meet the task required attentional demand by suppressing the activity in some brain areas and increasing the neural activity in some other areas. Thus, the deactivation in the current study may be a neural response for maintaining attention to unpleasant pictures. Interestingly, there is a recent study⁶⁹ that investigated negative BOLD responses to intermittent photic stimulation by integrating information from fNIRS and fMRI. The results from both techniques confirmed a negative hemodynamic response, which was related to a decrease in HbO concomitant to a lower increase in HbR response, corresponding to a decrease in HbT. This pattern is very similar to the current findings. Although we cannot make further inferences because of the differences of experimental tasks and observation sites, in the future, the multimodal approach used in this report seems a useful method for a comprehensive exploration of the negative hemodynamic responses. The second possible explanation to the deactivation may be related to the default network⁷⁰ that consists of the medial prefrontal cortex, the rostral anterior cingulate, the lateral frontal cortex, and some regions of the parietal and the temporal cortex. The instruction to the participants required them to immerse in the presented scenes but did not provide specific instructions of how to elicit the affective states. Some participants might perform the task by calling on some self-related events or memories. These self-referential processes possibly account for the result of deactivation. Third, it may be possible that the superficial noise mentioned above blurred the prefrontal response that was meant to be observed. From the fNIRS emotion literature, it can be seen that few studies have taken effective measures in data acquisition and processing steps to reduce superficial noise that can bias the results. We think that in future studies, the short channel method,⁷¹ i.e., adding additional short source–detector distance optodes to measure the superficial hemodynamic fluctuations and to regress out the superficial noise in subsequent analyses, may help to get purer cerebral responses.

Several other aspects should be taken into consideration in addition to the issues discussed above. First, we used the fNIRS reliability of prefrontal responses to unpleasant pictures as a representative of the fNIRS reliability in emotion studies. However, emotion research involves very complex and diverse situations. For example, there may be differences in the neural basis of pleasant and unpleasant emotions.⁷² There may also be differences between emotion induction and emotion regulation.^73,74 FNIRS stability under specific research questions needs more experimental support. In addition, some fNIRS-based emotion studies have selected some of the sensory areas, such as the visual cortex⁷⁵ and the auditory cortex,⁷⁶ as regions of interest. The reliability and validity in these cases also need to be tested. Second, in the current study, the reliability of the intersession was similar to that of the intrasession, but a previous study³¹ showed that the reliability of the intersession was higher. Because these two studies used different experimental tasks and analytical methods, it may not be appropriate to compare them. At present, there is little research regarding the relationship between time interval and reliability. In future studies, to set multiple time spans (e.g., a few hours, weeks, or months) in one study will help to explore this issue. Meanwhile, we should fully consider the consistency of the stimulation materials and the optode placement between the test and the retest sessions, and the psychological and the neural habituation along with the time going, which will help us understand further the role of time interval in reliability. Third, this study showed that the HbT signal, relative to HbO and HbR, was more stable, but we still cannot suggest that future emotion studies with fNIRS should rely more on the HbT signal. Given that existing research has not reported consistent results on this issue, it is unclear whether the stability of the three kinds of signal follows a certain rule. The stability may be influenced by various factors, such as experimental tasks, methods of data processing and analysis, NIRS systems or wavelengths used. To draw a more instructive conclusion, we must control the factors that may influence the results, and collect more evidence from similar studies. In addition, it was reported that the spatial specificity of the three types of fNIRS signal might be different.⁷⁷ HbR concentration changes were more spatially localized and the spatial specificity of HbT was relatively poor. From the existing findings, to simultaneously analyze all kinds of signals seems to be a more useful approach. Fourth, the validity of fNIRS in emotion studies is still unclear. At present, the fNIRS technique can only directly measure the superficial layer of the cortex; thus, many fNIRS emotion studies have used the prefrontal (especially lateral) cortex as their region of interest, which is a brain area that has a close relationship with emotional activity.^14,78–81 However, according to Pessoa’s review,⁸² the prefrontal cortex (excluding the orbital and ventromedial areas and the anterior cingulate cortex) is not regarded as one of the core emotional regions when compared with the amygdala and hypothalamus and other brain regions. This may also explain why previous fMRI reliability studies on emotion tend to choose the amygdala as the observation area, rather than the prefrontal lobe. As for the question whether the prefrontal lobe is a suitable area for the observation of emotional activity, or more precisely, what kind of emotional activity can be measured at the prefrontal lobe, relevant theoretical arguments and empirical work will have to continue for some time. In terms of the evaluation of the validity of fNIRS-based prefrontal responses to emotional stimuli, the simultaneous recording of fNIRS and fMRI may help to elucidate this issue.

In conclusion, under the current experimental design and data processing methods, the test–retest reliability of fNIRS-based prefrontal responses to affective pictures was acceptable at the group level for the mapwise and the clusterwise scales, which suggests that the fNIRS technique may be a reliable tool for emotion studies. Meanwhile, caution should be exercised when using the channelwise and individual-level fNIRS results because they may not be sufficiently stable according to the current study.