2.1.

Subjects

We studied six randomly selected healthy male volunteers (age, 25–30 years). All subjects were free of ocular diseases, had visual acuities between 0.8 and 1.0, and their color vision was tested with Ishihara and Stilling-Velhagen plates. The tests revealed no color vision abnormalities. None of the volunteers had a history of neurological or psychological disorders, and none was taking any medication or drugs. The subjects had excellent target fixation and demonstrated normal saccadic behavior. After receiving an explanation of the details and the purpose of the study, all the subjects gave their informed consent. All experiments were conducted in accordance with the Declaration of Helsinki.

2.2.

Stimulator

A suitable projector unit was selected for the fundus-controlled projection stimulator, which should be capable of producing SST chromatic stimuli. On the basis of the resolution, pixel pitch, aperture ratio, electro-optical response times, channel cross talk, and line flickering (Table 1), we chose a three-panel micro LCoS (3LCoS) projector (ASI6001-01, Aurora Systems, San Jose California). This projector was connected to the examination and stimulation unit (THERA PRAX system, neuroConn GmbH, Ilmenau, Germany) via a digital visual interface (DVI). The unit generates stimulations and records EEG data.

Fig. 8

PA versus central stimulation within the MA. (a) Corresponding stimulation areas containing (top) MA and (bottom) PA. (b) S-cone responses and (c) LM-cone responses. All responses are from electrode Oz.

Table 1

Parameters of the three-chip micro LCoS projector (ASI6001-01).

The electro-optical response time depends on the gray-scale value, respectively, the color value of the image. For SST stimulation, this produces a finite time difference between the start and end points of the presented color combination, which can cause contrast losses and distortions of the VEPs. The stimulator LCoS electronics must have a sufficiently large temporal bandwidth to achieve the desired maximum stimulus contrast. To analyze the contrast losses, we measured the temporal modulation transfer function (tMTF) using a PIN photodiode (BPX 65, Siemens, Munich, Germany). This photodiode was placed in the focal plane behind an artificial eye lens model (1177-617, Carl Zeiss Meditec, Jena, Germany) to simulate the target area of stimuli for a human retina. We detected the signal amplitude of the diode by increasing the flicker frequency in steps of the corresponding frame time. At each frequency, the average of 100 amplitude values was calculated. The computed standard deviation was <0.6% of the mean. We measured the tMTF for black−white flickering and for the color combinations for S-cone stimulation and LM-cone stimulation.

An additional problem is the frame delay generated by the LCoS electronic interface. It takes a certain time to convert the video signal into the LCoS driver signal. To correct this effect in the EEG, we used the same photodiode and analyzed its signal and a synchronization trigger from the stimulation unit. The trigger coincided with the onset of stimulus generation, while the photodiode signal coincided with stimulus imaging by the projector.

Signal analysis was performed using a digital oscilloscope (TDS 3054, Tektronix, Beaverton, Oregon). We also verified the LCoS refresh rate of 60 Hz with the same instrumentation by applying a 30-Hz flicker sequence generated by the stimulation system.

We used a fundus camera (VISUCAM lite, Carl Zeiss Meditec, Jena, Germany) to overcome the problems of stray light, direct reflections, and ghost reflections in the eye and in the instrument optics. To image chromatic SST stimuli onto the fundus, the 3LCoS projector was inserted in the illumination path of the fundus camera utilizing additional optics for the coupling. This created a new optical path, which is herein referred to as the stimulation path. Real-time fundus imaging and control could be achieved using the existing fundus camera port of the observation path. An intermediate fundus image produced by the ophthalmoscope lens is projected by a second lens onto a digital imager. The imager was connected to a control unit, which allows the observer to react on fundus conditions. To evaluate the optimal image quality, we performed sensitivity tests with three digital camera systems (see Section 2.4).

The dependence of the spatial performance on LCoS resolution and the optics was analyzed using the spatial modulation transfer function (sMTF). This was carried out for the stimulation path using the primary colors of the projector. We generated an incoherent image of a nearly ideal razor blade edge. The sMTF could be obtained by edge differentiation and then taking the Fourier transformation.²¹

Finally, we selected a projector light source. We used a high-power light emitting diode (LED) (OSTAR LE UW E3B, Osram, Munich, Germany) that has a smooth spectrum and emits high energy at lower wavelengths. The output spectra of the primary colors did not overlap due to the optimized dichroic layers in the LCoS core. The LED offers a luminous flux of up to 1120 lm and a white point with the CIE 1931 color coordinates x = 0.31 and y = 0.32. We used a cooling element to stabilize the diode temperature. The entire fundus-controlled LCoS projection stimulator is depicted in Fig. 1.

Fig. 1

(a) Optical schematic of the fundus-controlled LCoS projection stimulator. (b) The LCoS projection stimulator without housing. The stimulation path consists of the following components: light source (LED) with cooling element and optics (L1) that collimates the LED light (LO); three-channel (RGB) LCoS core (3LCoS); coupling optics (CO) with lenses (L2 and L3) to image the generated stimuli into the fundus camera (FC) and stop (S1) to form the entrance pupil of the FC; lenses (L4 and L5) and ophthalmoscope lens (OL) to image the stimuli onto the fundus; pinhole mirror (PM), which enables both simultaneous stimulation and control; and stop (S2), which is conjugate to the fundus, and limits its field of view. The observation path of the camera consists of the following components: ophthalmoscope lens (OL) to produce an intermediate image of the fundus (IF), PM, and lens (L) to project the fundus image onto the camera detector (EMCCD). The control unit (CU) displays the stimulation target area (TA) and the fundus, enabling adjustments to be made based on individual conditions. The stimulation unit (SU) is connected to the LCoS projector and controls the SST stimuli. EEG recording is not shown in this schematic.

2.3.

Energy Adjustment

To compare the measured VEPs to the data from our previous study, we adjusted the maximum light output of the LCoS projector. This working point calibration was performed by considering the full white spectra of the projector and the previous stimulator system (30–in. LCD stimulator). The adjustment is based on a radiometric parameter: the radiant flux Φ_{e_p} in the plane of the pupil. For the LCD, we computed the flux using the following equation:

To compare the VEPs generated from different stimulator systems, the retina irradiances (E _{e_r}) must be equal in all cases. Using the value of E _{e_r} for our LCD stimulator, we computed Φ_{e_p} as a set point of the LCoS projector. The retina area was obtained from the maximum projector target area of the stimuli and the geometry of Gullstrand's eye model. The computed radiant flux for adjusting the LCoS projector was 0.63 mW. This adjustment was realized by inserting neutral density filters in the illumination path.

2.4.

Digital Fundus Imaging

To make adjustments based on the condition of a patient's fundus, a high-quality image is important for controlling and positioning the SST stimuli on the retina. We performed sensitivity tests with three different camera systems: a complementary metal-oxide-semiconductor (CMOS) camera (Guppy F-036C, Allied Vision Technologies, Stadtroda, Germany), a charge-coupled device (CCD) camera (CF 8/5 MX, Kappa opto-electronics GmbH, Gleichen, Germany), and an electron-multiplying CCD (EMCCD) camera (Luca R 604, Andor Technology PLC, Belfast, United Kingdom). The main parameters that affect the sensitivity are listed in Table 2.

Table 2

Parameters of the tested camera systems.

The sensitivity was analyzed by varying the irradiance as the input signal and detecting the image gray value as the response signal. To illuminate the different imagers homogenously, we used the stimulation path of the stimulator and a barium-sulfate-coated integrating sphere (K-100W, LOT-Oriel GmbH, Darmstadt, Germany) was placed directly in front of the imagers. To enable the results to be compared, the CMOS imager was used in monochromatic mode and the cameras were set to an exposure time of 100 ms. We performed all measurements with a gain of 0 dB, without automatic gain control and without any pixel binning. The testing process was conducted using both the full white spectrum and the green spectrum of the stimulator, and it was performed three times to evaluate reproducibility. To estimate the average gray value, a region of interest (100×100 pixels) was centered in the image. We measured the input irradiance with the radiometer in the same plane as the camera imagers.

Because of the sensitivity results (see Section 3), the EMCCD system was used in the observation path. A universal serial bus interface provided the connection with the control unit.

2.5.

SST Stimulation

All stimuli sequences were imaged with the experimental LCoS projection stimulator. To avoid temporal and spatial distortions generated by applying interpolation algorithms to DVI input signals, we used only the native resolution of 1920×1080 pixels. The maximum target area of the stimuli on a human retina was about 27×15 deg. Image and video enhancement options were disabled, as were sharpness and noise reduction algorithms on the LCoS engine.

To selectively modulate activity in the S- and LM-cone pathways, we presented two different SST flash sequences in a full field. For one volunteer, we performed papilla stimulation to estimate the influence of stray light and geometrical correctness on optical imaging. Each stimulation sequence contained two different color stimuli. The silent substitution condition was realized by alternating these colors. The two colors correspond to two states, the ON and OFF states, which are related to different cone activations. The activation can be simplified using the following instantiations: L1 = L2, M1 = M2, S1 ≠ S2 (for S-cone condition) and L1 ≠ L2, M1 ≠ M2, S1 = S2 (for LM-cone condition). The color values and the exact cone activation were calculated using the Hunt fundamentals for 10-deg and larger viewing conditions [using Eq. 4]. ^{24, 25} Therefore, the input red, green, and blue (RGB) values of the stimulation unit were converted into the standard color values of virtual XYZ space [using Eq. 3].

As a precondition, the spectral distribution function for the RGB channels of the stimulator were measured using a compact array spectroradiometer (CAS 140B, Instruments Systems, Munich, Germany). The time constancy of the emission spectra for all three color channels was checked to ensure that the emission levels did not vary during the SST sequences. Gamma correction was performed by determining the characteristic gamma curve for the LCoS projector and the graphics card (8 bit). The mean quantification error was 0.96% for the red channel, 0.76% for the green channel, and 0.46% for the blue channel (given as percentages of the maximum light output of each channel). The transformation matrices [Eqs. 3, 4] for the applied stimulation system are as follows:

To suppress rod responses caused by scattered stimulation light outside the stimulation area, we studied light-adapted volunteers and employed an ambient room luminance of 95 cd/m². At the position of the volunteers’ pupil, the illuminance value was 132.5 lux.

The design of the color stimulator was aimed for the application of the silent substitution technique. To get clear evidences for a successful implementation the examination of deuter, protan, or tritanopes may be accomplished. The strongest indication can be achieved by investigating tritanopes, but they are too rare. Another way is the execution of an adaptation and bleaching experiment, which is based on the isolation technique from Stiles.²⁷ Therefore, a full-field adaptation box was designed which offers the opportunity to maximally suppress the unwanted cone types relative to the cone type to be isolated. The box has an adjustable RGB lighting system whose wavelength was fixed to selectively suppress the S-cones but spares the LM-cones and therefore simulates a tritanope. Using this adaptation box, we studied two healthy volunteers considering the same inclusion criteria as described in Section 2.1. In both volunteers, the right eye was dilated maximally by application of a mydriatic. After 10 min of S-cone bleaching, a selective stimulation was conducted with simultaneous recording of the EEG signal. We performed a balanced repetition sequence of the S- and LM-cone stimulations. A period of 60 min was implemented between the repetitions. Within the EEG signal, we used a moving average window of 25 trials without any overlap for the time analysis of the cone-regeneration dynamic. For the S-cone responses, there were no significant VEPs in the first window. Very small responses arose 37 s after the bleaching procedure. Clear S-cone VEPs were visible at the time window of 54 s. The LM-cone responses were not affected by the bleaching and were already visible in the first averaging window. The absence of S-cone responses up to nearly 50 s after the bleaching procedure in contrast to the unaffected LM-cone responses was an evidence for the successful selective cone isolation.

2.6.

Experimental Procedure

For all subjects, the right eye was maximally dilated by applying a mydriatic. The eye was kept light adapted before and during the EEG recordings. We performed a fundus survey imaging procedure before stimulation. Therefore, the green spectrum of the projector was used to obtain high-contrast fundus images, enabling optimal fundus survey and stimulus placement. Subsequently, selective S- and LM-cone stimulations were performed. Thus, two recordings were collected for each volunteer. During the stimulation sequences, the respective spectra were used for fundus illumination. The identical timings were used for the S- and LM-cone stimulations. The stimulus interval for the ON state was adjusted to 17 ms, and the interstimulus interval (ISI) for the OFF state was adjusted to 467 ms. To reduce the influence of habituation effects on the EEG, we employed an additional random ISI between 17 and 517 ms. A total of 150 stimuli were presented for all sequences. We applied a balanced sequence for the stimulation order.

2.7.

EEG Recording and Processing

The EEG signal was recorded simultaneously with the stimulation using the examination unit (sample rate 512 Hz). The electrodes were placed above the visual cortex at Oz and at the reference position Fz (10–20 system). An electrode cap and Ag/AgCl ring electrodes (EASYCAP GmbH, Herrsching, Germany) were used. To obtain the exact excitation time point, the hardware trigger generated by the stimulation unit was recorded and used. The time of this trigger was adjusted with respect to the measured frame delay.

Signal processing and analysis of EEG and VEP signals were performed (MATLAB Software, The MathWorks Incorporated, Natick Massachusetts). The raw EEG data were digitally filtered to remove electrode drifts (0.8 Hz high pass) and other signal distortions (30 Hz low pass). To prevent phase shifts, elliptic infinite impulse response filters were applied in the forward and backward directions. Trials with physiological signal distortions (e.g., muscle activity, increased α activity, and eye movements) were detected and excluded from analysis using artifact detectors.²⁸ After individual classification of the 150 recorded trials, selective averaging of the 100 best trials was performed. Therefore, an average signal of all the valid trials was computed and correlated with each single valid trial. The average was obtained by considering the highest correlation coefficient.

We compared the measured VEPs with the grand average (GA) responses for the S- and the LM-cone stimulations recorded in our previous study.²⁰ The GAs were obtained from 102 volunteers who had been medically diagnosed and found to be free of optical diseases or defective color perception. These volunteers had visual acuities between 0.8 and 1.0, intraocular pressures of <21 mm Hg, and normal visual fields. Recording and processing were performed in an identical manner to the methods described in this paper.

3.1.

Stimulator

Figure 2 shows the modulation transfer functions of the stimulator. The plot in Fig. 2a shows the contrast losses caused by the different electro-optical response times for temporal modulation transfer. In Fig. 2b, the dependence of the spatial modulation on the LCoS resolution and the optics is shown as a function of the spatial frequency. The tMTF of all three flicker sequences are nearly perfectly flat to 10 Hz. Small amplitude decay occurs at higher flicker frequencies. The tMTF measured with the color combinations for S-cone stimulation differs markedly from those measured with the color combinations for LM-cone stimulation and the black−white flicker. The S-cone flicker decays to nearly 93% of maximum at a stimulation frequency of 30 Hz, whereas the black−white flicker and LM-cone flicker remain at high amplitudes of ∼ 99%. Frequencies of >30 Hz could not be measured because of the sampling limitations of the DVI interface.

Fig. 2

(a) Temporal modulation transfer functions of the LCoS projection stimulator. Measurements were performed with black–white flicker (triangles), with the color combinations for LM-cone simulation (circles) and S-cone stimulation (asterisks). (b) Spatial modulation transfer functions of the LCoS projection stimulator. The functions of the red (dotted curve), green (dotted–dashed curve), and blue (solid curve) primaries of the projector were measured under best focus conditions for an image of a razor blade edge.

The measured frame delay caused by the LCoS electronic interface was 14 ms. The refresh rate of the projector (60 Hz) was verified. In a 120 min test, no variations in these measurements were observed over time.

The minimum sMTF amplitude for the projector primaries occurs at ∼120 cycles/deg [Fig. 2b]. The blue spectrum exhibits the fastest decay in amplitude, whereas the red and green sMTFs are close together, with higher amplitudes. Optimal performance was achieved for the green primary with a spatial frequency of ∼17 cycles/deg at a relative amplitude of 0.5. At the same amplitude level, red has a spatial frequency of 14 cycles/deg and blue has a spatial frequency of 10.8 cycles/deg.

3.2.

Digital Fundus Imaging

Figures 3a and 3b show the measured camera sensitivity functions for the green and white spectra, respectively. The sensitivity functions reveal differences between the camera imagers and the test spectra. For the EMCCD and the CCD imager, both spectra have linear response functions. The CMOS camera clearly exhibits a nonlinear response. Its green sensitivity function has three different linear regions, with transitions at approximately 1 and 2.5 μW/cm². In contrast, the white sensitivity function has only two linear regions, with a transition at 1.5 μW/cm². Compared to the EMCCD and the CCD system, the absolute sensitivity of the CMOS system is markedly lower. Their relative differences to the CMOS imager are given in Fig. 4. At most response-signal levels, the relative sensitivity differences to the CMOS system are between 500 and 1000%. The EMCCD imager has higher differences in both the green and white spectra. For the green spectrum at a typical response level of 0.5, the CCD and EMCCD imagers are 423 and 565% more sensitive, respectively.

Fig. 4

Relative sensitivity differences of the EMCCD (circles) and CCD (asterisks) imager to the CMOS imager: (a) differences in the green sensitivities and (b) differences in the white sensitivities.

In this study, fundus imaging was performed using the EMCCD camera. Figure 5 shows fundus images and various stimulation patterns acquired with the EMCCD camera.

Fig. 5

Fundus of one volunteer and various stimulation patterns: (a) shows the fundus with MA (center of image). A circular SST flash with a black fixation point is shown in (b). Two different states of a checkerboard sequence are shown in (c) and (d). For selective modulation activity in the S-cone and the LM-cone pathway, we only used the SST flash stimulation.

3.3.

EEG Recording

Figure 6 shows the measured VEPs and the GA responses (top) obtained in our previous study for S-cone simulation [Fig. 6a] and LM-cone [Fig. 6b] stimulation. The GAs from our previous study indicate a typical N1−P1 complex in the curve shapes of the S- and LM-cone responses. The mean peak-to-peak amplitude of the LM-cone response was ∼13 μV, which is higher than that of the S-cone response at 9 μV. The N1 latency of the LM-cone response (90 ms) is markedly smaller than that of the S-cone response (111 ms). The P1 latency was 140 ms for the LM-cone response and 182 ms for the S-cone response.

Fig. 6

Measured VEPs (V1–V6) and GA responses from our previous study²⁰. (a) S-cone responses and (b) LM-cone responses are plotted. N1 and P1 latencies are indicated by the left and right vertical lines, respectively. All responses are from electrode Oz.

All volunteers in the present study showed high LM-cone peak-to-peak amplitudes (mean = 15 μV) with very good agreement in the N1 (mean = 89 ms) and P1 (mean = 143 ms) latencies. In the S-cone responses, smaller peak-to-peak amplitudes (mean = 9 μV) and higher N1 (mean = 113 ms) and P1 (mean = 170 ms) latencies were measured. The S-cone response of volunteer V3 was markedly smaller with a broad first positive wave. For a complete comparison with the GA responses, the slopes of the N1−P1 complex were also computed. Figure 7 shows the positions of four parameters (N1 latency, P1 latency, slope, peak-to-peak amplitude) for both responses of the volunteers in box plots of the GA responses obtained from our previous study. Almost all the N1 and P1 values for both silent substitution conditions are located in the box near the GA median. In the plot of the slope, two values for the S-cone response and three values for the LM-cone response lie within 1.5 times of the lower interquartile range. Apart from two LM-cone responses and one S-cone response, all peak-to-peak amplitudes are located below the box median. Five of the 12 responses lie within the lower interquartile range. One S-cone amplitude is markedly smaller than the others, lying below the bottom of the whiskers.

Fig. 7

Volunteers’ VEP parameters: (a) N1 latency, (b) P1 latency, (c) slope, and (d) peak-to-peak amplitude compared with the distribution of the GA responses from our previous study²⁰. The GA box plots show (left) the S-cone response and (right) the LM-cone response. “+” symbols indicate the parameter positions of the measured VEPs (for better visualization, “+” symbols that overlap are plotted horizontally).

Figure 8 shows the VEPs of papilla (PA) versus those for central stimulation within the macula (MA) for volunteer V5. No response signals were measured after S- and LM-cone stimulations at the PA. In contrast, the central stimulation produces an S-cone VEP of ∼11 μV. For LM-cone modulation, the same position and area produces a VEP of ∼25 μV.