Noncontact imaging photoplethysmography to effectively access pulse rate variability

Yu Sun; Sijung Hu; Vicente Azorin-Peris; Roy Kalawsky; Stephen E. Greenwald

doi:10.1117/1.JBO.18.6.061205

30 October 2012 Noncontact imaging photoplethysmography to effectively access pulse rate variability

Yu Sun, Sijung Hu, Vicente Azorin-Peris, Roy Kalawsky, Stephen E. Greenwald

Author Affiliations +

Journal of Biomedical Optics, Vol. 18, Issue 6, 061205 (October 2012). https://doi.org/10.1117/1.JBO.18.6.061205

Abstract

Noncontact imaging photoplethysmography (PPG) can provide physiological assessment at various anatomical locations with no discomfort to the patient. However, most previous imaging PPG (iPPG) systems have been limited by a low sample frequency, which restricts their use clinically, for instance, in the assessment of pulse rate variability (PRV). In the present study, plethysmographic signals are remotely captured via an iPPG system at a rate of 200 fps. The physiological parameters (i.e., heart and respiration rate and PRV) derived from the iPPG datasets yield statistically comparable results to those acquired using a contact PPG sensor, the gold standard. More importantly, we present evidence that the negative influence of initial low sample frequency could be compensated via interpolation to improve the time domain resolution. We thereby provide further strong support for the low-cost webcam-based iPPG technique and, importantly, open up a new avenue for effective noncontact assessment of multiple physiological parameters, with potential applications in the evaluation of cardiac autonomic activity and remote sensing of vital physiological signs.

1. Introduction

Fluctuations in the interval between heartbeats, attributed to the continuous changes in the sympathetic-parasympathetic balance of the autonomic nervous system (ANS), are a sign of healthy cardiac function and a valuable tool to investigate the neural control of the heart. The quantification of these fluctuations—heart rate variability (HRV)—has proven its effectiveness in various research and clinical studies pertaining to cardiovascular disease, diabetic autonomic dysfunction, hypertension, and psychiatric and psychological disorders.¹^,² HRV has historically been obtained from the electrocardiograph (ECG). However, to obtain reliable recordings, multiple-lead ECG measurements are needed, requiring several electrodes to be carefully attached to the skin by a trained operator. Such a procedure could preclude frequent assessments of HRV for general use. Noncontact HRV measurements present an even greater challenge, and few attempts have been made.³^,⁴

Photoplethysmography (PPG) is an optical technique that noninvasively measures arterial pulsations in the microvascular tissue bed.⁵ The pulsatile component of the PPG signal oscillates with the heart cycle period. The ease of use, low cost, and convenience make PPG an attractive area of study in the biomedical community and has stimulated research into alternative methods for accessing HRV with the aim of providing a simple and reliable evaluation of ANS activity. A number of recent studies have shown that the functional characteristics of pulse rate variability (PRV) are comparable to those of HRV,⁶^–⁸ where the former can be easily acquired from PPG.

The need to apply a PPG sensor directly to the skin constrains its practicability in situations such as skin damage (burn/ulcer/trauma) or when free mobility is required. Several studies have been performed using spring-loaded contact PPG probes to investigate the effect of contact force on the amplitude and timing of the PPG signal.⁹^,¹⁰ It was shown that this force must be carefully controlled to obtain high-quality, clinically useful data from the signals. One potential solution to this kind of problem is to use noncontact imaging PPG (iPPG) to assess physiological variables remotely, i.e., by means of peripheral blood perfusion measurements. The use of an imaging sensor could provide additional functionality with respect to a point sensor, particularly motion compensation and region of interest selection, both of which enable a robust and flexible noncontact PPG system. The past decades have witnessed rapid growth in the literature pertaining to iPPG techniques.¹¹^–²⁰ Among these applications, Jonathan and Leahy have demonstrated the feasibility of a cellular phone-based iPPG technique in heart rate assessment, stimulating its application for personal healthcare, which might influence a broad population of users.¹⁹^,²⁰ Additionally, a growing list of personal physiological monitoring apps that utilize the camera modules in cellular phones have attracted significant public interest, even though they were not intended for clinical or diagnostic use.²¹ However successful iPPG has been in acquiring physiological parameters, including heart rate and respiration rate, few attempts have been made to assess ANS using noncontact PPG. Recently, Poh and colleagues have introduced a remote PRV assessment technique using a low-cost webcam with ambient light as the illumination source, which shows significant potential for advancing personal care and telemedicine.¹³ Nevertheless, a number of key questions remain concerning this pioneering feasibility study, the most important of which is the sample rate of the webcam (15 to 30 fps), which is much lower than the standard sample frequency in HRV analysis¹ (normally taken to be $> 100 Hz$ ). Although it is possible to increase the effective sample rate via interpolation in signal postprocessing, the influence of initial low sample frequency on the PRV analysis is still far from fully understood. A low sampling rate is likely to produce a jitter in the estimation of the pulse-pulse interval (PPI) sequences, which could alter their frequency spectrum considerably.

To further prove the practicability and feasibility of imaging PPG in remote PRV measurement, as well as to investigate the influence of sample rate on the PRV analysis, a high speed CMOS camera-based imaging system for capturing the iPPG signals at a sample frequency up to 200 fps is introduced and described here. The immediate objectives of this study were, first, to assess the practicability and the feasibility of the iPPG system in assessing PRV and, second, to investigate and quantify the influence of different sample rates on PRV analysis.

2. Materials and Methods

2.1.

Subjects

Ten volunteers [ $age = 37.8 \pm 15.3 years$ ( $mean \pm S.D.$ ), range 21 to 65 years, $male / female = 7 / 3$ , $height = 1.74 \pm 0.08 m$ , $weight = 71.3 \pm 10.3 kg$ , $BMI = 23.8 \pm 4.0 kg / m^{2}$ ] enrolled from The Blizard Institute, Barts & The London School of Medicine & Dentistry, participated in this study. None of them had any known cardiovascular disease, and none were diabetic. The investigation conformed to the principles outlined in the Declaration of Helsinki (1989) of the World Medical Association, and it was approved by the local Research Ethics Committee. The nature of the research was explained to the subjects before the recordings, and their informed consent was obtained. All subjects were asked to refrain from consuming caffeine or alcohol, and they were asked not to smoke or undertake strenuous exercise for the 2 h preceding the study.

2.2.

Instrument Setup

The iPPG system is schematically presented in Fig. 1(a). A monochrome CMOS camera (model: EoSens, MC 1363-63, Mikrotron GmbH, Unterschleissheim, Germany) with a spectral range of 500 to 1000 nm was focused on the palm of the participant’s hand using a standard F-mount lens (model: Nikkor 20 mm, $f / 2.8 D$ , Nikon, Japan). Two commercial infrared ( $λ = 880 nm$ , $Δ λ = 20 nm$ , 10 W) light sources (model: ABUS TV 6818, ABUS Security-Centre GmbH, German) were mounted on either side of the camera to provide a uniform illumination of the target area: the palm of the left hand. Images were captured for 4 min at a rate of 200 fps and an exposure time of 4 ms, producing a raw image size of $256 \times 384 pixels$ . The pixels were encoded in 10-bit gray scale, which allowed the camera to detect the weak pulsations of the microvascular tissue bed. During the image recording, a reference contact PPG signal was obtained simultaneously from a reference LED-PPG circuit, as shown in Fig. 1(b). Specifically, the reference PPG signal was initially obtained by means of a standard commercial contact PPG sensor (model: SA30014C, Shanghai Berry Electronic Tech. Co. Ltd., China) placed on the middle finger of the left hand and a DISCO4 data acquisition system (Dialog Devices, UK). This plethysmographic signal was then transferred to a custom-written LabVIEW (National Instruments Co., USA) software platform and served as the power intensity signal to trigger a red LED ( $λ = 650 nm$ , $Δ λ = 15 nm$ , RS Components, UK) through a DAQ (model: USB-6008, National Instruments Co., USA). The red LED probe was attached to the target area with double-sided adhesive tape during image acquisition, providing a clear and reliable reference signal for easy comparison with iPPG signals during postprocessing.

Fig. 1

(a) A schematic showing the experimental setup of the noncontact iPPG system, with the dashed box indicating the reference LED-PPG system, and (b) a picture of the reference LED-PPG system.

2.3.

Experimental Protocol

As mentioned above, the purpose of this study was to assess the practicability and the variability of the iPPG in assessing PRV and to investigate and quantify the influence of different sample rates on PRV analysis. To accomplish these objectives, 10 sets of recordings were taken from 10 healthy subjects enrolled during four successive days. The image sequences were taken with the subject at rest to minimize physiological variations and motion. A soft cushion was placed under the target hand to further minimize the motion. The palm of each subject was exposed to the infrared illumination source, and the distance between the camera lens and the skin was about 400 mm. All measurements were taken in a temperature-controlled (20°C to 22°C) darkroom by a trained operator. The activity of the ANS governing the PRV clearly reveals circadian rhythms.²² Hence, the recordings for this study were obtained within the period of 9:30 to 17:30 ( $median = 12 : 30$ ) for consistency.

2.4.

Image Processing

Motion artifacts were minimized by recording images under resting conditions. Hence, a simple yet efficient spatial averaging approach¹⁵ was adopted and was found to be adequate to attenuate relatively small motion artifacts.¹¹^,¹⁴ Once a set of recordings was acquired, the raw images were divided into discrete subwindows to produce a new set of reduced frames, as shown in Fig. 2(b). The value of each pixel in the reduced frame was set as the average of all the pixel values within each subwindow. Though it compromises the spatial resolution, such a procedure has been shown to significantly improve the signal-to-noise ratio.¹⁵ Subwindow size was set at $8 \times 8 pixels$ , resulting in a reduced frame size of $36 \times 48 pixels$ . PPG signals were then derived from each pixel position across a sequence of frames. A fifth-order Butterworth filter with cutoff frequencies set at [0.05, 4] Hz was employed to further attenuate noise.

Fig. 2

A representative figure showing (a) an original image (frame 11999, $t = 60 s$ ) and (b) the reduced image, each pixel of which was the average of an $8 \times 8 pixel$ subwindow from the original; (c) contact PPG signal with TFR results; and (d) iPPG signals with TFR plot. The inserts in (c) and (d), in which the time scale has been expanded, show the plethysmographic waveform in which RR and HR can be clearly seen. The upper TFR trace is from the cPPG, and the lower one is from iPPG, with a color bar indicating the power intensity. The position at which the iPPG signal was obtained is shown by the black box ( $1 \times 1 pixel$ ) and an arrow in (b), while the region where the cPPG reference signal was obtained is indicated with a larger black box ( $3 \times 3 pixel$ ). The signal is from Subject #1 (male, $age = 28 years$ ).

2.5.

Heart/Respiration Rate Extraction from iPPG

In our previous studies, a time-frequency analysis [time-frequency representation (TFR)] was used to assess the time-varying heart rate (HR).¹² The HR was first obtained through averaging the fundamental HR frequency from the TFR trace calculated from the reference contact PPG signal, and this HR was then treated as a reference. Successive calculations were then performed on the reduced frames, where if the difference between the current HR estimation and the reference value exceeded the threshold of 9 bpm, the algorithm isolated these regions as corrupt and rejected the invalid HR. The ${HR}_{iPPG}$ could then be obtained by averaging HRs within all the valid subwindows. For each set of recordings, a 1-min moving window (12,000 frames) was employed for this HR calculation with 0.5-min overlap (6,000 frames). This yielded 7 HR readings for each subject. A similar approach was employed to obtain the respiration rate ( ${RR}_{iPPG}$ ).

2.6.

Trough Detection

To analyze the PRV, the first step is to detect the pulse-pulse intervals. A revised technique based on wavelet transforms was used for detecting trough positions, where the PPG signal was flat and contained little disturbance, thus facilitating PRV analysis. After bandpass filtering, the wavelet transform was performed on the obtained PPG signals, where each peak and trough pair in the PPG signals corresponds to a positive maximum and negative minimum pair with two zero-crossing points indicating the peak (maximum value) and trough (minimum value) positions. The locations of these zero-crossing points ( $n$ ), where the minimum values were revealed in the original PPG signals, served as the raw trough positions. A semi-automated procedure was then employed to further eliminate the false positive and negative trough detections and to improve the accuracy. A manual validation was initially conducted to verify $n_{1}$ as the first trough position. HR was then calculated within a moving window $W_{t 1, t 2}$ ( $t 2 - t 1 = 10 s$ ), and then 0.9 to 1.1 of ${HR}^{- 1}$ was adopted as the PPI threshold. If $0.9 {HR}^{- 1} < | n_{i 1} - n_{i 2} | < 1.1 {HR}^{- 1}$ , where $t 1 \leq n_{i 1}$ , $n_{i 2} \leq t 2$ , the algorithm considers $n_{i 2}$ a valid trough position. If no valid trough position was found, a search back approach was performed to find the minimum value within the threshold range, and a warning index was saved for postvalidation.

2.7.

Pulse Rate Variability Analysis

A series of straightforward and efficient time domain and frequency domain measurements was employed to extract the PRV information. The mean values of pulse-pulse intervals (MPP) and standard deviation of the pulse-pulse intervals (SDPP) were employed in the time domain analysis. As a discrete event series, PPI is an unevenly sampled time series, hence an interpolation and resample approach was conducted before the frequency domain analysis. In practice, it is recommended that the duration of the recording be about 10 times the wavelength of the lowest-frequency band of the spectral component being investigated.¹ Therefore, only the low frequency (LF: 0.04 to 0.15 Hz), the high frequency (HF: 0.15 to 0.40 Hz), and the LF/HF ratios were calculated in this work. The HF component, which has a peak at respiratory frequency, corresponds to respiratory sinus arrhythmia (RSA) and reflects parasympathetic influence on the heart through efferent vagal activity. The LF component, consisting of fluctuations below 0.15 Hz and usually centered at about 0.1 Hz, is mediated by both cardiac vagal and sympathetic nerves.²³^,²⁴ Hence, the ratio of the LF and HF represents the sympatho-vagal interaction. LF and HF were also expressed in normalized units to account for inter-individual differences, denoted as ${LF}_{n} = LF / [Total power - VLF (very low frequency, 0.003 - 0.04 Hz)]$ and ${HF}_{n} = HF / (Total power - VLF)$ , respectively.¹ Postprocessing and analysis were performed with custom software in Matlab 2008a (MathWorks, Natick, Massachusetts, USA).

2.8.

Influence of Frame Rate on PRV Analysis

A down-sampling approach has been adopted (Fig. 3) to investigate the influence of sample rates on the PRV analysis. Once a set of reduced frames was acquired, three new frame sequences with sample frequencies of 100, 50, and 20 fps were obtained by down-sampling the original reduced frames. These new frame sequences used the iPPG recordings with the same image configuration yet different sample frequencies, which could therefore be treated independently. The PPG signal extraction and preprocessing procedures were repeated. To refine the trough fiducial point, especially for the low-sample-rate iPPG recordings, the PPG signals were interpolated with a spline function¹³ to reach an effective frequency of 200 Hz. These interpolated PPG signals were then ready for the trough detection and PRV analysis as described above.

Fig. 3

A representative figure showing the process and effects of investigating the influence of the frame rate on the PRV analysis. The PPG signal shown is from Subject #1 (male, $age = 28 years$ ).

2.9.

Statistical Analysis

To evaluate the reliability of the iPPG system for remote physiological assessments, Bland-Altman analysis²⁵ was performed to compare iPPG and contact PPG (cPPG). The difference between iPPG and cPPG was plotted against their average, as were the mean and the standard deviation (S.D.) of the differences and 95% limits of agreement ( $\pm 1.96 S.D.$ ). The Pearson’s correlation coefficients and the corresponding $p$ -value were also calculated to assess the physiological variables (HR, RR, and PRV) from both systems. Differences between the PRV measurements at different sample rates were assessed with one-way analysis of variance (ANOVA, $factor = sample rate$ ) to demonstrate the influence of sample frequency on the PRV analysis. The group differences were assessed by post hoc analysis using Fisher’s least significant difference (LSD), where groups represent the different sample frequency. All analyses were performed with $α$ (Type I error) set at 0.05 using the statistical software program SPSS for Windows, version 17.0 (IBM, Armonk, New York).

3. Results

3.1.

Heart/Respiration Rate Measurement

Figure 2 shows an example of the PPG signals and their corresponding TFR traces. The reference cPPG signal was obtained by calculating the mean intensity value within the region of the reference LED. The variations in the signal due to heartbeat and respiration are clearly visible for both PPG signals in the inserts of Fig. 2(c) and 2(d). It can be seen in the TFR traces that the oscillations of RR, HR, and the first harmonic component derived from the iPPG signals were in agreement with those obtained from the commercial pulse oximeter sensor readings. Bland-Altman analysis was used to evaluate the agreement of HR and RR between the imaging method and the contact readings. The results of the Bland Altman analysis are shown in Fig. 4. A close agreement between these two techniques was revealed. Specifically, in no case did the mean difference of the physiological measurements (HR/RR) between the two methods differ significantly from zero. The mean bias of RR is $0.03 breaths / \min$ with 95% limits of agreement $- 1.29$ to $+ 1.35 breaths / \min$ ; while the mean bias for the heart rate is $- 0.04 bpm$ with 95% confidence interval $- 3.00$ to $+ 2.93 bpm$ . Significant correlations between both techniques were also revealed (Pearson’s correlation for RR and HR, $r^{2} > 0.95$ , $p < 0.001$ ).

Fig. 4

Bland-Altman plots showing the average of (a) the respiration rate and (b) the heart rate measured by the imaging PPG and contact PPG, plotted against the difference between them.

3.2.

Pulse Rate Variability Measurement

A typical example of the imaging PPG signals is presented in Fig. 5(a) and 5(b) along with the contact PPG signals recorded with the reference LED-PPG system. The boxes in Fig. 5(a) and 5(b) show data at a magnified time scale (60 to 80 s), where clear plethysmographic waveforms with detected trough positions are shown in both PPG signals [cPPG (black), iPPG (green)]. The obtained pulse-pulse intervals are shown in the right panel of Fig. 5(a) and 5(b). It is evident that the two PPI sequences are in close agreement. The resulting power spectral density maps are shown in Fig. 5(c), where both spectra exhibit comparable functional characteristics and a dominant LF component. Table 1 summarizes the time and frequency domain results of the PRV analysis [ $values are mean \pm standard error of the mean$ (SEM)]. Significant correlations between cPPG and iPPG were found in all estimated results (Pearson’s correlation, $r > 0.85$ , $p < 0.001$ ).

Fig. 5

A representative figure showing (a) contact and (b) imaging PPG signals with detected troughs (red circles) and the corresponding pulse-pulse intervals and (c) the power spectral density map of the PPIs. This demonstration signal is from Subject #5 (male, $age = 30 years$ ).

Table 1

Overall results of the PRV analysis.

	cPPG	iPPG	Correlation
	( $mean \pm SEM$ )	( $mean \pm SEM$ )	$r$	$p$
MPP (ms)	$860.3 \pm 50.9$	$859.2 \pm 50.6$	0.998	$< 0.001$
SDPP (ms)	$73.3 \pm 9.5$	$72.7 \pm 8.7$	0.874	$< 0.001$
${LF}_{n}$	$0.536 \pm 0.057$	$0.532 \pm 0.065$	0.971	$< 0.001$
${HF}_{n}$	$0.431 \pm 0.056$	$0.437 \pm 0.057$	0.978	$< 0.001$
LF/HF	$1.932 \pm 0.562$	$1.967 \pm 0.613$	0.875	$< 0.001$

MPP: mean pulse-pulse interval; SDPP: standard deviation of pulse-pulse interval; LFn: power in low frequency range (0.04 to 0.15 Hz) in normalized units; HFn: power in high frequency range (0.15 to 0.4 Hz) in normalized units; SEM: standard error of the mean.

To further evaluate the performance of the imaging PPG in assessing PRV information, Bland-Altman analysis was employed to assess the agreement between contact and imaging readings of the estimated PRV results. Figure 6 shows that the mean biases for MPP and SDPP are $- 1.00$ and $- 0.49 ms$ , respectively. The corresponding 95% limits of agreement are from $- 9.94$ to 7.94 ms and $- 11.54$ to 10.56 ms, which shows that they do not differ significantly from zero. Similar statistically comparable results are revealed in the frequency analysis. The mean biases between cPPG and iPPG for the measurement of ${LF}_{n}$ , ${HF}_{n}$ , and LF/HF are $- 0.008$ , 0.005, and 0.046. The corresponding 95% confidence intervals are $- 0.093$ to 0.078, $- 0.076$ to 0.086, and $- 0.381$ to 0.473. Again, this shows that none of these biases differs significantly from zero.

Fig. 6

Bland-Altman plots of the comparison between contact and imaging readings of PRV. (a) Mean value and (b) standard deviation of pulse-pulse intervals, (c) low and (d) high frequency components, and (e) ratio of low and high frequency components.

3.3.

Effects of Frame Rate on PRV Analysis

Three additional down-sampled iPPG sequences were obtained as described previously (i.e., 100, 50, and 20 fps). Table 2 summarizes the PRV results for different sample frequencies. Further statistical analysis showed no significant influence of the sample rate on any of the PRV measurements: MPP ( $F = 0.03$ , $p = 0.998$ ), SDPP ( $F = 0.07$ , $p = 0.990$ ), ${LF}_{n}$ ( $F = 0.03$ , $p = 0.998$ ), ${HF}_{n}$ ( $F = 0.03$ , $p = 0.997$ ), or LF/HF ( $F = 0.01$ , $p = 0.999$ ). Treating the PRV measurements calculated from cPPG readings as the reference, significant correlations between cPPG and the three down-sampled iPPG sequences were also uncovered (Pearson’s correlation, $r > 0.81$ , $p < 0.001$ ). We note that MPP increases monotonically with increasing sample frequency ( ${MPP}_{_20 fps} < {MPP}_{_50 fps} < {MPP}_{_100 fps} < {MPP}_{_200 fps}$ ), whereas no such consistent trend is seen with the other related variables. The variations of PRV measurements in the down-sampled sequences indicate a negligible loss of data for reductions in sample frequency down to 20 fps.

Table 2

Effects of sample rate on PRV measurements (values are mean±SEM).

	cPPG	iPPG
	200 fps	200 fps	100 fps	50 fps	20 fps
MPP (ms)	$860.3 \pm 50.9$	$859.2 \pm 50.6$	$853.2 \pm 48.4$	$848.7 \pm 49.0$	$838.4 \pm 48.8$
SDPP (ms)	$73.3 \pm 9.5$	$72.7 \pm 8.7$	$68.9 \pm 9.2$	$67.8 \pm 10.3$	$69.2 \pm 8.5$
${LF}_{n}$	$0.536 \pm 0.057$	$0.532 \pm 0.065$	$0.521 \pm 0.056$	$0.521 \pm 0.051$	$0.515 \pm 0.059$
${HF}_{n}$	$0.431 \pm 0.056$	$0.437 \pm 0.057$	$0.421 \pm 0.056$	$0.412 \pm 0.053$	$0.415 \pm 0.053$
LF/HF	$1.932 \pm 0.562$	$1.967 \pm 0.613$	$1.891 \pm 0.519$	$1.870 \pm 0.508$	$1.898 \pm 0.537$

MPP: mean pulse-pulse interval; SDPP: standard deviation of pulse-pulse interval; LFn: power in low frequency range (0.04 to 0.15 Hz) in normalized units; HFn: power in high frequency range (0.15 to 0.4 Hz) in normalized units; SEM: standard error of the mean.

4. Discussion

This study considers the remote accessing of physiological information (i.e., RR, HR, and PRV) at a fast sample rate (200 fps) by means of iPPG and compares the results to those obtained from a commercial pulse oximeter sensor. The strong correlation and good agreement between these two methods for capturing PPG signals proves the value of an iPPG system in remotely measuring PRV. On the basis of these results, we have demonstrated the feasibility of using an iPPG system to appraise cardiac autonomic activity remotely. The influence of initial sample frequency on the PRV analysis has also been revealed. Statistical results showed nonsignificant difference among various sample frequencies. This finding thereby provides further support for a previous study that reported on a webcam-based system for measuring multiple physiological parameters.¹³

Since Huelsbusch and Blazek¹⁸ introduced a novel experimental charge coupled device (CCD) camera-based iPPG system in 2002, a steadily increasing number of iPPG systems have been described for a variety of noncontact physiological assessments.¹¹^–¹⁸ Although they have been successful in remote physiological measurements of heart/respiration rate,¹¹^,¹²^,¹⁴^,¹⁵ perfusion mapping, and blood oxygen saturation,¹⁵^,¹⁸ the maximum sample rate of these pioneering iPPG studies is $< 50 Hz$ , which is much less than the standard minimum sample rate¹ for reliable heart rate variability measurements (100 Hz). In the iPPG setup described here, the camera was configured to capture the images at a sample frequency of 200 fps, offering relatively accurate time domain resolution and ensuring reliable PRV analysis. As can be seen in Fig. 2, a clear imaging PPG signal has been obtained even from a small region ( $1 \times 1 pixel$ in the reduced frame, corresponding to $8 \times 8$ in the raw image). Furthermore, time-frequency analysis shows close agreement in respiration rate, heart rate, and the first harmonic components of the signals. Further statistical analysis of the measured physiological variables (RR and HR) shows that the iPPG system exhibits functional characteristics comparable to conventional contact PPG sensors, i.e., the maximum difference is about 3 bpm for the HR and $1.3 breaths / \min$ for RR, providing support for the derived PRV assessments. From the clinical point of view, the word “comparable” means that measurements by different instruments/techniques should be sufficiently close so as not to change the actions and decisions regarding the patient’s treatment.¹⁶ For instance, in the Emergency Severity Index triage, an adult patient with a $HR > 100 bpm$ is considered tachycardic.²⁶ In this scenario, a difference of $\sim 3 bpm$ (3% error) obtained by our remote iPPG system would most likely be acceptable, given the variability in this quantity however it is measured.

In contrast to various well-established QRS detection techniques in HRV analysis,²⁷ there are fewer publications pertaining to the characterization of PRV signals. In order to obtain the PPI sequences, a wavelet transform-based semi-automated trough detection algorithm has been introduced in this study. By adopting the short-window moving HR as the threshold, the present method achieved an accuracy of 95% in all subjects before the manual verification approach, which resulted in 100% accuracy. Statistical analysis of the time/frequency domain results calculated from the obtained PPI sequence showed a strong correlation ( $r > 0.85$ , $p < 0.001$ ) and close agreement between the imaging PPG and the standard contact PPG sensor. The results of this study thereby provide further and reliable support for the idea that an iPPG system can successfully obtain clinically useful information about the ANS. The last two decades have witnessed the recognition of a significant relationship between the ANS and cardiovascular mortality, including sudden cardiac death. One recent study of 763 elderly subjects has reported the time- and frequency-domain measures of HRV and the relationship of these measures to all-cause mortality during a four-year follow-up period.²⁸ However, the normal HRV standards for various age and gender subsets are still missing, and larger prospective population studies with longitudinal follow-up are needed.²⁹ Such studies might be significantly facilitated by employing imaging PPG technology.

One recent iPPG study has described a technique for the assessment of multiple physiological parameters using a basic webcam and provided evidence of its feasibility in appraising the autonomic nervous system.¹³ However, the low sample frequency (i.e., 15 fps) of the webcam used might influence the PRV analysis. Through down-sampling the iPPG recordings, we simulated three additional iPPG sequences with lower sample rates and calculated the PRV. An interpolation approach was then adopted before the PRV analysis to refine the troughs in the down-sampled iPPG signals and improve the timing estimation of pulse-to-pulse intervals. Statistical results presented no significant difference among the various sample rates, which was in keeping with the independent relationship between the variations of PRV measurements and sample frequency. These findings agree well with recent work by Poh, McDuff, and Picard¹³ and emphasize the necessity of interpolation in PRV assessments, especially when a low sample frequency is used, as in webcam-based iPPG.

One main source of error arises from the motion artifact that could occur because the sensor has no contact with the skin, consequently leading to corrupted signals and presenting problems for the subsequent signal processing. The primary aim of this study was to test the feasibility of the iPPG technique in assessing the ANS. The images were recorded from a palm that was supported with a soft cushion. We found that a simple spatial averaging approach was sufficient to minimize the minor motion artifacts observed under these experimental conditions. Nevertheless, the use of sophisticated and more effective motion artifact compensation methodologies is encouraged, and more generalized imaging registration¹²^,³⁰^,³¹ and blind source separation¹²^–¹⁴ methods should be used in the future. Originating from the pulse wave and governed by the cardiac cycle, PRV has been shown to be a good surrogate for HRV. A further study is under consideration to assess the performance and to appraise the accuracy of the iPPG-based PRV using the gold standard (ECG-based HRV) with more subjects. In addition, the recording time for this study was relatively short, and future work needs to extend the monitoring time to enable long-term, continuous, and comprehensive [e.g., VLF component (0.003 to 0.04 Hz)] measurements.

5. Conclusion

This study provides an important insight into the effective and comprehensive interpretation of remote physiological assessment. Imaging PPG technology was employed to investigate the feasibility of noncontact assessment of PRV, an index of ANS activity. By comparing the iPPG results with those from a standard contact sensor, we have demonstrated that the 200-fps iPPG system presented here provided measurements of HR, RR, and PRV that were closely comparable to those acquired from contact PPG. In addition, we found that the negative influence of a low initial sample rate could be compensated by interpolating the signal to improve the time domain resolution and PRV measurements. The results thereby offer further support for the applicability of the iPPG technique and constitute a further step toward effective, remote, and multiple physiological assessments, providing a promising alternative to conventional contact PPG techniques, with clear applications in triage, low cost home-based monitoring systems, and telemedicine.

Acknowledgments

This report presents independent research commissioned by the National Institute for Health Research (NIHR) under the Invention for Innovation (i4i) program (Grant No. II-FS-0109-11005). The views expressed here are those of the author(s) and not necessarily those of the NHS, the NIHR, or the Department of Health. The authors are grateful to Loughborough University and Shanghai Jiao Tong University for their support.

References

1.

A. J. Cammet al., “Heart rate variability: standards of measurement, physiological interpretation and clinical use,” Circulation, 93 (5), 1043 –1065 (1996). http://dx.doi.org/10.1161/01.CIR.93.5.1043 CIRCAZ 0009-7322 Google Scholar

2.

U. Rajendra Acharyaet al., “Heart rate variability: a review,” Med. Biol. Eng. Comput., 44 (12), 1031 –1051 (2006). http://dx.doi.org/10.1007/s11517-006-0119-0 MBECDY 0140-0118 Google Scholar

3.

G. Luet al., “Contact-free measurement of heart rate variability via a microwave sensor,” Sensors (Basel), 9 (12), 9572 –9581 (2009). http://dx.doi.org/10.3390/s91209572 1424-8220 Google Scholar

4.

S. Suzukiet al., “A novel autonomic activation measurement method for stress monitoring: non-contact measurement of heart rate variability using a compact microwave radar,” Med. Biol. Eng. Comput., 46 (7), 709 –714 (2008). http://dx.doi.org/10.1007/s11517-007-0298-3 MBECDY 0140-0118 Google Scholar

5.

J. Allen, “Photoplethysmography and its application in clinical physiological measurement,” Physiol. Meas., 28 (3), R1 –R39 (2007). http://dx.doi.org/10.1088/0967-3334/28/3/R01 PMEAE3 0967-3334 Google Scholar

6.

E. Gilet al., “Photoplethysmography pulse rate variability as a surrogate measurement of heart rate variability during non-stationary conditions,” Physiol. Meas., 31 (9), 1271 –1290 (2010). http://dx.doi.org/10.1088/0967-3334/31/9/015 PMEAE3 0967-3334 Google Scholar

7.

P. Shiet al., “Analysis of pulse rate variability derived from photoplethysmography with the combination of lagged Poincare plots and spectral characteristics,” Med. Eng. Phys., 31 (7), 866 –871 (2009). http://dx.doi.org/10.1016/j.medengphy.2009.05.001 MEPHEO 1350-4533 Google Scholar

8.

M. Nitzanet al., “The variability of the photoplethysmographic signal—a potential method for the evaluation of the autonomic nervous system,” Physiol. Meas., 19 (1), 93 –102 (1998). http://dx.doi.org/10.1088/0967-3334/19/1/008 PMEAE3 0967-3334 Google Scholar

9.

X. F. TengY. T. Zhang, “The effect of applied sensor contact force on pulse transit time,” Physiol. Meas., 27 (8), 675 –684 (2006). http://dx.doi.org/10.1088/0967-3334/27/8/002 PMEAE3 0967-3334 Google Scholar

10.

X. F. TengY. T. Zhang, “The effect of contacting force on photoplethysmographic signals,” Physiol. Meas., 25 (5), 1323 –1335 (2004). http://dx.doi.org/10.1088/0967-3334/25/5/020 PMEAE3 0967-3334 Google Scholar

11.

Y. Sunet al., “Use of ambient light in remote photoplethysmographic systems: comparison between a high-performance camera and a low-cost webcam,” J. Biomed. Opt., 17 (3), 037005 (2012). http://dx.doi.org/10.1117/1.JBO.17.3.037005 JBOPFO 1083-3668 Google Scholar

12.

Y. Sunet al., “Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise,” J. Biomed. Opt., 16 (7), 077010 (2011). http://dx.doi.org/10.1117/1.3602852 JBOPFO 1083-3668 Google Scholar

13.

M. Z. PohD. J. McDuffR. W. Picard, “Advancements in noncontact, multiparameter physiological measurements using a webcam,” IEEE Trans. Biomed. Eng., 58 (1), 7 –11 (2011). http://dx.doi.org/10.1109/TBME.2010.2086456 IEBEAX 0018-9294 Google Scholar

14.

M. Z. PohD. J. McDuffR. W. Picard, “Non-contact, automated cardiac pulse measurements using video imaging and blind source separation,” Opt. Express, 18 (10), 10762 –10774 (2010). http://dx.doi.org/10.1364/OE.18.010762 OPEXFF 1094-4087 Google Scholar

15.

W. VerkruysseL. O. SvaasandJ. S. Nelson, “Remote plethysmographic imaging using ambient light,” Opt. Express, 16 (26), 21434 –21445 (2008). http://dx.doi.org/10.1364/OE.16.021434 OPEXFF 1094-4087 Google Scholar

16.

K. HumphreysT. WardC. Markham, “Noncontact simultaneous dual wavelength photoplethysmography: a further step toward noncontact pulse oximetry,” Rev. Sci. Instrum., 78 (4), 044304 (2007). http://dx.doi.org/10.1063/1.2724789 RSINAK 0034-6748 Google Scholar

17.

F. P. WieringaF. MastikA. F. van der Steen, “Contactless multiple wavelength photoplethysmographic imaging: a first step toward “SpO2 camera” technology,” Ann. Biomed. Eng., 33 (8), 1034 –1041 (2005). http://dx.doi.org/10.1007/s10439-005-5763-2 ABMECF 0090-6964 Google Scholar

18.

M. HuelsbuschV. Blazek, “Contactless mapping of rhythmical phenomena in tissue perfusion using PPGI,” Proc. SPIE, 4683 110 –117 (2002). http://dx.doi.org/10.1117/12.463573 PSISDG 0277-786X Google Scholar

19.

E. JonathanM. J. Leahy, “Cellular phone-based photoplethysmographic imaging,” J. Biophotonics, 4 (5), 293 –296 (2011). http://dx.doi.org/10.1002/jbio.v4.5 1864-063X Google Scholar

20.

E. JonathanM. Leahy, “Investigating a smartphone imaging unit for photoplethysmography,” Physiol. Meas., 31 (11), N79 –83 (2010). http://dx.doi.org/10.1088/0967-3334/31/11/N01 PMEAE3 0967-3334 Google Scholar

21.

“Instant Heart Rate app passes 10M users,” (2011) http://mobihealthnews.com/13600/instant-heart-rate-app-passes-10m-users/ ,” ( Oct. 2012). Google Scholar

22.

R. Furlanet al., “Continuous 24- h assessment of the neural regulation of systemic arterial pressure and RR variabilities in ambulant subjects,” Circulation, 81 (2), 537 –547 (1990). http://dx.doi.org/10.1161/01.CIR.81.2.537 CIRCAZ 0009-7322 Google Scholar

23.

S. Akselrodet al., “Power spectrum analysis of heart rate fluctuation: a quantitative probe of beat-to-beat cardiovascular control,” Science, 213 (4504), 220 –222 (1981). http://dx.doi.org/10.1126/science.6166045 SCIEAS 0036-8075 Google Scholar

24.

R. PeriniA. Veicsteinas, “Heart rate variability and autonomic activity at rest and during exercise in various physiological conditions,” Eur. J. Appl. Physiol., 90 (3–4), 317 –325 (2003). http://dx.doi.org/10.1007/s00421-003-0953-9 EJAPFN 1439-6319 Google Scholar

25.

J. M. BlandD. G. Altman, “Statistical methods for assessing agreement between two methods of clinical measurement,” Lancet, 327 (8476), 307 –310 (1986). http://dx.doi.org/10.1016/S0140-6736(86)90837-8 LANCAO 0140-6736 Google Scholar

26.

P. Tanabeet al., “Reliability and validity of scores on the emergency severity index version 3,” Acad. Emerg. Med., 11 (1), 59 –65 (2004). http://dx.doi.org/10.1197/S1069-6563(03)00585-2 AEMEF5 1069-6563 Google Scholar

27.

B. U. KohlerC. HennigR. Orglmeister, “The principles of software QRS detection,” IEEE Eng. Med. Biol. Mag., 21 (1), 42 –57 (2002). http://dx.doi.org/10.1109/51.993193 IEMBDE 0739-5175 Google Scholar

28.

H. Tsujiet al., “Reduced heart rate variability and mortality risk in an elderly cohort. The Framingham Heart Study,” Circulation, 90 (2), 878 –883 (1994). http://dx.doi.org/10.1161/01.CIR.90.2.878 CIRCAZ 0009-7322 Google Scholar

29.

A. Algraet al., “Heart rate variability from 24-hour electrocardiography and the 2-year risk for sudden death,” Circulation, 88 (1), 180 –185 (1993). http://dx.doi.org/10.1161/01.CIR.88.1.180 CIRCAZ 0009-7322 Google Scholar

30.

D. I. BarneaH. F. Silverman, “A class of algorithms for fast digital image registration,” IEEE Trans. Comput., C-21 (2), 179 –186 (1972). http://dx.doi.org/10.1109/TC.1972.5008923 ITCOB4 0018-9340 Google Scholar

31.

L. G. Brown, “A survey of image registration techniques,” ACM Comput. Surveys, 24 (4), 325 –376 (1992). http://dx.doi.org/10.1145/146370.146374 ACSUEY 0360-0300 Google Scholar

© The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

Citation Download Citation

Yu Sun, Sijung Hu, Vicente Azorin-Peris, Roy Kalawsky, and Stephen E. Greenwald "Noncontact imaging photoplethysmography to effectively access pulse rate variability," Journal of Biomedical Optics 18(6), 061205 (30 October 2012). https://doi.org/10.1117/1.JBO.18.6.061205

Published: 30 October 2012

Access the abstract

JOURNAL ARTICLE
9 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

CITATIONS

Cited by 137 scholarly publications and 27 patents.

Explore citations on Lens.org

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Statistical analysis

Imaging systems

Photoplethysmography

Heart

Sensors

Signal detection

Cameras

1.

Introduction

2.

Materials and Methods