Accuracy of SmartWATCH-based ElectroCardioGraphic arrhythmia detection

Atrial fibrillation (AF) affects millions of patients worldwide and is a cause
of substantial morbidity and stroke. Asymptomatic AF is gaining worldwide
interest for its potentially serious clinical consequences.(1) Furthermore,
paroxysmal AF may evolve into persistent or permanent AF when left untreated.
Screening and early detection of this - in its paroxysmal stage somewhat
elusive - arrhythmia may lead to reductions in stroke, hospitalizations and
death due to early treatment initiation.

The Apple Watch (series 4) is among the first commercially available devices
capable of a single-lead electrocardiographic (ECG) registration using
electrodes embedded within components of the device (Figure 1). The ECG
application algorithm detects whether atrial fibrillation is present and
medical expertise should be consulted. Data from these recordings are encrypted
and users will be able to share a report with their doctors via PDF. *De Novo*
classification for this technology was recently obtained by the FDA, making it
the second consumer device to gain Class II clearance for ECG monitoring.(2)

Extensive research using deep neural networks has been conducted to test the
accuracy of heart rate detection using photoplethysmography (PPG) in older
models.(3) The model exhibited a C-statistic of 0.97 to detect AF against the
reference standard 12-lead ECG-diagnosed AF in a validation cohort of 51
patients undergoing cardioversion; sensitivity was 98.0% and specificity was
90.2%. The authors may be applauded for this effort to improve early detection
rhythm assessment, which will undoubtedly prove valuable in reducing AF-related
complications in the future. However, the performance of the new AF detection
algorithm using electrocardiographic data, especially in more challenging
real-world situations, is yet to be determined. Despite the fact that this is
promising technology, attention should be paid to its potential limited
accuracy in younger users and patients with concomitant heart disease.

Screening using the AliveCor Kardia monitor, which utilizes similar technology,
has proven to be significantly more effective in identifying AF than routine
care in patients > 65 years of age.(4) However, AF detection may be less
accurate at higher heart rates and during distorted signals and artifacts
caused by physical exertion. Therefore, younger and more physically active
users of this device - a large relatively healthy population - may be at risk
of receiving AF notifications during exercise and other causes of tachycardia,
potentially leading to unnecessary outpatient visits or even wrongful
diagnosis. Herein lies this new wearable technology*s main pitfall and source
of skepticism by some health care providers. We seek to further determine
whether this concern is justified and whether this commercially available
noninvasive screening tool has the potential to expand the diagnostic arsenal
for AF.

In the second study in this series (the first protocol (NL68180.018.18 -
2018_296) was approved on March 8th 2019), which is aimed to test the
real-world performance of the Apple Watch arrhythmia detection algorithm in a
resting state.

The objective of this study is to determine the performance, as defined by
sensitivity to detect AF and specificity to detect SR, of the commercially
available Apple Watch-based electrocardiographic rhythm classification
algorithm, validated against the gold standard 12-lead electrocardiography.

Type of study:
This is a prospective non-randomized two-arm monocenter observational study to
evaluate performance of a novel commercially available diagnostic tool in adult
patients that are admitted to our tertiary hospital with and without AF.

All data will be collected in the Amsterdam UMC, location Academic Medical
Center in Amsterdam. All participants will be asked for written informed
consent prior to enrolment.

Sample size:
Apple Inc. released a document on their website with results of an internal
(unpublished) study reporting a sensitivity of 85.2% and specificity of 90.5%
when including unreadable/unclassifiable results from the Apple Watch.(6) In
this real-world study, we assume the sensitivity and specificity might be
slightly lower. We therefore calculated that 265 enrolled patients in each
study arm (i.e., 530 patients in total) will provide enough data to determine
diagnostic accuracy. Patients with known AF are included to ensure an adequate
prevalence of AF in the total study population (0.5).

Study methodology:
During their admission to the inpatient clinic, but before undergoing 12-lead
electrocardiography, eligible subjects will be informed about the study. Only
patients who are already planned for 12-lead ECG testing or who have continuous
12-lead ECG monitoring will be screened for participation. Hence, the conduct
of this study will not influence the indication for ECG testing and will
therefore not bring to light any newly diagnosed arrhythmias, that might
otherwise have gone unnoticed. During the visit, the subject information sheet
(SIS) containing information on purpose, background, participation and risks of
the study will be handed for further reading. After sufficient time to read and
process the information the subject will asked for written informed consent.

Rhythm classification with the investigational device will be performed
simultaneously with the 12-lead ECG recording. In case of participation, the
subject will receive the device on their left wrist and place their right index
finger on the crown of the watch, as indicated by the manufacturer. In case
this not possible, the right arm may be used instead. Participants will be
assisted by the investigator in the Apple Watch placement and instructed to
keep their arm still, for instance by resting it on a table or leg. An
electrocardiographic registration of around 30 seconds will be made, after
which the device algorithm indicates the registered heart rhythm, being either
SR or AF (or unclassifiable or unreadable). This process may be repeated up to
three times in total, allowing for practice in adequate sample acquisition. The
12-lead ECG will be obtained simultaneously to ensure synchronous analysis of
heart rhythm between the two methods of diagnosis. No personal information will
be stored on the Apple Watch and no additional measurements of any kind will be
obtained from it.

A research fellow will be present to aid in device positioning and to record
its findings on the anonymized electronic case report form (eCRF), along with
baseline characteristics on age, gender, BMI and medical history.
Rhythm classification of the concurrent 12-lead ECG (using the same categories
as the device) will be performed by two investigators independently and will
serve as the reference standard in diagnosing AF. Subjects serve as their own
controls in the evaluation in difference in proportions between two
within-subject observations of the outcome.

Investigational product
The algorithm under investigation is featured within the rhythm detection
application of the Apple Watch series 4 (released September 2018 by Apple Inc.,
Cupertino, CA, USA), and has obtained clearance by the Food and Drug
Administration (FDA) as a class II medical device.

Follow-up
There will be no follow-up in this acute study. This study does not require any
follow-up visits or additional data collection beyond their current admission.

Statistical analysis
Continuous variables within the baseline characteristics are expressed as
median and interquartile range (IQR) or mean and standard deviation when
normally distributed. Categorical variables are expressed as frequency with
corresponding percentages. Of these baseline data, continuous values are
compared using the Mann-Whitney U test or unpaired t-test and categorical
variables with Fisher*s Exact test.
Sensitivity and specificity will be calculated from contingency tables
containing information about the test results (SR, AF and
unclassifiable/unreadable) of both the gold standard and investigational test.
Sinus rhythm could be wrongfully interpreted by the device as AF, counting as a
false-positive in our analysis. Conversely, AF could be labelled as sinus
rhythm, which would count as a false-negative.
The new diagnostic modality is thereby compared to the existing reference
standard by the proportion of the true positive and true negative samples that
it identifies. All reported p-values will be 2 tailed, and p-values <0.05 are
considered statistically significant. Statistical analyses will be performed in
IBM SPSS Statistics 24 and R version 3.5.1.

The participant will not benefit directly from this study. However, in the
future accumulating knowledge on the effectiveness of
screening for AF might be beneficial in reducing AF-related morbidity and
mortality on a population level. There are no additional
post-procedural limitations or visits required for this study.

Subjects will not be exposed to additional risks when choosing to participate
as electrocardiography was already indicated on their
physician*s discretion, and the additional burden that subjects will experience
is predicted to be low.