For people suffering from recurrent epileptic seizures, one of the most burdensome aspects of their condition is the unpredictability of their seizures. While medications, surgery, and novel neurostimulation methods can eliminate seizures in some cases, many people with epilepsy face the possibility of a seizure at any time, even when they occur only rarely. This has a disproportionate impact on their daily lives and profoundly limits even routine activities, such as driving a car, swimming, bathing, or performing any other activity where a momentary loss of consciousness could prove catastrophic. Even when epilepsy is well controlled with medications, people endure persistent side effects to prevent rare seizures.
For many years, experts in neurology, computer science, and engineering have worked toward developing algorithms to predict a seizure before it occurs. If an algorithm could detect subtle changes in the electrical activity of a person’s brain (measured by electroencephalography (EEG)) before a seizure occurs, people with epilepsy could take medications only when needed, and possibly reclaim some of those daily activities many of us take for granted. But algorithm development and testing requires substantial quantities of suitable data, and progress has been slow. Many early research reports developed and tested algorithms on relatively short intracranial EEG data segments from patients with epilepsy undergoing intracranial EEG before surgery. There are a number of problems with this. First, patients undergoing pre-surgical monitoring for epilepsy typically have their medications reduced to encourage seizures to occur, which causes a progressive decrease in the blood levels of medications which have been shown to affect the normal baseline pattern in a patient’s EEG. Second, hospital stays for pre-surgical monitoring by necessity rarely last more than two weeks, providing a very limited amount of data for any single patient. These short data segments with changing baseline EEG characteristics are particularly problematic when algorithm scientists attempt to measure an algorithm’s false positive rate, or the number of false alarms that a seizure forecasting algorithm might raise. Development of robust, reliable seizure prediction algorithms requires data on many seizures and many periods of baseline, non-seizure EEG with enough time between the seizures to allow the brain to recover. In addition, researchers are often reluctant to share algorithm data and programs; privacy concerns and the high cost of sharing large data sets makes testing and comparison very difficult.
In 2013 a group of physicians and scientists from Melbourne Australia reported a successful trial of an implanted device capable of measuring EEG from intracranial electrode strips, and telemetering the EEG data to a small external device about the size of a smart phone that could run seizure forecasting algorithms and provide warnings of impending seizures. The device used a proprietary seizure forecasting algorithm that performed well enough to be helpful for some patients in the trial, raising hopes that seizure forecasting might soon become clinically possible.
We recently made an effort to use Kaggle.com — a website that runs data science competitions to develop algorithms to predict everything from insurance rates to the Higgs Boson — to develop new algorithms for seizure forecasting. Our competition used intracranial EEG data from the same device in the Australian trial (implanted in eight dogs with naturally occurring epilepsy) as well as data from two human patients undergoing intracranial monitoring. In hope of winning $15,000 in prize money, plus bragging rights among elite data science circles, hundreds of algorithm developers, most with little or no experience with epilepsy or EEG, worked countless hours to build, test, and rebuild algorithms for seizure forecasting, and tested their algorithms on nearly 350 seizures recorded over more than 1,500 days. After four months, over half of these “crowdsourced” algorithms performed better than random predictions, and the winning algorithms accurately predicted over 70% of seizures with a 25% false positive rate. The data are available for researchers to continue developing new algorithms for predicting seizures, and can serve as a benchmark for new algorithms to be compared directly to one another and to the algorithms developed in this competition. The best performing algorithms in the competition used a mixture of conventional and complex approaches drawn from physics, engineering, and computer science, sometimes in unorthodox ways that proved to be surprisingly effective. The winning teams also made the source code for their algorithms publicly available, providing a benchmark and starting point for future algorithm developers.
While we applaud the talented algorithm scientists who took home the prize money, we hope the real winners of the contest will be our patients.
Featured image: Sky cloudy. Uploaded by Carla Nunziata. CC-BY-SA-3.0 via Wikimedia Commons