A new in-depth learning algorithm can provide advanced advancement when systems (from satellites to data centers) fall apart.
When you’re in charge of a multi-million dollar satellite for thousands of miles an hour, you need to make sure it works well. And time series can help.
The time series is a record of repeated measurements over time. It can track a system’s long-term trends and short-term blips. Examples include the Covid-19 curve of new daily cases and the Keeling curve, which has been tracking atmospheric carbon dioxide concentrations since 1958. In the age of big data, “time series are gathered everywhere, from satellites to turbines,” he tells Kalyan Veeramachanen. “All this machinery has sensors that cover these operating time series”
But it can be difficult to study these time series and mark data anomalies in them. Data can be noisy. If a satellite operator sees a high-temperature reading string, how do you know if it is a harmless incident or a sign that the satellite is about to warm up?
One problem is Veeramachaneni, who leads the Data-to-AI team WITH ONEHe hopes to fix the Information and Decision Systems Laboratory. The team has developed a new method based on in-depth learning to mark anomalies in time series data. The TadGAN approach went beyond competitive methods and could help operators detect and respond to major changes in a range of valuable systems, from a satellite flying through space to a computer server farm with rumors in the basement.
The research will be presented at this month’s IEEE BigData conference. The authors of the article include members of Veeramachaneni’s Data-to-AI team, Dr. Dongyu Liu, research student Alexander Geiger and master’s student Sarah Alnegheimish, and Alfredo Cuesta-Infante of Rey Juan Carlos University in Spain.
For a system as complex as a satellite, time series analysis must be automated. The SES satellite company, which is collaborating with Veeramachaneni, receives a flood of time series from its communications satellites – about 30,000 parameters per spacecraft. Human operators in the SES control room can only continue flashing on the screen for part of these time series. The rest are based on an alarm system to mark values outside the range. “So they told us,‘ can you do better? ’” Says Veeramachanenik. The company wanted its team to use in-depth learning to analyze all of these time series and express unusual behavior.
The bet of this request is great: if the deep learning algorithm does not detect an anomaly, the team may lose the opportunity to fix things. If the alarm sounds whenever there is a noisy data point, human reviewers will waste time constantly analyzing the algorithm that made the wolf cry. “So we have these two challenges,” Liu says. “And we need to balance them.”
Rather than achieving this balance only for satellite systems, the team sought to create a more general framework for detecting anomalies – one that can be applied to all industries. They turned to in-depth learning systems, called generative opposition networks (GANs), which are often used for image analysis.
GAN consists of a pair of neural networks. One network, the “creator,” creates fake images, while the second network processes the “discriminatory” images and tries to determine whether the images created by the generator are real or fake. Through many rounds of this process, the generator learns from the discriminator’s opinions and is adept at creating hyperrealistic fakes. The technique is considered an “unsupervised” study because it does not require a pre-tagged data set where images are tagged with topics. (Large sets of tagged data can be difficult.)
The team adapted this GAN approach to time series data. “Based on this training strategy, our model can tell which data points are normal and which are abnormal,” says Liu. To do this, it checks for discrepancies (possible anomalies) between real-time series and false time series generated by GAN. But the team saw that GANs alone were not sufficient to detect anomalies in time series, as they may be able to determine the actual time segment in which real-time false series should be compared. As a result, “If you only use GAN, you will create a lot of false positives,” he tells Veeramachanen.
To protect against false positives, the team complemented it with an algorithm called its GAN self-encoder, another technique for unsupervised deep learning. Compared to GAN’s tendency for wolves to cry, self-encoders tend to lose real anomalies. This is because auto-encoders capture too many patterns in time series, sometimes interpreting a real anomaly as a harmless incident – a problem called “over-fitting”. Combined with a GAN with an auto-encoder, the researchers created an anomaly detection system that achieved the perfect balance: TadGAN is alert, but does not generate too many false alarms.
Standing for the season series test
In addition, TadGAN won the competition. The traditional approach to time series prediction, called ARIMA, was developed in the 1970s. “We wanted to see how far we’ve come and whether deep learning models can really improve this classical method,” says Alnegheimish.
The group conducted 11 data set anomaly testing tests, pitting ARIMA against TadGAN and seven other methods, some of which were developed by companies such as Amazon and Microsoft. TadGAN surpassed ARIMA in detecting the anomaly in eight of the 11 data sets. The second best algorithm developed by Amazon was won only by ARIMA in six data sets.
Alnegheimish stressed that their goal is not only to develop an algorithm for detecting higher-level anomalies, but also to make them very useful. “We all know that AI has trouble playing,” he says. The team has made the TadGAN code available for free, and regularly updates it. They have also developed a benchmarking system for users to compare the performance of different models for detecting anomalies.
“This reference is open source, so anyone can go try it out. They can add their own model if they want,” says Alnegheimish. “We want to alleviate the stigma surrounding AI, which is not reproducible. We want to make sure everything is fine. “
Veeramachaneni hopes that one day TadGAN will serve many industries, not just satellite companies. For example, it could be used to monitor the performance of computer applications that are fundamental to the modern economy. “I have 30 applications to run a lab. Zoom, Slack, Github – you have a name, I have, ”he says. “And I rely on them all to work without problems and forever.” The same goes for millions of users around the world.
TadGAN can help companies like Zoom monitor time series signals in the data center – such as CPU usage or temperature – to prevent service disruptions, which could threaten the company’s market share. In future work, the team plans to package TadGAN in a user interface to help anyone in need provide timely time-series analysis.
Reference: Alexander Tiger, Dongyu Liu, Sarah Alnegheimish, Alfredo Cuesta-Infante and Kalyan Veeramachaneni, using “TadGAN: Time Series Anomaly Detection UsingGenerative Adversarial Networks”, 14 November 2020, Computer Science> Machine Learning.
This research was funded by SES and completed in collaboration.