6 years ago
Wise Practitioner – Predictive Analytics Interview Series: Przemek Maciołek, PhD at Collective Sense

By: Luba Gloukhova, Founding Chair, Deep Learning World

In anticipation of his upcoming conference presentation at Deep Learning World Las Vegas, June 16-20, 2019, we asked Przemek Maciołek, VP of
Research & Development at Collective Sense, a few questions about his work in deep learning. Catch a glimpse of his presentation, Elevating Deep Learning for Network Security and Performance with Real-Time Pattern Discovery in Logs, and see what’s in store at the DLW conference in Las Vegas.

Q: In your work with deep learning, what do you model (i.e., what is the dependent variable, the behavior or outcome your models predict)?

A:  At LogSense, what best describes our goal is detecting log sequences that should be brought to the operator attention. We look at many properties of incoming logs and do our best to answer the question if the behaviour is unusual and why it is so.

Q: How does deep learning deliver value at your organization – what is one specific way in which model outputs actively drive decisions or operations?
 
A:  Deep learning gives us a way to reduce the time spent on certain areas or events that might not be as relevant or critical, in turn accelerating the path to achieving certain goals, like troubleshooting. Anyone monitoring services or devices frequently has to deal with thousands or even millions of events each second, which makes it increasingly hard to filter out what is important from all of the noise. By using deep-learning based anomaly detection, we pick the most interesting patterns and thus drastically increase the chances of becoming aware of something important while reducing the time spent on achieving that goal. 

Q: Can you describe a quantitative result, such as the performance of your model or the ROI of the model deployment initiative?

A:  In our benchmarks, moving from simpler anomaly detection methods, based on Naive Bayes or Gaussian Mixtures, to a deep-learning based anomaly detection method, allowed us to decrease the false positive rate by an order of magnitude. Though to be fair in the process of doing that we also increased the number of features fed into the model.

Q: What surprising discovery or insight have you unearthed in your data?
 
A: One of the challenges, which I believe are true for many deep learning projects, is explaining the end result to the user. Because we are constantly improving our methods, we frequently find effects that we didn’t think about beforehand which make perfect sense though. For example, when we’ve been modeling an Advanced Persistent Threat scenario including a well-known attack against the bind DNS server, our model explained the anomaly not only by seeing segfaults of the service (which we anticipated) but also by unusual properties of domain names being queried before that (a side effect of the malicious code) – which we were somewhat (positively!) surprised to see as being picked up.

Q: What excites you most about the field of deep learning today?

A: I have been working with ML for over 15 years, and it’s amazing how sophisticated yet easy to use the tools are available these days. A complex Deep Learning network can be expressed in just a few lines of code using a library like Keras. Plenty of researchers publish their notebooks, which allow anyone to see how they handled the issue and solved the problem. There’s a lot of progress happening each day and the community is thriving.

Q: Sneak preview: Please tell us a take-away that you will provide during your talk at Deep Learning World.

A: Deep learning anomaly detection can have a dramatic impact on performance and security. During my talk, I will show how reconstructing the latent structure of text logs through pattern discovery allows us to think about the logs as events (patterns) with certain properties (parameters). This approach makes great features for deep learning anomaly detection, as presented on several real-life use cases in the performance and security area.