Distilled News

A software framework for embedded nonlinear model predictive control using a gradient-based augmented Lagrangian approach (GRAMPC)

A nonlinear MPC framework is presented that is suitable for dynamical systems with sampling times in the (sub)millisecond range and that allows for an efficient implementation on embedded hardware. The algorithm is based on an augmented Lagrangian formulation with a tailored gradient method for the inner minimization problem. The algorithm is implemented in the software framework GRAMPC and is a fundamental revision of an earlier version. Detailed performance results are presented for a test set of benchmark problems and in comparison to other nonlinear MPC packages. In addition, runtime results and memory requirements for GRAMPC on ECU level demonstrate its applicability on embedded hardware.

Machine Learning Explainability

The methods I am going to describe can be used with any model and are applied after a model is fit to the dataset. The following questions answered refer each to a section in the mentioned online course. The Titanic dataset can be used to train a classification model which predicts whether a passenger survived or died on the Titanic. I used a simple DecisionTree for this task and did not optimize it nor balanced the data, etc. It was simply trained in order to illustrate the methods. The code for the analyses can be found on this GitHub repository.

Efficient Machine Learning for Attack Detection

Detecting and fending off attacks on computer systems is an enduring problem in computer security. In light of a plethora of different threats and the growing automation used by attackers, we are in urgent need of more advanced methods for attack detection. In this thesis, we address the necessity of advanced attack detection and develop methods to detect attacks using machine learning to establish a higher degree of automation for reactive security. Machine learning is data-driven and not void of bias. For the effective application of machine learning for attack detection, thus, a periodic retraining over time is crucial. However, the training complexity of many learning-based approaches is substantial. We show that with the right data representation, efficient algorithms for mining substring statistics, and implementations based on probabilistic data structures, training the underlying model can be achieved in linear time. In two different scenarios, we demonstrate the effectiveness of so-called language models that allow to generically portray the content and structure of attacks: On the one hand, we are learning malicious behavior of Flash-based malware using classification, and on the other hand, we detect intrusions by learning normality in industrial control networks using anomaly detection. With a data throughput of up to 580 Mbit/s during training, we do not only meet our expectations with respect to runtime but also outperform related approaches by up to an order of magnitude in detection performance. The same techniques that facilitate learning in the previous scenarios can also be used for revealing malicious content, embedded in passive file formats, such as Microsoft Office documents. As a further showcase, we additionally develop a method based on the efficient mining of substring statistics that is able to break obfuscations irrespective of the used key length, with up to 25 Mbit/s and thus, succeeds where related approaches fail. These methods significantly improve detection performance and enable operation in linear time. In doing so, we counteract the trend of compensating increasing runtime requirements with resources. While the results are promising and the approaches provide urgently needed automation, they cannot and are not intended to replace human experts or traditional approaches, but are designed to assist and complement them.

Spectral Clustering

In this post, we’ll cover the ins and outs of spectral clustering for graphs and other data. Clustering is one of the main tasks in unsupervised machine learning. The goal is to assign unlabeled data to groups, where similar data points hopefully get assigned to the same group. Spectral clustering is a technique with roots in graph theory, where the approach is used to identify communities of nodes in a graph based on the edges connecting them. The method is flexible and allows us to cluster non graph data as well. Spectral clustering uses information from the eigenvalues (spectrum) of special matrices built from the graph or the data set. We’ll learn how to construct these matrices, interpret their spectrum, and use the eigenvectors to assign our data to clusters.

Reading between the layers (LSTM Network)

One of the most crucial part of building a Deep Neural Network is- to have a clear view on your data as it flows through layers undergoing change in dimensions, alteration in shape, flattening and then re-shaping… We will refer to the LSTM Architecture that we have seen earlier in our Sentiment Analysis Tutorial.

Data Anonymisation Software – Differences Between Static and Interactive Anonymisation

Which company would not like to get significantly improved insights into the market with detailed data and thus optimize its products and services? In most cases, the corresponding data is already available, but its use is often restricted for data protection reasons. Anonymisation is used when you want to use a data set with sensitive information for analysis without compromising the privacy of the user. Moreover, the principles of data protection do not apply to anonymised data. This means that this information can be analysed without risk. Therefore, anonymisation is an enormously helpful tool for drawing conclusions, for example, from bank transaction data or patient medical records.

The 7 Myths of Data Anonymisation

In our daily work as a company for a data anonymisation solution we are often asked to clarify data anonymisation with customers and in personal conversations. Alarmingly often we hear questions like ‘What is the difference between anonymisation and encryption again?’. But some questions demonstrate serious misunderstandings, which can have equally serious consequences.

Myth #1: Anonymisation has become important because of the GDPR

Myth #2: No PII means no personal data

Myth #3 Anonymisation destroys data and prevents innovation

Myth #4 ‘Differential Privacy’ makes data private

Myth #5 Synthetic data is anonymous

Myth #6 AI does not tolerate and does not need anonymous data

Myth #7 Anonymous data doesn’t exist at all

Artificial Neural Network Implementation using NumPy and Classification of the Fruits360 Image Dataset

This tutorial builds artificial neural network in Python using NumPy from scratch in order to do an image classification application for the Fruits360 dataset. Everything (i.e. images and source codes) used in this tutorial, rather than the color Fruits360 images, are exclusive rights for my book cited as ‘Ahmed Fawzy Gad ‘Practical Computer Vision Applications Using Deep Learning with CNNs’. Dec. 2018, Apress, 978-1-4842-4167-7 ‘. The book is available at Springer at this link: https://…/9781484241660.

Multiclass Classification with Word Bags and Word Sequences

SVM with tf-idf vectors edges out LSTM in quality and performance for classifying the 20-newsgroups text corpus. Document is a specific sequence of words. But not all sequences of words are documents. Teaching the difference to an algorithm is a tall order. Taking the sequence of words into account for text analysis is in general computationally expensive. Deep learning approaches such as LSTM allow us to model a document as a string- of-words and they have indeed found some success in NLP tasks recently. On the other hand when we shred the document and make bags by word, we end up with a vector of weights/counts of these words. The mapping from a document to this vector can be many-to-one as all possible sequences of words yield the same vector. So the deciphering of the meaning of the original document (much less resurrecting it!), from this vector is not possible. Nevertheless this decades old bags-of-words approach to modeling documents has been the main stay for NLP tasks. When the sequence of words is important in determining the class of a document, string-of-words approaches will outshine the bags-of-words. We have demonstrated this with synthetic documents where LSTM trounced the bags-of-words approach (Naive Bayes working with tf-idf vectors) for classification. But for a real text corpus of movie reviews for binary sentiment classification, we have shown that both LSTM and SVM (with tf-idf vectors) were comparable in quality even while the former took much longer. The objective of this post is to further evaluate ‘bags vs strings’ for a multiclass situation. We will work with the 20-newsgroups text corpus that is available from scikit-learn api. We will also look at the impact of using word-embeddings?-?both pre-trained and custom. We go through some code snippets here, but the complete code to reproduce the results can be downloaded from github.

Step-by-step Guide to Building Your Own Neural Network From Scratch

Learn the fundamentals of deep learning and build your very own neural network for image classification.

Understanding Logistic Regression Step by Step

Logistic Regression is a popular statistical model used for binary classification, that is for predictions of the type this or that, yes or no, A or B, etc. Logistic regression can, however, be used for multiclass classification, but here we will focus on its simplest application. As an example, consider the task of predicting someone’s gender (Male/Female) based on their Weight and Height. For this, we will train a machine learning model from a data set of 10,000 samples of people’s weight and height. The data set is taken from the Conway & Myles Machine Learning for Hackers book, Chapter 2, and can it can be directly downloaded here.

Developments in AzureR

The AzureR packages have now been on CRAN for a couple of months, so I thought I’d provide an update on developments in the works.

Deep Learning for Natural Language Processing (NLP) – using RNNs & CNNs

Wouldn’t it be cool if a computer could understand the actual human sentiment behind sarcastic texts that can sometimes even trump actual humans? Or what if computers could understand a human language so well that it can estimate a probability telling you how likely it is to encounter any random sentence that you give it? Or maybe it could generate completely fake code snippets of the Linux kernel that look so authentic that they are just as intimidating as the actual source code (well, unless you are a kernel programmer yourself)? What if computers could immaculately translate English to French or over 100 languages from all over the world?

Exciting Predictions For Where Big Data Analytics Are Headed By 2025

Data analytics are continuing to grow and become more technologically complex. Here are predictions for where big data analytics are headed in the future.

The Data Fabric for Machine Learning. Part 1-b: Deep Learning on Graphs.

We are in the process of defining a new way of doing machine learning, focusing on a new paradigm, the data fabric. In the past article I gave my new definition of machine learning: Machine learning is the automatic process of discovering hidden insights in data fabric by using algorithms that are able to find those insights without being specifically programmed for that, to create models that solves a particular (or multiple) problem(s). The premise for understanding this it’s that we have created a data fabric. For me the best tool out there for me for doing that is Anzo as I mentioned in other articles.