Geometric Understanding of Deep Learning
source link: https://www.tuicool.com/articles/hit/ANB3eeR
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Abstract: Deep learning is the mainstream technique for many machine learning tasks, including image recognition, machine translation, speech recognition, and so on. It has outperformed conventional methods in various fields and achieved great successes. Unfortunately, the understanding on how it works remains unclear. It has the central importance to lay down the theoretic foundation for deep learning.
In this work, we give a geometric view to understand deep learning: we show that the fundamental principle attributing to the success is the manifold structure in data, namely natural high dimensional data concentrates close to a low-dimensional manifold, deep learning learns the manifold and the probability distribution on it.
We further introduce the concepts of rectified linear complexity for deep neural network measuring its learning capability, rectified linear complexity of an embedding manifold describing the difficulty to be learned. Then we show for any deep neural network with fixed architecture, there exists a manifold that cannot be learned by the network. Finally, we propose to apply optimal mass transportation theory to control the probability distribution in the latent space.
Recommend
-
21
Vaishak V.Kumar One of the central problems in Computer Vision and Robotics is that of understanding how objects are positioned with re...
-
15
The following document provides a whirlwind tour of some fundamental concepts in geometric deep learning.
-
4
A Comparison Between Graph Neural Networks: DGL vs. PyTorch Geometric What is deep learning on graphs? In general, a graph is a system of nodes connected by edges. The nodes typically have some sort of internal state that can...
-
150
README.md
-
8
Calculating the average of a variable or a list of numbers is a common operation in machine learning. It is an operation you may use every day either directly, such as when summarizing data, or indirectly, such a...
-
44
ganja.js - Geometric Algebra for javascript. Geometric Algebra - Not Just Algebra Ganja.js is a Geometric Algebra code generator for javascript. I...
-
35
readme.md "To deal with hyper-...
-
35
README.md DGC-Net: Dense Geometric Correspondence Network This is a PyTorch implementation of our work "DGC-...
-
22
LLVM optimizes geometric sums, such as int sum(int count) { int result = 0; for (int j = 0; j < count; ++j) result += j*j; return result; } to code that calculates the re...
-
60
README.md leonsans Leon Sans is a geometric sans-serif typeface made with code in 2019 by Jongmin Kim. It allows to change font weight dynami...
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK