Why do we need a GPU for deep learning?
Many of the operations that go in deep learning computation (and neural networks in general) can be performed in parallel, which means they can be computed independently, then aggregated later. This is in part because most of the operations are on vectors.
A typical consumer CPU has 4 to 8 cores, and hyper-threading allows them to be handled as 8 or 16 respectively. Server processors can have 4 to 24 cores, 8 to 48 threads, respectively. In addition, most modern processors have SIMDs (Single-shot Multiple Data) which allow them to perform vector operations in parallel on a single thread. Depending on the type of data you are working with, an 8-core processor can do 8 * 2 * 4 = 64-8 * 2 * 8 = 128 vector computations at once.
The Nvidia new 1080ti has 3584 CUDA cores, which essentially means it can do 3584 vector computations at once (hyperthreads and SIMDs don't appear in the game here). This is 56-28 times more simultaneous operations than an 8-core processor. So, whether you train one network or multiply to tweak meta parameters, it will probably be significantly faster on GPU than CPU.
source to share
This can take a long time depending on what you are doing. I had 20x faster GPU usage. If you've read a few Computer Vision papers, they train their networks on ImageNet for about 1-2 weeks. Now imagine it took 20 times longer ...
Having said that: there are much easier tasks. For example, for my HASY dataset, you can train a reasonable network without a GPU in probably 3 hours. Such small datasets are MNIST, CIFAR-10, CIFAR-100.
source to share
The numerically intensive part of the neural network is the multiple matrix multiplications. And how do we do it faster? We can do this by performing all the operations at the same time, rather than doing it one by one. In a nutshell, why are we using a GPU (GPUs) instead of a CPU (Central Processing Unit).
Google had a powerful system that they built specifically to teach huge networks. This system is worth $ 5 billion with multiple processor clusters. Several years later, Stanford researchers built the same computationally-based system to train their deep networks using the GPU. They cut costs to 33 thousand dollars. This system was built using GPUs and provided the same processing power as the Googles system.
Source: https://www.analyticsvidhya.com/blog/2017/05/gpus-necessary-for-deep-learning/
source to share
Deep learning is about creating a mathematical model of reality or some part of reality for some specific use using a lot of training data, so you use a lot of training data from the real world that you have collected, and then you can train your model to your math model could have predicted different results when you give it new data as input, so you can basically train this math model, but it needs a lot of data and this training needs a lot of computation. So there are a lot of computational heavy operations to take place and you also need a lot of data. Therefore, for example, companies like Nvidia, which traditionally use GPUs for graphics,now they also get a huge chunk of the revenue from AI and Machine Learning, and all these scientists who want to train their models and you see companies like Google and Facebook all use GPUs nowadays to train their ML models.
source to share