This article needs additional citations for verification. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed. Find sources: "Convolutional neural network" – news · newspapers · books · scholar · JSTOR(June 2019) (Learn how and when to remove this message)
Part of a series on
Machine learning and data mining
Paradigms
Supervised learning
Unsupervised learning
Online learning
Batch learning
Meta-learning
Semi-supervised learning
Self-supervised learning
Reinforcement learning
Curriculum learning
Rule-based learning
Quantum machine learning
Problems
Classification
Generative modeling
Regression
Clustering
Dimensionality reduction
Density estimation
Anomaly detection
Data cleaning
AutoML
Association rules
Semantic analysis
Structured prediction
Feature engineering
Feature learning
Learning to rank
Grammar induction
Ontology learning
Multimodal learning
Supervised learning (classification • regression)
Apprenticeship learning
Decision trees
Ensembles
Bagging
Boosting
Random forest
k-NN
Linear regression
Naive Bayes
Artificial neural networks
Logistic regression
Perceptron
Relevance vector machine (RVM)
Support vector machine (SVM)
Clustering
BIRCH
CURE
Hierarchical
k-means
Fuzzy
Expectation–maximization (EM)
DBSCAN
OPTICS
Mean shift
Dimensionality reduction
Factor analysis
CCA
ICA
LDA
NMF
PCA
PGD
t-SNE
SDL
Structured prediction
Graphical models
Bayes net
Conditional random field
Hidden Markov
Anomaly detection
RANSAC
k-NN
Local outlier factor
Isolation forest
Artificial neural network
Autoencoder
Cognitive computing
Deep learning
DeepDream
Feedforward neural network
Kolmogorov–Arnold Network
Recurrent neural network
LSTM
GRU
ESN
reservoir computing
Restricted Boltzmann machine
GAN
Diffusion model
SOM
Convolutional neural network
U-Net
Transformer
Vision
Mamba
Spiking neural network
Memtransistor
Electrochemical RAM (ECRAM)
Reinforcement learning
Q-learning
SARSA
Temporal difference (TD)
Multi-agent
Self-play
Learning with humans
Active learning
Crowdsourcing
Human-in-the-loop
RLHF
Model diagnostics
Coefficient of determination
Confusion matrix
Learning curve
ROC curve
Mathematical foundations
Kernel machines
Bias–variance tradeoff
Computational learning theory
Empirical risk minimization
Occam learning
PAC learning
Statistical learning
VC theory
Machine-learning venues
ECML PKDD
NeurIPS
ICML
ICLR
IJCAI
ML
JMLR
Related articles
Glossary of artificial intelligence
List of datasets for machine-learning research
List of datasets in computer vision and image processing
Outline of machine learning
v
t
e
Convolutional neural network (CNN) is a regularized type of feed-forward neural network that learns feature engineering by itself via filters (or kernel) optimization. Vanishing gradients and exploding gradients, seen during backpropagation in earlier neural networks, are prevented by using regularized weights over fewer connections.[1][2] For example, for each neuron in the fully-connected layer, 10,000 weights would be required for processing an image sized 100 × 100 pixels. However, applying cascaded convolution (or cross-correlation) kernels,[3][4] only 25 neurons are required to process 5x5-sized tiles.[5][6] Higher-layer features are extracted from wider context windows, compared to lower-layer features.
They have applications in:
image and video recognition,[7]
recommender systems,[8]
image classification,
image segmentation,
medical image analysis,
natural language processing,[9]
brain–computer interfaces,[10] and
financial time series.[11]
CNNs are also known as Shift Invariant or Space Invariant Artificial Neural Networks (SIANN), based on the shared-weight architecture of the convolution kernels or filters that slide along input features and provide translation-equivariant responses known as feature maps.[12][13] Counter-intuitively, most convolutional neural networks are not invariant to translation, due to the downsampling operation they apply to the input.[14]
Feed-forward neural networks are usually fully connected networks, that is, each neuron in one layer is connected to all neurons in the next layer. The "full connectivity" of these networks makes them prone to overfitting data. Typical ways of regularization, or preventing overfitting, include: penalizing parameters during training (such as weight decay) or trimming connectivity (skipped connections, dropout, etc.) Robust datasets also increase the probability that CNNs will learn the generalized principles that characterize a given dataset rather than the biases of a poorly-populated set.[15]
Convolutional networks were inspired by biological processes[16][17][18][19] in that the connectivity pattern between neurons resembles the organization of the animal visual cortex. Individual cortical neurons respond to stimuli only in a restricted region of the visual field known as the receptive field. The receptive fields of different neurons partially overlap such that they cover the entire visual field.
CNNs use relatively little pre-processing compared to other image classification algorithms. This means that the network learns to optimize the filters (or kernels) through automated learning, whereas in traditional algorithms these filters are hand-engineered. This independence from prior knowledge and human intervention in feature extraction is a major advantage.[to whom?]
^Venkatesan, Ragav; Li, Baoxin (2017-10-23). Convolutional Neural Networks in Visual Computing: A Concise Guide. CRC Press. ISBN 978-1-351-65032-8. Archived from the original on 2023-10-16. Retrieved 2020-12-13.
^Balas, Valentina E.; Kumar, Raghvendra; Srivastava, Rajshree (2019-11-19). Recent Trends and Advances in Artificial Intelligence and Internet of Things. Springer Nature. ISBN 978-3-030-32644-9. Archived from the original on 2023-10-16. Retrieved 2020-12-13.
^Zhang, Yingjie; Soon, Hong Geok; Ye, Dongsen; Fuh, Jerry Ying Hsi; Zhu, Kunpeng (September 2020). "Powder-Bed Fusion Process Monitoring by Machine Vision With Hybrid Convolutional Neural Networks". IEEE Transactions on Industrial Informatics. 16 (9): 5769–5779. doi:10.1109/TII.2019.2956078. ISSN 1941-0050. S2CID 213010088. Archived from the original on 2023-07-31. Retrieved 2023-08-12.
^Chervyakov, N.I.; Lyakhov, P.A.; Deryabin, M.A.; Nagornov, N.N.; Valueva, M.V.; Valuev, G.V. (September 2020). "Residue Number System-Based Solution for Reducing the Hardware Cost of a Convolutional Neural Network". Neurocomputing. 407: 439–453. doi:10.1016/j.neucom.2020.04.018. S2CID 219470398. Archived from the original on 2023-06-29. Retrieved 2023-08-12. Convolutional neural networks represent deep learning architectures that are currently used in a wide range of applications, including computer vision, speech recognition, malware dedection, time series analysis in finance, and many others.
^Habibi, Aghdam, Hamed (2017-05-30). Guide to convolutional neural networks : a practical application to traffic-sign detection and classification. Heravi, Elnaz Jahani. Cham, Switzerland. ISBN 9783319575490. OCLC 987790957.{{cite book}}: CS1 maint: location missing publisher (link) CS1 maint: multiple names: authors list (link)
^Atlas, Homma, and Marks. "An Artificial Neural Network for Spatio-Temporal Bipolar Patterns: Application to Phoneme Classification" (PDF). Neural Information Processing Systems (NIPS 1987). 1. Archived (PDF) from the original on 2021-04-14.{{cite journal}}: CS1 maint: multiple names: authors list (link)
^Valueva, M.V.; Nagornov, N.N.; Lyakhov, P.A.; Valuev, G.V.; Chervyakov, N.I. (2020). "Application of the residue number system to reduce hardware costs of the convolutional neural network implementation". Mathematics and Computers in Simulation. 177. Elsevier BV: 232–243. doi:10.1016/j.matcom.2020.04.031. ISSN 0378-4754. S2CID 218955622. Convolutional neural networks are a promising tool for solving the problem of pattern recognition.
^van den Oord, Aaron; Dieleman, Sander; Schrauwen, Benjamin (2013-01-01). Burges, C. J. C.; Bottou, L.; Welling, M.; Ghahramani, Z.; Weinberger, K. Q. (eds.). Deep content-based music recommendation(PDF). Curran Associates, Inc. pp. 2643–2651. Archived (PDF) from the original on 2022-03-07. Retrieved 2022-03-31.
^Collobert, Ronan; Weston, Jason (2008-01-01). "A unified architecture for natural language processing". Proceedings of the 25th international conference on Machine learning - ICML '08. New York, NY, US: ACM. pp. 160–167. doi:10.1145/1390156.1390177. ISBN 978-1-60558-205-4. S2CID 2617020.
^Avilov, Oleksii; Rimbert, Sebastien; Popov, Anton; Bougrain, Laurent (July 2020). "Deep Learning Techniques to Improve Intraoperative Awareness Detection from Electroencephalographic Signals". 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC)(PDF). Vol. 2020. Montreal, QC, Canada: IEEE. pp. 142–145. doi:10.1109/EMBC44109.2020.9176228. ISBN 978-1-7281-1990-8. PMID 33017950. S2CID 221386616. Archived (PDF) from the original on 2022-05-19. Retrieved 2023-07-21.
^Tsantekidis, Avraam; Passalis, Nikolaos; Tefas, Anastasios; Kanniainen, Juho; Gabbouj, Moncef; Iosifidis, Alexandros (July 2017). "Forecasting Stock Prices from the Limit Order Book Using Convolutional Neural Networks". 2017 IEEE 19th Conference on Business Informatics (CBI). Thessaloniki, Greece: IEEE. pp. 7–12. doi:10.1109/CBI.2017.23. ISBN 978-1-5386-3035-8. S2CID 4950757.
^Zhang, Wei (1988). "Shift-invariant pattern recognition neural network and its optical architecture". Proceedings of Annual Conference of the Japan Society of Applied Physics. Archived from the original on 2020-06-23. Retrieved 2020-06-22.
^Zhang, Wei (1990). "Parallel distributed processing model with local space-invariant interconnections and its optical architecture". Applied Optics. 29 (32): 4790–7. Bibcode:1990ApOpt..29.4790Z. doi:10.1364/AO.29.004790. PMID 20577468. Archived from the original on 2017-02-06. Retrieved 2016-09-22.
^Mouton, Coenraad; Myburgh, Johannes C.; Davel, Marelie H. (2020). "Stride and Translation Invariance in CNNs". In Gerber, Aurona (ed.). Artificial Intelligence Research. Communications in Computer and Information Science. Vol. 1342. Cham: Springer International Publishing. pp. 267–281. arXiv:2103.10097. doi:10.1007/978-3-030-66151-9_17. ISBN 978-3-030-66151-9. S2CID 232269854. Archived from the original on 2021-06-27. Retrieved 2021-03-26.
^Kurtzman, Thomas (August 20, 2019). "Hidden bias in the DUD-E dataset leads to misleading performance of deep learning in structure-based virtual screening". PLOS ONE. 14 (8): e0220113. Bibcode:2019PLoSO..1420113C. doi:10.1371/journal.pone.0220113. PMC 6701836. PMID 31430292.
^Cite error: The named reference fukuneoscholar was invoked but never defined (see the help page).
^Cite error: The named reference hubelwiesel1968 was invoked but never defined (see the help page).
^Cite error: The named reference intro was invoked but never defined (see the help page).
^Matusugu, Masakazu; Katsuhiko Mori; Yusuke Mitari; Yuji Kaneda (2003). "Subject independent facial expression recognition with robust face detection using a convolutional neural network" (PDF). Neural Networks. 16 (5): 555–559. doi:10.1016/S0893-6080(03)00115-1. PMID 12850007. Archived (PDF) from the original on 13 December 2013. Retrieved 17 November 2013.
and 24 Related for: Convolutional neural network information
Convolutionalneuralnetwork (CNN) is a regularized type of feed-forward neuralnetwork that learns feature engineering by itself via filters (or kernel)...
certain existing neuralnetwork architectures can be interpreted as GNNs operating on suitably defined graphs. A convolutionalneuralnetwork layer, in the...
deep learning for computer vision. AlexNet is an eight-layer convolutionalneuralnetwork (CNN), and although CNNs had been around since at least LeNet...
"recurrent neuralnetwork" is used to refer to the class of networks with an infinite impulse response, whereas "convolutionalneuralnetwork" refers to...
A feedforward neuralnetwork (FNN) is one of the two broad types of artificial neuralnetwork, characterized by direction of the flow of information between...
Zhang et al. had discussed fast optical implementations of convolutionalneuralnetworks for alphabet recognition. In the 1990s, there were also attempts...
A Siamese neuralnetwork (sometimes called a twin neuralnetwork) is an artificial neuralnetwork that uses the same weights while working in tandem on...
recurrent neuralnetworks and convolutionalneuralnetworks, renewed interest in ANNs. The 2010s, saw the development of a deep neuralnetwork (a neural network...
a convolutionalneuralnetwork structure proposed by LeCun et al. in 1998. In general, LeNet refers to LeNet-5 and is a simple convolutionalneural network...
convolutionalneuralnetwork best performance was 0.25 percent error rate. As of August 2018, the best performance of a single convolutionalneural network...
optical neuralnetwork (ONN) is a physical implementation of an artificial neuralnetwork with optical components. Biological neuralnetworks are electrochemical...
closely mimic biological neural organization. The idea is to add structures called "capsules" to a convolutionalneuralnetwork (CNN), and to reuse output...
AlexNet is the name of a convolutionalneuralnetwork (CNN) architecture, designed by Alex Krizhevsky in collaboration with Ilya Sutskever and Geoffrey...
generator is typically a deconvolutional neuralnetwork, and the discriminator is a convolutionalneuralnetwork. GANs are implicit generative models, which...
Inception v3 is a convolutionalneuralnetwork for assisting in image analysis and object detection, and got its start as a module for GoogLeNet. It is...
types of artificial neuralnetworks (ANN). Artificial neuralnetworks are computational models inspired by biological neuralnetworks, and are used to approximate...
created by Google engineer Alexander Mordvintsev that uses a convolutionalneuralnetwork to find and enhance patterns in images via algorithmic pareidolia...
co-inventor, with Alex Krizhevsky and Geoffrey Hinton, of AlexNet, a convolutionalneuralnetwork. Sutskever is also one of the many co-authors of the AlphaGo...
Neural architecture search (NAS) is a technique for automating the design of artificial neuralnetworks (ANN), a widely used model in the field of machine...
optimizations for speech recognition. Convolutionalneuralnetwork – a convolutionalneural net where the convolution is performed along the time axis of...
representation of the minority class, improving model performance. When convolutionalneuralnetworks grew larger in mid-1990s, there was a lack of data to use, especially...