Minimum Description Length (MDL) is a model selection principle where the shortest description of the data is the best model. MDL methods learn through a data compression perspective and are sometimes described as mathematical applications of Occam's razor. The MDL principle can be extended to other forms of inductive inference and learning, for example to estimation and sequential prediction, without explicitly identifying a single model of the data.
MDL has its origins mostly in information theory and has been further developed within the general fields of statistics, theoretical computer science and machine learning, and more narrowly computational learning theory.
Historically, there are different, yet interrelated, usages of the definite noun phrase "the minimum description length principle" that vary in what is meant by description:
Within Jorma Rissanen's theory of learning, a central concept of information theory, models are statistical hypotheses and descriptions are defined as universal codes.
Rissanen's 1978[1] pragmatic first attempt to automatically derive short descriptions, relates to the Bayesian Information Criterion (BIC).
Within Algorithmic Information Theory, where the description length of a data sequence is the length of the smallest program that outputs that data set. In this context, it is also known as 'idealized' MDL principle and it is closely related to Solomonoff's theory of inductive inference, which is that the best model of a data set is represented by its shortest self-extracting archive.
^Rissanen, J. (September 1978). "Modeling by shortest data description". Automatica. 14 (5): 465–471. doi:10.1016/0005-1098(78)90005-5.
and 27 Related for: Minimum description length information
MinimumDescriptionLength (MDL) is a model selection principle where the shortest description of the data is the best model. MDL methods learn through...
Minimum message length (MML) is a Bayesian information-theoretic method for statistical model comparison and selection. It provides a formal information...
In the minimumdescriptionlength approach to statistics the goal is to describe data as compactly as possible where the length of a description is measured...
goodness of fit of the model and the simplicity of the model.) The minimumdescriptionlength (MDL) principle has been developed from ideas in information theory...
Dowe's Minimum Message Length (MML) and Occam's razor pages. Grunwald, P.; Pitt, M.A. (2005). Myung, I. J. (ed.). Advances in MinimumDescriptionLength: Theory...
boundaries. Minimumdescriptionlength: when forming a hypothesis, attempt to minimize the length of the description of the hypothesis. Minimum features:...
Scientific study of digital information Minimumdescriptionlength – Model selection principle Minimum message length – Formal information theory restatement...
single objective by means of a model selection principle such as minimumdescriptionlength. It has been proven that symbolic regression is an NP-hard problem...
S2CID 11715509. Hinton, G. E., & Zemel, R. S. (1994). Autoencoders, minimumdescriptionlength and Helmholtz free energy. In Advances in neural information processing...
complexity – Measure of algorithmic complexity Minimumdescriptionlength – Model selection principle Minimum message length – Formal information theory restatement...
Microsoft Design Language, a design language created by Microsoft Minimumdescriptionlength, a principle for inductive inference in information theory Mazagon...
images that can avoid measuring noise as complex by using the minimumdescriptionlength principle. There has also been interest in measuring the complexity...
coding length it attains. Texture is encoded by lossy compression in a way similar to minimumdescriptionlength (MDL) principle, but here the length of the...
of parameters in the model. It is approximately equal to the minimumdescriptionlength criterion but with negative sign. It can be used to choose the...
function for training auto-encoder based on the minimumdescriptionlength (MDL) principle". "The descriptionlength of an input vector using a particular code...
information matrix Likelihood-ratio test Mallows's Cp MinimumdescriptionlengthMinimum message length (MML) PRESS statistic, also known as the PRESS criterion...
(2005). "Minimum Message Length and Generalized Bayesian Nets with Asymmetric Languages". In Grünwald PD, Myung IJ, Pitt MA (eds.). Advances in Minimum Description...
Logical reasoning Logical positivism Marcus Hutter MinimumdescriptionlengthMinimum message length New riddle of induction Open world assumption Plausible...
Computational complexity theory Inductive inference Minimum message lengthMinimumdescriptionlength Schmidhuber, J. (2002) The Speed Prior: A New Simplicity...
value or cost of each model choice; to use minimum message length (MML). to use minimumdescriptionlength (MDL). A value of K > 1 means that M1 is more...
A minimum spanning tree (MST) or minimum weight spanning tree is a subset of the edges of a connected, edge-weighted undirected graph that connects all...
minimum wage (US$) rate by 52 weeks and then by the length of the standard hour workweek. Hourly wages were calculated by dividing the annual minimum...
other principles, such as information or coding theory (see e.g. minimumdescriptionlength) or frequentist statistics (so-called probability matching priors)...
Program, a distance learning program based in Mesa, Arizona. Minimumdescriptionlength principle This disambiguation page lists articles associated with...
Global Information