Parameter of a prior distribution in Bayesian statistics
This article has multiple issues. Please help improve it or discuss these issues on the talk page. (Learn how and when to remove these template messages)
This article relies largely or entirely on a single source. Relevant discussion may be found on the talk page. Please help improve this article by introducing citations to additional sources. Find sources: "Hyperparameter" – news · newspapers · books · scholar · JSTOR(July 2020)
This article may be too technical for most readers to understand. Please help improve it to make it understandable to non-experts, without removing the technical details.(December 2020) (Learn how and when to remove this message)
(Learn how and when to remove this message)
This article is about hyperparameters in Bayesian statistics. It is not to be confused with Hyperparameter (machine learning).
Part of a series on
Bayesian statistics
Posterior = Likelihood × Prior ÷ Evidence
Background
Bayesian inference
Bayesian probability
Bayes' theorem
Bernstein–von Mises theorem
Coherence
Cox's theorem
Cromwell's rule
Principle of indifference
Principle of maximum entropy
Model building
Weak prior ... Strong prior
Conjugate prior
Linear regression
Empirical Bayes
Hierarchical model
Posterior approximation
Markov chain Monte Carlo
Laplace's approximation
Integrated nested Laplace approximations
Variational inference
Approximate Bayesian computation
Estimators
Bayesian estimator
Credible interval
Maximum a posteriori estimation
Evidence approximation
Evidence lower bound
Nested sampling
Model evaluation
Bayes factor
Model averaging
Posterior predictive
Mathematics portal
v
t
e
In Bayesian statistics, a hyperparameter is a parameter of a prior distribution; the term is used to distinguish them from parameters of the model for the underlying system under analysis.
For example, if one is using a beta distribution to model the distribution of the parameter p of a Bernoulli distribution, then:
p is a parameter of the underlying system (Bernoulli distribution), and
α and β are parameters of the prior distribution (beta distribution), hence hyperparameters.
One may take a single value for a given hyperparameter, or one can iterate and take a probability distribution on the hyperparameter itself, called a hyperprior.
learning, hyperparameter optimization or tuning is the problem of choosing a set of optimal hyperparameters for a learning algorithm. A hyperparameter is a...
In Bayesian statistics, a hyperparameter is a parameter of a prior distribution; the term is used to distinguish them from parameters of the model for...
outperform hand-designed models. Common techniques used in AutoML include hyperparameter optimization, meta-learning and neural architecture search. In a typical...
a prior distribution on a hyperparameter, that is, on a parameter of a prior distribution. As with the term hyperparameter, the use of hyper is to distinguish...
built into deep learning libraries such as Keras. Hyperparameter (machine learning) Hyperparameter optimization Stochastic gradient descent Variable metric...
system: from a given set of hyperparameters, incoming data updates these hyperparameters, so one can see the change in hyperparameters as a kind of "time evolution"...
design (without constructing and training it). NAS is closely related to hyperparameter optimization and meta-learning and is a subfield of automated machine...
processes are popular surrogate models in Bayesian optimization used to do hyperparameter optimization. A genetic algorithm (GA) is a search algorithm and heuristic...
marginal likelihood, represents a convenient approach for setting hyperparameters, but has been mostly supplanted by fully Bayesian hierarchical analyses...
(-\infty ,\infty )} . Hyperparameters are various settings that are used to control the learning process. CNNs use more hyperparameters than a standard multilayer...
the models per se, but of the choice of specific hyperparameters. Transferring these hyperparameters to more 'traditional' approaches yields similar performances...
hyperparameters in turn greatly affecting convergence, HyFDCA's single hyperparameter allows for simpler practical implementations and hyperparameter...
optimizing decision trees for better performance, solving sudoku puzzles, hyperparameter optimization, causal inference, etc. In a genetic algorithm, a population...
Yamins, D. D. Cox (2013). Hyperopt: A Python Library for Optimizing the Hyperparameters of Machine Learning Algorithms. Proc. SciPy 2013. Chris Thornton, Frank...
different models on the same dataset and guide the optimization of hyperparameters, although it has been found sensitive to factors such as linguistic...
create a conditional prior of the mean on the unknown variance, with a hyperparameter specifying the mean of the pseudo-observations associated with the prior...
posterior distribution, namely: Hyperparameters: parameters of the prior distribution Hyperpriors: distributions of Hyperparameters Suppose a random variable...
reinforcement learning algorithms require hyperparameter tuning, PPO does not necessarily require hyperparameter tuning (0.2 for epsilon can be used in most...
algorithmic approaches to model selection include feature selection, hyperparameter optimization, and statistical learning theory. In its most basic forms...
{\boldsymbol {\alpha }}} is a set of parameters to the prior itself, or hyperparameters. Let E = ( e 1 , … , e n ) {\displaystyle \mathbf {E} =(e_{1},\dots...
Global Information
