Search results
Results from the WOW.Com Content Network
In probability theory, a Chernoff bound is an exponentially decreasing upper bound on the tail of a random variable based on its moment generating function. The minimum of all such exponential bounds forms the Chernoff or Chernoff-Cramér bound , which may decay faster than exponential (e.g. sub-Gaussian ).
Hoeffding's inequality was proven by Wassily Hoeffding in 1963. [1] Hoeffding's inequality is a special case of the Azuma–Hoeffding inequality and McDiarmid's inequality. It is similar to the Chernoff bound, but tends to be less sharp, in particular when the variance of the random variables is small. [2]
Thus, special cases of the Bernstein inequalities are also known as the Chernoff bound, Hoeffding's inequality and Azuma's inequality. The martingale case of the Bernstein inequality is known as Freedman's inequality [5] and its refinement is known as Hoeffding's inequality. [6]
Therefore, the theorem above gives a tighter bound than the Ahlswede–Winter result. The chief contribution of (Ahlswede & Winter 2003) was the extension of the Laplace-transform method used to prove the scalar Chernoff bound (see Chernoff bound#Additive form (absolute error)) to the case of self-adjoint
This is a generalization of Hoeffding's since it can handle random variables with not only almost-sure bound but both almost-sure bound and variance bound. 6. Chernoff bounds have a particularly simple form in the case of sum of independent variables, since [] = = [].
In probability theory, the Azuma–Hoeffding inequality (named after Kazuoki Azuma and Wassily Hoeffding) gives a concentration result for the values of martingales that have bounded differences. Suppose {: =,,,, …} is a martingale (or super-martingale) and
Hoeffding's inequality is the Chernoff bound obtained using this fact. Convolutions. Density of a mixture of three normal distributions μ ...
Chernoff's inequality; Sanov's theorem; Contraction principle (large deviations theory), a result on how large deviations principles "push forward" Freidlin–Wentzell theorem, a large deviations principle for Itō diffusions; Legendre transformation, Ensemble equivalence is based on this transformation.