Search results
Results from the WOW.Com Content Network
Chernoff bounds may also be applied to general sums of independent, bounded random variables, regardless of their distribution; this is known as Hoeffding's inequality. The proof follows a similar approach to the other Chernoff bounds, but applying Hoeffding's lemma to bound the moment generating functions (see Hoeffding's inequality).
Hoeffding's inequality is a special case of the Azuma–Hoeffding inequality and McDiarmid's inequality. It is similar to the Chernoff bound, but tends to be less sharp, in particular when the variance of the random variables is small. [2] It is similar to, but incomparable with, one of Bernstein's inequalities.
Thus, special cases of the Bernstein inequalities are also known as the Chernoff bound, Hoeffding's inequality and Azuma's inequality. The martingale case of the Bernstein inequality is known as Freedman's inequality [5] and its refinement is known as Hoeffding's inequality. [6]
Placing addition assumption that the summands in Matrix Azuma are independent gives a matrix extension of Hoeffding's inequalities. Consider a finite sequence { X k } {\displaystyle \{\mathbf {X} _{k}\}} of independent, random, self-adjoint matrices with dimension d {\displaystyle d} , and let { A k } {\displaystyle \{\mathbf {A} _{k}\}} be a ...
In multivariate statistics, random matrices were introduced by John Wishart, who sought to estimate covariance matrices of large samples. [15] Chernoff-, Bernstein-, and Hoeffding-type inequalities can typically be strengthened when applied to the maximal eigenvalue (i.e. the eigenvalue of largest magnitude) of a finite sum of random Hermitian matrices. [16]
Hoeffding's inequality is the Chernoff bound obtained using this fact. Convolutions. Density of a mixture of three normal distributions μ ...
Hoeffding's inequality yields the ... n with k < n), but Hoeffding's bound evaluates to a positive constant. A sharper bound can be obtained from the Chernoff ...
Such inequalities are of importance in several fields, including communication complexity (e.g., in proofs of the gap Hamming problem [13]) and graph theory. [14] An interesting anti-concentration inequality for weighted sums of independent Rademacher random variables can be obtained using the Paley–Zygmund and the Khintchine inequalities. [15]