Dependent and independent events. Production of events. The concept of conditional probability. Probability multiplication theorem (with proof).

Total probability and Bayes formulas (with proof). Examples.

Repeated independent tests. Bernoulli formula (with derivation). Examples.

Local Moivre-Laplace theorem, conditions for its applicability. Properties of the function Dx). Example.

Poisson's asymptotic formula and conditions for its applicability. Example.

Moivre-Laplace integral theorem and conditions of its applicability. The Laplace function φ(x) and its properties. Example.

Consequences from the Moivre-Laplace integral theorem (with derivation). Examples.

Mathematical expectation of a discrete random variable and its properties (with derivation). Examples.

Dispersion of a discrete random variable and its properties (with derivation). Examples.

The distribution function of a random variable, its definition, properties and graph.

Continuous random variable (new). The probability of a single value of nsv. Mathematical expectation and dispersion nsv.

Probability density of a continuous random variable, its definition, properties and graph.

A random variable distributed according to the binomial law, its mathematical expectation and variance. Poisson distribution law.

Mathematical expectation and variance of the number and frequency of occurrence of an event in n repeated independent trials (with inference).

Definition of the normal distribution law. Theoretical and probabilistic meaning of its parameters. Normal curve and dependence of its position and shape on parameters.

The distribution function of a normally distributed random variable and its expression in terms of the Laplace function.

Formulas for determining the probability: a) hitting a normally distributed random variable in a given interval; b) its deviations from the mathematical expectation. Three Sigma Rule.

The concept of a two-dimensional (/7-dimensional) random variable. Examples. Table of its distribution. One-dimensional distributions of its components. Conditional distributions and their finding in the distribution table.

Covariance and correlation coefficient of random variables. Relationship between e-correlation and independence of random variables.

The concept of a two-dimensional normal distribution law. Conditional mathematical expectations and variances.

Markov's inequality (Chebyshev's lemma) (with derivation). Example.

Chebyshev's inequality (with derivation) and its particular cases for a random variable distributed according to the binomial law and for the frequency of an event.

Chebyshev's theorem (with proof), its meaning and corollary. Example.

The law of large numbers. Bernoulli's theorem (with proof) and its meaning. Example.

Chebyshev's inequality for the arithmetic mean of random variables (with derivation).

Central limit theorem. The concept of Lyapunov's theorem and its meaning. Example.

Variation series, its varieties. Arithmetic mean and variance of the series. A simplified way to calculate them.

The concept of estimating the parameters of the general population. Evaluation properties: unbiasedness, consistency, efficiency.

Estimation of the general share on the basis of a random sample. Unbiasedness and consistency of the sample share.

Estimation of the general average for the proper random sample. Unbiasedness and consistency of the sample mean.

Estimation of the general variance for a proper random sample. Bias and consistency of sample variance (no inference). Corrected sample variance.

The concept of interval estimation. Confidence probability and confidence interval. Marginal sampling error. Sample representativeness errors (random and systematic).

Confidence formula for estimating the general average. The mean square error of repeated and non-repeated samples and the construction of a confidence interval for the general mean.

Determination of the required volume of repeated and non-repeated samples when estimating the general average and proportion.

Statistical hypothesis and statistical test. Errors of the 1st and 2nd kind. Significance level and power of the test. The principle of practical certainty.

Construction of a theoretical distribution law based on experimental data. The concept of the criteria of consent.

The x2-Pearson goodness-of-fit test and the scheme of its application.

Functional, statistical and correlation dependencies. Differences between them. The main tasks of the theory of correlation.

Linear pair regression. System of normal equations for determining the parameters of regression lines. Sample covariance. Formulas for calculating regression coefficients.

Simplified way:

Evaluation of the tightness of the connection. Correlation coefficient (selective), its properties and reliability assessment.

1. The concept of estimating the parameters of the general population. Evaluation properties: unbiasedness, consistency, efficiency.

Let us formulate the problem of estimating parameters in the general form . Let the distribution of the attribute X - the general population - be given by the function of verti (for discrete CV X) or the density of verti
(for continuous SW X), which contains an unknown parameter . For example, it is the parameter λ in the Poisson distribution or the parameters a and
For normal law distributions, etc.

To calculate the parameter it is not possible to study all the elements of the general population. Therefore, about the parameter trying to judge by a sample consisting of values ​​(options)
. These values ​​can be considered as private values ​​(realizations) of n independent random variables
each of which has the same distribution law as the SV X itself.

Definition . Appreciation parameter name any function of the results of observations over SV X (otherwise - statistics), with the help of which they judge the value of the parameter :

.

Because the
are random variables, then the estimate (as opposed to the estimated parameter - non-random, deterministic value) is a random variable that depends on the law of distribution of SV X and the number n.

The quality of an estimate should not be judged by its individual values, but only by the distribution of its values ​​in a large network of tests, i.e. according to the sample distribution of the estimate.

If the evaluation values concentrated around the true value of the parameter , i.e. the main part of the mass of the sample distribution of the estimate is concentrated in a small neighborhood of the estimated parameter , then with high probability we can assume that the estimate different from parameter only by a small amount. Therefore, in order for the value was close to , we must obviously require that the scattering random variablerelatively , expressed, for example, by the mathematical expectation of the squared deviation of the estimate from the estimated parameter
, was as small as possible. This is the basic condition that the "best" estimate must satisfy.

Rating properties.

Definition . Grade parameter called unbiased, if its mating expectation is equal to the estimated parameter, i.e.
.

otherwise, the estimate is called displaced.

If this equality is not satisfied, then the estimate , obtained from different samples, will average or overestimate the value (If
, or underestimate it (if
). The unbiasedness requirement ensures that there are no systematic errors in the estimation.

If for a finite sample size n
, i.e. estimate bias
, But
, then such an estimate called asymptotically unbiased.

Definition . Grade parameter called wealthy, if it satisfies the law of large numbers, i.e. converges in ver-ty to the estimated parameter:

, or .

In the case of using consistent estimates, an increase in the sample size is justified, since at the same time, significant errors in estimation become unlikely. Therefore, only consistent estimates have practical meaning. If the estimate is consistent, then it is almost certain that for sufficiently large n
.

If the score parameter is unbiased, and its variance
as n → ∞, then the estimate is also wealthy. This follows directly from the Chebyshev inequality:

.

Definition . Unbiased estimator parameter is called effective if it has the smallest variance among all possible unbiased estimates of the parameter calculated from samples of the same size n.

Because for unbiased estimator
is its variance , then eff is decisive property, which determines the quality of the assessment.

The effectiveness of the assessment is determined by the ratio: .

Where And - Corresponding to the variance of the effective and given estimates. The closer e is to 1, the more efficient the estimate. If e → 1 as n → ∞, then such an estimate is called asymptotically efficient.

"

to the parameter being evaluated.

Definitions

• Let $X\_1,\backslash ldots,\; X\_n,\backslash ldots$- sample for the distribution depending on the parameter $\backslash theta\; \backslash in\; \backslash Theta$. Then the estimate $\backslash hat(\backslash theta)\; \backslash equiv\; \backslash hat(\backslash theta)(X\_1,\backslash ldots,X\_n)$ is called wealthy if

$$ in probability at $n\; \backslash to\; \backslash infty$.

Otherwise, the estimate is called invalid.

• Grade $\backslash hat(\backslash theta)$ called very wealthy, If

$\backslash hat(\backslash theta)\; \backslash to\; \backslash theta,\backslash quad\; \backslash forall\; \backslash theta\backslash in\; \backslash Theta$ almost certainly at $n\; \backslash to\; \backslash infty$.

In practice, it is not possible to "see" convergence "almost probably" because the samples are finite. Thus, for applied statistics, it is sufficient to require the consistency of the estimate. Moreover, estimates that would be consistent, but not very consistent, "in life" are very rare. The law of large numbers for identically distributed and independent quantities with a finite first moment is also fulfilled in a strengthened version, all extreme order statistics also converge due to monotonicity not only in probability, but almost certainly.

sign

• If the estimate converges to the true value of the "root mean square" parameter, or if the estimate is asymptotically unbiased and its variance tends to zero, then such an estimate will be consistent.

Properties

• From the convergence properties of random variables, we have that a strongly consistent estimate is always consistent. The converse is generally not true.
• Since the variance of consistent estimates tends to zero, often at a rate of the order of 1/n, consistent estimates are compared with each other by the asymptotic variance of the random variable $\backslash sqrt\; (n)\; (\backslash hat(\backslash theta)-\backslash theta)$(the asymptotic expectation of this quantity is zero).

Related concepts

• The score is called super wealthy, if the variance of the random variable $n\; (\backslash hat(\backslash theta)-\backslash theta)$ tends to the final value. That is, the rate of convergence of the estimate to the true value is significantly higher than that of a consistent estimate. Superconsistent, for example, are estimates of regression parameters of cointegrated time series.

Examples

• sample mean $\backslash bar(X)\; =\; \backslash frac(1)(n)\; \backslash sum\backslash limits\_(i=1)^n\; X\_i$ is a strongly consistent estimate of the expectation $X\_i$.
• The periodogram is an unbiased but inconsistent estimate of the spectral density.

see also

Write a review on the article "Consistent assessment"

An excerpt characterizing the Consistent Evaluation

“Oh, Lord have mercy,” added the deacon again.
- You go here and there, they are there. She is. She was still crying, she was crying, - the woman said again. - She is. Here it is.
But Pierre did not listen to the woman. For several seconds he had been staring at what was happening a few steps away from him without taking his eyes off him. He looked at the Armenian family and the two French soldiers who had approached the Armenians. One of these soldiers, a small fidgety little man, was dressed in a blue overcoat, belted with a rope. He had a cap on his head and his feet were bare. The other, who especially struck Pierre, was a long, round-shouldered, blond, thin man with slow movements and an idiotic expression on his face. This one was dressed in a frieze hood, blue trousers and large torn over the knee boots. A little Frenchman, without boots, in blue, hissed, approaching the Armenians, immediately, saying something, took hold of the old man's legs, and the old man immediately began hastily taking off his boots. The other, in the hood, stopped in front of the beautiful Armenian woman and silently, motionless, holding his hands in his pockets, looked at her.
“Take, take the child,” Pierre said, giving the girl and imperiously and hastily addressing the woman. Give them back, give them back! he almost shouted at the woman, putting the screaming girl on the ground, and again looked back at the French and the Armenian family. The old man was already sitting barefoot. The little Frenchman took off his last boot and patted his boots one against the other. The old man, sobbing, said something, but Pierre only glimpsed it; all his attention was directed to the Frenchman in the hood, who at that moment, slowly swaying, moved towards the young woman and, taking his hands out of his pockets, took hold of her neck.
The beautiful Armenian woman continued to sit in the same motionless position, with her long eyelashes lowered, and as if she did not see and did not feel what the soldier was doing to her.
While Pierre ran those few steps that separated him from the French, a long marauder in a hood was already tearing the necklace that was on her from the neck of the Armenian woman, and the young woman, clutching her neck with her hands, screamed in a piercing voice.
– Laissez cette femme! [Leave this woman!] Pierre croaked in a frantic voice, grabbing a long, round-shouldered soldier by the shoulders and throwing him away. The soldier fell, got up and ran away. But his comrade, throwing down his boots, took out a cleaver and menacingly advanced on Pierre.
Voyons, pas de betises! [Oh well! Don't be stupid!] he shouted.
Pierre was in that ecstasy of fury in which he did not remember anything and in which his strength increased tenfold. He lunged at the barefoot Frenchman, and before he could draw his cleaver, he had already knocked him down and pounded him with his fists. Approving shouts of approval were heard from the surrounding crowd, at the same time, a horse patrol of French lancers appeared around the corner. The lancers rode up to Pierre and the Frenchman at a trot and surrounded them. Pierre did not remember anything from what happened next. He remembered that he was beating someone, he was being beaten, and that in the end he felt that his hands were tied, that a crowd of French soldiers were standing around him and searching his dress.

Definition.The random variable is called evaluation unknown parameter , if the value of this random variable, found from the results of a series of measurements, can be taken as an approximate value of this parameter, i.e. if equality is true.

Example. If the probability of the occurrence of some event is considered as an unknown parameter, then the estimate of this parameter is the frequency of the occurrence of the event in independent trials (see the statistical definition of probability and Bernoulli's theorem).

Example. Let random variables have the same mathematical expectation, i.e. . Then the arithmetic mean these random variables. An important special case of the considered situation is the following.

Example. An estimate of some parameter is the arithmetic mean results independent measurements of this parameter (see Chebyshev's theorem).

When using the approximate equality directly talking about point estimation unknown parameter.

Possibly also interval estimation unknown parameter. In order to explain what it consists of, we introduce the following concepts into consideration.

Definition.For an arbitrary interval is called confidence interval; the quantity itself is called in this case marginal sampling error.

Definition.The probability that an unknown value of the estimated parameter is covered by a confidence interval is called confidence probability.

Thus, if – parameter estimate , That

is the confidence level (we assume that the estimate is a continuous random variable).

Interval estimation consists, for example, in calculating a confidence level for a given marginal sampling error.

The solution of the problem of interval estimation is associated with determining the nature of the distribution law of the estimate used .

Let us now consider some properties of estimates.

Definition.The parameter estimate is called unbiased, if the mathematical expectation of this estimate is equal to the estimated parameter, i.e.

Definition.The parameter estimate is called wealthy, if the following limit relation holds for an arbitrary

In other words, an estimate for a parameter is consistent if this estimate converges in probability to the given parameter. (Recall that examples of this kind of convergence are given by Bernoulli's and Chebyshev's theorems; see § 6.2.)

Definition.An unbiased estimate of some parameter is called effective, if it has the smallest variance among all unbiased estimates found from a sample of a given size.

Example. Frequency occurrence of some event is an unbiased, consistent and effective estimate of the probability this event . Note that the properties of unbiasedness and consistency of frequency were actually considered by us earlier in a somewhat different context. Indeed, frequency unbiasedness—equality—is one of the properties of a binomially distributed random variable (see § 3.3). The consistency of the frequency is stated by Bernoulli's theorem (see § 6.2).

Example. The arithmetic mean of a certain number of independent and identically distributed random variables is an unbiased and consistent estimate of the total mathematical expectation of these random variables. Indeed, unbiasedness is property 5 of expectation (see § 3.3). Consistency is asserted by Chebyshev's theorem (see § 6.2).

) problems of mathematical statistics.

Let us assume that there is a parametric family of probability distributions (for simplicity, we will consider the distribution of random variables and the case of one parameter). Here, is a numeric parameter whose value is unknown. It is required to estimate it by the available sample of values ​​generated by this distribution.

There are two main types of assessments: point estimates And confidence intervals.

Point Estimation

Point estimation is a type of statistical estimation in which the value of an unknown parameter is approximated by a single number. That is, you must specify the function of the sample (statistics)

,

whose value will be considered as an approximation to the unknown true value .

Common methods for constructing point estimates of parameters include: maximum likelihood method, method of moments, quantile method.

Below are some properties that point estimates may or may not have.

solvency

One of the most obvious requirements for a point estimate is that one can expect a reasonably good approximation to the true value of the parameter for sufficiently large values ​​of the sample size . This means that the estimate must converge to the true value at . This evaluation property is called solvency. Since we are talking about random variables for which there are different types convergence, then given property can be precisely formulated in different ways:

When just using the term solvency, then we usually mean weak consistency, i.e., convergence in probability.

The consistency condition is practically obligatory for all estimates used in practice. Inconsistent estimates are rarely used.

Unbiasedness and asymptotic unbiasedness

The parameter estimate is called unbiased, if its mathematical expectation is equal to the true value of the estimated parameter:

.

The weaker condition is asymptotic unbiasedness, which means that the mathematical expectation of the estimate converges to the true value of the parameter with an increase in the sample size:

.

Unbiasedness is a recommended property of estimators. However, its importance should not be overestimated. Most often, unbiased parameter estimates exist, and then one tries to consider only them. However, there may be some statistical problems in which unbiased estimates do not exist. The most famous example is the following: consider a Poisson distribution with a parameter and set the problem of estimating the parameter . It can be proved that there is no unbiased estimator for this problem.

Grade Comparison and Efficiency

To compare different estimates of the same parameter with each other, the following method is used: choose some risk function, which measures the deviation of the estimate from the true value of the parameter, and the best one is considered to be the one for which this function takes a smaller value.

Most often, the mathematical expectation of the squared deviation of the estimate from the true value is considered as a risk function

For unbiased estimators, this is simply the variance.

There is a lower bound on this risk function called Cramer-Rao inequality.

(Unbiased) estimators for which this lower bound is met (i.e. having the smallest possible variance) are called effective. However, the existence of an effective estimate is a rather strong requirement for the problem, which is by no means always the case.

The weaker condition is asymptotic efficiency, which means that the ratio of the variance of the unbiased estimate to the lower Cramer-Rao bound tends to unity at .

Note that under sufficiently broad assumptions about the distribution under study, the maximum likelihood method gives an asymptotically efficient estimate of the parameter, and if there is an effective estimate, then it gives an efficient estimate.

Sufficient statistics

The statistic is called sufficient for the parameter if the conditional distribution of the sample provided that , does not depend on the parameter for all .

The importance of the concept of sufficient statistics is due to the following approval. If is a sufficient statistic and is an unbiased estimate of the parameter , then the conditional expectation is also an unbiased estimate of the parameter , and its variance is less than or equal to the variance of the original estimate .

Recall that the conditional expectation is a random variable that is a function of . Thus, in the class of unbiased estimators, it suffices to consider only those that are functions of sufficient statistics (provided that such a statistic exists for the given problem).

The (unbiased) effective parameter estimate is always a sufficient statistic.

We can say that a sufficient statistic contains all the information about the estimated parameter that is contained in the sample.

In order for statistical estimates to give a good approximation of the estimated parameters, they must be unbiased, efficient, and consistent.

unbiased is called the statistical estimate of the parameter , the mathematical expectation of which is equal to the estimated parameter for any sample size.

Displaced called statistical evaluation
parameter , whose mathematical expectation is not equal to the estimated parameter.

efficient called statistical evaluation
parameter , which for a given sample size has the smallest variance.

Wealthy called statistical evaluation
parameter , which at
tends in probability to the estimated parameter.

i.e. for any

.

For samples of different sizes, different values ​​of the arithmetic mean and statistical variance are obtained. Therefore, the arithmetic mean and statistical variance are random variables for which there is a mathematical expectation and variance.

Let's calculate the mathematical expectation of the arithmetic mean and variance. Denote by mathematical expectation of a random variable

Here, the following are considered as random variables: – S.V., the values ​​of which are equal to the first values ​​obtained for different volume samples from the general population
–S.V., the values ​​of which are equal to the second values ​​obtained for different volume samples from the general population, ...,
- S.V., whose values ​​are equal -th values ​​obtained for different volume samples from the general population. All these random variables are distributed according to the same law and have the same mathematical expectation.

It follows from formula (1) that the arithmetic mean is an unbiased estimate of the mathematical expectation, since the mathematical expectation of the arithmetic mean is equal to the mathematical expectation of the random variable. This estimate is also consistent. The efficiency of this estimate depends on the type of distribution of the random variable
. If, for example,
normally distributed, estimating the expected value using the arithmetic mean will be efficient.

Let us now find a statistical estimate of the variance.

The expression for the statistical variance can be transformed as follows

(2)

Let us now find the mathematical expectation of the statistical variance

. (3)

Given that
(4)

we get from (3) -

It can be seen from formula (6) that the mathematical expectation of the statistical variance differs by a factor from the variance, i.e. is a biased estimate of the population variance. This is because instead of the true value
, which is unknown, the statistical mean is used to estimate the variance .

Therefore, we introduce the corrected statistical variance

(7)

Then the mathematical expectation of the corrected statistical variance is

those. the corrected statistical variance is an unbiased estimate of the population variance. The resulting estimate is also consistent.