When Using A Significance Test Calculator, What Is The “margin Of Error” Called?

Statistic expressing the amount of random sampling error in a survey's results

Probability densities of polls of different sizes, each color-coded to its 95% confidence interval (below), margin of error (left), and sample size (right). Each interval reflects the range within which one may have 95% confidence that the true percentage may be found, given a reported percentage of 50%. The margin of error is half the confidence interval (also, the radius of the interval). The larger the sample, the smaller the margin of error. Also, the further from 50% the reported percentage, the smaller the margin of error.

The margin of error is a statistic expressing the amount of random sampling error in the results of a survey. The larger the margin of error, the less confidence one should have that a poll result would reflect the result of a survey of the entire population. The margin of error will be positive whenever a population is incompletely sampled and the outcome measure has positive variance, which is to say, the measure varies.

The term margin of error is often used in non-survey contexts to indicate observational error in reporting measured quantities. It is also used in colloquial speech to refer to the amount of space or amount of flexibility one might have in accomplishing a goal. For example, it is often used in sports by commentators when describing how much precision is required to achieve a goal, points, or outcome. A bowling pin used in the United States is 4.75 inches wide, and the ball is 8.5 inches wide, therefore one could say a bowler has a 21.75 inch margin of error when trying to hit a specific pin to earn a spare (e.g.,1 pin remaining on the lane).

Concept [edit]

Consider a simple yes/no poll ${\displaystyle P}$ as a sample of ${\displaystyle n}$ respondents drawn from a population ${\displaystyle N{\text{, }}(n<<N)}$ reporting the percentage ${\displaystyle p}$ of yes responses. We would like to know how close ${\displaystyle p}$ is to the true result of a survey of the entire population ${\displaystyle N}$ , without having to conduct one. If, hypothetically, we were to conduct poll ${\displaystyle P}$ over subsequent samples of ${\displaystyle n}$ respondents (newly drawn from ${\displaystyle N}$ ), we would expect those subsequent results ${\displaystyle p_{1},p_{2},\ldots }$ to be normally distributed about ${\displaystyle {\overline {p}}}$ . The margin of error describes the distance within which a specified percentage of these results is expected to vary from ${\displaystyle {\overline {p}}}$ .

According to the 68-95-99.7 rule, we would expect that 95% of the results ${\displaystyle p_{1},p_{2},\ldots }$ to fall within about two standard deviations ( ${\displaystyle \pm 2\sigma _{P}}$ ) either side of the true mean ${\displaystyle {\overline {p}}}$ .  This interval is called the confidence interval, and the radius (half the interval) is called the margin of error, corresponding to a 95% confidence level.

Generally, at a confidence level ${\displaystyle \gamma }$ , a sample sized ${\displaystyle n}$ of a population having expected standard deviation ${\displaystyle \sigma }$ has a margin of error

${\displaystyle MOE_{\gamma }=z_{\gamma }\times {\sqrt {\frac {\sigma ^{2}}{n}}}}$

where ${\displaystyle z_{\gamma }}$ denotes the quantile (also, commonly, a z-score), and ${\displaystyle {\sqrt {\frac {\sigma ^{2}}{n}}}}$ is the standard error.

Standard deviation and standard error [edit]

We would expect the normally distributed values ${\displaystyle p_{1},p_{2},\ldots }$ to have a standard deviation which somehow varies with ${\displaystyle n}$ . The smaller ${\displaystyle n}$ , the wider the margin. This is called the standard error ${\displaystyle \sigma _{\overline {p}}}$ .

For the single result from our survey, we assume that ${\displaystyle p={\overline {p}}}$ , and that all subsequent results ${\displaystyle p_{1},p_{2},\ldots }$ together would have a variance ${\displaystyle \sigma _{P}^{2}=P(1-P)}$ .

${\displaystyle {\text{Standard error}}=\sigma _{\overline {p}}\approx {\sqrt {\frac {\sigma _{P}^{2}}{n}}}\approx {\sqrt {\frac {p(1-p)}{n}}}}$

Note that ${\displaystyle p(1-p)}$ corresponds to the variance of a Bernoulli distribution.

Maximum margin of error at different confidence levels [edit]

For a confidence level ${\displaystyle \gamma }$ , there is a corresponding confidence interval about the mean ${\displaystyle \mu \pm z_{\gamma }\sigma }$ , that is, the interval ${\displaystyle [\mu -z_{\gamma }\sigma ,\mu +z_{\gamma }\sigma ]}$ within which values of ${\displaystyle P}$ should fall with probability ${\displaystyle \gamma }$ . Precise values of ${\displaystyle z_{\gamma }}$ are given by the quantile function of the normal distribution (which the 68-95-99.7 rule approximates).

Note that ${\displaystyle z_{\gamma }}$ is undefined for ${\displaystyle |\gamma |\geq 1}$ , that is, ${\displaystyle z_{1.00}}$ is undefined, as is ${\displaystyle z_{1.10}}$ .

${\displaystyle \gamma }$	${\displaystyle z_{\gamma }}$		${\displaystyle \gamma }$	${\displaystyle z_{\gamma }}$
0.68	0.994457 883 210		0.999	3.290526 731 492
0.90	1.644853 626 951		0.9999	3.890591 886 413
0.95	1.959963984540		0.99999	4.417173 413 469
0.98	2.326347 874 041		0.999999	4.891638 475 699
0.99	2.575829 303 549		0.9999999	5.326723 886 384
0.995	2.807033 768 344		0.99999999	5.730728 868 236
0.997	2.967737 925 342		0.999999999	6.109410 204 869

Since ${\displaystyle \max \sigma _{P}^{2}=\max P(1-P)=0.25}$ at ${\displaystyle p=0.5}$ , we can arbitrarily set ${\displaystyle p={\overline {p}}=0.5}$ , calculate ${\displaystyle \sigma _{P}}$ , ${\displaystyle \sigma _{\overline {p}}}$ , and ${\displaystyle z_{\gamma }\sigma _{\overline {p}}}$ to obtain the maximum margin of error for ${\displaystyle P}$ at a given confidence level ${\displaystyle \gamma }$ and sample size ${\displaystyle n}$ , even before having actual results.  With ${\displaystyle p=0.5,n=1013}$

${\displaystyle MOE_{95}(0.5)=z_{0.95}\sigma _{\overline {p}}\approx z_{0.95}{\sqrt {\frac {\sigma _{P}^{2}}{n}}}=1.96{\sqrt {\frac {.25}{n}}}=0.98/{\sqrt {n}}=\pm 3.1\%}$

${\displaystyle MOE_{99}(0.5)=z_{0.99}\sigma _{\overline {p}}\approx z_{0.99}{\sqrt {\frac {\sigma _{P}^{2}}{n}}}=2.58{\sqrt {\frac {.25}{n}}}=1.29/{\sqrt {n}}=\pm 4.1\%}$

Also, usefully, for any reported ${\displaystyle MOE_{95}}$

${\displaystyle MOE_{99}={\frac {z_{0.99}}{z_{0.95}}}MOE_{95}\approx 1.3\times MOE_{95}}$

Specific margins of error [edit]

If a poll has multiple percentage results (for example, a poll measuring a single multiple-choice preference), the result closest to 50% will have the highest margin of error. Typically, it is this number that is reported as the margin of error for the entire poll. Imagine poll ${\displaystyle P}$ reports ${\displaystyle p_{a},p_{b},p_{c}}$ as ${\displaystyle 71\%,27\%,2\%,n=1013}$

${\displaystyle MOE_{95}(P_{a})=z_{0.95}\sigma _{\overline {p_{a}}}\approx 1.96{\sqrt {\frac {p_{a}(1-p_{a})}{n}}}=0.89/{\sqrt {n}}=\pm 2.8\%}$ (as in the figure above)

${\displaystyle MOE_{95}(P_{b})=z_{0.95}\sigma _{\overline {p_{b}}}\approx 1.96{\sqrt {\frac {p_{b}(1-p_{b})}{n}}}=0.87/{\sqrt {n}}=\pm 2.7\%}$

${\displaystyle MOE_{95}(P_{c})=z_{0.95}\sigma _{\overline {p_{c}}}\approx 1.96{\sqrt {\frac {p_{c}(1-p_{c})}{n}}}=0.27/{\sqrt {n}}=\pm 0.8\%}$

As a given percentage approaches the extremes of 0% or 100%, its margin of error approaches ±0%.

Comparing percentages [edit]

Imagine multiple-choice poll ${\displaystyle P}$ reports ${\displaystyle p_{a},p_{b},p_{c}}$ as ${\displaystyle 46\%,42\%,12\%,n=1013}$ . As described above, the margin of error reported for the poll would typically be ${\displaystyle MOE_{95}(P_{a})}$ , as ${\displaystyle p_{a}}$ is closest to 50%. The popular notion of statistical tie or statistical dead heat, however, concerns itself not with the accuracy of the individual results, but with that of the ranking of the results. Which is in first?

If, hypothetically, we were to conduct poll ${\displaystyle P}$ over subsequent samples of ${\displaystyle n}$ respondents (newly drawn from ${\displaystyle N}$ ), and report result ${\displaystyle p_{w}=p_{a}-p_{b}}$ , we could use the standard error of difference to understand how ${\displaystyle p_{w_{1}},p_{w_{2}},p_{w_{3}},\ldots }$ is expected to fall about ${\displaystyle {\overline {p_{w}}}}$ . For this, we need to apply the sum of variances to obtain a new variance, ${\displaystyle \sigma _{P_{w}}^{2}}$ ,

${\displaystyle \sigma _{P_{w}}^{2}=\sigma _{P_{a}-P_{b}}^{2}=\sigma _{P_{a}}^{2}+\sigma _{P_{b}}^{2}-2\sigma _{P_{a},P_{b}}=p_{a}(1-p_{a})+p_{b}(1-p_{b})+2p_{a}p_{b}}$

where ${\displaystyle \sigma _{P_{a},P_{b}}=-P_{a}P_{b}}$ is the covariance of ${\displaystyle P_{a}}$ and ${\displaystyle P_{b}}$ .

Thus (after simplifying),

${\displaystyle {\text{Standard error of difference}}=\sigma _{\overline {w}}\approx {\sqrt {\frac {\sigma _{P_{w}}^{2}}{n}}}={\sqrt {\frac {p_{a}+p_{b}-(p_{a}-p_{b})^{2}}{n}}}=0.029,P_{w}=P_{a}-P_{b}}$

${\displaystyle MOE_{95}(P_{a})=z_{0.95}\sigma _{\overline {p_{a}}}\approx \pm {3.1\%}}$

${\displaystyle MOE_{95}(P_{w})=z_{0.95}\sigma _{\overline {w}}\approx \pm {5.8\%}}$

Note that this assumes that ${\displaystyle P_{c}}$ is close to constant, that is, respondents choosing either A or B would almost never chose C (making ${\displaystyle P_{a}}$ and ${\displaystyle P_{b}}$ close to perfectly negatively correlated). With three or more choices in closer contention, choosing a correct formula for ${\displaystyle \sigma _{P_{w}}^{2}}$ becomes more complicated.

Effect of finite population size [edit]

The formulae above for the margin of error assume that there is an infinitely large population and thus do not depend on the size of population ${\displaystyle N}$ , but only on the sample size ${\displaystyle n}$ . According to sampling theory, this assumption is reasonable when the sampling fraction is small. The margin of error for a particular sampling method is essentially the same regardless of whether the population of interest is the size of a school, city, state, or country, as long as the sampling fraction is small.

In cases where the sampling fraction is larger (in practice, greater than 5%), analysts might adjust the margin of error using a finite population correction to account for the added precision gained by sampling a much larger percentage of the population. FPC can be calculated using the formula^[1]

${\displaystyle \operatorname {FPC} ={\sqrt {\frac {N-n}{N-1}}}}$

...and so if poll ${\displaystyle P}$ were conducted over 24% of, say, an electorate of 300,000 voters

${\displaystyle MOE_{95}(0.5)=z_{0.95}\sigma _{\overline {p}}\approx {\frac {0.98}{\sqrt {72,000}}}=\pm 0.4\%}$

${\displaystyle MOE_{95_{FPC}}(0.5)=z_{0.95}\sigma _{\overline {p}}{\sqrt {\frac {N-n}{N-1}}}\approx {\frac {0.98}{\sqrt {72,000}}}{\sqrt {\frac {300,000-72,000}{300,000-1}}}=\pm 0.3\%}$

Intuitively, for appropriately large ${\displaystyle N}$ ,

${\displaystyle \lim _{n\to 0}{\sqrt {\frac {N-n}{N-1}}}\approx 1}$

${\displaystyle \lim _{n\to N}{\sqrt {\frac {N-n}{N-1}}}=0}$

In the former case, ${\displaystyle n}$ is so small as to require no correction. In the latter case, the poll effectively becomes a census and sampling error becomes moot.

See also [edit]

• Engineering tolerance
• Key relevance
• Measurement uncertainty
• Observational error
• Random error

Notes [edit]

1. ^ Isserlis, L. (1918). "On the value of a mean as calculated from a sample". Journal of the Royal Statistical Society. Blackwell Publishing. 81 (1): 75–81. doi:10.2307/2340569. JSTOR 2340569. (Equation 1)

References [edit]

• Sudman, Seymour and Bradburn, Norman (1982). Asking Questions: A Practical Guide to Questionnaire Design. San Francisco: Jossey Bass. ISBN 0-87589-546-8
• Wonnacott, T.H. and R.J. Wonnacott (1990). Introductory Statistics (5th ed.). Wiley. ISBN0-471-61518-8.

External links [edit]

• "Errors, theory of", Encyclopedia of Mathematics, EMS Press, 2001 [1994]
• Weisstein, Eric W. "Margin of Error". MathWorld.

When Using A Significance Test Calculator, What Is The "margin Of Error" Called?

Source: https://en.wikipedia.org/wiki/Margin_of_error

Posted by: davingoetted84.blogspot.com

Share this post