Biased sample

In statistics, sampling bias is a bias in which a sample is collected in such a way that some members of the intended population have a lower or higher sampling probability than others. It results in a biased sample of a population (or non-human factors) in which all individuals, or instances, were not equally likely to have been selected. If this is not accounted for, results can be erroneously attributed to the phenomenon under study rather than to the method of sampling.

Sample X is taken from Population Y. Conclusion Z is drawn from sample X. It is assumed that Z is also true about Y.

Take a biased or otherwise statistically invalid sample. Analyze the data. Draw conclusions and declare the results significant.

Most people believe they are pretty good at making statistical assessments. In fact we are generally pretty poor at it, and there are many traps into which we fall. Taking an unrepresentative sample is one of the most basic of these.

Where a sample is deliberately biased by leaving out data, this is the Fallacy of Exclusion.