{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [],
"source": [
"# Install datascience package if needed\n",
"try:\n",
" import datascience\n",
"except ImportError:\n",
" import micropip\n",
" await micropip.install('datascience')\n",
"from datascience import *\n",
"path_data = '../../../assets/data/'\n",
"import matplotlib\n",
"matplotlib.use('Agg')\n",
"%matplotlib inline\n",
"import matplotlib.pyplot as plots\n",
"plots.style.use('fivethirtyeight')\n",
"import numpy as np"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Random Sampling in Python \n",
"\n",
"This section summarizes the ways you have learned to sample at random using Python, and introduces a new way.\n",
"\n",
"## Review: Sampling from a Population in a Table \n",
"If you are sampling from a population of individuals whose data are represented in the rows of a table, then you can use the Table method `sample` to [randomly select rows](https://inferentialthinking.com/chapters/10/1/Empirical_Distributions.html#id1) of the table. That is, you can use `sample` to select a random sample of individuals.\n",
"\n",
"By default, `sample` draws uniformly at random with replacement. This is a natural model for chance experiments such as rolling a die."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr>\n",
" <th>Face</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>1 </td>\n",
" </tr>\n",
" <tr>\n",
" <td>2 </td>\n",
" </tr>\n",
" <tr>\n",
" <td>3 </td>\n",
" </tr>\n",
" <tr>\n",
" <td>4 </td>\n",
" </tr>\n",
" <tr>\n",
" <td>5 </td>\n",
" </tr>\n",
" <tr>\n",
" <td>6 </td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"Face\n",
"1\n",
"2\n",
"3\n",
"4\n",
"5\n",
"6"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"faces = np.arange(1, 7)\n",
"die = Table().with_columns('Face', faces)\n",
"die"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Run the cell below to simulate 7 rolls of a die."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr>\n",
" <th>Face</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>5 </td>\n",
" </tr>\n",
" <tr>\n",
" <td>3 </td>\n",
" </tr>\n",
" <tr>\n",
" <td>3 </td>\n",
" </tr>\n",
" <tr>\n",
" <td>5 </td>\n",
" </tr>\n",
" <tr>\n",
" <td>5 </td>\n",
" </tr>\n",
" <tr>\n",
" <td>1 </td>\n",
" </tr>\n",
" <tr>\n",
" <td>6 </td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"Face\n",
"5\n",
"3\n",
"3\n",
"5\n",
"5\n",
"1\n",
"6"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"die.sample(7)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Sometimes it is more natural to sample individuals at random without replacement. This is called a simple random sample. The argument `with_replacement=False` allows you to do this."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr>\n",
" <th>Actor</th> <th>Total Gross</th> <th>Number of Movies</th> <th>Average per Movie</th> <th>#1 Movie</th> <th>Gross</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>Harrison Ford </td> <td>4871.7 </td> <td>41 </td> <td>118.8 </td> <td>Star Wars: The Force Awakens</td> <td>936.7</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Samuel L. Jackson </td> <td>4772.8 </td> <td>69 </td> <td>69.2 </td> <td>The Avengers </td> <td>623.4</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Morgan Freeman </td> <td>4468.3 </td> <td>61 </td> <td>73.3 </td> <td>The Dark Knight </td> <td>534.9</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Tom Hanks </td> <td>4340.8 </td> <td>44 </td> <td>98.7 </td> <td>Toy Story 3 </td> <td>415 </td>\n",
" </tr>\n",
" <tr>\n",
" <td>Robert Downey, Jr.</td> <td>3947.3 </td> <td>53 </td> <td>74.5 </td> <td>The Avengers </td> <td>623.4</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Eddie Murphy </td> <td>3810.4 </td> <td>38 </td> <td>100.3 </td> <td>Shrek 2 </td> <td>441.2</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Tom Cruise </td> <td>3587.2 </td> <td>36 </td> <td>99.6 </td> <td>War of the Worlds </td> <td>234.3</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Johnny Depp </td> <td>3368.6 </td> <td>45 </td> <td>74.9 </td> <td>Dead Man's Chest </td> <td>423.3</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Michael Caine </td> <td>3351.5 </td> <td>58 </td> <td>57.8 </td> <td>The Dark Knight </td> <td>534.9</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Scarlett Johansson</td> <td>3341.2 </td> <td>37 </td> <td>90.3 </td> <td>The Avengers </td> <td>623.4</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>... (40 rows omitted)</p>"
],
"text/plain": [
"Actor | Total Gross | Number of Movies | Average per Movie | #1 Movie | Gross\n",
"Harrison Ford | 4871.7 | 41 | 118.8 | Star Wars: The Force Awakens | 936.7\n",
"Samuel L. Jackson | 4772.8 | 69 | 69.2 | The Avengers | 623.4\n",
"Morgan Freeman | 4468.3 | 61 | 73.3 | The Dark Knight | 534.9\n",
"Tom Hanks | 4340.8 | 44 | 98.7 | Toy Story 3 | 415\n",
"Robert Downey, Jr. | 3947.3 | 53 | 74.5 | The Avengers | 623.4\n",
"Eddie Murphy | 3810.4 | 38 | 100.3 | Shrek 2 | 441.2\n",
"Tom Cruise | 3587.2 | 36 | 99.6 | War of the Worlds | 234.3\n",
"Johnny Depp | 3368.6 | 45 | 74.9 | Dead Man's Chest | 423.3\n",
"Michael Caine | 3351.5 | 58 | 57.8 | The Dark Knight | 534.9\n",
"Scarlett Johansson | 3341.2 | 37 | 90.3 | The Avengers | 623.4\n",
"... (40 rows omitted)"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"actors = Table.read_table(path_data + 'actors.csv')\n",
"actors"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr>\n",
" <th>Actor</th> <th>Total Gross</th> <th>Number of Movies</th> <th>Average per Movie</th> <th>#1 Movie</th> <th>Gross</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <td>Morgan Freeman </td> <td>4468.3 </td> <td>61 </td> <td>73.3 </td> <td>The Dark Knight </td> <td>534.9</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Orlando Bloom </td> <td>2815.8 </td> <td>17 </td> <td>165.6 </td> <td>Dead Man's Chest</td> <td>423.3</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Cameron Diaz </td> <td>3031.7 </td> <td>34 </td> <td>89.2 </td> <td>Shrek 2 </td> <td>441.2</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Michael Caine </td> <td>3351.5 </td> <td>58 </td> <td>57.8 </td> <td>The Dark Knight </td> <td>534.9</td>\n",
" </tr>\n",
" <tr>\n",
" <td>Leonardo DiCaprio</td> <td>2518.3 </td> <td>25 </td> <td>100.7 </td> <td>Titanic </td> <td>658.7</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>"
],
"text/plain": [
"Actor | Total Gross | Number of Movies | Average per Movie | #1 Movie | Gross\n",
"Morgan Freeman | 4468.3 | 61 | 73.3 | The Dark Knight | 534.9\n",
"Orlando Bloom | 2815.8 | 17 | 165.6 | Dead Man's Chest | 423.3\n",
"Cameron Diaz | 3031.7 | 34 | 89.2 | Shrek 2 | 441.2\n",
"Michael Caine | 3351.5 | 58 | 57.8 | The Dark Knight | 534.9\n",
"Leonardo DiCaprio | 2518.3 | 25 | 100.7 | Titanic | 658.7"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Simple random sample of 5 rows\n",
"actors.sample(5, with_replacement=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Since `sample` gives you the entire sample in the order in which the rows were selected, you can use Table methods on the sampled table to answer many questions about the sample. For example, you can find the number of times the die showed six spots, or the average number of movies in which the sampled actors appeared, or whether one two specified actors appeared in the sample. You might need multiple lines of code to get some of this information."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Review: Sampling from a Population in an Array \n",
"\n",
"If you are sampling from a population of individuals whose data are represented as an array, you can use the NumPy function `np.random.choice` to [randomly select elements of the array](https://inferentialthinking.com/chapters/09/3/Simulation.html#example-number-of-heads-in-100-tosses).\n",
"\n",
"By default, `np.random.choice` samples at random with replacement."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([1, 2, 3, 4, 5, 6])"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# The faces of a die, as an array\n",
"faces"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([4, 1, 6, 3, 5, 4, 6])"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# 7 rolls of the die\n",
"np.random.choice(faces, 7)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The argument `replace=False` allows you to get a simple random sample, that is, a sample drawn at random without replacement."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"# Array of actor names\n",
"actor_names = actors.column('Actor')"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array(['Jonah Hill', 'Julia Roberts', 'Bruce Willis', 'Eddie Murphy',\n",
" 'Matt Damon'], dtype='<U22')"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Simple random sample of 5 actor names\n",
"np.random.choice(actor_names, 5, replace=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Just as `sample` did, so also `np.random.choice` gives you the entire sequence of sampled elements. You can use array operations to answer many questions about the sample. For example, you can find which actor was the second one to be drawn, or the number of faces of the die that appeared more than once. Some answers might need multiple lines of code."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Sampling from a Categorical Distribution \n",
"Sometimes we are interested in a categorical attribute of our sampled individuals. For example, we might be looking at whether a coin lands Heads or Tails. Or we might be interested in the political parties of randomly selected voters.\n",
"\n",
"In such cases, we frequently need the proportions of sampled voters in the different categories. If we have the entire sample, we can calculate these proportions. The function `sample_proportions` in the `datascience` library does that work for us. It is tailored for sampling at random with replacement from a categorical distribution and returns the proportions of sampled elements in each category.\n",
"\n",
"The `sample_proportions` function takes two arguments:\n",
"- the sample size\n",
"- the distribution of the categories in the population, as a list or array of proportions that add up to 1\n",
"\n",
"It returns an array containing the distribution of the categories in a random sample of the given size taken from the population. That's an array consisting of the sample proportions in all the different categories, in the same order in which they appeared in the population distribution.\n",
"\n",
"For example, suppose each plant of a species is red-flowering with chance 25%, pink-flowering with chance 50%, and white-flowering with chance 25%, regardless of the flower colors of all other plants. You can use `sample_proportions` to see the proportions of the different colors among 300 plants of the species."
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"array([0.24333333, 0.50333333, 0.25333333])"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Species distribution of flower colors:\n",
"# Proportions are in the order Red, Pink, White\n",
"species_proportions = [0.25, 0.5, .25]\n",
"\n",
"sample_size = 300\n",
"\n",
"# Distribution of sample\n",
"sample_distribution = sample_proportions(sample_size, species_proportions)\n",
"sample_distribution"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As you expect, the proportions in the sample sum to 1."
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1.0"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"sum(sample_distribution)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The categories in `species_proportions` are in the order Red, Pink, White. That order is preserved by `sample_proportions`. If you just want the proportion of pink-flowering plants in the sample, you can use `item`:"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0.5033333333333333"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Sample proportion of Heads\n",
"sample_distribution.item(1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can use `sample_proportions` and array operations to answer questions based only on the proportions of sampled individuals in the different categories. You will not be able to answer questions that require more detailed information about the sample, such as which of the sampled plants had each of the different colors."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.7"
}
},
"nbformat": 4,
"nbformat_minor": 4
}