5 Homework 1

Author

Shao-Ting Chiu (UIN:433002162)

Published

January 20, 2023

5.1 Homework Description

Read Chapters 1 and 2 in the Hoff book. Then, do the following exercises in Hoff (p. 225-226): 2.1, 2.2, 2.3, 2.5 You must turn in your solutions as a pdf file here on Canvas. Use the Submit Assignment button on the top right. If your solutions are on paper, please scan them to pdf using a scanner or a scanner app on your phone. Please do not take a regular photo, as this can result in very large file sizes. Make sure that everything is legible. Please note that late homework will not be accepted and will result in a score of zero. To avoid late submissions due to technical issues, we recommend turning in your homework the night before the due date.

Deadline: Sep. 8 by 12:01pm
PDF version

5.2 Problem 2.1

Marginal and conditional probability: The social mobility data from Section 2.5 gives a joint probability distribution on $(Y_1, Y_2)=$ (fathers’s occupation, son’s occupation)

Table 5.1: The social mobility data (Hoff 2009, 580:24)
			son’s occupation
father’s occupation	farm	operatives	craftsmen	sales	professional
farm	0.018	0.035	0.031	0.008	0.018
operatives	0.002	0.112	0.064	0.032	0.069
craftsman	0.001	0.066	0.094	0.032	0.084
sales	0.001	0.018	0.019	0.010	0.051
professional	0.001	0.029	0.032	0.043	0.130

5.2.1 (a) The marginal probability distribution of a father’s occupation

According to Table 5.1, let $\mathbb{Y_1}$ and $\mathbb{Y_2}$ be sets of father’s and son’s occupations:

\[\mathbb{Y_1} = \mathbb{Y_2} = \{\text{farm},\text{operatives},\text{craftsmen},\text{sales},\text{professional}\}\]

\[ \begin{aligned} Pr(Y_1 = \text{farm}) &= \sum_{y_2\in\mathbb{Y_2}}Pr(Y_1= \text{farm} \cap Y_2=y_2)\\ &= Pr(Y_1= \text{farm} \cap Y_2=\text{farm}) + Pr(Y_1= \text{farm} \cap Y_2=\text{operatives}) \\ & +Pr(Y_1= \text{farm} \cap Y_2=\text{craftsmen}) + Pr(Y_1= \text{farm} \cap Y_2=\text{sales}) \\ & +Pr(Y_1= \text{farm} \cap Y_2=\text{professional})\\ &= 0.018 + 0.035 + 0.031 + 0.008 + 0.018\\ &= 0.11 \end{aligned} \]

\[ \begin{aligned} Pr(Y_1 = \text{operatives}) &= \sum_{y_2\in\mathbb{Y_2}}Pr(Y_1= \text{operatives} \cap Y_2=y_2)\\ &= Pr(Y_1= \text{operatives} \cap Y_2=\text{farm}) + Pr(Y_1= \text{operatives} \cap Y_2=\text{operatives}) \\ & +Pr(Y_1= \text{operatives} \cap Y_2=\text{craftsmen}) + Pr(Y_1= \text{operatives} \cap Y_2=\text{sales}) \\ & +Pr(Y_1= \text{operatives} \cap Y_2=\text{professional})\\ &= 0.002 + 0.112 + 0.064 + 0.032 + 0.069\\ &= 0.279 \end{aligned} \]

\[ \begin{aligned} Pr(Y_1 = \text{craftsman}) &= \sum_{y_2\in\mathbb{Y_2}}Pr(Y_1= \text{craftsman} \cap Y_2=y_2)\\ &= Pr(Y_1= \text{craftsman} \cap Y_2=\text{farm}) + Pr(Y_1= \text{craftsman} \cap Y_2=\text{operatives}) \\ & +Pr(Y_1= \text{craftsman} \cap Y_2=\text{craftsmen}) + Pr(Y_1= \text{craftsman} \cap Y_2=\text{sales}) \\ & +Pr(Y_1= \text{craftsman} \cap Y_2=\text{professional})\\ &= 0.001+0.066+0.094+0.032+0.084\\ &= 0.277 \end{aligned} \]

\[ \begin{aligned} Pr(Y_1 = \text{sales}) &= \sum_{y_2\in\mathbb{Y_2}}Pr(Y_1= \text{sales} \cap Y_2=y_2)\\ &= Pr(Y_1= \text{sales} \cap Y_2=\text{farm}) + Pr(Y_1= \text{sales} \cap Y_2=\text{operatives}) \\ & +Pr(Y_1= \text{sales} \cap Y_2=\text{craftsmen}) + Pr(Y_1= \text{sales} \cap Y_2=\text{sales}) \\ & +Pr(Y_1= \text{sales} \cap Y_2=\text{professional})\\ &= 0.001 + 0.018 + 0.019 + 0.010 + 0.051\\ &= 0.099 \end{aligned} \]

\[ \begin{aligned} Pr(Y_1 = \text{professional}) &= \sum_{y_2\in\mathbb{Y_2}}Pr(Y_1= \text{professional} \cap Y_2=y_2)\\ &= Pr(Y_1= \text{professional} \cap Y_2=\text{farm}) + Pr(Y_1= \text{professional} \cap Y_2=\text{operatives}) \\ & +Pr(Y_1= \text{professional} \cap Y_2=\text{craftsmen}) + Pr(Y_1= \text{professional} \cap Y_2=\text{sales}) \\ & +Pr(Y_1= \text{professional} \cap Y_2=\text{professional})\\ &= 0.001 + 0.029 + 0.032 + 0.043 + 0.130\\ &= 0.235 \end{aligned} \]

Table 5.2: Marginal probability of father’s occupation
marginal probability	value
$p(Y_1=\text{farm})$	0.11
$p(Y_1=\text{operatives})$	0.279
$p(Y_1=\text{craftsmen})$	0.277
$p(Y_1=\text{sales})$	0.099
$p(Y_1=\text{professional})$	0.235
SUM	1.0

Table 5.2 shows that the sum of marginal probability is $1$.

5.2.1.0.1 (b) The marginal probability distribution of a son’s occupation

\[ \begin{aligned} Pr(Y_2 = \text{farm}) &= \sum_{y_1\in\mathbb{Y_1}}Pr(Y_2= \text{farm} \cap Y_1=y_1)\\ &= Pr(Y_2= \text{farm} \cap Y_1=\text{farm}) + Pr(Y_2= \text{farm} \cap Y_1=\text{operatives}) \\ & +Pr(Y_2= \text{farm} \cap Y_1=\text{craftsmen}) + Pr(Y_2= \text{farm} \cap Y_1=\text{sales}) \\ & +Pr(Y_2= \text{farm} \cap Y_1=\text{professional})\\ &= 0.018 + 0.002 + 0.001 + 0.001 + 0.001\\ &= 0.023 \end{aligned} \]

\[ \begin{aligned} Pr(Y_2 = \text{operatives}) &= \sum_{y_1\in\mathbb{Y_1}}Pr(Y_2= \text{operatives} \cap Y_1=y_1)\\ &= Pr(Y_2= \text{operatives} \cap Y_1=\text{farm}) + Pr(Y_2= \text{operatives} \cap Y_1=\text{operatives}) \\ & +Pr(Y_2= \text{operatives} \cap Y_1=\text{craftsmen}) + Pr(Y_2= \text{operatives} \cap Y_1=\text{sales}) \\ & +Pr(Y_2= \text{operatives} \cap Y_1=\text{professional})\\ &= 0.035 + 0.112 + 0.066 + 0.018 + 0.029\\ &= 0.26 \end{aligned} \]

\[ \begin{aligned} Pr(Y_2 = \text{craftsmen}) &= \sum_{y_1\in\mathbb{Y_1}}Pr(Y_2= \text{craftsmen} \cap Y_1=y_1)\\ &= Pr(Y_2= \text{craftsmen} \cap Y_1=\text{farm}) + Pr(Y_2= \text{craftsmen} \cap Y_1=\text{operatives}) \\ & +Pr(Y_2= \text{craftsmen} \cap Y_1=\text{craftsmen}) + Pr(Y_2= \text{craftsmen} \cap Y_1=\text{sales}) \\ & +Pr(Y_2= \text{craftsmen} \cap Y_1=\text{professional})\\ &= 0.031 + 0.064 + 0.094 + 0.019 + 0.032\\ &= 0.24 \end{aligned} \]

\[ \begin{aligned} Pr(Y_2 = \text{sales}) &= \sum_{y_1\in\mathbb{Y_1}}Pr(Y_2= \text{sales} \cap Y_1=y_1)\\ &= Pr(Y_2= \text{sales} \cap Y_1=\text{farm}) + Pr(Y_2= \text{sales} \cap Y_1=\text{operatives}) \\ & +Pr(Y_2= \text{sales} \cap Y_1=\text{craftsmen}) + Pr(Y_2= \text{sales} \cap Y_1=\text{sales}) \\ & +Pr(Y_2= \text{sales} \cap Y_1=\text{professional})\\ &= 0.008+0.032+0.032+0.010+0.043\\ &= 0.125 \end{aligned} \]

\[ \begin{aligned} Pr(Y_2 = \text{professional}) &= \sum_{y_1\in\mathbb{Y_1}}Pr(Y_2= \text{professional} \cap Y_1=y_1)\\ &= Pr(Y_2= \text{professional} \cap Y_1=\text{farm}) + Pr(Y_2= \text{professional} \cap Y_1=\text{operatives}) \\ & +Pr(Y_2= \text{professional} \cap Y_1=\text{craftsmen}) + Pr(Y_2= \text{professional} \cap Y_1=\text{sales}) \\ & +Pr(Y_2= \text{professional} \cap Y_1=\text{professional})\\ &= 0.018+ 0.069+0.084+0.051+0.130\\ &= 0.352 \end{aligned} \]

Table 5.3: Marginal probability of son’s occupation
marginal probability	value
$p(Y_2=\text{farm})$	0.023
$p(Y_2=\text{operatives})$	0.26
$p(Y_2=\text{craftsmen})$	0.24
$p(Y_2=\text{sales})$	0.125
$p(Y_2=\text{professional})$	0.352
SUM	1.0

Table 5.3 shows that the sum of marginal probability is $1$.

5.2.2 (c) The conditional distribution of a son’s occupation, given that the father is a farmer;

The conditional distribution of a son’s occupation can be expressed as $p(y_2 | y_1 = \text{farmer})$.

\[p(y_2= * | y_1 = \text{farmer}) = \frac{p(y_1=\text{farm} \cap y_2 = *)}{p(y_1 = \text{farm})}\]

where $* \in \mathbb{Y_2}$. As described in Table 5.2, $p(y_1=\text{farm})= 0.11$. Use Table 5.1 to calculate the distribution:

\[ \begin{aligned} p(y_2 = \text{farm} | y_1 = \text{farm}) &= \frac{0.018}{0.11} \approx 0.16\\ p(y_2 = \text{operatives} | y_1 = \text{farm}) &= \frac{0.035}{0.11} \approx 0.32\\ p(y_2 = \text{craftsman} | y_1 = \text{farm}) &= \frac{0.031}{0.11} \approx 0.28\\ p(y_2 = \text{sales} | y_1 = \text{farm}) &= \frac{0.008}{0.11} \approx 0.072\\ p(y_2 = \text{professional} | y_1 = \text{farm}) &= \frac{0.018}{0.11} \approx 0.16 \\ \end{aligned} \]

5.2.3 (d) The conditional distribution of a father’s occupation, given that the son is a farmer.

\[p(y_1= * | y_2 = \text{farm}) = \frac{p(y_1=\text{*} \cap y_2 = \text{farm})}{p(y_2 = \text{farm})}\]

According to Table 5.3, $p(y_2 = \text{farm}) = 0.023$. Use Table 5.1 to calculate the distribution:

\[ \begin{aligned} p(y_1 = \text{farm} | y_2 = \text{farm}) &= \frac{0.018}{0.023} \approx 0.78\\ p(y_1 = \text{operatives} | y_2 = \text{farm}) &=\frac{0.002}{0.023} \approx 0.09\\ p(y_1 = \text{craftsman} | y_2 = \text{farm}) &= \frac{0.001}{0.023} \approx 0.04\\ p(y_1 = \text{sales} | y_2 = \text{farm}) &= \frac{0.001}{0.023} \approx 0.04\\ p(y_1 = \text{professional} | y_2 = \text{farm}) &= \frac{0.001}{0.023} \approx 0.04\\ \end{aligned} \]

5.3 Problem 2.2

Expectations and variances: Let $Y_1$ and $Y_2$ be two independent random variables, such that $E[Y_i]=\mu_i$ and $Var[Y_i] = \sigma_{i}^2$. Using the definition of expectation and variance, computing the following quantities, where $a_1$ and $a_2$ are given constants.

Because $Y_1$ and $Y_2$ are independent,

5.3.1 (a) $E[a_1 Y_1 +a_2 Y_2]$, $Var[a_1 Y_1 + a_2 Y_2]$

Because $Y_1$ and $Y_2$ are independent: \[ E[Y_1 Y_2] = E[Y_1] E[Y_2] \]

Thus

\[ \begin{aligned} E[a_1 Y_1 + a_2 Y_2] &= E[a_1 Y_1] + E[a_2 Y_2]\\ &= a_1 E[Y_1] + a_2 E[Y_2]\\ &= \underline{a_1 \mu_1 + a_2 \mu_{2}}\\ \end{aligned} \]

\[ \begin{aligned} Var[a_1 Y_1 + a_2 Y_2] &= E[ [(a_1 Y_1 + a_2 Y_2) - E[a_1 Y_1 + a_2 Y_2]]^{2}]\\ &= E[(a_1 Y_1 + a_2 Y_2)^2] - E[a_1 Y_1 + a_2 Y_2]^2\\ &= E[a_{1}^{2}Y_{1}^{2} + 2a_1 a_2 Y_1 Y_2 + a_{2}^{2}Y_{2}^{2}] - (a_1 \mu_1 + a_2 \mu_2)^2\\ &= a^{2}_{1}E[Y_1^2] + 2a_1 a_2 \underbrace{\mu_1 \mu_2}_{E[Y_1 Y_2]=E[Y_1]E[Y_2]\quad \because \text{Indep.}} + a_{2}^{2}E[Y_{2}^{2}] - (a_1 \mu_1 + a_2 \mu_2)^2\\ &= a^{2}_{1}E[Y_1^2] + 2a_1 a_2\mu_1 \mu_2 + a_{2}^{2}E[Y_{2}^{2}] - (a_{1}^{2} \mu_{1}^{2}+ 2a_{1}a_{2}\mu_{1}\mu_{2} + a_{2}^{2} \mu_{2}^{2})\\ &= a_{1}^{2}(\underbrace{E[Y_{1}^2] - \mu_{1}^{2}}_{\sigma_{1}^{2}}) + a_{2}^{2}(\underbrace{E[Y^{2}_{2}] - \mu_{2}^{2}}_{\sigma_{2}^{2}})\\ &= \underline{a_{1}^{2}\sigma_{1}^{2} + a_{2}^{2}\sigma_{2}^{2}}\\ \end{aligned} \]

5.3.2 (b) $E[a_1 Y_1 - a_2 Y_2]$, $Var[a_1 Y_1 - a_2 Y_2]$

\[ \begin{aligned} E[a_1 Y_1 - a_2 Y_2] &= E[a_1 Y_1] - E[a_2 Y_2]\\ &= a_1 E[Y_1] - a_2 E[Y_2]\\ &= \underline{a_1 \mu_1 - a_2 \mu_{2}}\\ \end{aligned} \]

\[ \begin{aligned} Var[a_1 Y_1 - a_2 Y_2] &= E[ [(a_1 Y_1 - a_2 Y_2) - E[a_1 Y_1 - a_2 Y_2]]^{2}]\\ &= E[(a_1 Y_1 - a_2 Y_2)^2] - E[a_1 Y_1 - a_2 Y_2]^2\\ &= E[a_{1}^{2}Y_{1}^{2} - 2a_1 a_2 Y_1 Y_2 + a_{2}^{2}Y_{2}^{2}] - (a_1 \mu_1 - a_2 \mu_2)^2\\ &= a_{1}^{2}E[Y^{2}_{1}] + a_{2}^{2}E[Y^{2}_{2}] - 2a_1 2_2 \underbrace{E[Y_1Y_2]}_{=E[Y_1]E[Y_2]\quad\because \text{Indep.}} - (a_1 \mu_1 - a_2 \mu_2)^2\\ &= a_{1}^{2}E[Y^{2}_{1}] + a_{2}^{2}E[Y^{2}_{2}] - 2a_1 2_2 \mu_1\mu_2 - (a_{1}^2 \mu_{1}^2 - 2 a_{1}a_{2}\mu_1 \mu_2 + a_{2}^{2} \mu_{2}^{2})\\ &= a_{1}^{2}(\underbrace{E[Y_{1}^2] - \mu_{1}^{2}}_{\sigma_{1}^{2}}) + a_{2}^{2}(\underbrace{E[Y^{2}_{2}] - \mu_{2}^{2}}_{\sigma_{2}^{2}})\\ &= \underbrace{a_{1}^{2}\sigma_{1}^{2} + a_{2}^{2}\sigma^{2}_{2}} \end{aligned} \]

5.4 Problem 2.3

Full conditionals: Let $X$, $Y$, $Z$ be random variables with joint density (discrete or continuous) $p(x,y,z) \propto f(x,z)g(y,z)h(z)$. Show that

5.4.1 (a) $p(x|y,z) \propto f(x,z)$ i.e. $p(x|y,z)$ is a function of $x$ and $z$;

Let $c, d\in \mathbb{R}$ constants \[ \begin{aligned} p(x|y,z) &= \frac{p(x,y,z)}{p(y,z)}\\ &= \frac{c\cdot f(x,z)g(y,z)h(z)}{p(y,z)} &= \frac{c\cdot f(x,z)g(y,z)h(z)}{\int_{x\in\mathbb{X}} p(x,y,z)dx}\\ &= \frac{c\cdot f(x,z)g(y,z)h(z)}{d\cdot \int_{x\in \mathbb{X}} f(x,z)g(y,z)h(z) dx}\\ &= \frac{c\cdot f(x,z)g(y,z)h(z)}{d\cdot g(y,z)h(z)\int_{x\in \mathbb{X}} f(x,z) dx}\\ &= \frac{c\cdot f(x,z)}{d\cdot\int_{x\in \mathbb{X}} f(x,z) dx}\\ &\propto \underline{\frac{f(x,z)}{\int_{x\in \mathbb{X}} f(x,z) dx}} \end{aligned} \]

Thus, $p(x|y,z)$ is a function of $f(x,z)$.

5.4.2 (b) $p(y|x,z) \propto g(y,z)$ i.e. $p(y|x,z)$ is a function of $y$ and $z$;

Let $c, d\in \mathbb{R}$ constants \[ \begin{aligned} p(y|x,z) &= \frac{p(x,y,z)}{p(x,z)}\\ &= \frac{p(x,y,z)}{\int_{y\in \mathbb{Y}}p(x,y,z)dy}\\ &= \frac{c\cdot f(x,z)g(y,z)h(z)}{d\cdot \int_{y\in\mathbb{Y}} f(x,z)g(y,z)h(z)dy}\\ &= \frac{c\cdot f(x,z)g(y,z)h(z)}{d\cdot f(x,z)h(z)\int_{y\in\mathbb{Y}} g(y,z)dy}\\ &\propto \frac{f(y,z)}{\int_{y\in\mathbb{Y}}g(y,z)dy}\\ \end{aligned} \]

5.4.3 (c) $X$ and $Y$ are conditionally independent given $Z$.

Let $a_1, a_2 \in \mathbb{R}$ constant,

\[ \begin{aligned} p(x|z) &= \frac{p(x,z)}{p(z)}\\ &= \frac{\int_{y\in\mathbb{Y}}p(x,y,z)dy}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} p(x,y,z) dydx}\\ &= \frac{\int_{y\in\mathbb{Y}}f(x,z)g(y,z)h(z) dy}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} p(x,y,z) dydx}\\ &= \frac{f(x,z)h(z)\int_{y\in\mathbb{Y}}g(y,z)dy}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} p(x,y,z) dydx}\\ &= \frac{f(x,z)\int_{y\in\mathbb{Y}}g(y,z)dy}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} f(x,z)g(y,z) dydx} \end{aligned} \]

\[ \begin{aligned} p(y|z) &= \frac{p(y,z)}{p(z)}\\ &= \frac{\int_{x\in\mathbb{X}}p(x,y,z)dx}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} p(x,y,z) dydx}\\ &= \frac{\int_{x\in \mathbb{X}} f(x,z)g(y,z)h(z) dx}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} p(x,y,z) dydx}\\ &= \frac{g(y,z)h(z)\int_{x\in \mathbb{X}} f(x,z)dx}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} p(x,y,z) dydx}\\ &= \frac{g(y,z)\int_{x\in \mathbb{X}} f(x,z)dx}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} f(x,z)g(y,z) dydx}\\ \end{aligned} \]

\[ \begin{aligned} p(x|z)p(y|z) &= \frac{f(x,z)\int_{y\in\mathbb{Y}}g(y,z)dy}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} f(x,z)g(y,z) dydx} \cdot \frac{g(y,z)\int_{x\in \mathbb{X}} f(x,z)dx}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} f(x,z)g(y,z) dydx}\\ &= \frac{f(x,z)g(y,z)}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} f(x,z)g(y,z) dydx}\\ \end{aligned} \]

\[ \begin{aligned} p(x,y|z) &= \frac{p(x,y,z)}{p(z)}\\ &= \frac{f(x,z)g(y,z)h(z)}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} p(x,y,z) dy dx}\\ &= \frac{f(x,z)g(y,z)h(z)}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} f(x,z)g(y,z)h(z) dy dx}\\ &= \frac{f(x,z)g(y,z)h(z)}{\int_{x\in\mathbb{X}}f(x,z)h(z)\left[\int_{y\in\mathbb{Y}} g(y,z) dy\right] dx}\\ &= \frac{f(x,z)g(y,z)h(z)}{\int_{x\in\mathbb{X}}f(x,z)h(z)dx \cdot \int_{y\in\mathbb{Y}} g(y,z) dy}\\ &= \frac{f(x,z)}{\int_{x\in\mathbb{X}}f(x,z)dx}\frac{g(y,z)}{\int_{y\in\mathbb{Y}} g(y,z) dy}\\ &= \frac{f(x,z)g(y,z)}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} f(x,z)g(y,z) dydx}\\ &= \underline{p(x|z)p(y|z)}\\ \end{aligned} \]

Thus, $p(x,y|z) = p(x|z)p(y|z)$ that means $p(x,y|z)$ is conditionally independent given $z$.

5.5 Problem 2.5

Urns: Suppose urn $H$ is filled with $40\%$ green balls and $60\%$ red balls, and urn $T$ is filled with $60\%$ green balls and $40\%$ red balls. Someone will flip a coin and then select a ball from urn $H$ or urn $T$ depending on whether the coin lands heads or tails, respectively. Let $X$ be 1 or 0 if the coin lands heads or tails, and let $Y$ be $1$ or $0$ if the ball is green or red.

Table 5.4: Probability of choosing a certain ball in a given urn.
	Green	Red
$H$ (chosen if head[1])	0.4	0.6
$T$ (chosen if tail[0])	0.6	0.4

Event coding
	1	0
$X$	Head	Tail
$Y$	Green	Red

5.5.1 (a) Write out the joint distribution of $X$ and $Y$ in a table.

Suppose the coin is fair,

\[ \begin{aligned} p(X=0 \cap Y=0) &= p(X=0) p(Y=0|X=0)= 0.5\cdot 0.4 = 0.2\\ p(X=0 \cap Y=1) &= p(X=0) p(Y=1|X=0)= 0.5\cdot 0.6 = 0.3\\ p(X=1 \cap Y=0) &= p(X=1) p(Y=0|X=1)= 0.5\cdot 0.6 = 0.3\\ p(X=1 \cap Y=1) &= p(X=1) p(Y=1|X=1)= 0.5\cdot 0.4 = 0.2\\ \end{aligned} \]

5.5.2 (b) Find $E[Y]$. What is the probability that the ball is green?

\[ \begin{aligned} E[Y] &= \sum_{y\in \{0,1\}} p(Y=y)y\\ &= p(Y=1)\cdot 1\\ &= \sum_{x\in\{0,1\}} p(Y=1 | X=x)p(X=x)\\ &= p(Y=1|X=0)p(X=0) + p(Y=1|X=1)p(X=1)\\ &= 0.6\cdot 0.5 + 0.4 \cdot 0.5 \\ &= 0.3 + 0.2 \\ &= 0.5 \end{aligned} \]

5.5.3 (c) Find $Var[Y|X=0]$, $Var[Y|X=1]$ and $Var[Y]$. Thinking of variance as measuring uncertainty, explain intuitively why one of these variances is larger than others.

\[ \begin{aligned} E[Y|X=0] &= \sum_{y \in\{0,1\}} P_{Y|X=0}(Y=y|X=0)y\\ &= P_{Y|X=0}(Y=1|X=0)\cdot 1 \\ &= 0.6 \end{aligned} \]

\[ \begin{aligned} E[(Y|X=0)^2] &= \sum_{y \in\{0,1\}} P_{Y|X=0}(Y=y|X=0)y^2\\ &= P_{Y|X=0}(Y=1|X=0)\cdot 1 \\ &= 0.6 \end{aligned} \]

\[ \begin{aligned} E[Y|X=1] &= \sum_{y \in\{0,1\}} P_{Y|X=1}(Y=y|X=1)y\\ &= P_{Y|X=1}(Y=1|X=1)\cdot 1 \\ &= 0.4 \end{aligned} \]

\[ \begin{aligned} E[(Y|X=1)^2] &= \sum_{y \in\{0,1\}} P_{Y|X=0}(Y=y|X=1)y^2\\ &= P_{Y|X=1}(Y=1|X=1)\cdot 1 \\ &= 0.4 \end{aligned} \]

\[ \begin{aligned} E[Y^2] &= \sum_{y\in\{0,1\}} P(Y=y)y^2\\ &= P(Y=1)\cdot 1^2\\ &= P(Y=1 \cap X=1) + P(Y=1 \cap X=0)\\ &= 0.2 + 0.3 = 0.5\\ \end{aligned} \]

Thus,

\[Var[Y|X=0] = E[(Y|X=0)^2] - E[Y|X=0]^2 = 0.6 - 0.6^2 = \underline{0.24}\] \[Var[Y|X=1] = E[(Y|X=1)^2] - E[Y|X=1]^2 = 0.4 - 0.4^2 = \underline{0.24}\]

\[Var[Y] = E[Y^2] - E[Y]^2 = 0.5 - 0.5^2 = \underline{0.25}\]

Explaination

$Var[Y]$ is larger than $Var[Y|X=0]$ and $Var[Y|X=1]$ because $Y$ can be more determined by the information of $X$. With known $X$, the distribution of $Y$ is set, and less uncertain with single confirmed distribution.

5.5.4 (d) Suppose you see that the ball is green. What is the probability that the coin turned up tails?

\[\begin{aligned} p(X=0 | Y=1) &= \frac{p(X=0 \cap Y=1)}{p(Y=1)}\\ &= \frac{p(X=0 \cap Y=1)}{p(Y=1 | X=0)p(X=0) + p(Y=1|X=1)p(X=1)}\\ &= \frac{0.5 \cdot 0.6}{0.6\cdot 0.5 + 0.4\cdot 0.5}\\ &= \frac{0.3}{0.3+0.2}\\ &= \frac{0.3}{0.5}\\ &= \underline{0.6}\\ \end{aligned}\]

--- title: Homework 1 author: Shao-Ting Chiu (UIN:433002162) date: today bibliography: ../ref.bib --- ## Homework Description > Read Chapters 1 and 2 in the Hoff book. > Then, do the following exercises in Hoff (p. 225-226): 2.1, 2.2, 2.3, 2.5 > You must turn in your solutions as a pdf file here on Canvas. Use the Submit Assignment button on the top right. > If your solutions are on paper, please scan them to pdf using a scanner or a scanner app on your phone. Please do not take a regular photo, as this can result in very large file sizes. Make sure that everything is legible. > Please note that late homework will not be accepted and will result in a score of zero. To avoid late submissions due to technical issues, we recommend turning in your homework the night before the due date. - Deadline: Sep. 8 by 12:01pm - [PDF version](https://github.com/stevengogogo/STAT638_Applied-Bayes-Methods/blob/hw/hw1.pdf) ## Problem 2.1 Marginal and conditional probability: The social mobility data from Section 2.5 gives a joint probability distribution on $(Y_1, Y_2)=$ (fathers's occupation, son's occupation) ||||son's occupation||| |---|---|---|---|---|---| |**father's occupation**|farm|operatives|craftsmen|sales|professional| |farm|0.018|0.035|0.031|0.008|0.018| |operatives|0.002|0.112|0.064|0.032|0.069| |craftsman|0.001|0.066|0.094|0.032|0.084| |sales|0.001|0.018|0.019|0.010|0.051| |professional|0.001|0.029|0.032|0.043|0.130| : The social mobility data [@hoff2009first, pp. 24] {#tbl-social} ### (a) The marginal probability distribution of a father's occupation According to @tbl-social, let $\mathbb{Y_1}$ and $\mathbb{Y_2}$ be sets of father's and son's occupations: $$\mathbb{Y_1} = \mathbb{Y_2} = \{\text{farm},\text{operatives},\text{craftsmen},\text{sales},\text{professional}\}$$ $$ \begin{aligned} Pr(Y_1 = \text{farm}) &= \sum_{y_2\in\mathbb{Y_2}}Pr(Y_1= \text{farm} \cap Y_2=y_2)\\ &= Pr(Y_1= \text{farm} \cap Y_2=\text{farm}) + Pr(Y_1= \text{farm} \cap Y_2=\text{operatives}) \\ & +Pr(Y_1= \text{farm} \cap Y_2=\text{craftsmen}) + Pr(Y_1= \text{farm} \cap Y_2=\text{sales}) \\ & +Pr(Y_1= \text{farm} \cap Y_2=\text{professional})\\ &= 0.018 + 0.035 + 0.031 + 0.008 + 0.018\\ &= 0.11 \end{aligned} $$ $$ \begin{aligned} Pr(Y_1 = \text{operatives}) &= \sum_{y_2\in\mathbb{Y_2}}Pr(Y_1= \text{operatives} \cap Y_2=y_2)\\ &= Pr(Y_1= \text{operatives} \cap Y_2=\text{farm}) + Pr(Y_1= \text{operatives} \cap Y_2=\text{operatives}) \\ & +Pr(Y_1= \text{operatives} \cap Y_2=\text{craftsmen}) + Pr(Y_1= \text{operatives} \cap Y_2=\text{sales}) \\ & +Pr(Y_1= \text{operatives} \cap Y_2=\text{professional})\\ &= 0.002 + 0.112 + 0.064 + 0.032 + 0.069\\ &= 0.279 \end{aligned} $$ $$ \begin{aligned} Pr(Y_1 = \text{craftsman}) &= \sum_{y_2\in\mathbb{Y_2}}Pr(Y_1= \text{craftsman} \cap Y_2=y_2)\\ &= Pr(Y_1= \text{craftsman} \cap Y_2=\text{farm}) + Pr(Y_1= \text{craftsman} \cap Y_2=\text{operatives}) \\ & +Pr(Y_1= \text{craftsman} \cap Y_2=\text{craftsmen}) + Pr(Y_1= \text{craftsman} \cap Y_2=\text{sales}) \\ & +Pr(Y_1= \text{craftsman} \cap Y_2=\text{professional})\\ &= 0.001+0.066+0.094+0.032+0.084\\ &= 0.277 \end{aligned} $$ $$ \begin{aligned} Pr(Y_1 = \text{sales}) &= \sum_{y_2\in\mathbb{Y_2}}Pr(Y_1= \text{sales} \cap Y_2=y_2)\\ &= Pr(Y_1= \text{sales} \cap Y_2=\text{farm}) + Pr(Y_1= \text{sales} \cap Y_2=\text{operatives}) \\ & +Pr(Y_1= \text{sales} \cap Y_2=\text{craftsmen}) + Pr(Y_1= \text{sales} \cap Y_2=\text{sales}) \\ & +Pr(Y_1= \text{sales} \cap Y_2=\text{professional})\\ &= 0.001 + 0.018 + 0.019 + 0.010 + 0.051\\ &= 0.099 \end{aligned} $$ $$ \begin{aligned} Pr(Y_1 = \text{professional}) &= \sum_{y_2\in\mathbb{Y_2}}Pr(Y_1= \text{professional} \cap Y_2=y_2)\\ &= Pr(Y_1= \text{professional} \cap Y_2=\text{farm}) + Pr(Y_1= \text{professional} \cap Y_2=\text{operatives}) \\ & +Pr(Y_1= \text{professional} \cap Y_2=\text{craftsmen}) + Pr(Y_1= \text{professional} \cap Y_2=\text{sales}) \\ & +Pr(Y_1= \text{professional} \cap Y_2=\text{professional})\\ &= 0.001 + 0.029 + 0.032 + 0.043 + 0.130\\ &= 0.235 \end{aligned} $$ |marginal probability|value| |---|---| |$p(Y_1=\text{farm})$|0.11| |$p(Y_1=\text{operatives})$|0.279| |$p(Y_1=\text{craftsmen})$|0.277| |$p(Y_1=\text{sales})$|0.099| |$p(Y_1=\text{professional})$|0.235| |*SUM*|1.0| : Marginal probability of father's occupation {#tbl-father} @tbl-father shows that the sum of marginal probability is $1$. ##### (b) The marginal probability distribution of a son's occupation $$ \begin{aligned} Pr(Y_2 = \text{farm}) &= \sum_{y_1\in\mathbb{Y_1}}Pr(Y_2= \text{farm} \cap Y_1=y_1)\\ &= Pr(Y_2= \text{farm} \cap Y_1=\text{farm}) + Pr(Y_2= \text{farm} \cap Y_1=\text{operatives}) \\ & +Pr(Y_2= \text{farm} \cap Y_1=\text{craftsmen}) + Pr(Y_2= \text{farm} \cap Y_1=\text{sales}) \\ & +Pr(Y_2= \text{farm} \cap Y_1=\text{professional})\\ &= 0.018 + 0.002 + 0.001 + 0.001 + 0.001\\ &= 0.023 \end{aligned} $$ $$ \begin{aligned} Pr(Y_2 = \text{operatives}) &= \sum_{y_1\in\mathbb{Y_1}}Pr(Y_2= \text{operatives} \cap Y_1=y_1)\\ &= Pr(Y_2= \text{operatives} \cap Y_1=\text{farm}) + Pr(Y_2= \text{operatives} \cap Y_1=\text{operatives}) \\ & +Pr(Y_2= \text{operatives} \cap Y_1=\text{craftsmen}) + Pr(Y_2= \text{operatives} \cap Y_1=\text{sales}) \\ & +Pr(Y_2= \text{operatives} \cap Y_1=\text{professional})\\ &= 0.035 + 0.112 + 0.066 + 0.018 + 0.029\\ &= 0.26 \end{aligned} $$ $$ \begin{aligned} Pr(Y_2 = \text{craftsmen}) &= \sum_{y_1\in\mathbb{Y_1}}Pr(Y_2= \text{craftsmen} \cap Y_1=y_1)\\ &= Pr(Y_2= \text{craftsmen} \cap Y_1=\text{farm}) + Pr(Y_2= \text{craftsmen} \cap Y_1=\text{operatives}) \\ & +Pr(Y_2= \text{craftsmen} \cap Y_1=\text{craftsmen}) + Pr(Y_2= \text{craftsmen} \cap Y_1=\text{sales}) \\ & +Pr(Y_2= \text{craftsmen} \cap Y_1=\text{professional})\\ &= 0.031 + 0.064 + 0.094 + 0.019 + 0.032\\ &= 0.24 \end{aligned} $$ $$ \begin{aligned} Pr(Y_2 = \text{sales}) &= \sum_{y_1\in\mathbb{Y_1}}Pr(Y_2= \text{sales} \cap Y_1=y_1)\\ &= Pr(Y_2= \text{sales} \cap Y_1=\text{farm}) + Pr(Y_2= \text{sales} \cap Y_1=\text{operatives}) \\ & +Pr(Y_2= \text{sales} \cap Y_1=\text{craftsmen}) + Pr(Y_2= \text{sales} \cap Y_1=\text{sales}) \\ & +Pr(Y_2= \text{sales} \cap Y_1=\text{professional})\\ &= 0.008+0.032+0.032+0.010+0.043\\ &= 0.125 \end{aligned} $$ $$ \begin{aligned} Pr(Y_2 = \text{professional}) &= \sum_{y_1\in\mathbb{Y_1}}Pr(Y_2= \text{professional} \cap Y_1=y_1)\\ &= Pr(Y_2= \text{professional} \cap Y_1=\text{farm}) + Pr(Y_2= \text{professional} \cap Y_1=\text{operatives}) \\ & +Pr(Y_2= \text{professional} \cap Y_1=\text{craftsmen}) + Pr(Y_2= \text{professional} \cap Y_1=\text{sales}) \\ & +Pr(Y_2= \text{professional} \cap Y_1=\text{professional})\\ &= 0.018+ 0.069+0.084+0.051+0.130\\ &= 0.352 \end{aligned} $$ |marginal probability|value| |---|---| |$p(Y_2=\text{farm})$|0.023| |$p(Y_2=\text{operatives})$|0.26| |$p(Y_2=\text{craftsmen})$|0.24| |$p(Y_2=\text{sales})$|0.125| |$p(Y_2=\text{professional})$|0.352| |*SUM*|1.0| : Marginal probability of son's occupation {#tbl-son} @tbl-son shows that the sum of marginal probability is $1$. ### (c) The conditional distribution of a son's occupation, given that the father is a farmer; The conditional distribution of a son's occupation can be expressed as $p(y_2 | y_1 = \text{farmer})$. $$p(y_2= * | y_1 = \text{farmer}) = \frac{p(y_1=\text{farm} \cap y_2 = *)}{p(y_1 = \text{farm})}$$ where $* \in \mathbb{Y_2}$. As described in @tbl-father, $p(y_1=\text{farm})= 0.11$. Use @tbl-social to calculate the distribution: $$ \begin{aligned} p(y_2 = \text{farm} | y_1 = \text{farm}) &= \frac{0.018}{0.11} \approx 0.16\\ p(y_2 = \text{operatives} | y_1 = \text{farm}) &= \frac{0.035}{0.11} \approx 0.32\\ p(y_2 = \text{craftsman} | y_1 = \text{farm}) &= \frac{0.031}{0.11} \approx 0.28\\ p(y_2 = \text{sales} | y_1 = \text{farm}) &= \frac{0.008}{0.11} \approx 0.072\\ p(y_2 = \text{professional} | y_1 = \text{farm}) &= \frac{0.018}{0.11} \approx 0.16 \\ \end{aligned} $$ ### (d) The conditional distribution of a father's occupation, given that the son is a farmer. $$p(y_1= * | y_2 = \text{farm}) = \frac{p(y_1=\text{*} \cap y_2 = \text{farm})}{p(y_2 = \text{farm})}$$ According to @tbl-son, $p(y_2 = \text{farm}) = 0.023$. Use @tbl-social to calculate the distribution: $$ \begin{aligned} p(y_1 = \text{farm} | y_2 = \text{farm}) &= \frac{0.018}{0.023} \approx 0.78\\ p(y_1 = \text{operatives} | y_2 = \text{farm}) &=\frac{0.002}{0.023} \approx 0.09\\ p(y_1 = \text{craftsman} | y_2 = \text{farm}) &= \frac{0.001}{0.023} \approx 0.04\\ p(y_1 = \text{sales} | y_2 = \text{farm}) &= \frac{0.001}{0.023} \approx 0.04\\ p(y_1 = \text{professional} | y_2 = \text{farm}) &= \frac{0.001}{0.023} \approx 0.04\\ \end{aligned} $$ ## Problem 2.2 > Expectations and variances: Let $Y_1$ and $Y_2$ be two independent random variables, such that $E[Y_i]=\mu_i$ and $Var[Y_i] = \sigma_{i}^2$. Using the definition of expectation and variance, computing the following quantities, where $a_1$ and $a_2$ are given constants. Because $Y_1$ and $Y_2$ are independent, ### (a) $E[a_1 Y_1 +a_2 Y_2]$, $Var[a_1 Y_1 + a_2 Y_2]$ Because $Y_1$ and $Y_2$ are independent: $$ E[Y_1 Y_2] = E[Y_1] E[Y_2] $$ Thus $$ \begin{aligned} E[a_1 Y_1 + a_2 Y_2] &= E[a_1 Y_1] + E[a_2 Y_2]\\ &= a_1 E[Y_1] + a_2 E[Y_2]\\ &= \underline{a_1 \mu_1 + a_2 \mu_{2}}\\ \end{aligned} $$ $$ \begin{aligned} Var[a_1 Y_1 + a_2 Y_2] &= E[ [(a_1 Y_1 + a_2 Y_2) - E[a_1 Y_1 + a_2 Y_2]]^{2}]\\ &= E[(a_1 Y_1 + a_2 Y_2)^2] - E[a_1 Y_1 + a_2 Y_2]^2\\ &= E[a_{1}^{2}Y_{1}^{2} + 2a_1 a_2 Y_1 Y_2 + a_{2}^{2}Y_{2}^{2}] - (a_1 \mu_1 + a_2 \mu_2)^2\\ &= a^{2}_{1}E[Y_1^2] + 2a_1 a_2 \underbrace{\mu_1 \mu_2}_{E[Y_1 Y_2]=E[Y_1]E[Y_2]\quad \because \text{Indep.}} + a_{2}^{2}E[Y_{2}^{2}] - (a_1 \mu_1 + a_2 \mu_2)^2\\ &= a^{2}_{1}E[Y_1^2] + 2a_1 a_2\mu_1 \mu_2 + a_{2}^{2}E[Y_{2}^{2}] - (a_{1}^{2} \mu_{1}^{2}+ 2a_{1}a_{2}\mu_{1}\mu_{2} + a_{2}^{2} \mu_{2}^{2})\\ &= a_{1}^{2}(\underbrace{E[Y_{1}^2] - \mu_{1}^{2}}_{\sigma_{1}^{2}}) + a_{2}^{2}(\underbrace{E[Y^{2}_{2}] - \mu_{2}^{2}}_{\sigma_{2}^{2}})\\ &= \underline{a_{1}^{2}\sigma_{1}^{2} + a_{2}^{2}\sigma_{2}^{2}}\\ \end{aligned} $$ ### (b) $E[a_1 Y_1 - a_2 Y_2]$, $Var[a_1 Y_1 - a_2 Y_2]$ $$ \begin{aligned} E[a_1 Y_1 - a_2 Y_2] &= E[a_1 Y_1] - E[a_2 Y_2]\\ &= a_1 E[Y_1] - a_2 E[Y_2]\\ &= \underline{a_1 \mu_1 - a_2 \mu_{2}}\\ \end{aligned} $$ $$ \begin{aligned} Var[a_1 Y_1 - a_2 Y_2] &= E[ [(a_1 Y_1 - a_2 Y_2) - E[a_1 Y_1 - a_2 Y_2]]^{2}]\\ &= E[(a_1 Y_1 - a_2 Y_2)^2] - E[a_1 Y_1 - a_2 Y_2]^2\\ &= E[a_{1}^{2}Y_{1}^{2} - 2a_1 a_2 Y_1 Y_2 + a_{2}^{2}Y_{2}^{2}] - (a_1 \mu_1 - a_2 \mu_2)^2\\ &= a_{1}^{2}E[Y^{2}_{1}] + a_{2}^{2}E[Y^{2}_{2}] - 2a_1 2_2 \underbrace{E[Y_1Y_2]}_{=E[Y_1]E[Y_2]\quad\because \text{Indep.}} - (a_1 \mu_1 - a_2 \mu_2)^2\\ &= a_{1}^{2}E[Y^{2}_{1}] + a_{2}^{2}E[Y^{2}_{2}] - 2a_1 2_2 \mu_1\mu_2 - (a_{1}^2 \mu_{1}^2 - 2 a_{1}a_{2}\mu_1 \mu_2 + a_{2}^{2} \mu_{2}^{2})\\ &= a_{1}^{2}(\underbrace{E[Y_{1}^2] - \mu_{1}^{2}}_{\sigma_{1}^{2}}) + a_{2}^{2}(\underbrace{E[Y^{2}_{2}] - \mu_{2}^{2}}_{\sigma_{2}^{2}})\\ &= \underbrace{a_{1}^{2}\sigma_{1}^{2} + a_{2}^{2}\sigma^{2}_{2}} \end{aligned} $$ ## Problem 2.3 > Full conditionals: Let $X$, $Y$, $Z$ be random variables with joint density (discrete or continuous) $p(x,y,z) \propto f(x,z)g(y,z)h(z)$. Show that ### (a) $p(x|y,z) \propto f(x,z)$ i.e. $p(x|y,z)$ is a function of $x$ and $z$; Let $c, d\in \mathbb{R}$ constants $$ \begin{aligned} p(x|y,z) &= \frac{p(x,y,z)}{p(y,z)}\\ &= \frac{c\cdot f(x,z)g(y,z)h(z)}{p(y,z)} &= \frac{c\cdot f(x,z)g(y,z)h(z)}{\int_{x\in\mathbb{X}} p(x,y,z)dx}\\ &= \frac{c\cdot f(x,z)g(y,z)h(z)}{d\cdot \int_{x\in \mathbb{X}} f(x,z)g(y,z)h(z) dx}\\ &= \frac{c\cdot f(x,z)g(y,z)h(z)}{d\cdot g(y,z)h(z)\int_{x\in \mathbb{X}} f(x,z) dx}\\ &= \frac{c\cdot f(x,z)}{d\cdot\int_{x\in \mathbb{X}} f(x,z) dx}\\ &\propto \underline{\frac{f(x,z)}{\int_{x\in \mathbb{X}} f(x,z) dx}} \end{aligned} $$ Thus, $p(x|y,z)$ is a function of $f(x,z)$. ### (b) $p(y|x,z) \propto g(y,z)$ i.e. $p(y|x,z)$ is a function of $y$ and $z$; Let $c, d\in \mathbb{R}$ constants $$ \begin{aligned} p(y|x,z) &= \frac{p(x,y,z)}{p(x,z)}\\ &= \frac{p(x,y,z)}{\int_{y\in \mathbb{Y}}p(x,y,z)dy}\\ &= \frac{c\cdot f(x,z)g(y,z)h(z)}{d\cdot \int_{y\in\mathbb{Y}} f(x,z)g(y,z)h(z)dy}\\ &= \frac{c\cdot f(x,z)g(y,z)h(z)}{d\cdot f(x,z)h(z)\int_{y\in\mathbb{Y}} g(y,z)dy}\\ &\propto \frac{f(y,z)}{\int_{y\in\mathbb{Y}}g(y,z)dy}\\ \end{aligned} $$ ### (c) $X$ and $Y$ are conditionally independent given $Z$. Let $a_1, a_2 \in \mathbb{R}$ constant, $$ \begin{aligned} p(x|z) &= \frac{p(x,z)}{p(z)}\\ &= \frac{\int_{y\in\mathbb{Y}}p(x,y,z)dy}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} p(x,y,z) dydx}\\ &= \frac{\int_{y\in\mathbb{Y}}f(x,z)g(y,z)h(z) dy}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} p(x,y,z) dydx}\\ &= \frac{f(x,z)h(z)\int_{y\in\mathbb{Y}}g(y,z)dy}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} p(x,y,z) dydx}\\ &= \frac{f(x,z)\int_{y\in\mathbb{Y}}g(y,z)dy}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} f(x,z)g(y,z) dydx} \end{aligned} $$ $$ \begin{aligned} p(y|z) &= \frac{p(y,z)}{p(z)}\\ &= \frac{\int_{x\in\mathbb{X}}p(x,y,z)dx}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} p(x,y,z) dydx}\\ &= \frac{\int_{x\in \mathbb{X}} f(x,z)g(y,z)h(z) dx}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} p(x,y,z) dydx}\\ &= \frac{g(y,z)h(z)\int_{x\in \mathbb{X}} f(x,z)dx}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} p(x,y,z) dydx}\\ &= \frac{g(y,z)\int_{x\in \mathbb{X}} f(x,z)dx}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} f(x,z)g(y,z) dydx}\\ \end{aligned} $$ $$ \begin{aligned} p(x|z)p(y|z) &= \frac{f(x,z)\int_{y\in\mathbb{Y}}g(y,z)dy}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} f(x,z)g(y,z) dydx} \cdot \frac{g(y,z)\int_{x\in \mathbb{X}} f(x,z)dx}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} f(x,z)g(y,z) dydx}\\ &= \frac{f(x,z)g(y,z)}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} f(x,z)g(y,z) dydx}\\ \end{aligned} $$ $$ \begin{aligned} p(x,y|z) &= \frac{p(x,y,z)}{p(z)}\\ &= \frac{f(x,z)g(y,z)h(z)}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} p(x,y,z) dy dx}\\ &= \frac{f(x,z)g(y,z)h(z)}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} f(x,z)g(y,z)h(z) dy dx}\\ &= \frac{f(x,z)g(y,z)h(z)}{\int_{x\in\mathbb{X}}f(x,z)h(z)\left[\int_{y\in\mathbb{Y}} g(y,z) dy\right] dx}\\ &= \frac{f(x,z)g(y,z)h(z)}{\int_{x\in\mathbb{X}}f(x,z)h(z)dx \cdot \int_{y\in\mathbb{Y}} g(y,z) dy}\\ &= \frac{f(x,z)}{\int_{x\in\mathbb{X}}f(x,z)dx}\frac{g(y,z)}{\int_{y\in\mathbb{Y}} g(y,z) dy}\\ &= \frac{f(x,z)g(y,z)}{\int_{x\in\mathbb{X}}\int_{y\in\mathbb{Y}} f(x,z)g(y,z) dydx}\\ &= \underline{p(x|z)p(y|z)}\\ \end{aligned} $$ Thus, $p(x,y|z) = p(x|z)p(y|z)$ that means $p(x,y|z)$ is conditionally independent given $z$. ## Problem 2.5 > Urns: Suppose urn $H$ is filled with $40\%$ green balls and $60\%$ red balls, and urn $T$ is filled with $60\%$ green balls and $40\%$ red balls. Someone will flip a coin and then select a ball from urn $H$ or urn $T$ depending on whether the coin lands heads or tails, respectively. Let $X$ be 1 or 0 if the coin lands heads or tails, and let $Y$ be $1$ or $0$ if the ball is green or red. ||Green|Red| |---|---|---| |$H$ (chosen if head[1])|0.4|0.6| |$T$ (chosen if tail[0])|0.6|0.4| : Probability of choosing a certain ball in a given urn. {#tbl-balls} ||1|0| |---|---|---| |$X$|Head|Tail| |$Y$|Green|Red| : Event coding ### (a) Write out the joint distribution of $X$ and $Y$ in a table. Suppose the coin is fair, $$ \begin{aligned} p(X=0 \cap Y=0) &= p(X=0) p(Y=0|X=0)= 0.5\cdot 0.4 = 0.2\\ p(X=0 \cap Y=1) &= p(X=0) p(Y=1|X=0)= 0.5\cdot 0.6 = 0.3\\ p(X=1 \cap Y=0) &= p(X=1) p(Y=0|X=1)= 0.5\cdot 0.6 = 0.3\\ p(X=1 \cap Y=1) &= p(X=1) p(Y=1|X=1)= 0.5\cdot 0.4 = 0.2\\ \end{aligned} $$ ### (b) Find $E[Y]$. What is the probability that the ball is green? $$ \begin{aligned} E[Y] &= \sum_{y\in \{0,1\}} p(Y=y)y\\ &= p(Y=1)\cdot 1\\ &= \sum_{x\in\{0,1\}} p(Y=1 | X=x)p(X=x)\\ &= p(Y=1|X=0)p(X=0) + p(Y=1|X=1)p(X=1)\\ &= 0.6\cdot 0.5 + 0.4 \cdot 0.5 \\ &= 0.3 + 0.2 \\ &= 0.5 \end{aligned} $$ ### (c) Find $Var[Y|X=0]$, $Var[Y|X=1]$ and $Var[Y]$. Thinking of variance as measuring uncertainty, explain intuitively why one of these variances is larger than others. $$ \begin{aligned} E[Y|X=0] &= \sum_{y \in\{0,1\}} P_{Y|X=0}(Y=y|X=0)y\\ &= P_{Y|X=0}(Y=1|X=0)\cdot 1 \\ &= 0.6 \end{aligned} $$ $$ \begin{aligned} E[(Y|X=0)^2] &= \sum_{y \in\{0,1\}} P_{Y|X=0}(Y=y|X=0)y^2\\ &= P_{Y|X=0}(Y=1|X=0)\cdot 1 \\ &= 0.6 \end{aligned} $$ $$ \begin{aligned} E[Y|X=1] &= \sum_{y \in\{0,1\}} P_{Y|X=1}(Y=y|X=1)y\\ &= P_{Y|X=1}(Y=1|X=1)\cdot 1 \\ &= 0.4 \end{aligned} $$ $$ \begin{aligned} E[(Y|X=1)^2] &= \sum_{y \in\{0,1\}} P_{Y|X=0}(Y=y|X=1)y^2\\ &= P_{Y|X=1}(Y=1|X=1)\cdot 1 \\ &= 0.4 \end{aligned} $$ $$ \begin{aligned} E[Y^2] &= \sum_{y\in\{0,1\}} P(Y=y)y^2\\ &= P(Y=1)\cdot 1^2\\ &= P(Y=1 \cap X=1) + P(Y=1 \cap X=0)\\ &= 0.2 + 0.3 = 0.5\\ \end{aligned} $$ Thus, $$Var[Y|X=0] = E[(Y|X=0)^2] - E[Y|X=0]^2 = 0.6 - 0.6^2 = \underline{0.24}$$ $$Var[Y|X=1] = E[(Y|X=1)^2] - E[Y|X=1]^2 = 0.4 - 0.4^2 = \underline{0.24}$$ $$Var[Y] = E[Y^2] - E[Y]^2 = 0.5 - 0.5^2 = \underline{0.25}$$ **Explaination** $Var[Y]$ is larger than $Var[Y|X=0]$ and $Var[Y|X=1]$ because $Y$ can be more determined by the information of $X$. With known $X$, the distribution of $Y$ is set, and less uncertain with single confirmed distribution. ### (d) Suppose you see that the ball is green. What is the probability that the coin turned up tails? $$\begin{aligned} p(X=0 | Y=1) &= \frac{p(X=0 \cap Y=1)}{p(Y=1)}\\ &= \frac{p(X=0 \cap Y=1)}{p(Y=1 | X=0)p(X=0) + p(Y=1|X=1)p(X=1)}\\ &= \frac{0.5 \cdot 0.6}{0.6\cdot 0.5 + 0.4\cdot 0.5}\\ &= \frac{0.3}{0.3+0.2}\\ &= \frac{0.3}{0.5}\\ &= \underline{0.6}\\ \end{aligned}$$ ::: {.content-hidden when-format="html"} ## References :::

5 Homework 1

5.1 Homework Description

5.2 Problem 2.1

5.2.1 (a) The marginal probability distribution of a father’s occupation

5.2.1.0.1 (b) The marginal probability distribution of a son’s occupation

5.2.2 (c) The conditional distribution of a son’s occupation, given that the father is a farmer;

5.2.3 (d) The conditional distribution of a father’s occupation, given that the son is a farmer.

5.3 Problem 2.2

5.3.1 (a) \(E[a_1 Y_1 +a_2 Y_2]\), \(Var[a_1 Y_1 + a_2 Y_2]\)

5.3.2 (b) \(E[a_1 Y_1 - a_2 Y_2]\), \(Var[a_1 Y_1 - a_2 Y_2]\)

5.4 Problem 2.3

5.4.1 (a) \(p(x|y,z) \propto f(x,z)\) i.e. \(p(x|y,z)\) is a function of \(x\) and \(z\);

5.4.2 (b) \(p(y|x,z) \propto g(y,z)\) i.e. \(p(y|x,z)\) is a function of \(y\) and \(z\);

5.4.3 (c) \(X\) and \(Y\) are conditionally independent given \(Z\).

5.5 Problem 2.5

5.5.1 (a) Write out the joint distribution of \(X\) and \(Y\) in a table.

5.5.2 (b) Find \(E[Y]\). What is the probability that the ball is green?

5.5.3 (c) Find \(Var[Y|X=0]\), \(Var[Y|X=1]\) and \(Var[Y]\). Thinking of variance as measuring uncertainty, explain intuitively why one of these variances is larger than others.

5.5.4 (d) Suppose you see that the ball is green. What is the probability that the coin turned up tails?