Neural Network Basics

1. In logistic regression given the input $\mathbf{x}$, and parameters $w\in {\mathbb{R}}^{n_x}$, $b\in \mathbb{R}$，how do we generate the output $\hat{y}$?

• $\sigma (Wx)$
• $Wx+b$
• $\sigma (Wx+b)$
• $tanh(Wx+b)$

2. suppose that $\hat{y}=0.9$ and $y=1$. What is the value of the “Logistic Loss”? Choose the best option.

• $0.005$
• $0.105$
• $+\infty$
• $L(\hat{y},y)=-(\hat{y}logy+(1-\hat{y})log(1-y))$

3. Suppose img is a (32,32,3) array, representing a 32x32 image with 3 color channels red, green and blue. How do you reshape this into a column vector $x$?

• x = img.reshape((3,32*32))
• x = img.reshape((32*32,3))
• x = img.reshape((1,32*32,3))
• x = img.reshape((32*32*3,1))

4. Consider the following random arrays $a$ and $b$, and $c$:
   $a=np.random.randn(3,4)$ # $a.shape=(3,4)a.shape=(3,4)$
   $b=np.random.randn(1,4)$ # $b.shape=(1,4)b.shape=(1,4)$
   $c=a+b$
   What will be the shape of $c$?

• The computation cannot happen because it is not possible to broadcast more than one dimension.
• c.shape = (3, 4)
• c.shape = (1, 4)
• c.shape = (3, 1)

5. Consider the two following random arrays aa and bb:
   $a=np.random.randn(1,3)$ # $a.shape=(1,3)a.shape=(1,3)$
   $b=np.random.randn(3,3)$ # $b.shape=(3,3)b.shape=(3,3)$
   $c=a\ast b$
   What will be the shape of #c#?

• The computation cannot happen because the sizes don’t match.
• c.shape = (3, 3)
• c.shape = (1, 3)
• The computation cannot happen because it is not possible to broadcast more than one dimension.

6. Suppose you have $n_x$ input features per example. Recall that $X=[x^{(1)}{x}^{(2)}...x^{(m)}]$. What is the dimension of $X$?

• $(m,n_x)$
• $(m,1)$
• $(n_x,m)$
• $(1,m)$

7. Consider the following array:
   $a=np.array([[2,1],[1,3]])$
   What is the result of np.dot(a,a)?

• $\left(\begin{array}{cc}5& 5\\ 5& 10\end{array}\right)$
• $\left(\begin{array}{cc}4& 2\\ 2& 6\end{array}\right)$
• The computation cannot happen because the sizes don’t match. It’s going to be an “Error”!
• $\left(\begin{array}{cc}4& 1\\ 1& 9\end{array}\right)$

8. Consider the following code snippet:

$a.shape=(4,3)$
$b.shape=(4,1)$

for i in range(3):
	for j in range(4):
 		c[i][j] = a[j][i] + b[j]

How do you vectorize this?

• c = a + b.T
• c = a + b
• c = a.T + b.T
• c = a.T + b

9. Consider the following code:
   $a=np.random.randn(3,3)$
   $b=np.random.randn(3,1)$
   $c=a\ast b$
   What will be cc? (If you’re not sure, feel free to run this in python to find out).

• This will multiply a 3x3 matrix a with a 3x1 vector, thus resulting in a 3x1 vector. That is, c.shape = (3,1).
• This will invoke broadcasting, so b is copied three times to become (3, 3), and ∗ invokes a matrix multiplication operation of two 3x3 matrices so c.shape will be (3, 3)
• This will invoke broadcasting, so b is copied three times to become (3,3), and ∗ is an element-wise product so c.shape will be (3, 3)
• It will lead to an error since you cannot use “*” to operate on these two matrices. You need to instead use np.dot(a,b)

10. Consider the following computational graph.
    

• $(a+c),(b-1)$
• $ab+bc+ac$
• $(a-1),(b+c)$
• $(c-1),(a+c)$