Generative Adversarial Networks- Given a random samples will generate image
Single Layer Network
Activation function you choose depends on the convergence of the NN
Sigmoid is always positive and never too large. Somewhat robust to outliers.
Rectified Linear Unit is essentially the linear function with negative rectification.
Creating Logic Gates from Single Layer Perceptrons
The decision boundaries here are arbitrary and simply represent our choice of weights. There are infinitely many weights that will satisfy our decision boundary conditions here.
Classification vs. Regression (as it applies to ML)
Generically this is determined by asking whether the estimated output is continuous or discrete. Regression representing the continuous output case say if you are fitting a line to data or discrete if you are trying to identify between say 2 colors red and blue.
How to Determine Goodness of Model
Training data is known good data. The objective function measures the difference between the target and the model (NN) output. The objective function is what we try to minimize over the weights (w). The loss function is in other mathematical language the residual sum of error squared (RSS).