Day 73: Neural Networks

Why Should I Care?

One neuron makes one tiny decision. But one neuron cannot recognize your face, translate a sentence, or recommend your next PUBG squad. For that, you need thousands of neurons working together in layers — each layer passing smarter information to the next. That is a neural network. And it is the engine behind every smart app you use today.

Core Concept

A neural network is layers of neurons stacked together, each transforming data into something more useful than what it received. Raw data goes into the first layer. Each layer finds something new. The final layer gives you the answer. No layer skips the line. No backward information flow. Data moves in one direction — left to right — station by station — until the answer comes out at the end.

How It Works

Think of a car factory with four stations on an assembly line. Station 1 gets raw steel — that is your Input Layer receiving raw data. Station 2 builds the basic frame — Hidden Layer 1 finds basic patterns. Station 3 assembles a running car — Hidden Layer 2 refines those patterns. Station 4 delivers the finished car — the Output Layer gives the final answer. Each station only sees what the previous station produced. No worker handles raw steel after Station 1. A neural network works the exact same way. Layer by layer. Transformation by transformation.

Input Layer  (3 nodes)  -- receives raw data
     |
Hidden Layer 1 (16 nodes) -- finds basic patterns
     |
Hidden Layer 2  (8 nodes) -- refines patterns
     |
Output Layer    (1 node)  -- final answer (y-hat)

Real World Connection

When you open Snapchat and a filter tracks your face in real time — a neural network is running. Input Layer gets raw pixels from your camera. Hidden layers find edges, then eyes, then your full face shape. Output Layer places the filter exactly where it belongs. Every layer added more understanding. Without the hidden layers, the app would see nothing but a blur of numbers.

Examples

import tensorflow as tf

model = tf.keras.Sequential([
    tf.keras.layers.Dense(16, activation="relu", input_shape=(3,)),
    tf.keras.layers.Dense(8,  activation="relu"),
    tf.keras.layers.Dense(1)
])

model.compile(optimizer="adam", loss="mse")
model.fit(X, y, epochs=5, verbose=0)

predictions = model.predict(X)
print("Predictions:", predictions.flatten().round(2))

# OUTPUT:
# Predictions: [0.82  0.31  0.76]
# Three inputs. Three outputs. Network learned the pattern.

The input layer shape matches your data shape — 3 features means input_shape=(3,). Hidden layers must have activation="relu" — this is what gives each layer real transforming power. The output layer has no activation — it just returns the raw final number.

Common Mistakes

Mistake 1 — Thinking more layers always means better results:

-- WRONG:
Add 10 hidden layers to every model because deeper = smarter.

-- CORRECT:
More layers = more complexity.
Useful for complex data like images and audio.
Harmful for simple problems -- causes overfitting and slow training
without improving accuracy at all.

Mistake 2 — Forgetting activation functions on hidden layers:

-- WRONG:
tf.keras.layers.Dense(16)   # missing activation
tf.keras.layers.Dense(8)    # missing activation
tf.keras.layers.Dense(1)

-- CORRECT:
tf.keras.layers.Dense(16, activation="relu")
tf.keras.layers.Dense(8,  activation="relu")
tf.keras.layers.Dense(1)

-- WHY IT MATTERS:
Without activation, all layers collapse into one mathematically.
Depth disappears completely. Hidden layers mean nothing.

Mini Challenge

Quick Quiz

Q: What is the job of a hidden layer in a neural network?
A: Hidden layers find and refine patterns in the data — each one transforming what it receives into something more useful before passing it to the next layer.

Q: Why does the output layer not need an activation function?
A: The output layer just returns the final number or prediction directly — no transformation needed, just the raw result.

Q: What happens if you remove relu from all hidden layers?
A: All layers mathematically collapse into a single layer — depth disappears and the network loses its ability to learn complex patterns.

Bonus Knowledge

ReLU stands for Rectified Linear Unit. It is the most popular activation function in deep learning. All it does is this: if the number coming in is negative, output zero. If it is positive, pass it through unchanged. That tiny rule is what stops all layers from collapsing into one. It gives each layer its own real transformation. Without activation functions, a network with 100 layers would behave exactly like a network with 1 layer. That is why hidden layers need activation. Without it, depth means nothing.

Key Takeaways