Memory tricks that make AI, ML & Deep Learning click

The three terms are nested, not synonyms. Artificial Intelligence (AI): the broadest category — any technique enabling machines to mimic human intelligence. Machine Learning (ML): a subset of AI where machines learn patterns from data without explicit programming. Deep Learning (DL): a subset of ML using multi-layered neural networks. Key exam point: everything deployed today — ChatGPT, self-driving cars, medical imaging AI — is Narrow AI.

Artificial Narrow Intelligence (ANI): all current AI — excels at ONE specific task. Cannot transfer skills. ChatGPT writes text but cannot drive a car. Artificial General Intelligence (AGI): hypothetical — human-level ability across ALL tasks. Does not yet exist. Artificial Superintelligence (ASI): theoretical — surpasses human intelligence in every domain. Key point: everything deployed today is Narrow AI.

Picture three vivid scenes. Supervised: a TEACHER with the answer key — every example has a correct label. Learns to match inputs to outputs (spam filter, image classifier). Unsupervised: an EXPLORER dropped in an unknown jungle with no map — discovers hidden structure with no labels (clustering, topic modeling). Reinforcement: a DOG trained with treats — takes an action, gets a reward or correction, learns what works (game AI, robotics, RLHF for ChatGPT).

Bias: error from wrong assumptions — model too simple, misses real patterns. A linear model on curved data has high bias (underfitting). Variance: error from sensitivity to training noise — model memorizes training data, fails on new data (overfitting). Goal: find the sweet spot — right complexity for the data. Underfitting fix: more complex model, more features. Overfitting fix: more data, regularization, simpler model.

Visualize a river of regret flowing upstream. Forward pass: data flows forward through the network producing a prediction. Loss: the network measures how wrong it was — this is the regret. Backward pass (backpropagation): the regret flows BACKWARD using the chain rule — assigns blame to each weight. Weight update: each weight is nudged to reduce future regret (gradient descent). Repeat thousands of times. The river of regret gets smaller as the network learns.

Dartmouth Conference (1956): John McCarthy coins 'Artificial Intelligence' — field is born. AI Winters (1970s, 1980s): funding collapses when expectations outstrip results — twice. Deep Learning Breakthrough (2012): AlexNet wins ImageNet by a massive margin using deep CNNs and GPUs — modern AI era begins. LLM Era (2017–present): Transformers → BERT → GPT → ChatGPT → explosion of generative AI. Key names: Turing (test), McCarthy (AI term), Hinton/LeCun/Bengio (deep learning pioneers — 2018 Turing Award).

Token: the unit of text an LLM processes — roughly ¾ of a word. Temperature: controls randomness — 0.0 = deterministic/predictable, 1.0+ = creative/random. Context window: maximum tokens the model can see at once — Claude: 200K, Gemini 1.5: 1M. Hallucination: when an LLM confidently states false information — a fundamental limitation, not a bug. Models predict plausible next tokens, not verified facts.

Fairness: AI should not discriminate based on protected characteristics. Accountability: when AI causes harm, who is responsible? Transparency (Explainability): can we understand why the model made a decision? Ethics: broader societal impact — jobs, privacy, power, autonomy, surveillance. These principles are in tension — no pure algorithmic solution. EU AI Act (2024) is the first comprehensive AI regulation globally.

Classification: predict a category (spam/not spam, cat/dog). Output = discrete label. Regression: predict a number (house price, temperature). Output = continuous value. Clustering: group similar items without labels (customer segments). Unsupervised. Generation: produce new content (text, images, code). LLMs and diffusion models. Knowing the task type immediately narrows which algorithms, loss functions, and evaluation metrics are appropriate.

Linear Algebra: vectors (data points), matrices (weight matrices), dot products (similarity), eigenvalues (PCA). Calculus: derivatives (gradients), chain rule (backpropagation), gradient descent finds minimum of loss function. Matrix Operations: matrix multiplication is the core operation of neural networks — GPUs are optimized for it. Probability: Bayes theorem, probability distributions, expected value — underpins everything from Naive Bayes to diffusion models.

Frameworks: PyTorch (dominant in research and industry), TensorFlow/Keras (Google, production deployment). Hardware: NVIDIA GPUs (CUDA), Google TPUs, Apple Silicon. Open-source models: Hugging Face Hub — 500K+ pretrained models (BERT, Llama, Stable Diffusion). Libraries: scikit-learn (classical ML), NumPy/Pandas (data), Matplotlib (visualization). For beginners: scikit-learn for ML fundamentals, PyTorch + Hugging Face for deep learning.

AI is genuinely excellent at: pattern recognition in images and audio, language generation and translation, game playing, protein structure prediction (AlphaFold), recommendation systems, code generation, speech recognition. AI still struggles with: robust common-sense reasoning, reliable arithmetic and logic, consistent long-term planning, physical world understanding, verified factual accuracy. Key insight: AI excels at interpolating within its training distribution — it struggles when situations require true generalization beyond what it has seen.