Personal Learning Notes · BRUR · Statistics · M.Sc.
Postgraduate Course Learning Notes
A growing personal reference covering M.Sc.-level statistics courses — friendly, in-depth explanations, derivations, definitions, and applications from lectures, textbooks, and real-world examples at Begum Rokeya University, Rangpur. Written so anyone, statistician or not, can actually enjoy reading it.
STAT5103 · Statistical Genomics & BioinformaticsSTAT5102 · Statistical Signal Processing+ More M.Sc. courses coming soon
Bioinformatics combines biology, computer science, mathematics, and statistics to collect, store, manage, analyse, and interpret biological data — DNA sequences, RNA transcripts, protein structures, gene-expression profiles. It develops algorithms and statistical methods to turn raw biological data into biological knowledge.
⚙️
Statistics' Contribution
The Inferential Backbone
Experimental design: replication, randomisation, blocking in microarray/RNA-seq studies
Hypothesis testing: t-tests, ANOVA, LOD-score tests for linkage
Multiple testing correction: FDR (Benjamini–Hochberg) across thousands of genes/markers
Probability models: Markov chains/HMMs for sequence alignment; Poisson/negative-binomial for read counts
Multivariate & ML: clustering, PCA, classification for -omics data
Bayesian inference: incorporating prior biological knowledge into gene mapping
⚠️
Why It Matters
Trust in the Signal
Modern biology produces extremely large, high-dimensional, noisy datasets. Without statistics' inferential rigour (significance, confidence, error control), patterns found by bioinformatics tools could not be trusted as real rather than due to chance.
Memory TipBio + Informatics = using computational and statistical tools to make sense of biological information. Think of it as Excel for life itself, except the spreadsheet is 3 billion rows long and was written by evolution, not your colleague in accounting.
😂 Lab HumorWhy did the statistician break up with the biologist? Too much unexplained variance in the relationship. Also: a biologist walks into a bar, orders a beer, then orders half a beer, then a quarter, an eighth... the bartender says "I think you don't understand limits," and the statistician at the next table mutters, "I think you don't understand sample size."
· · ·
02
Cell Biology
Mitosis vs Meiosis
🔁
Mitosis
Growth, Repair & Asexual Reproduction
One division (PMAT: Prophase, Metaphase, Anaphase, Telophase)
2 daughter cells, diploid (2n)
Genetically identical to the parent cell
No homologous pairing or crossing over
Occurs in somatic cells
🧬
Meiosis
Production of Gametes
Two successive divisions (Meiosis I & II)
4 daughter cells, haploid (n)
Genetically unique — recombined
Homologous pairing (synapsis) in Prophase I
Crossing over present — generates recombinants
Occurs in germ cells (gonads)
💡
Key Distinction
Why Both Exist
Mitosis maintains genetic constancy across somatic cell generations. Meiosis introduces genetic variation (via independent assortment and crossing over) while halving chromosome number, so fertilisation restores the diploid state in offspring.
Meiosis result1 cell (2n) → 4 cells (n, n, n, n) — recombined
Plain-English RecapMitosis is your body photocopying cells for everyday repairs — same content, just more copies. Meiosis is your body shuffling a deck of cards (your chromosomes), cutting it twice, and dealing out four hands that no one has ever seen before — that's why siblings (except identical twins) never look exactly alike.
😂 Lab HumorMitosis and meiosis walk into a bar. Mitosis orders one drink and gets two identical copies of it. Meiosis orders one drink and somehow leaves with four completely different cocktails, half-finished, and no memory of how. Also — meiosis is basically your cells going "let's mix things up" while mitosis is your cells going "no thanks, I like things exactly the way they are."
Two or more loci located close together on the same chromosome tend to be inherited together more often than expected under independent assortment — because physically close loci are less likely to be separated by a crossover. Linkage is an exception to Mendel's Law of Independent Assortment.
📐
Recombination Fraction (r)
Proportion of Recombinants
r = (number of recombinant offspring) / (total offspring). r = 0 → complete linkage; r = 0.5 → independent assortment (unlinked); 0 < r < 0.5 → partial linkage (smaller r = loci physically closer).
💡
Map Distance
Morgans & centiMorgans
Map distance expresses physical separation in terms of expected crossover frequency (Morgans/cM); 1 cM ≈ 1% recombination for closely-spaced loci. For distant loci, multiple crossovers can cancel out and go undetected, so observed r underestimates true distance — mapping functions correct for this.
Map unit1 cM ≈ 1% recombination (short distances only)
Think of it like roommatesGenes that live close together on a chromosome are like roommates who always show up to parties together — they rarely get "separated" because there's so little chromosome between them for a crossover to slip in. Genes far apart, or on different chromosomes entirely, are more like distant acquaintances — they show up independently of each other.
😂 Lab HumorTwo genes are sitting next to each other on a chromosome. One says, "We've been inherited together for 40 generations, I think it's time we made it official." The other says, "Careful, one crossover event and we're getting separated."
· · ·
04
Molecular Biology
Central Dogma & Roles of RNA
🧪
The Central Dogma
Crick (1958)
Describes the directional flow of genetic information: DNA → (transcription) → RNA → (translation) → Protein. DNA also replicates (DNA→DNA). In retroviruses, reverse transcription (RNA→DNA) is an exception to the classical dogma.
⚙️
5 Steps
Replication to Maturation
1. Replication: semi-conservative copying of DNA
2. Transcription: RNA polymerase synthesises pre-mRNA from the DNA template (nucleus)
Plain-English RecapDNA is the master recipe book locked safely in the kitchen office (the nucleus). RNA is the photocopy of one recipe page that's allowed to leave the office and go to the kitchen counter (the ribosome). The protein is the actual dish that gets cooked from that recipe. Reverse transcription is a retrovirus sneaking in and rewriting a page of the recipe book using its own photocopy — which is exactly how HIV works.
😂 Lab HumorDNA never leaves the house, RNA is the intern who runs errands, and the ribosome is the over-caffeinated chef who reads the intern's handwriting and somehow still makes a five-star protein. Also: why did the mRNA break up with the ribosome? It said, "You only ever read me once and then degrade me."
· · ·
05
Quantitative Genetics
Quantitative Trait Loci (QTL) & Regression Models
📍
What is a QTL?
A Mapped Genomic Region
A QTL is a genome region statistically associated with variation in a continuous trait (body weight, milk yield, blood pressure). Identified not by direct observation of a single gene but by testing association between marker genotypes and phenotype in a segregating population.
⚖️
QTL vs Mendelian Locus
Key Differences
Trait type: continuous (QTL) vs discrete (Mendelian)
Genes: many, small effects (QTL) vs single, major effect (Mendelian)
Detection: regression/association (QTL) vs segregation ratios & χ² (Mendelian)
Penetrance: incomplete, polygenic background (QTL) vs complete (Mendelian)
💡
Simple QTL Regression
Marker-Based Mapping
Phenotype regressed on genotype score at a marker: yᵢ = μ + b·xᵢ + eᵢ, eᵢ~N(0,σ²). If the marker is linked to a QTL, b differs significantly from zero.
🔢
F2 Additive–Dominance Model
Three Genotype Classes
For AA / Aa / aa with additive score x = +1, 0, −1 and dominance score z = 0, 1, 0: yᵢ = μ + a·xᵢ + d·zᵢ + eᵢ. a = additive effect (half the homozygote-mean difference); d = dominance effect (heterozygote deviation from mid-parent).
Pooled variancesp² = [(n₁−1)s₁² + (n₀−1)s₀²] / (n₁+n₀−2)
One-Way ANOVA for 3 F2 Genotype GroupsH₀: μ_AA=μ_Aa=μ_aa. SST=SSB+SSW; F = MSB/MSW ~ F(2, n−3). If F exceeds the critical value, the marker shows an overall genotype effect — evidence it is linked to a QTL. Follow-up t-tests on â and d̂ separately test the additive and dominance components.
📐 Where â and d̂ come from
Plug the coding scores (x=+1,0,−1 for AA,Aa,aa; z=0,1,0) into yᵢ = μ + a·xᵢ + d·zᵢ + eᵢ and take group means. For the AA group (x=1,z=0): ȳ_AA = μ+a. For aa (x=−1,z=0): ȳ_aa = μ−a. Subtracting and dividing by 2 isolates a cleanly:
ȳ_AA − ȳ_aa = 2a ⇒ â = (ȳ_AA − ȳ_aa)/2
For the heterozygote Aa (x=0,z=1): ȳ_Aa = μ+d. Since μ = (ȳ_AA+ȳ_aa)/2 from the two homozygote means, substituting gives:
d̂ = ȳ_Aa − μ̂ = ȳ_Aa − (ȳ_AA+ȳ_aa)/2
In words: a is literally half the gap between the two homozygous extremes (how far each allele copy pushes the trait), and d is how far the heterozygote's mean sits away from the midpoint of the two homozygotes (zero if inheritance is purely additive, non-zero if there's dominance).
Additive (a) & Dominance (d) Effects — Visualised
Plain-English RecapUnlike a single Mendelian gene that flips a trait on/off (like pea colour), a QTL is more like one of many tiny dimmer switches that together control something continuous, like height or milk yield. We can't see the switch directly, so we watch a nearby DNA "marker" instead — if individuals with a particular marker genotype consistently have higher/lower trait values, that marker is probably sitting close to a real QTL.
😂 Lab HumorA single Mendelian gene is like a light switch — on or off, tall or short, round or wrinkled. A QTL is like a room with 200 dimmer switches, all slightly randomly set, and your job is to figure out which switch is connected to the lamp just by watching the brightness change. Geneticists call this "polygenic inheritance." Everyone else calls it "why does my family reunion look like a genetics lecture."
· · ·
06
Mapping Mechanics
Crossing Over & Two-Point Linkage Analysis (LOD Score)
✂️
Crossing Over
Physical Exchange of Segments
Crossing over is the exchange of chromosomal segments between non-sister chromatids of homologous chromosomes during pachytene (Prophase I). A crossover between loci A and B produces recombinant chromatids (Ab, aB) differing from parental (AB, ab). P(crossover) — and hence r — increases with physical distance, the basis of genetic mapping.
🧮
F1 Gamete Classes
Parental vs Recombinant
From F1 = AB/ab (coupling phase): gamete AB (parental, freq (1−r)/2), ab (parental, (1−r)/2), Ab (recombinant, r/2), aB (recombinant, r/2). With a fully informative marker, every offspring gamete is unambiguously classified, giving r̂ = (# recombinant)/(total).
💡
LOD Score
Testing H₀: r = 0.5
Likelihood L(r) = (1−r)^nP · r^nR is binomial in the parental (nP) and recombinant (nR) offspring counts; r̂ = nR/n. LOD(r) = log₁₀[L(r̂)/L(0.5)]. LOD ≥ 3 is the conventional threshold for declaring significant linkage (1000:1 odds); LOD ≤ −2 is significant evidence against linkage.
🔢
LOD ↔ Likelihood Ratio
Connecting the Two Frameworks
LR = L(r̂)/L(0.5) = 10^Z where Z is the LOD score. The likelihood-ratio chi-square statistic Λ = 2ln(LR) = 2ln(10)·Z ≈ 4.605·Z is approximately χ²(1) under H₀ — so LOD=3 ⇔ Λ≈13.8, far exceeding the χ²(1) critical value of 3.84 at α=0.05.
Likelihood (binomial)L(r) = (1−r)^nP · r^nR
MLE of rr̂ = nR / n (n = nP + nR)
LOD scoreZ(r) = log₁₀[L(r̂)/L(0.5)] significant if Z ≥ 3
LR ↔ LODLR = 10^Z ; Λ = 2ln(LR) = 4.605·Z ~ χ²(1) under H₀
📐 Where the binomial likelihood and r̂ = nR/n come from
Every offspring gamete is independently classified as parental (probability 1−r) or recombinant (probability r) — a Bernoulli trial repeated n times, so the count of recombinants nR follows a Binomial(n, r) distribution:
L(r) = P(nR recombinants out of n) ∝ (1−r)^(n−nR) · r^(nR) = (1−r)^nP · r^nR
To find the MLE, take the log and differentiate with respect to r:
ln L(r) = nP·ln(1−r) + nR·ln(r) ⇒ d(lnL)/dr = −nP/(1−r) + nR/r = 0⇒ nR(1−r) = nP·r ⇒ nR = r(nP+nR) = r·n ⇒ r̂ = nR/n
So the "just count the fraction of recombinants" formula is, again, not a shortcut — it's exactly what maximising the binomial likelihood produces. The LOD score log₁₀[L(r̂)/L(0.5)] then compares how much more plausible the best-fitting r̂ is versus the "no linkage" baseline of r=0.5, expressed on a log₁₀ scale purely for historical convenience (easy to add LOD scores across markers when building a full genetic map).
LOD Score — How Confidently r̂ Beats r = 0.5
Plain-English RecapA LOD score of 3 means the data is 1,000 times more likely to occur if the genes really are linked than if they were totally independent — that's why "LOD ≥ 3" became the gold-standard bar for claiming linkage, much like p < 0.05 in ordinary stats, except scaled in powers of 10 instead of probabilities.
😂 Lab HumorGetting a LOD score of 3 is geneticists' version of a court verdict: "beyond reasonable doubt, these two genes are definitely hanging out together." A LOD score of 0.5 is more like, "well, they were at the same party once, that proves nothing."
· · ·
07
Mapping Theory
Mendel's Laws & Mapping Functions
⚖️
Mendel's Two Laws
Segregation & Independent Assortment
Segregation: the two alleles of a gene separate during gamete formation — each gamete gets one allele
Independent assortment: alleles of different genes assort independently if on different chromosomes (or far apart) — gives the 9:3:3:1 dihybrid F2 ratio
Linkage violates independent assortment for genes on the same chromosome
📐
Why a Map Function?
Correcting for Hidden Crossovers
A map function m=f(r) converts observed recombination fraction r into additive map distance d. Needed because double crossovers between distant loci restore parental combinations and go undetected — so observed r underestimates d and saturates at 0.5 even as d grows.
💡
Morgan vs Haldane
Two Mapping Functions
Morgan: d = r (no correction) — accurate only for small r (≲0.10)
Haldane: r = ½(1−e^(−2d)) ⇔ d = −½ln(1−2r) — assumes Poisson crossovers, no interference; valid over a wider range
🧮
Derivation of Haldane's Function
From Poisson Crossovers
If N~Poisson(d) crossovers occur in interval AC, a gamete is recombinant iff N is odd. Using P(N even)−P(N odd)=e^(−2d) and P(even)+P(odd)=1 gives P(N odd) = ½(1−e^(−2d)) = r_AC. For small d, r≈d (consistent with 1 cM≈1%); as d→∞, r→0.5.
Morgan functiond = r (valid only for short distances)
📐 Deriving Haldane's r = ½(1−e^(−2d)) step by step
Model the number of crossovers N in an interval of map length d as N ~ Poisson(d) — the standard assumption of no interference, crossovers occurring independently and uniformly. A gamete ends up recombinant exactly when an odd number of crossovers occurred (each crossover flips which parental strand you're tracking; an even number flips back to where you started).
P(N even) − P(N odd) = Σₖ (−1)^k P(N=k) = Σₖ (−d)^k/k! · e^(−d) = e^(−d)·e^(−d) = e^(−2d)
(using the Poisson pmf and recognising the alternating sum as the series for e^(−d))
Also P(N even) + P(N odd) = 1. Solving the two equations together:P(N odd) = [1 − e^(−2d)] / 2 = r
That's the full derivation of Haldane's function. Check the limits: as d→0, expanding e^(−2d)≈1−2d gives r≈d (matches "1 cM ≈ 1% recombination" for short distances). As d→∞, e^(−2d)→0 so r→0.5 — confirming that r can never exceed 0.5 no matter how far apart two loci are, even though map distance d keeps growing without bound.
Recombination Fraction r vs Map Distance d — Morgan vs Haldane
Plain-English RecapRecombination fraction r can never exceed 0.5, no matter how far apart two genes are — once they're far enough apart, crossovers happen so often between them that they behave as if unlinked. Map distance d, however, just keeps adding up the more crossovers occur. A mapping function is the translator that converts "what we can observe" (r, capped at 0.5) into "what's physically true" (d, unbounded).
😂 Lab HumorRecombination fraction r is like a strict speed limit sign that caps out at 50 mph no matter how hard you press the gas. Map distance d is the actual odometer — it keeps climbing even after r has long since given up. Haldane basically built the GPS that converts "miles you think you drove" back into "miles you actually drove."
Gene expression = abundance of the RNA transcript (and ultimately protein) in a cell/tissue. A gene is differentially expressed (DE) when its observed change between conditions (disease vs healthy, treated vs untreated) is unlikely to be due to random/technical variation alone.
⚙️
Detection Methods
From Fold-Change to GLMs
Fold-change ratio with cut-off
Two-sample / Welch's t-test (gene-by-gene)
ANOVA for >2 groups
SAM: regularised t-statistic + permutation FDR
Empirical Bayes (limma): borrows info across genes, stabilises variance for small n
edgeR / DESeq2: negative-binomial GLMs for over-dispersed RNA-seq counts
💡
SAM's Innovation
The "Fudge Factor"
SAM adds a small constant s₀ to the SE denominator, stabilising the test statistic for genes with very small variance (preventing spurious significance), and estimates FDR directly via permutation of group labels — more robust than a plain t-test in small-sample microarray settings.
⚠️
Why DE Analysis Matters
Importance
Identifies candidate genes/pathways for disease or treatment response
Discovers biomarkers for diagnosis, prognosis, drug targets
Multiple testingBenjamini–Hochberg FDR correction across thousands of genes
cDNA Microarray WorkflowRNA extraction → reverse transcription to fluorescently-labelled cDNA → hybridisation onto the array → scanning to quantify signal intensity → normalisation (correcting dye/array bias) → statistical DE testing (t-test/SAM/limma) with FDR correction.
Plain-English RecapRunning a plain t-test separately on 20,000 genes is like flipping 20,000 coins and being shocked that a few hundred land on six heads in a row by pure chance — that's why multiple-testing correction (FDR) is non-negotiable in genomics. SAM's "fudge factor" simply stops genes with almost-zero natural variability from looking falsely dramatic.
😂 Lab HumorTesting 20,000 genes without correcting for multiple testing is like buying 20,000 lottery tickets and announcing to the world that you've discovered a "winning strategy" because three of them happened to match a few numbers. FDR correction is statistics gently tapping you on the shoulder and saying, "maybe sit down."
· · ·
09
Sequencing Technology
Genome Sequencing — Sanger vs NGS
🧫
What it is
Determining Base Order
Genome sequencing determines the complete, precise order of nucleotide bases (A,T,G,C) across all chromosomes of an organism. Methods include Sanger (chain-termination), Next-Generation Sequencing (NGS), and third-generation long-read sequencing (PacBio, Oxford Nanopore).
⚖️
Sanger vs NGS
Key Trade-offs
Throughput: low, one fragment at a time (Sanger) vs millions in parallel (NGS)
Read length: long & accurate ~700–1000bp (Sanger) vs short ~100–300bp (short-read NGS)
Cost/base: high (Sanger) vs dramatically lower (NGS)
Accuracy: very high per-read (Sanger, gold standard) vs lower per-read but high aggregate accuracy via deep coverage (NGS)
Use: single-gene confirmation (Sanger) vs whole-genome/exome, RNA-seq, population studies (NGS)
💡
WGS, WES & Shotgun Strategy
Strategic Choices
Whole Genome Sequencing (WGS) covers everything; Whole Exome Sequencing (WES) targets only protein-coding exons — cheaper, focused on coding variants. Shotgun sequencing randomly fragments the genome, sequences overlapping pieces, and computationally reassembles them.
Plain-English RecapSanger sequencing is like carefully reading one sentence out loud, word by word, very accurately — perfect for double-checking a single gene. NGS is like photocopying the whole book into a million tiny shredded scraps, reading each scrap fast and cheaply, then using a computer to puzzle them all back together based on overlaps.
😂 Lab HumorShotgun sequencing got its name because the strategy is basically: blast the genome into tiny pieces first, ask questions later. It's the scientific equivalent of shredding a novel, reading each scrap, and reassembling the whole story from the torn edges — and somehow it works.
· · ·
10
Genome-Wide Mapping
Linkage Analysis, LD vs LE & OMICS Overview
🔬
Linkage Analysis
Goals in Statistical Genomics
Detect & confirm linkage between trait/disease loci and markers
Construct genetic (linkage) maps — order & distances of loci
Localise disease genes/QTLs to chromosomal regions
Estimate recombination fractions and modes of inheritance
⚖️
LD vs LE
Linkage Disequilibrium vs Equilibrium
LE: alleles at two loci associate by chance (D≈0) — loci far apart or many generations of recombination
LD: non-random allele association (D≠0) — loci physically close, recent mutation, selection, drift, admixture
LD is the basis of GWAS tag-SNP mapping; strong LD complicates fine-mapping of the causal variant
💡
Huntington's Disease
A Landmark Linkage Success
Gusella et al. (1983) mapped Huntington's disease to chromosome 4p16.3 using RFLP marker G8 in a Venezuelan family via LOD-score linkage analysis — before the causal gene (HTT) was known. HTT was cloned a decade later (1993), revealing the CAG-repeat expansion. This established the strategy of positional cloning for many Mendelian diseases.
🧬
OMICS Family
Beyond the Genome
Genomics: the complete DNA/gene set
Transcriptomics: the complete RNA transcript set
Proteomics: the complete protein set
Metabolomics: the complete small-molecule metabolite set
Metagenomics: genetic material from environmental/microbial communities
Pharmacogenomics: genome's influence on drug response
LD measuresD = pᴬᴮ − pᴬ·pᴮ (also expressed as D' or r² for normalised LD)
LE conditionD ≈ 0 (alleles associate as expected by chance)
Why Bioinformatics Matters in HealthDisease diagnosis & biomarker discovery, precision medicine, drug discovery & design, genetic counselling, pathogen surveillance, vaccine development, cancer genomics, and large-scale clinical/EHR data mining all rely on the OMICS disciplines above, integrated through statistical genomics methods.
Plain-English RecapLinkage equilibrium is two strangers who happen to sit together on a bus by pure coincidence — no real connection. Linkage disequilibrium is two old friends who always sit together because they live on the same street (i.e., are close on the chromosome) — that predictable pairing is exactly what GWAS exploits to find disease-associated regions without testing every single SNP.
😂 Lab HumorOMICS researchers have officially run out of restraint: first it was genomics, then transcriptomics, proteomics, metabolomics, epigenomics, metagenomics... at this rate we're one bad grant-writing day away from "snackomics" — the complete statistical study of what's in the lab fridge.
A signal is any quantity that varies with an independent variable (usually time or space) and carries information — a voltage, a sound pressure wave, a pixel-intensity grid, a voltage trace from the heart. Signal processing is the statistical and mathematical toolkit for extracting the information hidden inside that variation, despite noise.
🗂️
Classification by Nature
Continuous vs Discrete
Continuous-time (analog): defined at every instant, e.g. voice pressure wave
Discrete-time: defined only at sample instants, e.g. a sampled ECG trace
Continuous-valued vs quantised (digital): amplitude can take any value vs a finite set of levels
Digital signal = discrete-time AND quantised (what computers actually store)
💡
Classification by Behaviour
Deterministic vs Random
Deterministic: exactly reproducible by a formula, e.g. a pure sine wave
Random (stochastic): only describable statistically, e.g. thermal noise, biological signals with trial-to-trial variability
Periodic vs aperiodic: repeats after a fixed period or not
Deterministic vs stochastic distinction is exactly why statistics enters signal processing — most real signals are noisy and must be estimated, not read off directly
🔢
Classification by Energy
Energy vs Power Signals
An energy signal has finite total energy E=∫|x(t)|²dt and zero average power (e.g. a single pulse). A power signal has infinite energy but finite average power (e.g. a sinusoid that goes on forever). Most man-made communication and biomedical signals of interest are treated as power or finite-duration energy signals depending on context.
General modelx(t) = s(t) + n(t) (signal + noise)
Signal energyE = ∫ |x(t)|² dt
Signal powerP = lim_(T→∞) (1/T) ∫_(-T/2)^(T/2) |x(t)|² dt
Plain-English RecapAlmost nothing you record in the real world is a "clean" signal — a microphone, an ECG electrode, a satellite antenna all pick up your signal of interest tangled up with noise. Statistical signal processing exists precisely because we rarely see s(t) directly; we see x(t) = s(t) + n(t) and have to work backward.
😂 Lab HumorA deterministic signal is that one friend who always shows up exactly on time and says exactly the same thing. A random signal is the friend who shows up "sometime between 2 and 9pm" and you can only describe their behaviour using a probability distribution. Engineers call the second one "stochastic." Everyone else calls it "Dhaka traffic."
· · ·
02
Applications
Real-World Analog Signals — Audio, Video & Biomedical
🔊
Audio Signals
Acoustic Pressure Waves
Sound is a continuous variation in air pressure over time, roughly 20 Hz–20 kHz for human hearing. Microphones convert pressure to a continuous voltage signal — the classic 1-D time-varying analog signal, the textbook starting point for sampling theory.
🎞️
Video Signals
Intensity Varying in Space AND Time
Video is a sequence of 2-D image frames (spatial intensity/colour signals) sampled rapidly in time — a 3-D signal (x, y, t). Statistically, each pixel is itself noisy, and frame-to-frame estimation/detection (motion, edges, objects) is a core signal-processing problem.
🩺
Biomedical Signals
EEG, ECG, PET, CT, US
EEG: scalp electrical activity from neuronal firing — low amplitude (μV), heavily contaminated by muscle/eye-blink noise
ECG: electrical activity of the heart — quasi-periodic, used for arrhythmia/QRS detection
PET: detects gamma-ray pairs from a radioactive tracer — signal is a photon-count (Poisson) process, reconstructed into a metabolic image
CT: X-ray attenuation measured from many angles, reconstructed via inverse transforms (Radon transform) into a cross-sectional image
US (Ultrasound): reflected high-frequency sound pulses; signal = time-delay & amplitude of echoes
🛰️
SAR & Microarray Signals
Non-Traditional "Signals"
SAR (Synthetic Aperture Radar): a moving radar emits pulses and records the phase/amplitude of returned echoes; synthetic combination of many pulses over the platform's motion creates a high-resolution image — a classic statistical estimation problem, since raw radar echoes are extremely noisy
Microarray signal: fluorescence intensity at each spot on a chip is treated as a "signal" whose true expression level must be estimated from noisy optical measurement — the same estimation theory used for radar or biomedical data applies directly to gene-expression quantification
Plain-English RecapWhatever the source — sound waves, heartbeats, radar echoes, or glowing spots on a microarray chip — the underlying statistical problem is identical: a true quantity of interest is buried under measurement noise, and we need principled estimation/detection tools to dig it out. That's why the same estimation theory (ML, EM, CRLB, Bayesian estimators) shows up across audio engineering, medical imaging, radar, and genomics.
😂 Lab HumorAn EEG signal is basically your brain trying to whisper something important while a blinking eyelid stands right next to the microphone shouting. Untangling the whisper from the shout is 90% of what a signal-processing statistician does for a living.
· · ·
03
Digitisation
Digital Representation of Analog Signals
⏱️
Sampling
Continuous → Discrete Time
Sampling reads the analog signal's value at regular instants tₙ = nTₛ, producing a discrete-time sequence x[n] = x(nTₛ). The sampling rate fₛ = 1/Tₛ must be chosen carefully — too low, and information is permanently lost (aliasing).
📏
Quantisation
Continuous → Discrete Amplitude
Quantisation rounds each sample's amplitude to the nearest of a finite set of levels (e.g. 2ᴮ levels for a B-bit ADC), introducing quantisation error/noise. Finer quantisation (more bits) reduces this error but increases data storage and bandwidth needs.
💡
Nyquist–Shannon Sampling Theorem
The Golden Rule
A band-limited signal with maximum frequency fₘₐₓ can be perfectly reconstructed from its samples only if fₛ > 2fₘₐₓ (the Nyquist rate). Sampling below this rate causes aliasing — high frequencies masquerade as false low frequencies in the reconstructed signal.
🔢
ADC Pipeline
End-to-End Digitisation
Analog signal → anti-aliasing low-pass filter (removes frequencies above fₛ/2) → sample-and-hold → quantiser (ADC) → digital code (binary sequence) → stored/processed digital signal x[n].
Sampling & Aliasing — Adequate vs Insufficient Rate
Plain-English RecapSampling too slowly doesn't just give you a "blurry" version of the signal — it gives you a completely wrong, fake signal that looks perfectly legitimate (aliasing). That's the classic "wagon-wheel effect" in old movies where spinning wheels appear to rotate backward: the camera's frame rate (its sampling rate) is too slow for the wheel's true rotation speed.
😂 Lab HumorUndersampling is like trying to summarise a 3-hour movie using only 5 screenshots taken at random — you might end up believing the hero spent the entire film standing perfectly still, when really they were sprinting the whole time. The Nyquist theorem is just statistics politely saying, "take more screenshots."
· · ·
04
Transform Domain
Role of Transformation in Signal Processing
🔁
Why Transform?
A Different View of the Same Signal
A signal viewed in the time domain can hide structure that becomes obvious in another domain (frequency, scale, etc.). Transformations re-express a signal in a basis where the properties we care about — periodicity, bandwidth, sparsity, noise separation — are easier to see and to exploit.
🌊
Fourier Transform
Time → Frequency
Decomposes a signal into a sum/integral of sinusoids at different frequencies: X(f) = ∫x(t)e^(−j2πft)dt. Reveals which frequencies are present and how strong they are — essential for filtering noise (e.g. removing 50/60 Hz power-line interference from EEG) and for compression (most energy concentrated in few frequencies).
💡
Other Key Transforms
Beyond Fourier
Z-transform: discrete-time generalisation of the Laplace transform; central to digital filter design and stability analysis
Wavelet transform: localises both time AND frequency — ideal for non-stationary signals like ECG/EEG where frequency content changes over time
Radon transform: maps an image into line-integral "projections" — the mathematical basis of CT reconstruction
⚙️
Role in Estimation & Detection
Why This Matters Statistically
Many estimation/detection problems become far simpler in a transform domain: noise that is complicated in time may be simple (e.g. white/uncorrelated) in frequency; a weak periodic signal buried in noise may show a sharp, easily-detected peak in its spectrum. Transformations are therefore a key pre-processing step before applying ML, Bayesian, or detection-theoretic methods.
Plain-English RecapThink of the time domain as hearing an entire orchestra play at once, and the frequency domain as being handed the actual sheet music — suddenly you can see exactly which instrument (frequency) is playing how loudly, and mute the one you don't want (noise).
😂 Lab HumorThe Fourier Transform is the ultimate gossip: it takes one messy, tangled-up time signal and instantly tells you "here's exactly who's involved and how loud each one is being." Engineers have been quietly obsessed with this one trick for about 200 years, and honestly, fair enough.
· · ·
05
Estimation Theory
Signal Estimation Theory — The Basics
🎯
The Estimation Problem
Recovering θ from Noisy Data
Given observed data x = [x[0],...,x[N−1]] generated from a model x[n] = s(n;θ) + w[n] (signal depending on unknown parameter(s) θ, plus noise w[n]), estimation theory seeks a function θ̂(x) — an estimator — that produces a "best" guess of θ from the data.
⚖️
Estimator Properties
Desirable Estimator Properties
Unbiasedness: E[θ̂] = θ (correct "on average")
Efficiency: achieves the lowest possible variance among unbiased estimators
Consistency: θ̂ → θ as sample size N → ∞
Minimum Mean Square Error (MSE): MSE = Var(θ̂) + Bias(θ̂)² — the fundamental bias–variance trade-off
💡
Two Philosophies
Classical vs Bayesian
Classical (frequentist): θ is an unknown but fixed constant; only the data is random (leads to ML, MVU, CRLB, BLUE)
Bayesian: θ is itself a random variable with a prior distribution p(θ); we compute a posterior p(θ|x) (leads to MAP, MMSE, MAE)
🔢
Common Signal Models
What θ Might Represent
θ could be a DC level in noise, an amplitude/phase/frequency of a sinusoid (radar, sonar), a time-delay (ultrasound, SAR ranging), or an expression-intensity level (microarray spot). The estimation machinery is identical — only the model s(n;θ) changes.
Signal-plus-noise modelx[n] = s(n;θ) + w[n], n = 0,...,N−1
BiasBias(θ̂) = E[θ̂] − θ
Mean square errorMSE(θ̂) = E[(θ̂−θ)²] = Var(θ̂) + Bias(θ̂)²
📐 Where the MSE formula comes from
We want a single number that measures "how wrong is θ̂ typically?" The natural choice is the expected squared error, E[(θ̂−θ)²]. Add and subtract E[θ̂] inside the square, then expand:
E[(θ̂−θ)²] = E[((θ̂−E[θ̂]) + (E[θ̂]−θ))²]= E[(θ̂−E[θ̂])²] + 2(E[θ̂]−θ)·E[θ̂−E[θ̂]] + (E[θ̂]−θ)²
The middle cross-term vanishes because E[θ̂−E[θ̂]] = 0 by definition of the mean. What's left is exactly Var(θ̂) + Bias(θ̂)². This is why every estimator faces a trade-off: you can sometimes shrink variance by accepting a little bias, or vice versa — MSE is what tells you if that trade was worth it.
Bias vs Variance — the Dartboard Picture
Plain-English RecapEstimation is just an educated, mathematically optimal form of guessing. We can't remove noise from a single measurement, but with a good model and enough data, we can construct a rule (the estimator) that gets us as close to the truth as physically possible, on average.
😂 Lab HumorAn unbiased-but-high-variance estimator is like a dart player who, over a thousand throws, averages a perfect bullseye — but any single throw could land anywhere on the board, including the neighbour's yard. A biased-but-low-variance estimator is the player who always hits 2cm left of centre, every single time. Statisticians have strong opinions about which player they'd rather hire.
· · ·
06
Estimation Methods
Maximum Likelihood Estimation of Signal Parameters
📈
The ML Idea
Pick the θ that Makes the Data Most Probable
Given the likelihood p(x;θ) — the probability density of the observed data as a function of θ — the Maximum Likelihood Estimate (MLE) is the value θ̂ that maximises p(x;θ). Intuitively: "which parameter value would have made this exact data most likely to occur?"
🧮
Practical Recipe
Log-Likelihood & Calculus
Since log is monotonic, maximising p(x;θ) is equivalent to maximising the log-likelihood ln p(x;θ) — usually easier since products of exponentials (e.g. Gaussian noise terms) become sums. Set ∂(ln p(x;θ))/∂θ = 0 and solve for θ̂ (checking it's a maximum).
💡
Worked Example
DC Level in White Gaussian Noise
Model x[n] = A + w[n], w[n]~N(0,σ²) i.i.d. The MLE of A turns out to be simply the sample mean: Â = (1/N)Σx[n]. This matches intuition — average out the noise — and is also the MVU estimator here, achieving the CRLB exactly.
✅
Large-Sample Properties
Why ML Is So Popular
Asymptotically unbiased as N → ∞
Asymptotically efficient — attains the CRLB as N → ∞
📐 Where  = sample mean comes from
For x[n] = A + w[n] with w[n] ~ N(0,σ²) i.i.d., each observation has density p(x[n];A) = (1/√(2πσ²))·exp(−(x[n]−A)²/2σ²). Because the samples are independent, the joint likelihood is a product, and the log-likelihood turns that product into a sum:
ln p(x;A) = −(N/2)ln(2πσ²) − (1/2σ²) Σₙ (x[n]−A)²
Only the last term depends on A, so maximising the likelihood is the same as minimising Σₙ(x[n]−A)². Take the derivative with respect to A and set it to zero:
∂/∂A [ −(1/2σ²) Σₙ (x[n]−A)² ] = (1/σ²) Σₙ (x[n]−A) = 0⇒ Σₙ x[n] = N·A ⇒ Â = (1/N) Σₙ x[n]
That's the whole reason "just average the data" turns out to be mathematically optimal here — it directly falls out of maximising the Gaussian likelihood, not just intuition.
The Likelihood Function — Finding Its Peak
Plain-English RecapML estimation flips the usual probability question around. Normally we ask "given θ, what's the chance of seeing this data?" ML instead asks "given this data, which θ made it most plausible?" — running probability in reverse to point back at the most believable explanation.
😂 Lab HumorMaximum Likelihood Estimation is basically a detective walking into a crime scene and asking, "of all possible suspects, who would have made THIS EXACT mess most likely?" — then arresting that suspect. It works shockingly often, which says something either about the method or about detectives.
· · ·
07
Iterative Estimation
The EM Algorithm
🔄
Why EM Exists
ML When Data Is "Incomplete"
Sometimes the log-likelihood has no closed-form maximiser — often because some relevant data is missing, hidden, or mixed together (e.g. which of several sources produced each observed sample). The Expectation-Maximization (EM) algorithm handles exactly this "incomplete data" ML problem iteratively.
⚙️
The Algorithm
Two-Step Iteration
E-step (Expectation): given the current parameter estimate θ⁽ᵏ⁾, compute the expected value of the complete-data log-likelihood over the missing/hidden data
M-step (Maximization): maximise that expected log-likelihood over θ to obtain θ⁽ᵏ⁺¹⁾
Repeat E and M steps until θ converges
💡
Guaranteed Property
Monotonic Improvement
Each EM iteration is guaranteed to never decrease the observed-data log-likelihood — the algorithm climbs uphill (or stays flat) every step, though it may converge to a local rather than global maximum, so initialisation matters.
🧬
Signal-Processing Uses
Where EM Shows Up
Separating overlapping/mixed source signals (e.g. multiple biological or radar sources)
Gaussian Mixture Model fitting for clustering noisy signal features
Missing-sample or corrupted-sample reconstruction
Channel/parameter estimation in communications when the transmitted symbol sequence is also unknown
📐 Why Q(θ|θ⁽ᵏ⁾) is defined this way
We actually want to maximise the observed-data log-likelihood ln p(x;θ), but the hidden variable z makes that hard directly. Since p(x;θ) = p(x,z;θ)/p(z|x;θ), taking logs gives ln p(x;θ) = ln p(x,z;θ) − ln p(z|x;θ). Averaging both sides over the current best guess of z's distribution, p(z|x,θ⁽ᵏ⁾), removes z from the left side (it doesn't depend on z) and produces exactly the Q-function on the right:
ln p(x;θ) = Q(θ|θ⁽ᵏ⁾) − E_{z|x,θ⁽ᵏ⁾}[ln p(z|x;θ)]
A key inequality (from Jensen's inequality / Gibbs' inequality) guarantees the second term is maximised — not minimised — when θ=θ⁽ᵏ⁾. So any θ that increases Q also increases (or at least doesn't decrease) the true log-likelihood. That's precisely why "maximise Q instead of the real thing" is a safe, monotonically-improving substitute.
EM Algorithm — The Iteration Loop
Plain-English RecapEM is what you do when direct maximum likelihood hits a wall because some crucial piece of the puzzle is hidden. Instead of solving the hard problem directly, EM alternates between (1) making its best guess about the missing piece given current parameters, and (2) updating parameters given that guess — repeating until both stop changing.
😂 Lab HumorEM is basically two roommates splitting a chore they don't fully understand: one guesses "I think this is probably what needs doing" (E-step), the other says "okay, given that guess, here's the best way to actually do it" (M-step) — and they keep swapping notes until the apartment stops getting messier. Somehow, this converges to a clean house.
· · ·
08
Optimality Bounds
Minimum Variance Unbiased Estimation — CRLB & BLUE
🏆
MVU Estimator
The Best Unbiased Estimator
Among all unbiased estimators of θ, the Minimum Variance Unbiased (MVU) estimator is the one with the smallest variance for every possible true θ. It's the theoretical "gold standard" — if it exists and can be found, no other unbiased estimator does better.
📐
Cramér–Rao Lower Bound (CRLB)
The Speed Limit on Variance
The CRLB gives the theoretical minimum possible variance any unbiased estimator can achieve, based on the Fisher Information I(θ) — how "sharply peaked" the likelihood is around the true θ. No unbiased estimator can beat this bound; an estimator that achieves it exactly is called efficient.
💡
The Practical Alternative
BLUE — Best Linear Unbiased Estimator
When the exact MVU estimator is too hard to derive (e.g. noise PDF is unknown or non-Gaussian), the BLUE restricts the search to estimators that are linear functions of the data and finds the one with minimum variance among those — a practical compromise that only requires knowing the noise mean and covariance, not its full distribution.
🔢
The Information Measure
Fisher Information
Fisher Information I(θ) quantifies how much "information" the data carries about θ — formally, the expected curvature (negative second derivative) of the log-likelihood. Higher I(θ) → sharper likelihood peak → lower achievable variance → more precise estimation is possible.
BLUE (general linear model)θ̂_BLUE = (Hᵀ C⁻¹ H)⁻¹ Hᵀ C⁻¹ x (H = model matrix, C = noise covariance)
📐 Why the bound is 1/I(θ) — and why I(θ) is called "information"
Start from the fact that the score function ∂lnp(x;θ)/∂θ has mean zero (a standard regularity result) — it's a random quantity that, on average, doesn't push you away from the true θ. Its own variance, by definition, is:
Var(score) = E[(∂lnp/∂θ)²] = −E[∂²lnp/∂θ²] = I(θ)
(the middle equality is a standard identity for well-behaved likelihoods). Now apply the Cauchy-Schwarz inequality to Cov(θ̂, score) — the correlation between your estimator and the score can never exceed 1 in magnitude. Working through that inequality and using E[score]=0 leads directly to:
Var(θ̂) ≥ 1 / I(θ)
Intuitively: I(θ) measures how sharply the log-likelihood curves near the true θ (its negative curvature). A sharp, peaked likelihood (high curvature, high I(θ)) means even a tiny change in θ makes the data look very different — so the data carries a lot of "information" about θ, and a low-variance estimator is possible. A flat, broad likelihood (low I(θ)) means many θ values explain the data almost equally well — little information, and no estimator can pin θ down precisely, however clever it is.
CRLB — The Variance Floor No Unbiased Estimator Can Beat
Plain-English RecapThe CRLB is a "no free lunch" statement: however clever your unbiased estimator is, there's a hard floor on how small its variance can get, set entirely by how informative the data/model is. The BLUE is what you reach for when you want a good, mathematically simple estimator without needing to know the full noise distribution — you just need its mean and covariance.
😂 Lab HumorThe CRLB is the universe's way of saying, "I don't care how smart your estimator is — there's a floor, and you WILL hit it." It's basically a cosmic credit limit for precision. BLUE is the estimator equivalent of "I don't know your whole financial history, just tell me your average income and how erratic your spending is, and I'll do my best."
· · ·
09
Bayesian Estimation
Bayesian Estimators — MAP, MMSE & MAE
🎲
The Bayesian Shift
θ Is Random, Not Fixed
Instead of treating θ as an unknown constant, the Bayesian approach assigns it a prior distribution p(θ) representing what's known before seeing data, then combines it with the likelihood p(x|θ) via Bayes' rule to get the posterior p(θ|x) — the updated belief about θ after observing the data.
🏔️
Bayesian Point Estimators
MAP — Maximum A Posteriori
Chooses θ̂ as the value that maximises the posterior p(θ|x) — the single most probable value of θ given the data. Equivalent to ML when the prior is flat (non-informative), but MAP can incorporate genuine prior knowledge to stabilise noisy estimates.
💡
Bayesian Point Estimators
MMSE — Minimum Mean Square Error
Chooses θ̂ to minimise E[(θ−θ̂)²] under the posterior — the optimal solution turns out to be simply the posterior mean, E[θ|x]. MMSE penalises large errors quadratically, making it sensitive to outliers but easy to compute for Gaussian models.
📊
Bayesian Point Estimators
MAE — Minimum Absolute Error
Chooses θ̂ to minimise E[|θ−θ̂|] under the posterior — the optimal solution is the posterior median. MAE weighs errors linearly rather than quadratically, making it more robust to occasional large outliers than MMSE.
MAE estimatorθ̂_MAE = median of p(θ|x) (posterior median)
📐 Why each estimator lands where it does
All three estimators solve the same kind of problem — minimise E[Loss(θ,θ̂)|x] — just with a different loss function, and each has a clean, provable minimiser:
MMSE (squared-error loss): minimise E[(θ−θ̂)²|x]. Differentiate w.r.t. θ̂ and set to zero: −2E[θ−θ̂|x]=0 ⟹ θ̂=E[θ|x]. The posterior mean is the unique minimiser of squared error — this is the exact same algebra as ordinary least squares, just conditioned on x.MAE (absolute-error loss): minimise E[|θ−θ̂||x]. Splitting the expectation at θ̂ and differentiating shows the derivative is zero exactly when P(θ<θ̂|x) = P(θ>θ̂|x) = 0.5 — by definition, that point is the median.MAP (0-1 / "all-or-nothing" loss): as the tolerance window around θ̂ shrinks to zero, the estimator that maximises the chance of being "exactly right" converges to wherever p(θ|x) is highest — the posterior mode.
Since p(x) doesn't depend on θ, maximising the posterior is the same as maximising the numerator p(x|θ)·p(θ) alone — which is why the MAP formula only needs the likelihood times the prior, never the (often hard to compute) denominator p(x).
One Posterior, Three Different "Best Guesses"
Why the three points differFor a symmetric, single-peaked posterior (e.g. Gaussian), the mode, mean, and median all coincide — MAP = MMSE = MAE. They only pull apart when the posterior is skewed or multi-modal, as drawn above, which is exactly when it matters most which loss function you actually care about minimising.
Plain-English RecapMAP asks "what's the single most likely value, given everything I know?" MMSE asks "what value minimises my average squared regret?" and lands on the posterior's centre of mass. MAE asks the same but for absolute regret, landing on the posterior's midpoint. All three are "correct" — they just optimise different notions of what a "good guess" means.
😂 Lab HumorMAP, MMSE, and MAE walk into a bar and are asked to guess a stranger's age from a blurry photo. MAP says "I'm going with my single most confident guess." MMSE says "I'll average all my plausible guesses, weighted by how sure I am." MAE says "I'll pick the guess right in the middle, so I'm never TOO embarrassingly wrong." All three leave the bar reasonably satisfied, which never happens with actual strangers guessing ages.
· · ·
10
Practical Bayesian Estimation
Linear Bayesian Estimator
📏
Why "Linear"?
A Practical Compromise
Computing the true MMSE/MAP estimator generally requires knowing the full joint distribution of x and θ. The Linear Bayesian estimator restricts θ̂ to be a linear (affine) function of the data, θ̂ = a + Bx, and finds the a, B that minimise Bayes MSE — needing only means and covariances, much like the BLUE does in the classical setting.
🧮
Closed-Form Solution
General Form
For jointly Gaussian x and θ, the Linear (and in fact optimal) Bayesian estimator has a clean closed form using the cross-covariance between θ and x and the covariance of x — this is exactly the structure behind the Kalman filter and Wiener filter, two workhorses of applied signal processing.
💡
A Happy Coincidence
Special Case: Jointly Gaussian
When x and θ are jointly Gaussian, the MMSE estimator IS linear — so the Linear Bayesian estimator and the true MMSE estimator coincide exactly, no approximation needed. This is a major reason Gaussian noise assumptions are so convenient and widespread in signal processing.
🛰️
Where It's Used
Real Applications
Kalman filtering — recursive linear Bayesian estimation for tracking (radar target tracking, GPS navigation)
Wiener filtering — optimal linear filter for signal-from-noise recovery (audio denoising, image restoration)
Gaussian special caseIf (θ, x) jointly Gaussian ⇒ θ̂_MMSE = θ̂_linear (exact, not approximate)
📐 Where the linear formula comes from
Restrict the estimator to the affine form θ̂ = a + Bx and choose a, B to minimise the Bayes MSE, E[(θ−θ̂)²]. Two conditions from calculus pin the answer down:
Unbiasedness in expectation forces a = E[θ] − B·E[x], so θ̂ = E[θ] + B(x − E[x]).The "orthogonality principle" requires the estimation error (θ−θ̂) to be uncorrelated with the data x — otherwise there'd be leftover structure in x you failed to use. Writing this out, Cov(θ − B x, x) = 0, gives C_θx − B·C_xx = 0, i.e. B = C_θx · C_xx⁻¹.
Substituting B back gives the formula above. It only needs first and second moments (means, variances, covariances) — never the full joint density — which is exactly why it's the practical, "Bayesian BLUE" workhorse behind the Kalman and Wiener filters.
Plain-English RecapThe Linear Bayesian estimator is the pragmatic Bayesian's toolkit: rather than needing the entire probability landscape, it just needs to know how θ and the data typically move together (their covariance) and adjusts a simple straight-line rule accordingly. When everything happens to be Gaussian, this "simple approximation" turns out to be the perfect answer anyway.
😂 Lab HumorThe Linear Bayesian estimator is the friend who doesn't need your entire life story to give good advice — just tell them roughly how two things tend to move together, and they'll draw a surprisingly accurate straight line through your problems. Also, secretly, this friend IS the Kalman filter guiding your phone's GPS every time you walk somewhere.
· · ·
11
Decision Theory
Signal Detection Theory
🚦
Detection vs Estimation
A Different Kind of Question
Estimation asks "what is the value of θ?" Detection asks a yes/no question: "is a signal present at all?" — formulated as choosing between two (or more) competing hypotheses based on noisy observations, e.g. H₀: x[n]=w[n] (noise only) vs H₁: x[n]=s[n]+w[n] (signal present).
⚖️
The Optimal Test
Neyman-Pearson & the Likelihood Ratio Test
The Neyman-Pearson approach fixes an acceptable false-alarm probability P_FA and finds the test that maximises the probability of detection P_D subject to that constraint. The optimal test compares the likelihood ratio Λ(x)=p(x|H₁)/p(x|H₀) against a threshold γ — decide H₁ if Λ(x) > γ.
💡
Decision Table
Four Outcomes
Detection (true positive): say H₁, H₁ true — probability P_D
False alarm (false positive): say H₁, H₀ true — probability P_FA
Miss (false negative): say H₀, H₁ true — probability P_M = 1−P_D
Correct rejection (true negative): say H₀, H₀ true
📈
Visualising Trade-offs
ROC Curve
The Receiver Operating Characteristic (ROC) plots P_D against P_FA as the decision threshold γ varies, visually summarising the detector's fundamental trade-off — a better detector "bows" further toward the top-left corner (high P_D at low P_FA).
Matched filter (known signal in WGN)T(x) = Σₙ x[n]s[n] ≷ γ (optimal detector)
📐 Why the likelihood ratio is the optimal test (Neyman-Pearson)
We want to maximise P_D subject to a hard ceiling on P_FA — a constrained optimisation problem. Using a Lagrange multiplier λ to fold the constraint into the objective, we maximise:
P_D − λ(P_FA − α) = ∫_R p(x|H₁)dx − λ[∫_R p(x|H₀)dx − α]
over the choice of decision region R (which x values trigger "decide H₁"). Rearranging, we want to include a point x in R whenever p(x|H₁) − λ·p(x|H₀) > 0, i.e. whenever p(x|H₁)/p(x|H₀) > λ. That ratio is exactly the likelihood ratio Λ(x), and λ becomes the threshold γ — so the Neyman-Pearson Lemma proves the likelihood-ratio test isn't just convenient, it's provably the single best possible test for any fixed P_FA.
For the matched-filter case (known signal s[n] in white Gaussian noise), plugging in the two Gaussian densities and taking logs of Λ(x) > γ, all the terms that don't involve the data cancel, leaving only Σₙ x[n]s[n] > γ' — correlating the received data against a copy of the signal you're looking for. That's why "matched filtering" (cross-correlation with a template) is the provably optimal way to detect a known signal in noise.
Why Detection Always Trades Off P_D against P_FA
The ROC Curve — Sweeping the Threshold γ
Plain-English RecapDetection theory is the mathematics behind every "is it there or not?" decision under uncertainty — a radar operator deciding if that blip is a real aircraft or just noise, a doctor deciding if a faint signal on an ECG is a genuine arrhythmia, or a spam filter deciding if an email is real or junk. There's always a trade-off: crank up sensitivity and you catch more true signals, but you also trigger more false alarms.
😂 Lab HumorA detector with a super low threshold is like a smoke alarm that goes off every time you make toast — technically it never misses a real fire, but everyone's ripped it off the ceiling by week two. A detector with a super high threshold is the smoke alarm that stays dead silent even as the kitchen genuinely burns down. The Neyman-Pearson approach is just statistics negotiating a reasonable middle ground between "paranoid" and "asleep at the wheel."
Detecting differentially expressed genes — a detection-theoretic hypothesis test dressed up in biological language
📡
Signal Recovery
Communications & Audio
Channel estimation and equalisation (Linear Bayesian / Kalman-type estimators)
Speech/audio denoising (Wiener filtering)
Digital demodulation — deciding which symbol was sent is literally a detection problem
Why This Course MattersEstimation and detection theory form the shared statistical backbone underneath fields that look completely unrelated on the surface — a cardiologist reading an ECG, a radar engineer tracking a plane, and a geneticist screening a microarray are all, mathematically, solving the same kind of "find θ" or "is H₁ true?" problem, just dressed in different domain-specific clothing.
😂 Lab HumorBy the end of this course you'll realise that a cardiologist reading an ECG, a radar operator watching a screen for enemy planes, and a bioinformatician squinting at a microarray chip are all, deep down, doing the exact same statistics homework — they just get very different amounts of respect at parties for it.
📚 More M.Sc. Courses Coming Soon
This page will be extended with additional postgraduate course notes over time.
📚 Reference Texts
[1]
An Introduction to Statistical Genetic Data Analysis
Laird, N.M. & Lange, C. — MIT Press
[2]
Bioinformatics and Functional Genomics
Pevsner, J. — John Wiley & Sons · 3rd Ed.
[3]
Statistical Methods in Bioinformatics: An Introduction
Ewens, W.J. & Grant, G.R. — Springer · 2nd Ed.
[4]
Mapping and Analysis of Quantitative Trait Loci
Lynch, M. & Walsh, B. — Sinauer Associates
[5]
Statistics for Microarrays — Design, Analysis and Inference