SECAI Core · phase 2 of 15 · noun

Differentially private stochastic gradient descent

A method used for machine learning data encryption to protect data privacy of the data endpoints by calculating a gradient of the training data and limiting it to ensure a single data point in the data set is not being used to dominate the training process

The Explain card

Plain English
DP-SGD is a training method that clips each example's gradient and adds calibrated noise, so no single data point can dominate what the model learns, protecting the privacy of individuals in the dataset.
Example
A hospital trains a diagnostic model with DP-SGD so that a patient's rare record cannot be reconstructed or confirmed from the finished model.
Why it matters
Models memorise, and membership inference and extraction attacks exploit that. DP-SGD gives a measurable privacy guarantee, at some cost in accuracy and training time.
Hook
Clip the loud voices, add a little static, and no single person can be heard.

Word knowledge

How the word is built, where it came from, and what it sits beside in memory.

In a sentence

In the Opacus training loop on the hospital GPU cluster, differentially private stochastic gradient descent updates the imaging model after each minibatch.

Why these words
  • differentially Latin differentia, a difference, from differre, to carry apart turns private into a measured adverb, so the claim is about how much two neighboring runs may differ
  • private Latin privatus, withdrawn from public life, from privare, to set apart names the withdrawn-from-public quality the adverb is scaling
  • stochastic Greek stokhastikos, skilled at aiming, from stokhos, a target marks the descent as a sampled walk, so each step draws a random batch rather than the full set
  • gradient Latin gradiens, walking, from gradi, to step names the slope the walk follows, the steepest-change direction
  • descent Latin descensus, a climbing down, from descendere, to go down names the downhill walk, the head noun the rest of the stack modifies
Where it came from
Origin
English computing compound by Martin Abadi and colleagues, from Latin differentia (a difference, from differre, to carry apart), Latin privatus (withdrawn from public life), Greek stokhastikos (skilled at aiming, from stokhos, a target), Latin gradiens (walking, from gradi, to step), and Latin descensus (a climbing down)
Entered the language
2016
What changed
Differential had meant a difference since the 1300s, then a calculus increment, then, after Dwork's 2006 paper, a privacy guarantee measured in distances. Stochastic had meant skilled at aiming, then conjectural, then random in the statistical sense. Gradient had meant a slope you walk, and descent the walk downhill. Labs had already stacked stochastic onto gradient descent for sampling. Abadi's 2016 paper hung differentially private in front of that stack, and the five words slid from a conference heading to the everyday label for that style of training.
How it is spelled
Pattern
-ly turns an adjective into an adverb
Pattern
Latin ferre keeps a double f after the prefix in differ
Pattern
Greek stokhos writes ch for /k/ in stochastic
Pattern
-ent from Latin present participles stays -ent
Pattern
Latin scandere keeps sc in descent, ascent, and transcend
Breaks the pattern
the noun phrase stays five words; hyphens appear only when the stack modifies another noun
Breaks the pattern
stochastic writes ch for a /k/ that Greek spelled with chi
Breaks the pattern
differentially keeps the double f of differ, which English often simplifies in other ferre compounds such as refer

Spelled like

  • privately
  • different
  • chronic
  • ascent
Broken into chunks
  • differ
    • different
    • indifferent
    • difference
  • fer
    • transfer
    • confer
    • refer
  • tial
    • sequential
    • essential
    • confidential
  • ly
    • privately
    • locally
    • recently
  • priv
    • privacy
    • privilege
    • deprive
  • stoch
    • stochasticity
    • stochastically
    • stochastics
  • grad
    • gradual
    • graduate
    • degrade
  • scend
    • ascend
    • descend
    • transcend
What it sits beside

Same subject

Same shape

  • stochastic gradient descent
  • batch gradient descent
  • minibatch gradient descent
  • projected gradient descent

Privacy-preserving training

Gradient-descent family

  • stochastic gradient descent
  • batch gradient descent
  • minibatch gradient descent

Stochastic family

  • stochastic process
  • stochastic approximation
  • stochastic rounding

Where it sits in the deck

Phase 2: How Models Are Built: Architectures and Learning Mechanics

With the paradigms named, zoom in on the architectural building blocks that turn data into trained artifacts.

Neighbours in this phase