Deep Learning · Memory

LSTM Gates

Set the gates. Forget≈1 holds the cell state on the highway; low forget clears it. Input writes, output reads.

forget σ 0.90 input σ 0.85 output σ 0.80 input
Cₜ xₜ
memory
—
read-out
—
regime
held
cell Cₜoutput hₜinput xₜ
tip: set forget = 1 and watch the cell state hold flat — then drop the output gate to hide it without erasing it
lstm.py