What is Recurrent Neural Network (RNN)?
Recurrent Neural Network (RNN) — A type of neural network specialized for processing sequential data like text or time series.
RNNs process data sequentially, maintaining an internal memory of previous inputs. They were the dominant architecture for language tasks before Transformers. Today, RNNs are still used for time series forecasting, signal processing, and applications where Transformers are overkill.
Frequently Asked Questions
Why were RNNs replaced by Transformers?
RNNs process data one step at a time, making them slow to train and prone to forgetting long-range patterns. Transformers process entire sequences in parallel, enabling faster training and better long-range understanding.
Are RNNs still useful?
Yes, for specific use cases. Time series forecasting, audio processing, and small-scale sequence tasks where Transformer overhead is unnecessary. LSTMs (a type of RNN) remain popular for these applications.
What is an LSTM?
Long Short-Term Memory — an improved RNN variant that can remember information over long sequences. It solved the vanishing gradient problem that limited standard RNNs.