Transformer
Based on the attention mechanism, the structure of the nervous network, which constitutes the architectural basis of today’s large language models.
Transformer is the nervous network structure, which was introduced by the article "Attention Is All You Need" published by Google researchers in 2017 and created almost the architectural foundation of today's large language models. The previous repetitive nerve networks (RNN) and LSTMs were forced to capture both slowed and long-distance dependency as word, sorted. Transformer calculates its relationship with all other words of each word in a number, using the mechanism of self-dickat (self-attention). This has made it much faster in the GPUs of both training and protects the consistency of the model in very long contexts.
A Transformer block is typically composed of increased kneeling of advanced-feeded nerve net layers with self-solid layers; no longer links (residual connections) and layer normalization helps very deep networks to be stable trained. Original architectural encoder-coder (encoder-decoder) was in its structure; models such as GPT only use coder, BERT only the encoder part. The scalability of the transformer - i.e. showing continuous performance increase with more data and parameters - the large language model has been directly triggered by the age, the image beyond the language processing (Vision Transformer) and audio processing are also spread to areas.
A Transformer block is typically composed of increased kneeling of advanced-feeded nerve net layers with self-solid layers; no longer links (residual connections) and layer normalization helps very deep networks to be stable trained. Original architectural encoder-coder (encoder-decoder) was in its structure; models such as GPT only use coder, BERT only the encoder part. The scalability of the transformer - i.e. showing continuous performance increase with more data and parameters - the large language model has been directly triggered by the age, the image beyond the language processing (Vision Transformer) and audio processing are also spread to areas.