Self-Attention
The type of attention that calculates the relationship of each element in a number with all other items in the same directory.
Öz-dikkat (self-attention) is a kind of attention mechanism that calculates the relationship with all other elements in a number (e.g. in a sentence) and in parallel. The difference from the classic remarkable mechanism is derived from the same array of query, key and value vectors; that is, when processing a word in a sentence, they update the contextual meaning of this word by looking at other words in the sentence itself. For example, the word "bank" can focus on a different understanding (finance institution) in a phrase "nehir" and "kenar" in the sentence "we sat the bank on the edge of the river."
The biggest technical advantage of the essence-dickatin is that it can be paralleled: Unlike RNNs, the account for all locations in the array can be done at the same time, with matrix palpitations; which provides a lot of acceleration in the GPUs, and the very long-distance dependency of the model - for example, the relationship between the verb at the end of a paragraph - allows direct capture. In the technique called "Multi-head" (multi-head) attention, the model simultaneously learns in parallel with the different aspects of the language (specific, meaningal). Öz-dikkat is the direct source of the processing power and scalability of the transformer architecture and therefore large language models of today.
The biggest technical advantage of the essence-dickatin is that it can be paralleled: Unlike RNNs, the account for all locations in the array can be done at the same time, with matrix palpitations; which provides a lot of acceleration in the GPUs, and the very long-distance dependency of the model - for example, the relationship between the verb at the end of a paragraph - allows direct capture. In the technique called "Multi-head" (multi-head) attention, the model simultaneously learns in parallel with the different aspects of the language (specific, meaningal). Öz-dikkat is the direct source of the processing power and scalability of the transformer architecture and therefore large language models of today.