Tokenization and Embedding: Science Behind Large Language Model Every input that we are providing to GPT is nothing but a token (numerical id) or a sequence of tokens. GPT doesn’t understand the language the way humans do but it just processes sequence of numerical ids, that we call tokens. But how does it find the association among words(tokens) and provide human like response, here comes the c..

Basic Models input 프랑스 단어들을 받는 부분 -> 인코더 output 영어 단어들을 출력 -> 디코더 + 충분한 양의 input / output 단어 쌍들이 있다면, 해당 구조도 working 많이 길지 않은 문장을 output으로 낸다면 image captioning도 가능 sequence to seq image to seq Picking the Most Likely Sentence condition으로 프랑스어 단어가 들어왔을 때, 영단어의 확률을 예측하는 것 -> conditional probablity 랜덤하게 뽑아내다가는 이상한 문장을 만들 수 있다. 따라서 확률 값을 최대화하는 문장을 예측하는 것이 적합함 따라서 most likely english sentence 윗 문장이 더..

Transformer Abstract The dominant sequence transduction models are based on complex recurrent or convolutional neural networks that include an encoder and a decoder. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and conv..