W가 only parameter, nx dim vector. b는 real number loss func - single training example에 대한 error cost func - cost of your params (전체 데이터에 대해, Parameter W, b의 평균 에러를 의미) Gradient Descent slope of the function Derivatives 직선이라면 (1차 함수) a의 값에 무관하게, 함수의 증가량은 변수 증가량의 3배 즉 3으로 미분값이 일정하다 Computation Graph Derivatives with a Computation Graph Logistic Regression Gradient Descent Gradient Descent on m Exam..

Neural Networks and Deep Learning 1. Introduction to Deep Learning Supervised Learning with Neural Networks Why is Deep Learning taking off? + 생각보다 영어 퀴즈가 굉장히 까다롭다 . . .

Swin Transformer Hierarchical Vision Transformer using Shifted Windows Abstract This paper presents a new vision Transformer, called Swin Transformer, that capably serves as a general-purpose backbone for computer vision. Challenges in adapting Transformer from language to vision arise from differences between the two domains, such as large variations in the scale of visual entities and the high..

Transformers for image recognition Model overview. We split an image into fixed-size patches, linearly embed each of them, add position embeddings, and feed the resulting sequence of vectors to a standard Transformer encoder. In order to perform classification, we use the standard approach of adding an extra learnable “classification token” to the sequence. Abstract While the Transformer archite..

Transformer Abstract The dominant sequence transduction models are based on complex recurrent or convolutional neural networks that include an encoder and a decoder. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and conv..

DenseNet Abstract Recent work has shown that convolutional networks can be substantially deeper, more accurate, and efficient to train if they contain shorter connections between layers close to the input and those close to the output. In this paper, we embrace this observation and introduce the Dense Convolutional Network (DenseNet), which connects each layer to every other layer in a feed-forw..

Inception Module The fundamental way of solving both issues would be by ultimately moving from fully connected to sparsely connected architectures, even inside the convolutions. adding an alternative parallel pooling path As these “Inception modules” are stacked on top of each other, their output correlation statistics are bound to vary; as features of higher abstraction are captured by higher l..

ResNet Intro Short Cut connections adding neither extra params nor computational complexity 100 ~ 1000 layers / 3.57% Conclusion 장점 아이디어가 독창적이다. 간단한 아이디어로 네트워크의 깊이를 매우 깊게 쌓았다. optimization 문제를 해결, 성능 향상 단점 왜 이러한 skip connection을 사용하면 성능이 더 잘 나오는 지 기존의 gradient descent 의 정의와 방향이 달라지지 않나? 개선할 점 다른 function들도 skip 해보기 굳이 - x, + x 되어야 하는 이유? + x, - x 순서로 진행되면 어떠한 차이가 있을지 잔차라는 개념이 어디에서 처음 나왔을까(relate..

참고한 코드: https://github.com/mrzhu-cool/pix2pix-pytorch GitHub - mrzhu-cool/pix2pix-pytorch: PyTorch implementation of "Image-to-Image Translation Using Conditional Adversarial Network PyTorch implementation of "Image-to-Image Translation Using Conditional Adversarial Networks". - GitHub - mrzhu-cool/pix2pix-pytorch: PyTorch implementation of "Image-to-Image Transl... github.com network.py import ..