개발의숲

Transformer architectures have become one of the predominant architecture of processing text in NLP. It is typically pre-trained on a large text corpus and then fine-tuned on a task-specific dataset, allowing it to scale effectively to large-scale applications. Inspired by the success in NLP, many attempts have been made in the vision domain to move away from conventional convolution-based appr..