Title of education
Learn your own LLM: BERT
Architecture and principles of deep mesh. The principles of learning the deep web. Activation functions. Functions of loss.
Differences of recurrent network and transformer network. Encoders, decoders and encoders. Attention mechanisms: Self attention and Multihead attention.
Content of the training:
Learning a language model on recurrent and transformer architecture. Authoregressive and masked learning.
Static and dimenic (contextualized) vectors for the representation of languages - embeddings.
BERT model with variations: RoBERT, AlBERT, SpanBERT.
Transfer learninig principles – fine-tuning the model to a user task from the domain of sequence prediction, classification or sentence generation.
Duration: 4h
Location: Faculty of Informatics and Digital Technologies, Radmile Matejčić 2, 51000 Rijeka
Term: 27.3.2024, 22.05.2024, 14-19h
