{"id":6178,"date":"2024-01-11T11:46:08","date_gmt":"2024-01-11T10:46:08","guid":{"rendered":"https:\/\/edihadria.eu\/?page_id=6178"},"modified":"2024-10-21T16:44:31","modified_gmt":"2024-10-21T14:44:31","slug":"naucite-vlastiti-llm-bert","status":"publish","type":"page","link":"https:\/\/edihadria.eu\/en\/naucite-vlastiti-llm-bert\/","title":{"rendered":"Learn your own LLM: BERT"},"content":{"rendered":"
\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Our services<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Education of experts
the IT Industry<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t

\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t
\n\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Title of education<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Learn your own LLM: BERT<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Architecture and principles of deep mesh. The principles of learning the deep web. Activation functions. Functions of loss.
Differences of recurrent network and transformer network. Encoders, decoders and encoders. Attention mechanisms: Self attention and Multihead attention.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t

\n\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Content of the training:<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Learning a language model on recurrent and transformer architecture. Authoregressive and masked learning.
Static and dimenic (contextualized) vectors for the representation of languages - embeddings.
BERT model with variations: RoBERT, AlBERT, SpanBERT.
Transfer learninig principles \u2013 fine-tuning the model to a user task from the domain of sequence prediction, classification or sentence generation.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Duration: 4h<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\tContractors: Prof. dr. sc. Sanda Martin\u010di\u0107-Ip\u0161i\u0107 \/ Andrija Poleksi\u0107, Faculty of Informatics and Digital Technologies\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Location: Faculty of Informatics and Digital Technologies, Radmile Matej\u010di\u0107 2, 51000 Rijeka<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Term: 27.3.2024, 22.05.2024, 14-19h<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Log in<\/a><\/h1>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\n\t\t\t\t\t\t<\/span>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t
\n\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

<- return<\/a><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>","protected":false},"excerpt":{"rendered":"

Title of education Learn your own LLM: BERT Architecture and Deep Grid Principles. Principles of deep network learning. Activation functions. Loss functions.Differences of recurrent network and transformer network. Encoders, decoders and encoder-decoders. Attention mechanisms: Self attention and Multihead attention. Content of the training: Learning the language model on recurrent and transformer architecture. Authoregressive and masked learning.Statistical and dimenic (contextualized) vectors for representation [...]<\/p>","protected":false},"author":14,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"elementor_header_footer","meta":{"footnotes":""},"class_list":["post-6178","page","type-page","status-publish","hentry"],"yoast_head":"\nNau\u010dite vlastiti LLM: BERT<\/title>\n<meta name=\"description\" content=\"EDIH Adria (European Digital Innovation HUB), projekt financiran iz programa Digitalna Europa i Nacionalnog plana oporavka i otpornosti \u010diji je glavni cilj omogu\u0107avanje malim poduzetnicima i jedinicama lokalne samouprave ubrzanje procesa digitalizacije njihovih organizacija, pospje\u0161ivanje inovacijskog procesa, stvaranje prilika za testiranje ideja i rje\u0161enja prije po\u010detka investicijskog procesa.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/edihadria.eu\/en\/naucite-vlastiti-llm-bert\/\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Nau\u010dite vlastiti LLM: BERT\" \/>\n<meta property=\"og:description\" content=\"EDIH Adria (European Digital Innovation HUB), projekt financiran iz programa Digitalna Europa i Nacionalnog plana oporavka i otpornosti \u010diji je glavni cilj omogu\u0107avanje malim poduzetnicima i jedinicama lokalne samouprave ubrzanje procesa digitalizacije njihovih organizacija, pospje\u0161ivanje inovacijskog procesa, stvaranje prilika za testiranje ideja i rje\u0161enja prije po\u010detka investicijskog procesa.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/edihadria.eu\/en\/naucite-vlastiti-llm-bert\/\" \/>\n<meta property=\"article:modified_time\" content=\"2024-10-21T14:44:31+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/edihadria.eu\/wp-content\/uploads\/2023\/07\/Roboti-02.png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/edihadria.eu\\\/naucite-vlastiti-llm-bert\\\/\",\"url\":\"https:\\\/\\\/edihadria.eu\\\/naucite-vlastiti-llm-bert\\\/\",\"name\":\"Nau\u010dite vlastiti LLM: BERT\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/edihadria.eu\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/edihadria.eu\\\/naucite-vlastiti-llm-bert\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/edihadria.eu\\\/naucite-vlastiti-llm-bert\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/edihadria.eu\\\/wp-content\\\/uploads\\\/2023\\\/07\\\/Roboti-02.png\",\"datePublished\":\"2024-01-11T10:46:08+00:00\",\"dateModified\":\"2024-10-21T14:44:31+00:00\",\"description\":\"EDIH Adria (European Digital Innovation HUB), projekt financiran iz programa Digitalna Europa i Nacionalnog plana oporavka i otpornosti \u010diji je glavni cilj omogu\u0107avanje malim poduzetnicima i jedinicama lokalne samouprave ubrzanje procesa digitalizacije njihovih organizacija, pospje\u0161ivanje inovacijskog procesa, stvaranje prilika za testiranje ideja i rje\u0161enja prije po\u010detka investicijskog procesa.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/edihadria.eu\\\/naucite-vlastiti-llm-bert\\\/#breadcrumb\"},\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/edihadria.eu\\\/naucite-vlastiti-llm-bert\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\\\/\\\/edihadria.eu\\\/naucite-vlastiti-llm-bert\\\/#primaryimage\",\"url\":\"https:\\\/\\\/edihadria.eu\\\/wp-content\\\/uploads\\\/2023\\\/07\\\/Roboti-02.png\",\"contentUrl\":\"https:\\\/\\\/edihadria.eu\\\/wp-content\\\/uploads\\\/2023\\\/07\\\/Roboti-02.png\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/edihadria.eu\\\/naucite-vlastiti-llm-bert\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Po\u010detna stranica\",\"item\":\"https:\\\/\\\/edihadria.eu\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Nau\u010dite vlastiti LLM: BERT\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/edihadria.eu\\\/#website\",\"url\":\"https:\\\/\\\/edihadria.eu\\\/\",\"name\":\"\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/edihadria.eu\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-GB\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Nau\u010dite vlastiti LLM: BERT","description":"EDIH Adria (European Digital Innovation HUB), projekt financiran iz programa Digitalna Europa i Nacionalnog plana oporavka i otpornosti \u010diji je glavni cilj omogu\u0107avanje malim poduzetnicima i jedinicama lokalne samouprave ubrzanje procesa digitalizacije njihovih organizacija, pospje\u0161ivanje inovacijskog procesa, stvaranje prilika za testiranje ideja i rje\u0161enja prije po\u010detka investicijskog procesa.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/edihadria.eu\/en\/naucite-vlastiti-llm-bert\/","og_locale":"en_GB","og_type":"article","og_title":"Nau\u010dite vlastiti LLM: BERT","og_description":"EDIH Adria (European Digital Innovation HUB), projekt financiran iz programa Digitalna Europa i Nacionalnog plana oporavka i otpornosti \u010diji je glavni cilj omogu\u0107avanje malim poduzetnicima i jedinicama lokalne samouprave ubrzanje procesa digitalizacije njihovih organizacija, pospje\u0161ivanje inovacijskog procesa, stvaranje prilika za testiranje ideja i rje\u0161enja prije po\u010detka investicijskog procesa.","og_url":"https:\/\/edihadria.eu\/en\/naucite-vlastiti-llm-bert\/","article_modified_time":"2024-10-21T14:44:31+00:00","og_image":[{"url":"https:\/\/edihadria.eu\/wp-content\/uploads\/2023\/07\/Roboti-02.png","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_misc":{"Estimated reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/edihadria.eu\/naucite-vlastiti-llm-bert\/","url":"https:\/\/edihadria.eu\/naucite-vlastiti-llm-bert\/","name":"Nau\u010dite vlastiti LLM: BERT","isPartOf":{"@id":"https:\/\/edihadria.eu\/#website"},"primaryImageOfPage":{"@id":"https:\/\/edihadria.eu\/naucite-vlastiti-llm-bert\/#primaryimage"},"image":{"@id":"https:\/\/edihadria.eu\/naucite-vlastiti-llm-bert\/#primaryimage"},"thumbnailUrl":"https:\/\/edihadria.eu\/wp-content\/uploads\/2023\/07\/Roboti-02.png","datePublished":"2024-01-11T10:46:08+00:00","dateModified":"2024-10-21T14:44:31+00:00","description":"EDIH Adria (European Digital Innovation HUB), projekt financiran iz programa Digitalna Europa i Nacionalnog plana oporavka i otpornosti \u010diji je glavni cilj omogu\u0107avanje malim poduzetnicima i jedinicama lokalne samouprave ubrzanje procesa digitalizacije njihovih organizacija, pospje\u0161ivanje inovacijskog procesa, stvaranje prilika za testiranje ideja i rje\u0161enja prije po\u010detka investicijskog procesa.","breadcrumb":{"@id":"https:\/\/edihadria.eu\/naucite-vlastiti-llm-bert\/#breadcrumb"},"inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/edihadria.eu\/naucite-vlastiti-llm-bert\/"]}]},{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/edihadria.eu\/naucite-vlastiti-llm-bert\/#primaryimage","url":"https:\/\/edihadria.eu\/wp-content\/uploads\/2023\/07\/Roboti-02.png","contentUrl":"https:\/\/edihadria.eu\/wp-content\/uploads\/2023\/07\/Roboti-02.png"},{"@type":"BreadcrumbList","@id":"https:\/\/edihadria.eu\/naucite-vlastiti-llm-bert\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Po\u010detna stranica","item":"https:\/\/edihadria.eu\/"},{"@type":"ListItem","position":2,"name":"Nau\u010dite vlastiti LLM: BERT"}]},{"@type":"WebSite","@id":"https:\/\/edihadria.eu\/#website","url":"https:\/\/edihadria.eu\/","name":"","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/edihadria.eu\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-GB"}]}},"_links":{"self":[{"href":"https:\/\/edihadria.eu\/en\/wp-json\/wp\/v2\/pages\/6178","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/edihadria.eu\/en\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/edihadria.eu\/en\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/edihadria.eu\/en\/wp-json\/wp\/v2\/users\/14"}],"replies":[{"embeddable":true,"href":"https:\/\/edihadria.eu\/en\/wp-json\/wp\/v2\/comments?post=6178"}],"version-history":[{"count":16,"href":"https:\/\/edihadria.eu\/en\/wp-json\/wp\/v2\/pages\/6178\/revisions"}],"predecessor-version":[{"id":8863,"href":"https:\/\/edihadria.eu\/en\/wp-json\/wp\/v2\/pages\/6178\/revisions\/8863"}],"wp:attachment":[{"href":"https:\/\/edihadria.eu\/en\/wp-json\/wp\/v2\/media?parent=6178"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}