BERT:Bidirectional Encoder Representations from Transformers Main ideas Merits of BERT Propose a new pre-training objective Just fine-tune BERT model for specific so that a deep bidirectional tasks to achieve state-of-the-art Transformer can be trained performance ·The“masked language model" BERT advances the state-of-the-art for (MLM):the objective is to predict the eleven NLP tasks original word of a masked word based only on its context "Next sentence prediction" 里)亲之大2024/5/13 16 BERT: Bidirectional Encoder Representations from Transformers Main ideas Propose a new pre-training objective so that a deep bidirectional Transformer can be trained • The “masked language model” (MLM): the objective is to predict the original word of a masked word based only on its context • ”Next sentence prediction” Merits of BERT Just fine-tune BERT model for specific tasks to achieve state-of-the-art performance BERT advances the state-of-the-art for eleven NLP tasks