Newbie_step_llm

Keeping a record about how I begin to pick up LLM

Doc. written in Obsidian, which makes it inconvenient to make updates here due to multiple exclusive link format. I will update once and all when I finish and export html or pdf version.

Log

1. Architecture
   1. simple version & Basic idea
      1. overview
      2. Encoder, Decoder
      3. Attention
         • Coding: Attention
   2. comprehensive version
      1. Encoder-only: BERT
         1. architecture
         2. pre-training
      2. Decoder-only: GPT
         1. Brief history: What's new, what's the differences
         2. pre-training
         • tokenizer, embedding, transformer block, im_head
      3. Encoder-Decoder
      4. Comparation
         • bert vs. GPT
         • decoder-only vs. encoder-decoder
      • Coding <- I'm here, have been preparing for an exam recently, catch up soon
2. Training process
   1. Overview
   2. [ every step ]
3. Use LLM
   • 实战1
   • 实战2
4. Research