✨ Awesome-Anything

Awesome-AI-Researchs

Awesome-GitHub-Repos

Awesome-AI-Examples

🖼️ Digital Image Processing

DIP I Image Demosaicing and Histogram Manipulation

DIP II Image Denoising

DIP III Edge Detection

DIP IV Digital Half-toning

DIP V Geometric Image Modification

🍃 LLM Components

1.1 Tokenization

1.2 Embedding

1.3 Attention

1.4 FFN, Residual Addition, and LN

1.5 Positional Encoding

1.6 Architecture & Decoding Policy

🌱 LLM Pre-training

2.1 Pre-training Definition

2.2 Pre-training Data

2.3 Pre-training Workflow

2.4 Continued Pre-training (CPT)

2.5 Pre-training Evaluation

☘️ LLM Post-Training

3.1 Supervised Fine-Tuning (SFT)

3.2 Reinforcement Learning

3.3 RLHF: Reinforcement Learning from Human Feedback

3.4 Optimizing PPO-Based Algorithms

3.5 Direct Preference Optimization (DPO)

🍀 LLM Popular Models

🪴 LLM Applications

5.1 Prompt Engineering

5.2 LLM-based Agents

5.3 Retrieval-Augmented Generation (RAG)

5.4 Enterprise-level RAG Practice (I)

5.5 Enterprise-level RAG Practice (II)

🌿 LLM Optimization

6.1 GPU Usage Analysis

6.2 Latency Analysis for Inference

6.3 FlashAttention

6.4 PagedAttention

6.5 Packing for Efficient Training

6.6 PEFT

🌾 LLM Compression

7.1 Quantization

7.2 Pruning

7.3 Knowledge Distillation

🌵 LLM Hands-on Practice

8.1 nanoGPT: Training a GPT from Scratch

8.2 Instruction Fine-Tuning Qwen3-0.6B: A Minimal Working Example

8.3 OpenRLHF: SFT Qwen3-0.6B and Code Analysis

8.4 OpenRLHF: Reward Model Training and Code Analysis

8.5 OpenRLHF: PPO Tutorial and Code Analysis

🌴 LLM Must-read Papers

9.1 Technical Reports Explained

9.2 MedLLM

9.3 o1 Techinical Paper

9.4 Prompting

9.5 LLM-as-Judge

🌳 LLM Q&A

10.1 Why GRPO training loss starts at 0

10.2 VLM-R1 Reproduction Results

10.3 KL(P||Q) or KL(Q||P)?

10.4 Left Padding or Right Padding?

10.5 What is QK-Norm?

10.6 Why LLM training rarely adopts Dropout?

10.7 Dropout and DropPath

🐝 VLM Image Encoders

11.0 Introduction

11.1 Vision Transformer

📝 MISC.

🤔 Large Language Models Pass the Turing Test.

OpenAI ChatGPT Progression

⏳ Claude Sonnet 4 Test with Sandtris

🔍 What DeepSeek-R1's Surge Teaches Us About SEO

💰 CivitAI’s Payment Issue

🎓 arXivSub, an Academia Assistant

8.2 Instruction Fine-Tuning Qwen3-0.6B: A Minimal Working Example

Qwen3 was released on April 29, 2025. Where Qwen3-0.6B is a 0.6 billion-parameter model in the Qwen series. Its small size makes it feasible to fine-tune on Google Colab (T4 GPU with 15 GB memory), or a local environment (at least have 16GB GPU memory). We'll walk through how to perform supervised fine-tuning (SFT) on Qwen3-0.6B using Google Colab. Full code is provided in the end.

Before fine-tuning, we need to ensure our Colab runtime has a GPU and all required libraries are installed. In Colab, go to Runtime > Change runtime type > Hardware Accelerator and select GPU (see Chapter 8.1 nanoGPT). Then, install a new version of Hugging Face Transformers and other libraries we'll use. During installation, it may has package version conflict error, we can simply ignore the error.

We will:

Comparing an instructed Qwen3-0.6B model with the base model.

Prepare a small dataset of example instructions and answers, and fine-tune the model.

Finally, test the fine-tuned model to observe the improvements in its responses.

8.2.1 Comparing an Instructed Model Witha Base Model

Test a Instructed Model

Loading the Qwen3-0.6B Model: We use the AutoTokenizer and AutoModelForCausalLM classes to download the tokenizer and model weights. Enable device_map="auto" to automatically put the model on the GPU, and torch_dtype="auto" to use the GPU's preferred data type (which will be BP16 for Qwen3)

Testing the Base Model: let's see how the instructed model behaves

The output is:

apply_chat_template function fits the message into the chatml template, with special token (<|im_start|>, <|im_end|>, <think>, </think>) padding.

Let’s inspect the model’s output token by token:

The second column is the subword unit (sub) or raw token string as stored in the vocabulary (often includes special marks like Ġ for space or continuation). The third column is the Python string representation (repr(txt)) of the decoded token — how it would look if you printed it in code, showing spaces, escapes, etc.

We can find that token id 151645 represents the <EOS> token, which is used as a mark to stop generating. We can find how Qwen3-0.6B introduce this token in Qwen3-0.6B model’s tokenizer.

Test a Base Model

Before fine-tuning, let's see how the base model behaves. Remember that without SFT, the model might not follow the instructions properly. It could produce irrelevant text, incomplete answers, or even some gibberish.

Loading the Qwen3-0.6B-Base Model

Testing the Instructed Model With the Template

The model’s output is gibberish because it doesn’t know the meaning of special tokens.

Now, we try to keep only the input content, without any special tokens:

The model are able to keep writing, but since it doesn’t know <EOS> token, the generation do not stop until reach the maximum limit of output token.

<ins/>

8.2.2 Preparing the Fine-Tuning Dataset

We will use alpaca dataset to fine-tune the base model, teaching Qwen3-0.6B how to follow instructions. alpaca dataset has 52k training data, we will use 500 data to train our model. Observe that this dataset has the following format:

The instruction is the prompt, and input is an optional supplemental prompt. The output is the model’s answer. text combines all others together with its own template. This template doesn’t match chatml template, so, we will use the instruction, input, and output to compose chatml template

Let’s check the original raw dataset, the chat dataset, and the dataset after tokenization:

Creating a Data Collator for SFT: We need to mask out the prompt portion in the labels, so the loss is only computed on the response part.

<ins/>