Update README.md

README.md

@@ -1,21 +1,108 @@
-MIT License
-
-Copyright (c) 2023 DeepSeek
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
+---
+license: mit
+base_model:
+- deepseek-ai/DeepSeek-OCR
+---
+
+# Model Overview
+
+- **Model Architecture:** DeepSeek-OCR
+  - **Input:** Text
+  - **Output:** Text
+- **Supported Hardware Microarchitecture:** AMD MI350/MI355
+- **ROCm:** 7.1.0
+- **Operating System(s):** Linux
+- **Inference Engine:** [vLLM](https://docs.vllm.ai/en/latest/)
+- **Model Optimizer:** [AMD-Quark](https://quark.docs.amd.com/latest/index.html) (V0.11)
+  - **Weight quantization:** Language model, MoE only, OCP MXFP4, Static
+  - **Activation quantization:** Language model, MoE only, OCP MXFP4, Dynamic
+- **Calibration Dataset:** [Pile](https://huggingface.co/datasets/mit-han-lab/pile-val-backup)
+
+This model was built with DeepSeek-OCR model by applying AMD-Quark for MXFP4 quantization.
+
+# Model Quantization
+
+The model was quantized from [amd/DeepSeek-OCR](https://huggingface.co/amd/DeepSeek-OCR) using [AMD-Quark](https://quark.docs.amd.com/latest/index.html). The weights and activations are quantized to MXFP4.
+
+**Quantization scripts:**
+
+Note that deepseek_vl_v2 is not in the built-in model template list in Quark V0.11, it has to be registered before quantization.
+
+```python
+import torch
+from transformers import AutoModel, AutoTokenizer, AutoProcessor
+from quark.torch import LLMTemplate, ModelQuantizer, export_safetensors
+from datasets import load_dataset
+from quark.contrib.llm_eval import ppl_eval
+
+# Register DeepSeek-OCR template
+deepseek_ocr_template = LLMTemplate(
+    model_type="deepseek_vl_v2",
+    kv_layers_name=["*k_proj", "*v_proj"],
+    q_layer_name="*q_proj",
+    exclude_layers_name=["lm_head", "model.sam_model*", "model.vision_model*", "model.projector*"],
+)
+LLMTemplate.register_template(deepseek_ocr_template)
+
+# Configuration
+ckpt_path = "amd/DeepSeek-OCR"
+output_dir = "amd/DeepSeek-OCR-MXFP4"
+quant_scheme = "mxfp4"
+exclude_layers = ["*self_attn*", "*mlp.gate", "lm_head", "*mlp.gate_proj", "*mlp.up_proj", 
+                  "*mlp.down_proj", "*shared_experts.*", "*sam_model*", "*vision_model*", "*projector*"]
+
+# Load model
+model = AutoModel.from_pretrained(ckpt_path, use_safetensors=True, trust_remote_code=True, 
+                                   _attn_implementation='flash_attention_2', device_map="cuda:0", torch_dtype=torch.bfloat16)
+model.eval()
+tokenizer = AutoTokenizer.from_pretrained(ckpt_path, trust_remote_code=True)
+processor = AutoProcessor.from_pretrained(ckpt_path, trust_remote_code=True)
+
+# Get quant config from template
+template = LLMTemplate.get(model.config.model_type)
+quant_config = template.get_config(scheme=quant_scheme, exclude_layers=exclude_layers)
+
+# Quantize
+quantizer = ModelQuantizer(quant_config)
+model = quantizer.quantize_model(model)
+model = quantizer.freeze(model)
+
+# Export hf_format
+export_safetensors(model, output_dir, custom_mode="quark")
+tokenizer.save_pretrained(output_dir)
+processor.save_pretrained(output_dir)
+
+# Evaluate PPL (optional)
+testdata = load_dataset("wikitext", "wikitext-2-raw-v1", split="test")
+testenc = tokenizer("\n\n".join(testdata["text"]), return_tensors="pt")
+ppl = ppl_eval(model, testenc, model.device)
+print(f"Perplexity: {ppl.item()}")
+```
+
+### Perplexity
+
+<table>
+  <tr>
+   <td><strong>Benchmark</strong>
+   </td>
+   <td><strong>DeepSeek-OCR </strong>
+   </td>
+   <td><strong>DeepSeek-OCR-MXFP4(this model) </strong>
+   </td>
+   <td><strong>Recovery</strong>
+   </td>
+  </tr>
+  <tr>
+   <td>ppl
+   </td>
+   <td>11.178650856018066
+   </td>
+   <td>11.88680648803711
+   </td>
+   <td>94.04%
+   </td>
+  </tr>
+</table>
+
+# License
+Modifications Copyright(c) 2025 Advanced Micro Devices, Inc. All rights reserved.