[WIP]Add FP8KV for DS/QWEN

[yiliu30]

• QWEN
• DS
  depends on
• Add FP8 KV Support for DS auto-round#1180
• Deepseek MXFP4/MXFP8 + FP8KV  yiliu30/vllm-fork#84

cc @thuang6

[PRAgent4INC]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Typo There is a typo in the usage function where each should be echo. each " $0 --kv_cache_dtype fp8 -m /path/to/my/model" Redundant Comment The comment "Set environment variables based on quantization type" is repeated twice. # Set environment variables based on quantization type if [[ "$QUANT_TYPE_UPPER" == "MXFP4" ]]; then

[PRAgent4INC]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
General	Remove duplicate section Remove the duplicate NVFP4 section. examples/pytorch/nlp/huggingface_models/language-modeling/quantization/auto_round/deepseek/README.md [34-40] - NVFP4 ```bash bash run_quant.sh --model $MODEL -t nvfp4 --output_dir ./qmodels -+ -+``` -+ -+- NVFP4 -+```bash -+bash run_quant.sh --model $MODEL -t nvfp4 --output_dir ./qmodels +``` Suggestion importance[1-10]: 8 __ Why: The suggestion is correct and important as it removes a duplicate section, improving the clarity and maintainability of the README file.	Medium
	Fix usage example typo Correct the typo in the usage example. examples/pytorch/nlp/huggingface_models/language-modeling/quantization/auto_round/deepseek/run_generate.sh [26] -each " $0 --kv_cache_dtype fp8 -m /path/to/my/model" +echo " $0 --kv_cache_dtype fp8 -m /path/to/my/model" Suggestion importance[1-10]: 3 __ Why: The suggestion corrects a typo in the usage example, improving readability but offering minimal functional impact.	Low