Skip to content

Conversation

@abukhoy
Copy link
Contributor

@abukhoy abukhoy commented Dec 9, 2025

The following models have been successfully tested without using model cache:

  • TinyLlama/TinyLlama-1.1B-Chat-v1.0
  • gpt2
  • Salesforce/codegen-350M-mono
  • microsoft/Phi-3-mini-4k-instruct
  • tiiuae/falcon-7b
  • Qwen/Qwen2-0.5B
  • Qwen/Qwen3-0.6B
  • bigcode/starcoder2-3b
  • Felladrin/Minueza-32M-Base
  • wtang06/mpt-125m-c4
  • hakurei/gpt-j-random-tinier
  • meta-llama/Llama-3.2-1B
  • unsloth/gemma-2b
  • unsloth/gemma-2-2b
  • TheBloke/TinyLlama-1.1B-Chat-v0.3-AWQ (AWQ model)
  • neuralmagic/Llama-3.2-3B-Instruct-FP8 (float quantized compressed-tensor per tensor for weights and activations)
  • neuralmagic/Qwen2-0.5B-Instruct-FP8 (FP8 quant method, static, LM head ignored)
  • ibm-granite/granite-3.1-2b-instruct
  • ibm-granite/granite-guardian-3.1-2b
  • allenai/OLMo-2-0425-1B

Signed-off-by: Abukhoyer Shaik <abukhoye@qti.qualcomm.com>
@abukhoy abukhoy marked this pull request as draft December 15, 2025 06:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant