Skip to content

TPUΒΆ

Supported ModelsΒΆ

Text-only Language ModelsΒΆ

Model Architecture Supported
mistralai/Mixtral-8x7B-Instruct-v0.1 MixtralForCausalLM 🟨
mistralai/Mistral-Small-24B-Instruct-2501 MistralForCausalLM βœ…
mistralai/Codestral-22B-v0.1 MistralForCausalLM βœ…
mistralai/Mixtral-8x22B-Instruct-v0.1 MixtralForCausalLM ❌
meta-llama/Llama-3.3-70B-Instruct LlamaForCausalLM βœ…
meta-llama/Llama-3.1-8B-Instruct LlamaForCausalLM βœ…
meta-llama/Llama-3.1-70B-Instruct LlamaForCausalLM βœ…
meta-llama/Llama-4-* Llama4ForConditionalGeneration ❌
microsoft/Phi-3-mini-128k-instruct Phi3ForCausalLM 🟨
microsoft/phi-4 Phi3ForCausalLM ❌
google/gemma-3-27b-it Gemma3ForConditionalGeneration 🟨
google/gemma-3-4b-it Gemma3ForConditionalGeneration ❌
deepseek-ai/DeepSeek-R1 DeepseekV3ForCausalLM ❌
deepseek-ai/DeepSeek-V3 DeepseekV3ForCausalLM ❌
RedHatAI/Meta-Llama-3.1-8B-Instruct-quantized.w8a8 LlamaForCausalLM βœ…
RedHatAI/Meta-Llama-3.1-70B-Instruct-quantized.w8a8 LlamaForCausalLM βœ…
Qwen/Qwen3-8B Qwen3ForCausalLM βœ…
Qwen/Qwen3-32B Qwen3ForCausalLM βœ…
Qwen/Qwen2.5-7B-Instruct Qwen2ForCausalLM βœ…
Qwen/Qwen2.5-32B Qwen2ForCausalLM βœ…
Qwen/Qwen2.5-14B-Instruct Qwen2ForCausalLM βœ…
Qwen/Qwen2.5-1.5B-Instruct Qwen2ForCausalLM 🟨

βœ… Runs and optimized.
🟨 Runs and correct but not optimized to green yet.
❌ Does not pass accuracy test or does not run.