--- language: - en license: apache-2.0 license_link: https://huggingface.co/Qwen/Qwen3-1.7B/blob/main/LICENSE pipeline_tag: text-generation base_model: Qwen/Qwen3-1.7B base_model_relation: quantized library_name: litert-lm tags: - litert-lm - litertlm - qwen - Qwen3 --- # Qwen3-1.7B LiteRT-LM Model This repository contains LiteRT-LM variants of [Qwen/Qwen3-1.7B](https://huggingface.co/Qwen/Qwen3-1.7B) optimized for on-device text generation. ## Available Artifact | File | Quantization Recipe | Context | Size | |---|---|---:|---:| | `Qwen3_1.7B.litertlm` | dynamic_wi8_afp32 | - | 2.1 GB | ## How to Use ### Command-Line Interface 1. Install the prerequisites: ```bash pip install litert-lm ``` 2. Run the command in CLI: ```bash litert-lm run --from-huggingface-repo=litert-community/Qwen3-1.7B Qwen3_1.7B.litertlm --prompt="Write me a poem on nature" ``` ### Python 1. Install the prerequisites: ```bash pip install litert-lm huggingface_hub ``` 2. Download the model file: ```python from huggingface_hub import hf_hub_download model_path = hf_hub_download( repo_id="litert-community/Qwen3-1.7B", filename="Qwen3_1.7B.litertlm" ) ``` 3. Run inference: ```python import litert_lm litert_lm.set_min_log_severity(litert_lm.LogSeverity.ERROR) # Hide log for TUI app with litert_lm.Engine(model_path) as engine: with engine.create_conversation() as conversation: while True: user_input = input("\n>>> ") for chunk in conversation.send_message_async(user_input): print(chunk["content"][0]["text"], end="", flush=True) ``` ## Integration Ready to integrate this into your product? Get started in the [LiteRT-LM documentation](https://ai.google.dev/edge/litert-lm/overview).
