--- language: - en license: mit tags: - medical - llm - pytorch - text-generation - custom-model pipeline_tag: text-generation library_name: pytorch --- # MediMind-411M MediMind-411M is a custom medical language model trained from scratch for biomedical and clinical text generation. This model was trained and uploaded by **Koyeliya Ghosh** under the Hugging Face account `koyelog`. ## Overview MediMind-411M is a 411M-parameter transformer-based language model designed to generate medical-style text. It was trained on a large medical text collection and uses a custom tokenizer. ## Training Summary - Model name: `MediMind-411M` - Parameters: approximately 411.1M - Training device: Kaggle GPU T4 x2 - Total texts loaded: 171,047 - Training samples tokenized: 50,000 - Total batches: 12,500 - Final average loss: 4.9253 - Total runtime: about 5536.5 seconds (~92 minutes) ## Architecture This model uses a decoder-only transformer architecture with: - Embedding dimension: 1024 - Layers: 24 - Attention heads: 16 - KV heads: 8 - RoPE positional encoding - RMSNorm - SwiGLU-style feed-forward layers ## Files in this Repository - `medimind_final.pt` — final trained model weights - `checkpoint_latest.pt` — latest training checkpoint - `vocab.json` — tokenizer vocabulary - `merges.txt` — tokenizer merges ## Testing The model was tested locally in a Kaggle notebook by: 1. Downloading the model files from this Hugging Face repository 2. Loading the tokenizer using `vocab.json` and `merges.txt` 3. Rebuilding the training architecture in PyTorch 4. Loading `medimind_final.pt` 5. Generating outputs from medical prompts ### Example test prompts - `Patient presents with fever and cough. Diagnosis:` - `Symptoms of diabetes include` - `Treatment for hypertension includes` ### Observed behavior The model successfully generates medical-style text and terminology. Outputs show that the model has learned domain vocabulary and sentence patterns, but generations can still be noisy, mixed-topic, or clinically unreliable. ## Limitations - This is an early-stage base language model, not an instruction-tuned chatbot. - It may produce incorrect, incomplete, or hallucinated medical statements. - It should **not** be used for real medical diagnosis, treatment, or decision-making. - Output quality can vary depending on prompt style and decoding settings. ## Intended Use This model is intended for: - learning and experimentation - research practice - testing custom LLM training pipelines - educational exploration of medical text generation This model is **not intended** for direct clinical deployment or patient-facing use. ## Example Usage ```python from huggingface_hub import hf_hub_download from tokenizers import ByteLevelBPETokenizer import torch model_path = hf_hub_download(repo_id="koyelog/MediMind-411M", filename="medimind_final.pt") vocab_path = hf_hub_download(repo_id="koyelog/MediMind-411M", filename="vocab.json") merges_path = hf_hub_download(repo_id="koyelog/MediMind-411M", filename="merges.txt") tokenizer = ByteLevelBPETokenizer(vocab_path, merges_path) print("Load tokenizer and model architecture, then run generation.") ``` ## Future Work Planned next improvements: - cleaner inference pipeline - better decoding and stopping rules - further training epochs - instruction tuning on medical QA data - model card improvements and benchmark evaluation ## Author Created by **Koyeliya Ghosh** Hugging Face: [koyelog](https://huggingface.co/koyelog) ## Disclaimer This model is for research and educational purposes only. It must not be used as a substitute for licensed medical advice or professional healthcare judgment.
