--- language: - en license: mit tags: - finance - continual-learning - qwen2 - causal-lm - ewc base_model: Qwen/Qwen2.5-0.5B datasets: - gbharti/finance-alpaca - sujet-ai/Sujet-Finance-Instruct-177k - nvidia/OpenMathInstruct-2 - HuggingFaceFW/fineweb-edu - yahma/alpaca-cleaned library_name: transformers pipeline_tag: text-generation --- # Meridian.AI — Continual-Learning Finance LLM Meridian.AI is a finance-specialized language model that continuously fine-tunes a **Qwen2.5-0.5B** backbone every hour on 25+ finance and math datasets, using **Elastic Weight Consolidation (EWC)** to prevent catastrophic forgetting across training sessions. The entire pipeline runs unattended on free GitHub Actions infrastructure — no GPUs. - **Base model:** [`Qwen/Qwen2.5-0.5B`](https://huggingface.co/Qwen/Qwen2.5-0.5B) (~494M params, Qwen2 architecture) - **Continual learning:** Elastic Weight Consolidation (diagonal Fisher) - **Training cadence:** hourly GitHub Actions CI on CPU runners - **Source code & full docs:** [github.com/MeridianAlgo/FinAI](https://github.com/MeridianAlgo/FinAI) ## Usage The deployed checkpoint is a **standard Qwen2 model** — `trust_remote_code=True` is **not** required. ```python import torch from transformers import AutoModelForCausalLM, AutoTokenizer repo_id = "meridianal/FinAI" tokenizer = AutoTokenizer.from_pretrained(repo_id, subfolder="checkpoint") model = AutoModelForCausalLM.from_pretrained( repo_id, subfolder="checkpoint", torch_dtype=torch.float32, low_cpu_mem_usage=True, ) model.eval() prompt = """### Instruction: Explain the difference between a bond's yield to maturity and its coupon rate. ### Response: """ inputs = tokenizer(prompt, return_tensors="pt") with torch.no_grad(): output = model.generate( **inputs, max_new_tokens=200, do_sample=True, temperature=0.8, top_p=0.92, repetition_penalty=1.3, no_repeat_ngram_size=3, pad_token_id=tokenizer.pad_token_id, eos_token_id=tokenizer.eos_token_id, ) print(tokenizer.decode(output[0], skip_special_tokens=True)) ``` Inputs are formatted with the `### Instruction: / ### Response:` template used during training. ## Model details | Specification | Value | |:---|:---| | Base model | Qwen2.5-0.5B | | Architecture | Qwen2ForCausalLM | | Parameters | ~494M | | Context window | 32,768 tokens (Qwen2.5 default) | | Training dtype | bfloat16 | | Continual learning | Elastic Weight Consolidation (EWC) | ## Training data A weighted streaming mix of 25+ finance and instruction datasets, including `gbharti/finance-alpaca`, `sujet-ai/Sujet-Finance-Instruct-177k`, `nvidia/OpenMathInstruct-2`, `HuggingFaceFW/fineweb-edu`, `yahma/alpaca-cleaned`, and the FinanceMTEB suite. See the [repository README](https://github.com/MeridianAlgo/FinAI#dataset-curriculum) for the full curriculum and weights. ## Limitations & disclaimer This is an experimental research project on continual learning for financial NLP. Outputs may contain factual errors and are intended for academic and research purposes only. **Nothing generated by this model constitutes financial advice. Do not use outputs to make real financial decisions or execute trades.**
