--- base_model: - NeverSleep/Lumimaid-v0.2-12B - mistralai/Mistral-Nemo-Instruct-2407 - MuXodious/Mistral-Nemo-Instruct-2407-absolute-heresy library_name: transformers tags: - mergekit - merge --- # Mistral-Heretica-12B This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit). ## Merge Details ### Merge Method This model was merged using the [Task Arithmetic](https://arxiv.org/abs/2212.04089) merge method using [mistralai/Mistral-Nemo-Instruct-2407](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407) as a base. ### Models Merged The following models were included in the merge: * [NeverSleep/Lumimaid-v0.2-12B](https://huggingface.co/NeverSleep/Lumimaid-v0.2-12B) * [MuXodious/Mistral-Nemo-Instruct-2407-absolute-heresy](https://huggingface.co/MuXodious/Mistral-Nemo-Instruct-2407-absolute-heresy) ### Configuration The following YAML configuration was used to produce this model: ```yaml models: - model: mistralai/Mistral-Nemo-Instruct-2407 - model: NeverSleep/Lumimaid-v0.2-12B parameters: weight: 0.5 - model: MuXodious/Mistral-Nemo-Instruct-2407-absolute-heresy parameters: weight: 0.5 merge_method: task_arithmetic base_model: mistralai/Mistral-Nemo-Instruct-2407 parameters: normalize: true dtype: bfloat16 ```
