--- base_model: - google/gemma-4-31B-it - ApocalypseParty/G4-31B-SFT-Diversity-1-5ep library_name: transformers tags: - mergekit - merge --- # G4-31B-configPB This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit). ## Merge Details ### Merge Method This model was merged using the [Linear DELLA](https://arxiv.org/abs/2406.11617) merge method using [google/gemma-4-31B-it](https://huggingface.co/google/gemma-4-31B-it) as a base. ### Models Merged The following models were included in the merge: * [ApocalypseParty/G4-31B-SFT-Diversity-1-5ep](https://huggingface.co/ApocalypseParty/G4-31B-SFT-Diversity-1-5ep) ### Configuration The following YAML configuration was used to produce this model: ```yaml merge_method: della_linear base_model: google/gemma-4-31B-it dtype: bfloat16 parameters: lambda: 1.0 normalize: false rescale: false density: 1.0 models: - model: ApocalypseParty/G4-31B-SFT-Diversity-1-5ep parameters: weight: - filter: vision_tower value: 0.0 - filter: embed_vision value: 0.0 - filter: mlp. value: 1.0 - value: 0.0 ```
