This repository contains relevant information of our research article 'Supervised fine-tuning enhances unsupervised learning from 45 million amino acids in TCR and peptide sequences' . Embedding examples Transformers You can use RoBERTpep with Transformers. To get started, install the necessary dependencies to setup your environment:
--- license: mit datasets: - keiwoo/peptide base_model: - FacebookAI/roberta-base pipeline_tag: feature-extraction tags: - biology - medical metrics: - accuracy --- This repository contains relevant information of our research article *'Supervised fine-tuning enhances unsupervised learning from 45 million amino acids in TCR and peptide sequences'*. ### Embedding examples #### Transformers You can use RoBERTpep with Transformers. To get started, install the necessary dependencies to setup your environment: ``` pip install -U transformers torch ``` Once setup you can proceed to run the model by running the snippet below: ``` from transformers import BertTokenizer, RobertaModel import torch device = 'cuda:0' if torch.cuda.is_available() else 'cpu' tokenizer = BertTokenizer.from_pretrained('keiwoo/RoBERTpep', do_lower_case = False) model = RobertaModel.from_pretrained('keiwoo/RoBERTpep').to(device) outputs = model(**tokenizer(' '.join('KLGGALQAK'), return_tensors="pt").to(device)) print(outputs.last_hidden_state[0].shape) # torch.Size([11, 1024]) [CLS+N+SEP, 1024] ``` ### Fine-tuning Please refer to [https://github.com/keiwoo/RoBERTcr](https://github.com/keiwoo/RoBERTcr)
This repository contains relevant information of our research article 'Supervised fine-tuning enhances unsupervised learning from 45 million amino acids in TCR and peptide sequences' . Embedding examples Transformers You can use RoBERTpep with Transformers.…
