RoBERTpep

--- license: mit datasets: - keiwoo/peptide base_model: - FacebookAI/roberta-base pipeline_tag: feature-extraction tags: - biology - medical metrics: - accuracy --- This repository contains relevant information of our research article *'Supervised fine-tuning enhances unsupervised learning from 45 million amino acids in TCR and peptide sequences'*. ### Embedding examples #### Transformers You can use RoBERTpep with Transformers. To get started, install the necessary dependencies to setup your environment: ``` pip install -U transformers torch ``` Once setup you can proceed to run the model by running the snippet below: ``` from transformers import BertTokenizer, RobertaModel import torch device = 'cuda:0' if torch.cuda.is_available() else 'cpu' tokenizer = BertTokenizer.from_pretrained('keiwoo/RoBERTpep', do_lower_case = False) model = RobertaModel.from_pretrained('keiwoo/RoBERTpep').to(device) outputs = model(**tokenizer(' '.join('KLGGALQAK'), return_tensors="pt").to(device)) print(outputs.last_hidden_state[0].shape) # torch.Size([11, 1024]) [CLS+N+SEP, 1024] ``` ### Fine-tuning Please refer to [https://github.com/keiwoo/RoBERTcr](https://github.com/keiwoo/RoBERTcr)

Source context: 385 downloads · 0 likes · Pipeline feature-extraction · Repo keiwoo/RoBERTpep

Source context: 42 downloads · 0 likes · Pipeline feature-extraction · Repo keiwoo/RoBERTpep