Abstract The identification of T cell neo-epitopes is fundamental and computational challenging in tumor immunotherapy study. As the binding of pMHC - T cell receptor (TCR) is the essential condition for neo-epitopes to trigger the cytotoxic T cell reactivity, several computational studies have been proposed to predict neo-epitopes from the perspective of pMHC-TCR binding recognition. However, they often failed with the inaccurate binding prediction for a single pMHC -TCR pair due to the highly diverse TCR space. In this study, we proposed a novel weakly-supervised learning framework, i . e ., TCRBagger , to facilitate the personalized neo-epitope identification with weakly-supervised peptide-TCR binding prediction by bagging a sample-specific TCR profile. TCRBagger integrates three carefully designed learning strategies, i . e . a self-supervised learning strategy, a denoising learning strategy and a Multi-Instance Learning (MIL) strategy in the modeling of peptide-TCR binding. Our comprehensive tests revealed that TCRBagger exhibited great advances over existing tools by modeling interactions between peptide and TCR profiles. We further applied TCRBagger in different clinical settings, including (1) facilitating the peptide-TCR binding prediction under MIL using single-cell TCR-seq data. (2) improving the patient-specific neoantigen prioritization compared to the existing neoantigen identification tools. Collectively, TCRBagger provides novel perspectives and contributions for identifying neo-epitopes as well as discovering potential pMHC-TCR interactions in personalized tumor immunotherapy.