Using multimodal foundational models to predict neoantigen immunogenicity and vaccine effectiveness across different tumor types
DOI:
https://doi.org/10.63808/ghc.v2i1.498Keywords:
Neoantigen, Immunogenicity prediction, Multimodal foundation model, Cancer vaccine, Cross-tumor, Precision immunotherapyAbstract
Neoantigen vaccines are a key component of personalised cancer immunotherapy; however, existing prediction techniques are primarily restricted to a single tumour type and have difficulty integrating multi-modal biological data, which leads to inadequate accuracy in immunogenicity evaluation and vaccine efficacy prediction. In order to predict neoantigen immunogenicity and customised vaccination clinical outcomes across various tumour types, this study attempts to develop and verify a universal multi-modal foundation model.Neoantigen peptide sequences, mass spectrometry-derived pHLA binding patterns, single-cell TCR repertoires, tumour transcriptomes, and clinical vaccination trial follow-up records were among the extensive paired data we gathered from 15 solid tumour types. We present the NeoVAX-FM multi-modal foundation model, which first embeds peptide sequences, pHLA complex 3D structures, and gene expression into a single semantic space using a contrastive language-image pretraining paradigm. It is then fine-tuned on downstream tasks to concurrently score immunogenicity and predict progression-free survival.The model was assessed in one prospective clinical trial and three external validation cohorts. NeoVAX-FM showed strong performance in melanoma, non-small cell lung cancer, and microsatellite stable colorectal cancer, with an average AUC of 0.94 for cross-tumor neoantigen immunogenicity prediction—a 12.3% increase over the best currently available techniques. Patients in the prospective vaccination cohort who were projected by the model to be “high responders” had a considerably higher median progression-free survival (HR = 0.28, p < 0.001), and the model was successful in identifying tumour microenvironment characteristics and universal TCR motifs that drive long-term responses.This work is the first to use a multi-modal foundation model for neoantigen vaccines, overcoming the tumor-type-specific constraints of conventional approaches and facilitating joint modelling of immunogenicity and clinical efficacy, thereby offering an AI decision engine for precision cancer vaccine design that is applicable to all cancer types.
References
[1] Reynisson, B., Alvarez, B., Paul, S., Peters, B., & Nielsen, M. (2020). NetMHCpan-4.1 and NetMHCIIpan-4.0: Improved predictions of MHC antigen presentation by concurrent motif deconvolution and integration of MS MHC eluted ligand data. Nucleic Acids Research, 48(W1), W449–W454. https://doi.org/10.1093/nar/gkaa379
[2] O’Donnell, T. J., Rubinsteyn, A., & Laserson, U. (2020). MHCflurry 2.0: Improved pan-allele prediction of MHC class I-presented peptides by incorporating antigen processing. Cell Systems, 11(1), 42–48.e7. https://doi.org/10.1016/j.cels.2020.06.010
[3] Jurtz, V., Paul, S., Andreatta, M., Marcatili, P., Peters, B., & Nielsen, M. (2017). NetMHCpan 4.0: Improved peptide-MHC class I interaction predictions integrating eluted ligand and peptide binding affinity data. Journal of Immunology, 199(9), 3360–3368. https://doi.org/10.4049/jimmunol.1700893
[4] Abelin, J. G., Keskin, D. B., Sarkizova, S., Hartigan, C., Zhang, W., Sidney, J., Stevens, J., Lane, W., Zhang, G., Eisenhaure, T., Clauser, K., Hacohen, N., Rooney, M., Carr, S., & Wu, C. J. (2017). Mass spectrometry profiling of HLA-associated peptidomes in mono-allelic cells enables more accurate epitope prediction. Immunity, 46(2), 315–326. https://doi.org/10.1016/j.immuni.2017.02.007
[5] Bassani-Sternberg, M., Bräunlein, E., Klar, R., Engleitner, T., Sinitcyn, P., Audehm, S., Straub, M., Weber, J., Slotta-Huspenina, J., Specht, K., Martignoni, M. E., Werner, A., Hein, R., Busch, D. H., Peschel, C., Rad, R., Cox, J., Mann, M., & Krackhardt, A. M. (2016). Direct identification of clinically relevant neoepitopes presented on native human melanoma tissue by mass spectrometry. Nature Communications, 7, Article 13404. https://doi.org/10.1038/ncomms13404
[6] Bulik-Sullivan, B., Busby, J., Palmer, C. D., Davis, M. J., Murphy, T., Clark, A., Busby, M., Duke, F., Yang, A., Young, L., … Yelensky, R. (2019). Deep learning using tumor HLA peptide mass spectrometry datasets improves neoantigen identification. Nature Biotechnology, 37(1), 55–63. https://doi.org/10.1038/nbt.4313
[7] Wu, J., Wang, W., Zhang, J., Zhou, B., Zhao, W., Su, Z., Gu, X., Wu, J., Zhou, Z., & Chen, S. (2019). DeepHLApan: A deep learning approach for neoantigen prediction considering both HLA-peptide binding and immunogenicity. Frontiers in Immunology, 10, Article 2559. https://doi.org/10.3389/fimmu.2019.02559
[8] Ott, P. A., Hu, Z., Keskin, D. B., Shukla, S. A., Sun, J., Bozym, D. J., Zhang, W., Luoma, A., Giobbie-Hurder, A., Peter, L., … Wu, C. J. (2017). An immunogenic personal neoantigen vaccine for patients with melanoma. Nature, 547(7662), 217–221. https://doi.org/10.1038/nature22991
[9] Sahin, U., Derhovanessian, E., Miller, M., Kloke, B.-P., Simon, P., Löwer, M., Bukur, V., Tadmor, A. D., Luxemburger, U., Schrörs, B., … Türeci, Ö. (2017). Personalized RNA mutanome vaccines mobilize poly-specific therapeutic immunity against cancer. Nature, 547(7662), 222–226. https://doi.org/10.1038/nature23003
[10] Keskin, D. B., Anandappa, A. J., Sun, J., Tirosh, I., Mathewson, N. D., Li, S., Oliveira, G., Giobbie-Hurder, A., Felt, K., Gjini, E., … Reardon, D. A. (2019). Neoantigen vaccine generates intratumoral T cell responses in phase Ib glioblastoma trial. Nature, 565(7738), 234–239. https://doi.org/10.1038/s41586-018-0792-9
[11] Ding, Z., Li, Q., Zhang, R., Xie, L., Shu, Y., Gao, S., Song, W., Wang, P., Su, X., Qin, Y., … Xu, H. (2021). Personalized neoantigen pulsed dendritic cell vaccine for advanced lung cancer. Signal Transduction and Targeted Therapy, 6(1), Article 26. https://doi.org/10.1038/s41392-020-00448-5
[12] Hu, Z., Leet, D. E., Allesøe, R. L., Oliveira, G., Li, S., Luoma, A. M., … Wu, C. J. (2021). Personal neoantigen vaccines induce persistent memory T cell responses and epitope spreading in patients with melanoma. Nature Medicine, 27(3), 515–525. https://doi.org/10.1038/s41591-020-01206-4
[13] Cafri, G., Gartner, J. J., Zaks, T., Hopson, K., Levin, N., Paria, B. C., Parkhurst, M. R., Yossef, R., Lowery, F. J., Jafferji, M. S., Prickett, T. D., Goff, S. L., McGowan, C. T., Seitter, S., Shindorf, M. L., Parikh, A., Chatani, P. D., Robbins, P. F., & Rosenberg, S. A. (2020). mRNA vaccine–induced neoantigen-specific T cell immunity in patients with gastrointestinal cancer. Journal of Clinical Investigation, 130(11), 5976–5988. https://doi.org/10.1172/JCI134915
[14] Liu, Y., Zugazagoitia, J., Ahmed, F. S., Henick, B. S., Gettinger, S. N., Herbst, R. S., Schalper, K. A., & Rimm, D. L. (2020). Immune cell PD-L1 colocalizes with macrophages and is associated with outcome in PD-1 pathway blockade therapy. Clinical Cancer Research, 26(4), 970–977. https://doi.org/10.1158/1078-0432.CCR-19-1040
[15] Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C. L., Ma, J., & Fergus, R. (2021). Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. Proceedings of the National Academy of Sciences, 118(15), Article e2016239118. https://doi.org/10.1073/pnas.2016239118
[16] Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., Verkuil, R., Kabeli, O., Shmueli, Y., … Rives, A. (2023). Evolutionary-scale prediction of atomic-level protein structure with a language model. Science, 379(6637), 1123–1130. https://doi.org/10.1126/science.ade2574
[17] Cui, H., Wang, C., Maan, H., Pang, K., Luo, F., Duan, N., & Wang, B. (2024). scGPT: Toward building a foundation model for single-cell multi-omics using generative AI. Nature Methods, 21(9), 1470–1480. https://doi.org/10.1038/s41592-024-02201-0
[18] Theodoris, C. V., Xiao, L., Chopra, A., Chaffin, M. D., Al Sayed, Z. R., Hill, M. C., Mantineo, H. M., Brydon, E. M., Zeng, Z., Liu, X. S., & Ellinor, P. T. (2023). Transfer learning enables predictions in network biology. Nature, 618(7965), 616–624. https://doi.org/10.1038/s41586-023-06139-9
[19] Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., & Sutskever, I. (2021). Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning (Vol. 139, pp. 8748–8763). PMLR. https://doi.org/10.48550/arXiv.2103.00020
[20] Lu, M. Y., Chen, B., Zhang, A., Williamson, D. F. K., Chen, R. J., Liang, I., Ding, T., Jaume, G., Odintsov, I., Le, L. P., … Mahmood, F. (2024). A visual-language foundation model for computational pathology. Nature Medicine, 30(3), 863–874. https://doi.org/10.1038/s41591-024-02856-4
[21] Wang, Y., Wang, J., Cao, Z., & Barati Farimani, A. (2022). Molecular contrastive learning of representations via graph neural networks. Nature Machine Intelligence, 4(3), 279–287. https://doi.org/10.1038/s42256-022-00447-x
[22] Moor, M., Banerjee, O., Abad, Z. S. H., Krumholz, H. M., Leskovec, J., Topol, E. J., & Rajpurkar, P. (2023). Foundation models for generalist medical artificial intelligence. Nature, 616(7956), 259–265. https://doi.org/10.1038/s41586-023-05881-4
[23] Wells, D. K., van Buuren, M. M., Dang, K. K., Hubbard-Lucey, V. M., Sheehan, K. C. F., Campbell, K. M., Lamb, A., Ward, J. P., Sidney, J., Blazquez, A. B., Rech, A. J., Zaretsky, J. M., Comin-Anduix, B., Ng, A. H. C., Chour, W., Yu, T. V., Rizvi, H., Chen, J. M., … Defranoux, N. A. (2020). Key parameters of tumor epitope immunogenicity revealed through a consortium approach improve neoantigen prediction. Cell, 183(3), 818–834.e13. https://doi.org/10.1016/j.cell.2020.09.015
[24] Auslander, N., Zhang, G., Lee, J. S., Frederick, D. T., Miao, B., Moll, T., Tian, T., Wei, Z., Madan, S., Sullivan, R. J., Boland, G., Flaherty, K., Herlyn, M., & Ruppin, E. (2018). Robust prediction of response to immune checkpoint blockade therapy in metastatic melanoma. Nature Medicine, 24(10), 1545–1549. https://doi.org/10.1038/s41591-018-0157-9
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Gang Liu, Jia Wang, Jia Zhu
This work is licensed under a Creative Commons Attribution 4.0 International License.