Geneformer is a recently introduced and powerful AI model that learns gene network dynamics and interactions using transfer learning from vast single-cell transcriptome data. This tool enables researchers to make accurate predictions about gene behavior and disease mechanisms even with limited data, accelerating drug target discovery and advancing understanding of complex genetic networks in various biological contexts.

Developed by researchers at the Broad Institute of MIT and Harvard and their collaborators, the AI model Geneformer uses the highest-expressed genes in sc-RNA expression data to generate a dense representation of each cell, which can be used as features for various downstream predictive tasks. What makes Geneformer unique, however, are the capabilities its architecture enables, even when trained on very little data.

Geneformer has a BERT-like transformer architecture and was pre-trained on data from about 30M single-cell transcriptomes across various human tissues. Its attention mechanism enables it to focus on the most relevant parts of the input data. With this context-aware approach, the model can make predictions by considering ‌relationships and dependencies between genes.