A bstract Spatial transcriptomics (ST) technologies have gained increasing popularity due to their ability to provide positional context of gene expressions in a tissue. One major limitation of current commercially available ST methods such as the 10X Genomics Visium platform is the lack of single cell resolution. Cell type deconvolution for ST data is critical in order to fully reveal underlying biological mechanisms. Existing ST data deconvolution methods share two common limitations: first, few of them utilize spatial neighborhood information. Existing methods such as RCTD and SPOTlight intrinsically treat each spatial spot as independent of neighboring spots, although we anticipate nearby spots to share similar cell type compositions based on clinical evidence of tissue structures. Such limitation could be amplified when sequencing depths at single spots are relatively low so that borrowing information from neighboring spots is necessary in order to obtain reliable deconvolution results. Second, although Visium data provide us with a histological image which could add additional information regarding spot heterogeneity, most existing methods do not utilize this H&amp;E image. To solve these two limitations, we developed Multiscale Adaptive ST Deconvolution (MAST-Decon), a smooth deconvolution method for ST data. MAST-Decon uses a weighted likelihood approach and incorporates both gene expression data, spatial neighborhood information and H&amp;E image features by constructing different kernel functions to obtain a smooth deconvolution result. We showcased the strength of MAST-Decon through simulations based on real data, including a single-cell dataset of mouse brain primary visual cortex, and real-world Visium datasets to demonstrate its robust and superior performance compared with other state-of-the-art methods.

MAST-Decon: Smooth Cell-type Deconvolution Method for Spatial Transcriptomics Data