Introduction The gut microbiota significantly influences the host’s production performance and health status, with different gastrointestinal tissues exhibiting functional diversity reflected in their microbial diversity. Methods In this study, five adult male Tianzhu white yaks (4.5 years old) were selected and fed under the same nutritional conditions. After the feeding experiment, the yaks were slaughtered, and chyme samples were collected from the rumen, abomasum, jejunum, and colon for 16S rRNA full-length sequencing and volatile fatty acid analysis. Results The results showed that the microbial composition and diversity of the rumen and abomasum were similar, with close genetic distances and functional projections. In contrast, the jejunum and colon had distinct microbial compositions and diversity compared to the rumen and abomasum. At the phylum level, the dominant phyla in the rumen, abomasum, and colon were Firmicutes and Bacteroidetes, while in the jejunum, the dominant phyla were Firmicutes and Proteobacteria. The abundance of Firmicutes differed significantly between the jejunum (87.24%) and the rumen (54.67%), abomasum (67.70%), and colon (65.77%). Similarly, Bacteroidetes showed significant differences between the jejunum (2.21%) and the rumen (36.54%), abomasum (23.81%), and colon (28.12%). At the genus level, Rikenellaceae_RC9_gut_group and Christensenellaceae_R-7_group were dominant in both the rumen and abomasum. In the jejunum, Romboutsia and Paeniclostridium were dominant, while Rikenellaceae_RC9_gut_group and UCG-005 were the dominant genera in the colon. At the species level, rumen_bacterium_g_Rikenellaceae_RC9_gut_group and rumen_bacterium_g_Christensenellaceae_R-7_group were dominant in both the rumen and abomasum, while Clostridium_sp._g_Romboutsia and bacterium_g_Paeniclostridium were unique to the jejunum. Ruminococcaceae_bacterium_g_UCG-005 and bacterium_g_Rikenellaceae_RC9_gut_group were unique to the colon. KEGG functional prediction of the microbiota indicated that the dominant functions in the rumen, abomasum, colon, and jejunum were amino acid metabolism, glycan biosynthesis and metabolism, carbohydrate metabolism, and membrane transport, respectively, reflecting the digestive functions of these organs. Volatile fatty acid analysis showed that the concentrations of acetic acid, propionic acid, and butyric acid in the rumen were significantly higher than those in the abomasum, jejunum, and colon ( p < 0.05). Among these, the propionic acid concentration in the jejunum was significantly lower than in the abomasum and colon. Additionally, correlation analysis results indicated that acetic acid and butyric acid were significantly positively correlated with the ruminal bacterial community ( p < 0.05). The total volatile fatty acid concentration was highest in the rumen, decreased to less than one-fifth of the rumen’s total volatile fatty acid concentration in the abomasum and jejunum, and then reached a second peak in the colon. Conclusion This study explored the microbial composition and differential bacterial genera in the rumen and intestines of Tianzhu white yak, comparing the differences in volatile fatty acid levels and microbial composition and function across different regions. This is important for understanding their gastrointestinal microbiota’s spatial heterogeneity.