Abstract Lymph node (LN) lipomatosis is a common, but rarely discussed phenomenon, associated with aging, involving a gradual exchange of the LN parenchyma into adipose tissue. The mechanisms behind these changes and the effects on the LN have been unknown. We show that LN lipomatosis starts in the medullary regions of the human LN and link the initiation of lipomatosis to transdifferentiation of LN medullary fibroblasts into adipocytes. The latter is associated with a downregulation of lymphotoxin beta expression. We also show that, medullary fibroblasts, in contrast to the reticular cells in the T-cell zone, display an inherent higher sensitivity for adipogenesis. Progression of lipomatosis leads to a gradual loss of the medullary lymphatic network, but at later stages, collecting-like lymphatic vessels, are found inside the adipose tissue. The stromal dysregulation includes a dramatic remodeling and dilation of the high endothelial venules associated with reduced density of naïve T-cells. Abnormal clustering of plasma cells is also observed. Thus, LN lipomatosis causes widespread stromal dysfunction with consequences for the immune contexture of the human LN. Our data warrant an increased awareness of LN lipomatosis as a factor contributing to decreased immune functions in the elderly and in disease. Graphical abstract In lymph nodes (LNs) of young patients there is a normal lymph flow, normal and functioning high endothelial venules (HEVs) with a high density of surrounding naïve T-cells. With aging lymphotoxin beta (LTB) is downregulated in the medulla of the LN and the fibroblasts of the medulla, namely the medullary reticular cells (MedRCs), transdifferentiate into adipocytes inducing LN lipomatosis. LN lipomatosis leads to loss of lymphoid tissue, medullary sinuses and can be predicted to result in a shortcut of the lymph flow based on the presence of collecting-like vessels in the adipose tissue in late stage lipomatosis. Lipomatosis also induce extensive vascular remodeling with loss of medullary lymphatic vessels and dysfunctional, highly dilated HEVs with lower density of naïve T-cells and trapped plasma cells.