Abstract The tumor stroma is a tissue composed primarily of extracellular matrix, fibroblasts, immune cells, and vasculature. Its structure and functions, such as nutrient support and waste removal, are altered during malignancy. Tumor cells transform fibroblasts into cancer-associated fibroblasts, which have an important immunosuppressive activity on which growth, invasion, and metastasis depend. These activated fibroblasts prevent immune cell infiltration into the tumor nest, thereby promoting cancer progression and inhibiting T-cell-based immunotherapy. To understand these complex interactions, we measure the density of different cell types in the stroma using immunohistochemistry techniques on tumor samples from lung cancer patients. We incorporate these data, and also known information on cell proliferation rates and relevant biochemical interactions, into a minimal dynamical system with few parameters. A spatio-temporal approach to the inhomogeneous environment explains the cell distribution and fate of lung carcinomas. The model reproduces that cancer-associated fibroblasts act as a barrier to tumor growth, but also reduce the efficiency of the immune response. The final outcome depends on the parameter values for each patient and leads to either tumor invasion, persistence or eradication as a result of the interplay between cancer cell growth, T-cell cytotoxic activity and fibroblast attraction, activation and spatial dynamics. Our conclusion is that a wide spectrum of scenarios exists as a result of the competition between the characteristic times of cancer cell growth and the activity rates of the other species. Nevertheless, distinct trajectories and patterns allow quantitative predictions that may help in the selection of new therapies and personalized protocols. Graphical Abstract