Accumulation of senescent cells is an important contributor to chronic inflammation upon aging. While cytoplasmic DNA was shown to drive the inflammatory phenotype of senescent cells, an equivalent role for RNA has never been explored. Here, we show that some senescent cells accumulate long promoter RNAs and 3 gene extensions, rich in retrotransposon sequences. Accordingly, these cells display increased expression of genes involved in detecting double stranded RNA of viral origin downstream of the interferon pathway. The RNA accumulation is correlated with signs of reduced RNA turn-over, including in some cases, reduced expression of RNA exosome subunits. Reciprocally, engineered inactivation of RNA exosome subunit Exosc3 induces expression of multiple senescence markers. A senescence-like RNA accumulation is also observed in cells exposed to oxidative stress, an important trigger of cellular senescence. Altogether, we propose that in a subset of senescent cells, repeat-containing transcripts stabilized by oxidative stress or reduced RNA exosome activity participate, possibly in combination with cytoplasmic DNA, in driving and maintaining the permanent inflammatory state characterizing cellular senescence.