Flowering time is a key trait for plant reproductive success ensuring both overlap with pollinator activity and favorable conditions for fruit development. To quantify selection on flowering time, we individually marked 1250 plants of the Small Spider Orchid, Ophrys araneola RchB., in six populations in Northern Switzerland and surveyed them during three years. We recorded the date of first flowering, frost damage, and fruiting success of individual plants. In addition, we analyzed historical records of the orchid and its only verified pollinator, the solitary bee Andrena combinata in Northern Switzerland, to estimate potential desynchronization due to climate change. We documented strong selection for later flowering driven by frost damage, with all populations showing significant selection for later flowering in at least one year. Selection for later flowering driven by pollination (fruit set) could only be analyzed in one population due to the overall low fruit set, where it was significant in one year. The historical data from between 1970 and 2019 indicated low synchronization between orchid flowering and bee occurrence, with mean flowering three weeks earlier than the mean peak of bee occurrence, corroborating selection for later flowering through fruit set. The data also showed a significant advance of flowering time and bee-occurrence in the last decades, but to a similar degree in orchids and bees, hence without an indication of increasing desynchronization through climate change. Our study shows that selection for later flowering is mostly caused by frost damage, but also by the little synchronized flowering and pollinator activity, which is however unlikely to be a consequence of climate change in this orchid.