While the capsaicin receptor TRPV1 channel is a polymodal nociceptor for heat, capsaicin, and proton, its responses to each of these stimuli are profoundly regulated by membrane potential, damping or even prohibiting its response at negative voltages and amplifying its response at positive voltages. Though voltage sensitivity plays an important role is shaping pain responses, how voltage regulates TRPV1 activation remains unknown. Here we showed that the voltage sensitivity of TRPV1 does not originate from the S4 segment like classic voltage-gated ion channels; instead, outer pore acidic residues directly partake in voltage-sensitive activation, with their negative charges collectively constituting the observed gating charges. Voltage-sensitive outer pore movement is titratable by extracellular pH and is allosterically coupled to channel activation, likely by influencing the upper gate in the ion selectivity filter. Elucidating this unorthodox voltage-gating process provides a mechanistic foundation for understanding polymodal gating and opens the door to novel approaches regulating channel activity for pain managements.