Hybrid digital-analog precoding is essential for balancing communication performance, energy efficiency, and hardware costs in millimeter wave (mmWave) massive multiple-input multiple-output (MIMO) systems. However, most existing designs rely on the Shannon capacity and assume infinite blocklengths, which are impractical for emerging applications, such as massive machine-type communications, operating with finite blocklength (FBL). To address this gap, this paper pioneers a novel hybrid precoding design for mmWave massive MIMO in the FBL regime. We meticulously optimize hybrid precoding based on both the weighted sum-rate (WSR) and the max-min fairness (MMF) criteria, while fulfilling the transmit power budget and users' minimum rate requirements. Both continuous and discrete phase shifters are considered for analog precoding. The formulated optimization problems are highly challenging to solve due to the nonconvex objective functions and nonconvex constraints. These challenges are further intensified by the nonconcave FBL rate function and the intricate coupling between analog and digital precoders. By proposing novel problem transformation and decomposition techniques, we reformulate the original complex problems into forms solvable with the penalty dual decomposition (PDD) method. We then develop two efficient iterative algorithms with parallel, and even closed-form variable updates, and guaranteed convergence to solve the WSR and MMF optimization problems, applicable to both continuous and discrete phase shifters. Simulation results show that our proposed hybrid precoding designs significantly outperform several baseline schemes, especially those adopting the Shannon capacity and infinite blocklength. Additionally, our proposed optimization algorithms enable hybrid precoding exploiting discrete phase shifters with limited quantization resolution (e.g., 3-bit) to closely match the performance of fully digital precoding in FBL scenarios.