Abstract: To expand application of acidic aggregates in hydraulic asphalt concrete, this study employed organic anti-stripping agents (OA and OB) and inorganic anti-stripping agents (cement and hydrated lime) to enhance the adhesion between acidic aggregates and asphalt. A series of tests, including Fourier transform infrared spectroscopy (FTIR), conventional asphalt properties, rolling thin film oven aging (RTFOT), contact angle measurement, and water stability test, was conducted to evaluate the influence patterns of anti-stripping agents on the performance of asphalt and mixtures. The results indicate that the abundant phosphate esters and aromatic components in the OA anti-stripping agent are the key components responsible for enhancing adhesion and thermal stability, respectively, as revealed by infrared spectroscopy. Compared with OB, OA-modified asphalt exhibited superior anti-aging properties. At a dosage of 0.4%, the softening point of OA-modified asphalt increased by only 0.5°C after aging, whereas that of OB-modified asphalt increased by 3.4°C. Surface energy tests demonstrated that increasing the OA content improved the cohesive work, dry adhesive work, and water damage resistance index (R-value) of asphalt. For asphalt concrete, OA reduced the splitting strength and stiffness modulus, whereas 100% cement replacement of mineral powder restored the splitting strength to the baseline level while maintaining the stiffness modulus at 90% of the reference. In summary, the strategy combining 0.4% OA organic anti-stripping agent with 100% cement filler significantly enhances the adhesion and water stability between asphalt and acidic aggregate while maintaining good mechanical properties. The findings provide a feasible technical pathway for the cost-effective utilization of acidic aggregates in hydraulic engineering.