Palace put Southampton under sustained pressure to open the second-half, with Odsonne Edouard forcing Caballero into a strong double save.
The home side finally broke through after a lengthy period on top; Jordan Ayew struck in on the rebound from a tight angle to end his goal drought.
Southampton forced the game to a better balance, and the game ended in a tense deadlock.
The draw left Palace 11th on the table, while the Saints are 15th.
In another match, Romain Saiss scored the only goal as Wolves beat Brighton 1-0 at the American Express Community Stadium.
The Moroccan found the back of the net in the first-half added time to halt Wolves’ two-match losing run and move them eighth.