The spelling of /p/ is <p> in this set. The /ʒ/ sound is spelled <ge> or <su>.
The contrast is between a voiceless stop and a voiced fricative, well separated in the mouth. It is unlikely to be a problem.
The consonant /p/ can occur in any position in the syllable, but /ʒ/ only occurs finally or intervocalically. /ʒ/ is in any case a very rare phoneme, found in only 334 words in the source dictionary. Figures for mean density and semantic loading are insignificant with such a small set.
leopard leisured leper leisure loop luge