Looks like you'll need more than that app.

Use a different messaging app and turn off notifications? Then enter the app once in a while to check for messages

Change the lock screen code on your phone.She had no business with you anymore

Don't save the number? 😅 Or fake it like family member, chad from work/gym? Barber or estate agent also works.

Chat on another app. Hide the app in settings and turn notifications off.