Should notifications be deleted if the action that created them is reversed?

I'm building a web application which allows a user (the first user) to follow another user (the second user.) That second user is sent a notification when they are followed by the first user.

If that first user then unfollows the second user, should the notification to the second user stating that the first user followed them be automatically deleted? Why or why not?