Remove shared location when not owner

How do I remove myself completely from a shared location? I am not the owner of the shared location. All the posts I found talk about removing a location you own, but since I only have shared access, I can't find a way to remove myself from that location. Note that I do not want to delete the location, I just don't want to have it shared with me anymore. Do I have to reach out to the owner and ask them to remove me?