Block/dampen/filter infrared light "whiteout" on a brickwall?

Have a video doorbell 2, which is mounted on the frondoor frame. On the left side there is a red brickwall for the first 25 cm, so view on the left is blocked. This is fine as i'm not interested in having a view in that direction, and camera can not be mounted in other positions. However this brickwall causes the night views to be most white on the left due to IR on this wall. The camera then probably just turns down the exposure as there so much white content, and then the right part is more or less completely black, making night vies almost unusable. I have tried to hang a matt black rubber piece on the brick wall covering the left view, to kind of absorb the IR light. It works somewhat, so the right part got brighter, but still not near usable. Any tips how to get this IR light on the left side part eliminated? br Lars