You’ll only need to use two of the original wires to provide power. The rest of the wires were most likely to provide data communication between the main panel and the keypad.
Look at the main panel and determine which colored wires are providing power to the keypad. It’s most likely a black and red wire. There may be a schematic diagram on the inside door of the panel that shows which wires provide power to the keypads.
In addition, you’ll need to ensure the panel is providing DC power and not AC power. The adapter you have is expecting DC power to convert down to 5V DC power.