I was mistaken - you can actually do this if you have the Ring Alarm, it is just not that obvious in the documentation. Not sure if it is supported with the new modes feature of the app in general. My guess is this will come soon as it seems like if would be useful even if you only have Ring Cameras.