Preheat oven to 350 degrees F (175 degrees C). Grease a 9x13-inch baking pan with cooking spray.
Stir flour, white sugar, brown sugar, cocoa powder, baking powder, and salt together in a large bowl. Pour in water, vegetable oil, milk, and vanilla extract; mix until well blended. Spread into the prepared pan.
Bake in the preheated oven until top is dry and edges have started to pull away from the sides of the pan, about 30 minutes. Let cool completely, about 30 minutes.
Stir confectioners' sugar, butter, and cocoa powder together in a bowl until smooth. Mix in enough milk to reach a spreading consistency.
Spread frosting over cooled brownies; drizzle caramel topping on top. Cut into squares.
