Preheat an oven to 325 degrees F (165 degrees C). Grease a 9x13-inch baking dish.
Chop the peaches and lightly mash them in a mixing bowl. Combine the sweetened condensed milk and the eggs; add them to the peaches and mix well. Stir in the hot water, melted butter, cinnamon, and vanilla. Stir the French bread into to the custard mixture until the bread is completely moistened. Turn the pudding into the prepared baking dish.
Bake until a knife inserted in the center of the pudding comes out clean, about 1 hour and 10 minutes.
While the pudding is baking, combine the brown sugar, 1/2 cup butter, corn syrup, and rum in a saucepan. Bring to a boil over medium heat and simmer for 3 to 4 minutes or until just slightly thickened. Let cool slightly.
Remove the pudding from the oven and let it cool for about ten minutes before serving. Serve warm with the caramel sauce. Cool and cover any leftover pudding and store it in the refrigerator.
