Preheat the oven to 325 degrees F (165 degrees C). Grease an 9x5 inch loaf pan.
In a medium bowl, cream together the butter and sugar until light and fluffy. Gradually beat in the egg yolks until well blended. Combine the flour, baking powder, and nutmeg; stir into the batter alternately with the port wine and brandy. Fold in the candied fruit.
In a separate clean dry bowl, whip egg whites to soft peaks. Fold 1/4 of the egg whites into the batter to lighten, then fold in the remaining whites. Pour into the prepared loaf pan.
Bake for 1 hour in the preheated oven, or until a toothpick inserted into the center of the loaf comes out clean. Cool cake in the pan, and pour the bourbon over it. When the bourbon has soaked in, remove it from the pan, and wrap with aluminum foil. Let it sit for at least 1 day before serving.
