Preheat oven to 350 degrees F (175 degrees C). Grease and flour a 9x13 inch cake pan.
Cream together 1/2 cup butter and 1/2 cup peanut butter. Stir in white and brown sugars and beat well, then add the beaten eggs and 3/4 teaspoon of vanilla. Sift together flour, baking powder and salt. Add the flour mixture into the batter alternately with the milk.
Pour batter into the prepared pan and bake for 35 minutes. Frost when cool.
To Make Frosting: Blend together 1/8 cup butter or margarine, 3 cups confectioners' sugar, 4 1/2 tablespoons evaporated milk and 1 1/2 teaspoons vanilla extract, stirring slowly until mixture holds its shape. Then beat until very smooth. If necessary, gradually add tiny amounts of evaporated milk to bring to spreading consistency.