Preheat oven to 350 degrees F (175 degrees C). Grease and flour two 9 inch round cake pans.
Combine flour, oats, baking soda, baking powder and salt in a medium size bowl.
In a large bowl, cream the butter or margarine. Add brown sugar, sugar, allspice and nutmeg; beat until light and fluffy. Mix in eggs and vanilla extract. Add 1/3 of the dry ingredients to the butter mixture; mix well. Add the sour cream; mix well. Add another 1/3 of the dry ingredients; mix well. Add half of the bananas; mix well. Repeat with remaining 1/3 of the dry ingredients and remaining bananas. Pour batter into prepared pans.
Bake until cake tests done, about 30 minutes. Let it cool in the pan for 5 minutes, and then turn onto wire rack to cool. Frost with Allspice Cream Cheese Frosting.