Preheat oven to 350 degrees F (175 degrees C). Grease and flour a 9 inch springform pan. In a medium bowl, combine blueberries, 1/2 cup sugar, 2 tablespoons flour and 2 tablespoons grated lemon zest. Toss to coat berries, then pour into the prepared pan.
In a large bowl, cream together the butter and 1 cup sugar until light and fluffy. Beat in the eggs one at a time, then stir in the almond extract and 1 tablespoon grated lemon zest. Stir together 2 cups flour and the baking powder. Beat into butter mixture alternately with the milk, mixing just until incorporated. Pour batter over the berries in the pan.
Bake in the preheated oven for 60 to 70 minutes, or until a toothpick inserted into the center of the cake comes out clean. Place serving plate over the top of the pan and carefully invert cake and plate together, so that the berries end up on top. Carefully unlock the buckle on the side of the pan and remove. Sprinkle toasted almonds on top of cake and serve warm.