Preheat oven to 350 degrees F (175 degrees C). Grease a 9x13-inch baking pan.
Mix 3 cups flour, milk, white sugar, eggs, baking powder, 2 teaspoons vanilla extract, and salt together in a large bowl until well-combined. Stir in 1/4 cup melted butter. Pour batter into the prepared baking pan.
Beat 1 cup softened butter, brown sugar, 2 tablespoons flour, and cinnamon together in a large bowl until smooth. Drop by spoonfuls over the batter in the baking pan. Swirl the top layer into the bottom layer with a knife to create a marble effect.
Bake in the preheated oven until a toothpick inserted into the center comes out nearly clean, 25 to 30 minutes.
Mix confectioners' sugar, milk, and 1 teaspoon vanilla extract in a large bowl until smooth. Drizzle over warm cake.
