Combine milk, banana pudding mix, and vanilla extract in a bowl and mix until smooth. Refrigerate banana filling at least 30 minutes before using.
Combine milk, vanilla pudding mix, and vanilla extract in a bowl. Whisk in confectioners' sugar. Fold in whipped topping until no streaks remain. Refrigerate frosting for at least 30 minutes before using.
Preheat the oven to 375 degrees F (190 degrees C).
While filling and frosting are chilling, prepare the cupcake batter. Whisk flour, baking soda, cinnamon, and salt together in a bowl and set aside.
Cream white sugar, brown sugar, and butter together in the bowl of a stand mixer until light and creamy. Add eggs, 1 at a time, beating well after each addition. Add vanilla extract. Alternate adding flour mixture, mashed bananas, and yogurt. Mix on low speed until just moistened; batter may be lumpy, but do not overmix. Fill 24 muffin cups 3/4 full with batter.
Bake in the preheated oven until a toothpick inserted in the center of a cupcake comes out clean, 17 to 20 minutes. Place on a rack to cool completely, at least 30 minutes.
Use a paring knife to cut a small circle out of the top of each cooled cupcake. Fill with banana filling. Top each cupcake with vanilla frosting.
