Place chocolate in top of a double boiler over simmering water. Stir frequently, scraping down the sides with a rubber spatula to avoid scorching, until chocolate is melted, about 5 minutes. Remove from heat and whisk in milk until combined.
Distribute melted chocolate amongst mini muffin cups. Push 1 banana slice in each muffin cup and push to the bottom of the cup with the tip of a knife just until chocolate covers banana. Decorate with sprinkles.
Refrigerate until chocolate is set, 15 to 30 minutes. Pop chocolate treats out by stabbing a knife into the side of the muffin cups.
