Pour the cold milk into a large bowl; whisk the pudding mix into the cold milk until dissolved, about 2 minutes. Allow to rest until nearly set, 4 to 5 minutes.
Mash the peanut butter into the pudding; stir to distribute the peanut butter evenly throughout the mixture. Spoon the mixture into popsicle molds, tapping the mold on a hard surface to allow any air bubbles to escape. Store in freezer until completely frozen solid, 5 hours to overnight. Running warm water over the outside of the mold will make it easier to remove the pops.
