Heat milk in a saucepan over medium heat without boiling until steaming, about 4 minutes. Beat egg yolks and sugar in a large bowl. Slowly whisk hot milk into yolk mixture until well mixed; return mixture to saucepan and continue to cook on medium heat.
Gradually stir peanut butter and vanilla extract into milk mixture until thoroughly combined, about 5 minutes. Pour peanut mixture into large bowl and cool for 15 minutes.
Stir heavy cream and peanut butter cups into milk mixture, pour into an ice cream maker, and freeze according to the manufacturer's instructions.
