Combine mandarin oranges, sugar, pina colada mix, banana, milk, and food coloring in a saucepan. Bring to a gentle boil, stirring until fully combined. Transfer immediately to a small bowl. Cover with plastic wrap and freeze until firm, 8 hours to overnight.