Mix water and sugar in a saucepan. Bring water to a boil, reduce heat to medium-low, and cook, stirring regularly, until sugar is dissolved completely into a simple syrup, about 5 minutes.
Stir 6 tablespoons pumpkin pie filling, cinnamon sticks, nutmeg, ginger, cloves, vanilla extract, and pumpkin pie spice into the syrup. Bring liquid again to a simmer and cook until the syrup is well-spiced, 8 to 10 minutes.
Strain the mixture through a sheet of cheesecloth. Return liquid to the saucepan over low heat.
Stir sweetened condensed milk and remaining 2 tablespoons pumpkin pie filling into the spiced syrup and cook, stirring regularly, until the pumpkin dissolves into the liquid, about 5 minutes.
