Stick 2 or 3 cloves into each of the crabapples, and set aside.
In a large pot, stir together the water, sugar, and vinegar. Place allspice berries, cinnamon sticks, and lemon zest in cheesecloth, and tie cheesecloth to make a small bag; add to pot. Bring to a boil, then reduce heat and simmer for 5 minutes. Add the crabapples,and simmer until tender, about 20 minutes.
Use a slotted spoon to lift out crabapples and place them into sterile jars. Cover with hot syrup, and seal in a hot water bath for 10 minutes, or until the centers of the lids are depressed. If the syrup seems too thick, add more water. Refrigerate after opening.
