Whisk the chicken soup base and water together in a large pot. Add the potatoes, carrots, and celery, and bring to a boil; cook until the vegetables are tender, 10 to 15 minutes. Reduce heat to simmer, and stir in the evaporated milk.
In a mixing bowl, mix together the egg, flour, and milk using your hands. Drop by teaspoonful into the hot soup. Allow the dumplings to cook in the soup until firm.
