De-bone chicken and cut into chunks or shred. Set aside.
Melt 2 tablespoons butter in a large Dutch oven over medium heat; cook and stir onion and celery until soft, about 10 minutes. Sprinkle in 2 tablespoons flour and whisk continuously to make a thick roux, about 2 minutes. Slowly pour in chicken broth, whisking to remove any lumps. Add 1 teaspoon salt, black pepper, basil, thyme, potatoes, and mixed vegetable. Cover and cook the stew over medium heat until vegetables are tender, 15 to 20 minutes. Stir in chicken meat and continue to simmer.
Meanwhile, combine 1 1/2 cups flour, baking powder, and 1/2 teaspoon salt in a large bowl; cut in 3 tablespoons butter until the mixture resembles coarse crumbs. Stir in milk and dill. Drop rounded tablespoonfuls of dough into the simmering stew. Cook, uncovered, for 10 minutes. Cover and cook until the dumplings are tender, 8 to 10 minutes more.