Preheat the oven to 375 degrees F (190 degrees C).
Combine mushrooms, celery, onion, rice, and garlic in an ungreased 10-inch baking dish. Sprinkle thyme, poultry seasoning, salt, and pepper on top.
Add enough water to reserved mushroom liquid to measure 3/4 cup and pour over rice mixture in the baking dish. Place chicken on top, skin-side up. Brush with olive oil and season with salt and pepper to taste. Cover baking dish tightly with aluminum foil.
Bake in the preheated oven for 1 hour. Remove aluminum foil and bake until skin is browned to your preference, 30 to 45 minutes more.
