Preheat oven to 350 degrees F (175 degrees C). Grease a 9-inch square baking dish.
Bring rice and 1 1/4 cup chicken broth to a boil in a saucepan. Reduce heat to medium-low, cover, and simmer until the rice is tender and the liquid has been absorbed, 20 to 25 minutes.
Heat olive oil in a saucepan over medium heat. Cook and stir mushrooms and onion in hot oil until mushrooms are soft and onion is translucent, 5 to 7 minutes. Stir chicken, chicken soup, sour cream, milk, 1/2 cup chicken broth, seasoned salt, and black pepper into the mushroom mixture. Reduce heat to low. Add cooked rice to the saucepan; stir until rice is evenly mixed in. Transfer mixture to prepared baking dish.
Bake in preheated oven 15 minutes. Top casserole with Cheddar cheese and crushed croutons; bake until top begins to brown, about another 10 minutes. Let cool 5 minutes before serving.
