In a large pot over high heat, combine the celery, onions, rice, bouillon, broth and water and bring to a boil. Reduce heat to low, cover and simmer for 30 minutes, or until the rice has absorbed most of the liquid. Remove from heat and set aside.
In a medium saucepan over medium heat, melt the butter or margarine. Slowly add the flour, stirring often, to make a roux. Add 4 cups of milk, 1/2 cup at a time, while constantly stirring. Add this and the chicken to the rice mixture and return the rice mixture to the stovetop over low heat.
If the soup seems too thick, add some or all of the remaining 2 cups of milk. Season with salt and pepper to taste and allow to simmer for at least an hour, stirring every 15 minutes.
