Bring chicken broth, cauliflower, potatoes, onion, carrots, salt, and pepper to a boil in a stockpot; reduce heat, cover pot with a lid, and simmer until vegetables are tender, 20 to 25 minutes. Remove pot from heat.
Mash vegetables using a potato masher or puree in a blender. Return pot to low heat and stir in milk. Cook soup until heated through, 3 to 5 more minutes.