Place potatoes into a large pot of salted water; bring to a boil, reduce heat to medium-low, and simmer just until tender, about 20 minutes. Drain potatoes, transfer to a plate, and refrigerate until cool enough to touch.
Preheat oven to 350 degrees F (175 degrees C).
Dice cooled potatoes. Spread potatoes into the bottom of a 13x8-inch cake pan; top with diced kielbasa and diced egg. Season mixture with pepper.
Stir sour cream and milk together in a bowl and pour over the ingredients in the cake pan. Sprinkle Cheddar cheese over sour cream layer.
Bake in preheated oven until the cheese is melted and the casserole is hot in the middle, about 30 minutes.
