Place the potatoes into a large pot and cover with salted water. Bring to a boil over high heat, then reduce heat to medium-low, cover, and simmer until tender, about 20 minutes. Drain and allow to steam dry for a minute or two. Allow to cool, peel, and cut into cubes.
Place the eggs into a saucepan in a single layer and fill with water to cover the eggs by 1 inch. Cover the saucepan and bring the water to a boil over high heat. Once the water is boiling, remove from the heat and let the eggs stand in the hot water for 15 minutes. Pour out the hot water, then cool the eggs under cold running water in the sink. Peel and slice once cold. Reserve 1 attractive slice of egg for garnish.
Mix the mayonnaise and mustard together in a bowl. In a large salad bowl, lightly mix together the potatoes, eggs, onion, celery, sweet gherkins, and olives until thoroughly combined. Pour the mayonnaise dressing over the salad, and gently toss gain to coat the ingredients with dressing. Season to taste with salt, and top the salad with slices of green pepper and reserved egg slice in the center for garnish. Chill until ready to serve.