Squeeze 3 tablespoons juice from lemons; reserve squeezed lemon halves. Combine the lemon juice, parsley, mustard, garlic, oil, rosemary, pepper, and salt in a small bowl; blend well.
Toss the potatoes with 2 tablespoons of the parsley mixture in a large bowl. Reserve. Gently slide fingers between the skin and meat of the chicken to loosen skin. Spoon parsley mixture between skin and meat. Secure breast skin with toothpicks, if necessary. Place lemon halves in the cavity of chicken. Place chicken on a rack in a roasting pan.
Bake chicken in preheated oven for 30 minutes. Arrange potatoes around the chicken; bake until chicken juices run clear, about 1 additional hour. Allow chicken to rest for 10 minutes before serving.
