Pour olive oil into a heavy baking dish (such as an enameled cast iron dish). Place potatoes into the baking dish and toss to coat with olive oil. Sprinkle with fresh thyme sprigs, kosher salt, black pepper, cayenne pepper, and red bell pepper pieces.
Bake in the preheated oven for 30 minutes. Toss vegetables to turn over. Some potatoes may stick; don't try to turn them. Return to oven and bake for 20 more minutes.
Toss potatoes again, turning potatoes so unbrowned sides contact the bottom of the dish. Continue baking until potatoes are browned and tender, about 15 more minutes.
Turn oven off and leave potatoes in the oven for 15 more minutes. Toss once more and adjust levels of salt, black pepper, and cayenne pepper before transferring potatoes to a serving bowl.
