Soak cured lemons briefly in a bowl with cold water to remove curing brine, about 10 minutes. Dry lemons with paper towels and finely mince.
Stir minced lemon, mayonnaise, tarragon, and lemon juice together in a bowl. Cover bowl with plastic wrap and chill aioli in the refrigerator until cold, at least 15 minutes.
Preheat an outdoor grill for high heat, and lightly oil the grate.
Put shrimp in a bowl. Drizzle olive oil and sprinkle paprika and salt over the shrimp, stir to coat shrimp completely.
Cook the shrimp on the preheated grill until they are bright pink on the outside and the meat is no longer transparent in the center, about 2 minutes per side. Transfer shrimp to a plate and serve with lemon aioli.
