Combine cauliflower, black olives, green olives, green bell pepper, celery, pimento peppers, onion, sugar, salt, and pepper in a large bowl.
Whisk olive oil, vinegar, and lemon juice together in a small bowl. Pour over cauliflower mixture and toss to combine. Refrigerate salad until flavors combine, 4 hours to overnight.
