I have a report that has conditional formatting of >10 = yellow and >12 = red for a field.

When the report is run, all the numbers are correctly color coded, but when a user uses an olap dimension to drill into the report, the colors remain even though the values are definitely less than 10 or 12.

I would understand why this would be a good idea in some cases, so you can remember where there was a problem while you are drilling... but I want to perform conditional formatting for each 'drill down' in tabbed olap. Is this possible?

Works fine for me. I have a regular report that I use to drill into an OLAP Tabbed report.
When the OLAP report comes up, the background color of the numerical field shows appropriately. Am I missing something in your situation here?