Hi,
Thank you very much for the details!
Usually the issue in these cases is that there might be taxonomy term inclusions set. Make sure to check that and unset that option.
Those will override the filters under specific circumstances. It seemingly works with checkboxes, because the unchecked ones are excluded – but with drop-down it has obviously no effect as a drop-down is inclusive and not exclusive.