Hi!
Sorry about the late response.
I think you are looking for this option, try turning it off: https://i.imgur.com/H23qlEg.png
The plugin displays items by default, that does not have any of the terms from the filtered taxonomy assigned. Turning this option off will make sure those are excluded as well.
