It can be done. Say your Date field is called fldDate. In a blank column in the query builder, put something like the following in the "Field:" line:
IIf(IsNull([ENTER DATE]),[fldDate] Like "*",[fldDate]=[ENTER DATE])
Uncheck the "Show:" check box, and in the "Criteria" line, put:
<>False
Delete anything you currently have in the fldDate criteria (but make sure you have fldDate in your query if you want it to show).
Hope this helps,
Russell