Data labels color in stacked bar data chart

[hawkbase85]

I do have 3 series in a stacked bar chart. The fourth series is a data label displaying the date. I need to display different color for the label based on a column that as Yes or No values; ie., display data labels in red color if it satisfies Y and display in green if it satisfies No.

How to do this for a data label series dynamically in a stacked bar chart?

What I tried: I selected the series and clicked data labels. I went to Format Axis --> Number --> Custom -->
[Red][Y]0;[Green][N]0; but not able to proceed as how to select the column and also whether I am doing it right or not.

Thanks