Yes and no.
It really depends on the the class of tea that is being brewed.
Usually teas that are designated Breakfast teas are made from black teas.
They tend to be darker and more roasted teas.
Teas that are grown and processed in India and Sri Lanka
are designed to take on additions like spices and ginger.
These teas are particularly designed to be had with milk, sweeteners or lemon slices.
The general rule of thumb is that black teas are best for any of these additions.
Moderate to high quality teas that are mildly to almost no roasting do not require additions.
It is about the flavor and aroma of the tea.
Some are purist and would not think of tainting the tea with additions of sweeteners, milk or lemon.
This is considered tantamount to adding salt and pepper to a dish that is already perfectly flavored.
At the end of the day, you are the final arbiter of taste in your tea.