In a saucepan, heat 4 cups water to a simmer. Turn off the heat and add the tea bags. Allow to steep for 5 minutes.
Remove the tea bags (and stir in the sugar until dissolved, if making sweet tea). Add the remaining 4 cups cold water to the pan. Cool until room temperature, about 10 minutes.
Transfer to a pitcher. Refrigerate at least 2 hours before drinking. Store up to 4 days.