Yellow River source water has long suffered from odor problems in winter. In this study, odor characteristics, potential odorants, and algae in the source water of six cities (Lanzhou, Yinchuan, Hohhot, Zhengzhou, Jinan and Dongying) along the Yellow River were determined in winter (February to March 2014). According to flavor profile analysis (FPA), moderate to strong fishy odors occurred in all cities, except for Lanzhou. At the same time, mild earthy/musty odors and septic/swampy odors were also detected. The strong fishy odor (FPA intensity, 8.5) in Yinchuan was attributed to the abnormal growth of Dinobryon (cell density, 5.7 × 104 cells/mL), while the fishy odors in Hohhot, Zhengzhou, Jinan, and Dongying might be caused by Melosira and Cyclotella, Cryptomonas, Dinobryon, and Synedra, respectively. Unsaturated aldehydes, which have been reported to cause fishy odors, were not detected in all samples. However, some saturated aldehydes, including hexanal, heptanal, nonanal, decanal, and benzaldehyde, were detected with a total concentration range of 690 ng/L to 2166 ng/L, and might have partly contributed to the fishy odors. In addition, 2-MIB (5.77–21.12 ng/L) and geosmin (2.26–9.73 ng/L) were responsible for the earthy/musty odors in the Yellow River source waters, and dimethyl disulfide (648.2 ng/L) was responsible for the rancid/swampy odor (FPA intensity, 8.0) episode in Yinchuan. This is a comprehensive study reporting on the occurrence and possible reasons for the odor issues in the Yellow River source water during winter.