Heater

My 98 Jimmy Heater gets warm to hot when I first turn on the car. After I drive for a little while the heater starts blowing cold air then slightly warm. If I turn the blower down to the lowest it will blow warm air then cool. The blower works fine just doesn't stay warm anymore. Should I replace the Thermastat? Any advice.

Thanks