The best 308 is the fiberglass GTB --- lighter, weber carbs... Nothing sounds like the Ferrari V8 playing thru a TUBI exhaust with the Webers.
BUT, the 328 offers more than just 3.2L. It gives you updated interior, better engine performance(compared to the smogged last years of the 308), 16" stock wheels(308s had 14" and some of those oddball 390mm TRX), better cooling system, heavy duty clutch, better water pump as well as a lot of other small improvements. Most 308s were USA models so the bumpers got pretty strange because of the requirements. Euro versions were always more appealing. I think the 328 also looks better(subjective of course) because of the louvers in the front and the rear spoiler bar across the back(some 308s had variations of these). The 328 is a much more refined car.