My headlights just went out but Im sure if its a fuse or the bulb how can I check them?

Well......You have a right and left headlight fuse in a plastic box close to the battery. If both the high and low beams went out on it, the solution would be
new fuses. If just the low beams went out on it, the solution would be new bulbs. Another way to check the filament is to whack the headlight housing with your hand with the low beams turned on. If doing that makes the low beam come on you know the problem is the bulb. The vibration will reconnect a broken filament temporarily.