Hi bud,
I did my bulb change on my A3 and you can get to them without taking the bumper off... however... if you want to play around with it I'd suggest having it loose off so it gives you a wider access point!
There's a few guides on how to take it off with all the screws and tools needed. Not too hard if you have the time!
Thanks,