New member here! 

Just bought 2004 A3 SE and wanting to upgrade the headlights. Have found some OEM Xenon headlight units already but how easy are they to change? Could you get similar improvements from changing just the bulbs? 

The OEM units will be quite expensive and a reasonable amount of work. To get it working properly as per the factory you need to also make sure the auto levelling is working and also the headlight washers will need to be fitted. If you get all this done it will be the best possible upgrade though.

Cheapest and easiest solution would be to just upgrade the bulbs for something like Osram Nightbreakers or similar. These will give a decent improvement for a fairly small cost but obviously will not be as good as xenon.

The other option is an after market xenon/HID kit using your existing headlights. Also not very expensive if sourced online but a little more involved to fit. You will need to find canbus compatible ones for Audi too. This option will give a good light output but can be a grey area legally as like the OEM xenons you are meant to have auto levelling and headlight washers. They are also not designed for you type of headlight.

Hope that helps.