My audi a3 heaters only get luke warm and seem slightly warmer on the passenger side of the dash.  Both heater matrix hoses are hot after working temperature is reached which makes me think the matrix is okay.

When the heater fails, a typical symptom is insufficient or inoperative heating. If, after the heating temperature has been set to maximum and the blower is working properly, cold air is blowing all the time, it can be assumed that the heater is not giving off heat to the flowing air. Remember that the heater may also be dirty, and you will only need to clean it and not replace it with a new one. 

This could be caused by a sticking thermostat, is the engine getting up to temperature? 

It could be the heater matrix being blocked. 

Or an issue with the heater control unit. 

The engine is getting upto temperature and can feel the inlet and outlet hoses on the matrix are hot which is why I was thinking the matrix is okay. The blower on the passenger side of the car is a lot warmer than the one on the drivers side. 

Could be an issue with blower motor or control unit then. Might be worth getting a diagnostic check to see if any fault codes flag up