A couple clicks of time alignment is insignificant phase change in the lower frequencies where phase becomes more meaningful. If you can improve your stage with a small time adjustment, go for it?
I would also focus on what frequencies in the stage are not centered. Have you tried playing bandwidth limited pink noise tracks? I would try that with only one pair of channels engaged at a time, like woofers only, then mids only, etc. Somewhere around here we have some Tuning Stones that work great for this