You can check the Timing Advance of your cell. Sometimes, when you have phantom rachs, you have very big TA which are not justified (but check that this is not a resurgence case.
Generally, the problemm can be solved by reducing the cell size (rach_ta_filter)