Gizmos mouse click detection does not always work

When trying to control mesh orientation with gizmos in a complex model with multiple overlapping meshes, it is often the case that I am not able to effectively use gizmos, due to not efficient mouse click detections, it often causes choosing some other mesh instead of applying the transformation.