Subj. It's standard problem to get the best vector - even formula exists, but for linearly independent vectors. But how then minimize second tieresolver in case of linearly dependent system?
My iterative approach needs to find subsystems of linearly dependent vectors. Is there easier approach, may be formula?

There is a formula, but it is really a mess and it is quite non-trivial to obtain. There are different apporaches possible, one of them is to notice that to minimize the sum of squares you should do almost the same thing as when you minimize the difference vector.