I have WA, but I don't guess at the feature of this test. Does anyone have a problem with this test? Please comment your trick.

You should be careful with the point (X, 0) that lies on line A + (B - A). The angle should be 0 there, but because of precision issues we all love this case can break ternary search.