Slow Ac O(N^3*log(N))
O(N^2) -choice plane through two poins and the centre;
O(N*log(N)) -searching semicircle with max number of points
How make more quickly?

To AC it I've used some analog of hash tables when applied second of your algos (O(N^2)). If points A and B are between two points that we check (on a small arc), then we may not check them in the future. Such a trick helped me to AC the problem in 0.14 secs.