Mathematician Timothy Gowers, a Fields medallist, takes the risk of extinction from AI seriously

Timothy Gowers, a Fields medallist, published a long review on his blog on 16 October of the arguments about whether AI could wipe out humanity. Gowers sits on the scientific advisory panel of the Mathematical AI Safety Institute, founded by another Fields medallist, Jacob Tsimerman, after he left mathematical research for AI safety.

He says he used to dismiss the risk with simple arguments, and no longer does: he is convinced that a trained AI acts as if it had wants of its own. He gives no number, but says he is more worried than when he started and leans towards pausing development to do the theoretical safety work first, because the benefits don't justify the risk.