Consent

The argument that a risk imposed on everyone needs everyone’s agreement, or at least that of most people. A surgeon may operate only with the patient’s informed consent. Building superintelligence, on this view, is an operation performed on all of humanity, so the people building it should not decide alone.

Where it comes up

• Against the labs. In the Against SURA debate, Vaniver argued that even if the labs’ own estimates of the risk are right, imposing that risk on everyone without their agreement is illegitimate.
• Against arguments for speed. Olle Häggström holds that something this momentous needs the informed consent of at least a solid majority of the world’s population. He faults Optimal Timing for Superintelligence for saying nothing about consent while comparing superintelligence to surgery: in surgery, the patient decides.
• In public campaigns. The October 2025 superintelligence statement asks for a prohibition until superintelligence can be built safely and the public supports it. See The politics of AI risk.

Why it is hard to apply

• Whose consent? Eight billion people cannot be asked one by one, and future generations cannot be asked at all. Requiring everyone’s agreement amounts to a permanent ban, which even advocates of consent such as Häggström find too strong.
• How would it be given? Elections rarely turn on AI, and most governments did not choose to host the race. Referendums, treaties and opinion polls each capture something, but none is an obvious way for humanity to decide.
• Not building is also a choice. People die every day of diseases advanced AI might cure, and no one consented to that either. Defenders of fast development argue that consent cuts both ways; critics reply that dying of old age and being killed by a machine someone else built are not morally alike.
• Consent to what? People can consent to a risk only if they understand it. Most of the public has never heard the arguments, and the experts disagree about the size of the risk.

Why it matters

The consent argument moves the debate from whether building superintelligence is wise to who has the right to decide. Even someone who thinks the risk is small can accept that a few companies should not take it on everyone’s behalf. It connects to the question of who would hold power if it succeeds; see Who should control aligned AI.