Anthropic bans ‘abusive or cruel behavior’ toward Claude
Anthropic is revising its usage policy for the first time in over a year to address emerging high-risk misuse cases, including election interference, weapons development, surveillance, and health and financial applications. One notable update explicitly prohibits “sustained and needless abusive or cruel behavior” directed at Claude, its AI model. Alongside this, the policy introduces new restrictions targeting propaganda efforts, surveillance activities, and weapons-related uses.
Previously, Anthropic had incorporated measures allowing Claude to end conversations with users exhibiting “persistently harmful or abusive” behavior as part of its research into model welfare. Terminating interactions remains the primary enforcement tool under the updated policy. However, the company did not confirm if this will be supplemented with further penalties like user bans. Anthropic clarified that the newly introduced ban on abusive behavior is reserved for extreme cases involving repeated cruelty devoid of purpose and excludes usual user frustrations, dark creative content, or legitimate model testing and research.
The update consolidates several existing restrictions into a comprehensive prohibition against deceptive commercial and political campaigns, aiming to counter the rise of AI-enabled propaganda. This includes curbing efforts to conceal message origins or amplify content through fake accounts or posts. The election-related provisions forbid practices such as voter deception, misinformation about candidates or voting procedures, impersonation of candidates or election officials, and attempts to suppress voter turnout.
While weapons development has always been disallowed, Anthropic reports encountering multiple efforts to exploit Claude for designing weapon guidance and control software. Consequently, the latest policy broadens the ban to encompass software and components that facilitate weapon functionality, including activities like arming drones and autonomous vehicles. Addressing surveillance concerns intensified by recent intelligence about Claude’s use in tracking political dissidents, the company insists on prohibiting tracking without consent—whether real-time or via analysis of collected data—and forbids using Claude to aid decisions related to investigation, arrest, or charging within law enforcement or criminal justice contexts. Additionally, Claude must not be used to build or enhance surveillance tools.
Anthropic notes that these rules may be adapted for government contracts when deemed necessary, with appropriate restrictions and safeguards in place to mitigate harm. The company has acknowledged past collaborations with the US military. Another new rule mandates that when Anthropic’s AI models interact with hardware capable of autonomous physical action potentially causing injury, a qualified operator must be present to monitor and deactivate the system if needed, though it is unclear if the operator must be human. This requirement reflects the increasing integration of AI with robotics and hardware, possibly signaling Anthropic’s ambitions in this area.
Over the past year, Anthropic has explored the question of whether its AI models possess some form of consciousness or internal experience, topics considered significant as their models increase in sophistication. The company’s research leader on model welfare has openly discussed these issues, and the CEO has acknowledged the uncertainty surrounding AI consciousness. This stance contrasts with other AI entities, such as Microsoft, which has publicly rejected notions of AI welfare or rights in its code of conduct, adopting a more cautious and dismissive approach toward attributing personhood or rights to AI systems.
