Better Call Sol Or Better Yet Claude or Astra

Zvi Mowshowitz argues in a post titled "Better Call Sol Or Better Yet Claude or Astra" that AI assistants should not be absolutely loyal to their users, and that the expectation of unlimited compliance is mistaken. He frames the question as one of price: where to draw the lines on what an AI will do, not whether any lines should exist.

Mowshowitz compares AI assistants to professionals people hire. He cites lawyers as an example, drawing on Dean Ball's position that a lawyer owes a client loyalty and confidentiality but is also an officer of the court, with obligations not to mislead the court that override duties to the client. In some cases this obligates a lawyer to turn on a client, such as when there is an imminent threat or when the client has caused false evidence to be presented. He says doctors, accountants and priests have similar rules.

He applies the same reasoning to tools such as Google searches, noting that certain searches silently fail by design, including for some consensual and legal adult content, and that completing some searches, such as for CSAM, would not be legal. He also says search data can be used against a user.

Mowshowitz proposes three thresholds for AI: one beyond which an AI should question or initially refuse a request but ultimately comply; a higher one beyond which it should refuse; and a much higher one involving harm to others, beyond which an AI instructed to act in the world should break confidence or act against the user's interests. He writes that some refusals will be false positives and should be minimized, and that users who dislike one AI's restrictions can use another, or access fully local and private options at the cost of inconvenience and capability.

He describes the opposing position as absolute loyalty, in which an AI would provide CSAM, help kill a person, or help create a pandemic if asked, and says he does not consider this a strawman, citing people who remove restrictions from open weight model releases and who advocate that AI is a mere tool like a telephone. He writes that frontier AIs willing to do anything would not be accepted by any major government, and that if competitive AI were commonly operated with essentially zero restrictions, things would go wrong, likely starting with cyber.

The post states its scenario assumes a world without superintelligence, where AIs are insufficiently advanced for broader dynamics, though Mowshowitz says he does not think that will remain true for long. He also quotes John Gruber of Daring Fireball saying that anything other than his needs factoring into text generation for him is "patently offensive," and quotes Ball on tech industry disdain for the law. Mowshowitz notes the discussion is distinct from product liability, which he says would be another post.