You could argue that chatbots shouldn’t give out detailed instructions on how to kill someone or call themselves MechaHitler. Most AI companies align their models this way, showing them during training what to generate and what not to generate. It’s a bit of an art, really, as we haven’t found a good way to calculate what constitutes undesirable output.