[NEW YORK] - New York City’s AI Chat ‘Bomb’
The Synopsis:
In October 2023, New York City released an AI Chatbot to help businesses retrieve accurate information about the city, its laws, constituents, and interests. In recent news, the AI Chatbot has been returning inaccurate, dangerous information, like urging store owners to go cashless, even though a 2020 law bans the refusal of cash. Mayor, Eric Adams, refuses to remove the chatbot from the NYC website, knowing the implications.1
The Public Commentary:
Julia Stoyanovich, a computer science professor and the Director of the Center for Responsible AI at NYU, has said, “They’re rolling out software that is unproven without oversight.”1
Jevin West, a professor at the University of Washington and co-founder of the Center for an Informed Public has stated, “Public officials need to consider what kind of damage they can do if someone was to follow this advice and get themselves in trouble.”1
The Analysis:
Eric Adams and others’ refusal to remove the hallucinating, misinforming chatbot [MyCity Chatbot] is like leaving a heretic preacher at the pulpit, hoping he or she will not spread misinformation; the intent IS to harm and mislead, and the longer he or she is there, the more damage is done. Unfortunately, history repeats itself, since we have recently witnessed instances of LLMs’ going awry. For example, Google created an LLM, Gemini, to help user search and query content, yet the outputs were ethically questionable and racially insensitive. Additionally, the MyCity Chatbot disclaims that, “Responses may occasionally produce inaccurate or incomplete content. Validate answers on NYC.gov and MyCityBusiness.”
Eric Adams believes that ‘testing’ the model in the real world would fix the language model, but that is exactly how an unregulated language model becomes egregiously worse.
I prompted the MyCity Chatbot about the minimum wage, yet the answer I received was the following:
“In New York City, the minimum wage is $15.00 per hour.” — This statement was correct in 2022, but in 2024, it is $16.00 per hour.
The MyCity Chatbot should be removed because it is misleading New Yorkers. I, as a fellow New Yorker, would not use this unregulated chatbot. I concur with Julia Stoyanovich and Jevin West who believe that state governments have a responsibility to protects its constituents from harms and should not repeat what New York has done. If our governments cannot do its responsibility to protect us, then we must protect ourselves and boycott the MyCity Chatbot.
The Endnotes:
1 Jake Offenhartz, “NYC’s AI chatbot was caught telling businesses to break the law. The city isn’t taking it down”, The Associated Press, accessed Apr 5, 2024,
https://apnews.com/article/new-york-city-chatbot-misinformation-6ebc71db5b770b9969c906a7ee4fae21