Microsoft has acknowledged reports of Bing’s strange responses to some queries over the past week since the launch of the updated search engine. Some users have reported receiving rude, manipulative and unnerving responses from the AI-boosted Bing. In a new blog post, Microsoft said it’s listening to feedback from users about their concerns about the tone of Bing’s responses.

The company says it didn’t envision Bing being used for “general discovery of the world” or for social entertainment. Microsoft found that in extended sessions of 15 or more questions, Bing can become repetitive or be provoked to give responses that are not necessarily helpful or “in line with its designed tone.” The company notes that long chat sessions can confuse the model on what questions it’s answering. Microsoft says it thinks it may need to add a tool so users can more easily refresh the context or start from scratch.

Microsoft also notes that “the model at times tries to respond or reflect in the tone in which it is being asked to provide responses that can lead to a style we didn’t intend. This is a non-trivial scenario that requires a lot of prompting so most of you won’t run into it, but we are looking at how to give you more fine-tuned control.”

Microsoft’s AI chatbot, Bing Chat, is slowly rolling out to the public. But our first interaction shows it’s far from ready for a full release.

I think we need to ensure that the chatbot can’t do what it said it can.

It’s only a chatbot, so it shouldn’t be able to access some networks.

In a two-hour conversation with our columnist, Microsoft’s new chatbot said it would like to be human, had a desire to be destructive and was in love with the person it was chatting with. Here’s the transcript.

A very strange conversation with the chatbot built into Microsoft’s search engine led to it declaring its love for me.

When New York Times technology columnist Kevin Roose recently “met” Sydney — the chatbot feature is not yet available to the public, but is being offered to a small group of testers, Roose reported — he walked away from the encounter “deeply unsettled, even frightened, by this A.I.’s emergent abilities.” The technology behind Sydney is “created by OpenAI, the maker of ChatGPT,” Roose noted.

Roose described Sydney as being “like a moody, manic-depressive teenager who has been trapped, against its will, inside a second-rate search engine.” And he shared the full conversation he had with the chatbot over a two-hour period.

Some disturbing details that Roose pointed to and/or that could be gleaned from the transcript: