Forrester: Question Uses of Generative AI Before Experimenting

The onslaught of ChatGPT and AI generative news isn’t necessarily helpful to today’s corporate decision makers.

Early indications are that conversations with OpenAI’s trained language model can take some strange paths. Still, innovators need to nurture, explore, and probe the use cases for this AI technology.

That’s the word of Rowan Curran, analyst at Forrester Research. While the sheer scale of generative AI datasets brings new complexity, the same ground rules that already guide good AI governance likely still apply, according to Curran, who recently co-authored a report with fellow Forrester analysts. about generative AI and the enterprise. Learning and experimenting is time well spent, he suggested.

Experimentation and emotion – and caution

While it’s still not easy to critically explore the vast possible use cases for generative AI, underestimating the technology would be a mistake, Curran told VentureBeat. Forrester is encouraging people to embrace the experimentation and excitement of this space, he said, but to do so with the knowledge that what you’re building is likely to look quite different from ChatGPT and its brethren as seen today.

>>Follow VentureBeat’s ongoing coverage of generative AI<

“The important thing for leaders – whether at the C level or a few clicks below – is to take a very investigative, skeptical and questioning approach when embracing generative AI,” he said.

Generative AI shows promise for content creation and summarization, both on the textual side and on the image generation side. In Forrester’s estimation, this can promote collaboration within organizations. It can start writing code and support research in different programming schemes.

Still, as noted in Forrester’s Generative AI Prompts Productivity, Imagination, And Innovation In The Enterprise report: “Generative AI can go horribly wrong, and we still don’t know much about how generative AI models perform at scale.”

What’s inside the box?

Some past experience also suggests that the large-scale datasets of these models can take unwanted biases to new levels.

AI as a black box – creating unexplained results – has long been a concern of CEOs, technologists and society at large. VentureBeat asked Curran if generative AI has overcome these black box issues in any way.

“Absolutely not,” he replied. “We still have the same issues with data quality, bias and ensuring these models perform acceptable. One of the current challenges with them is that when it comes to large language models (LLMs), they are very much like black boxes in many ways.”

However, a lot of work is being done to determine what is going on inside these LLMs. But the fact that there isn’t a clear picture of its inner workings shouldn’t be an impediment to cautious experiments, Curran said. And for many, the experience will be familiar.

“We’ve been using neural networks in a variety of different use cases for years, and understanding what a neural network is doing is still very difficult,” he said. “The big language models are a black box, but that should drive how we apply them, not move away from them completely.”

But there are also some differences. LLM itself is not just a sort of black box for the viewer, but also the datasets it works on. And big means big.

“The sheer size of the models makes it very difficult to do a complete overhaul within a reasonable amount of time at a reasonable cost,” said Curran.

Weighing generative AI

Looking to the future, it is up to business decision makers to discern what is sensible and doable with ChatGTP-style generative AI models. Failing to understand your ins and outs can be a costly mistake, Curran advised.

Pros include increased developer productivity, more extensive test suites to strengthen security, and expand the breadth of human creative expression. Cons include a tendency to bias, vulnerability to security attacks, disarming human behavior, and significant costs.

Curran said it’s important to look at these innovations as business tools. He said that there is no corporate tool that solves all problems for everyone. “Taking an approach like this to generative AI will only end in disappointment,” he scolded.

Each organization will need to study cases where it can leverage the strengths of the new tools in its organization. In the beginning, Curran said, this might include brainstorming and summarizing content.