Walker Management Library
Owen Graduate School of Management
401 21st. Avenue South
Nashville, TN 37203
Exploring the Business Value of Generative AI
Generative AI potentially offers much value to business. The 2024 McKinsey Global Survey indicates that use of Gen AI in organizations has rapidly increased in the course of a year. 65% of respondents say that Gen AI is used in their organizations (Singla et al., 2024). Most of that use is in marketing and sales and product and service development (Singla et al., 2024). Still, we are in the early stages of Gen AI's use in organizations, and time will tell how these tools can best be leveraged as helpful assistants to human work.
Below you will find a number of sources discussing various aspects of Gen AI in business.
- The State of AI in Early 2024: Gen AI Adoption Spikes and Starts to Generate ValueThis article presents findings from McKinsey and Company's survey on Generative AI, which charts the adoption of Gen AI in organizations globally.
- A Reality check of the benefits of LLM in businessThis paper thoroughly examines the usefulness and readiness of LLMs for business processes. The limitations and capacities of LLMs are evaluated through experiments conducted on four accessible LLMs using real-world data. The findings have significant implications for organizations seeking to leverage generative AI and provide valuable insights into future research directions.
- How People Can Create—and Destroy—Value with Generative AIA first-of-its-kind scientific experiment finds that people mistrust generative AI in areas where it can contribute tremendous value and trust it too much where the technology isn’t competent.
- Competencies Needed by Business Professionals in the AI Age: Character and Communication Lead the WayA study by Cardon et al. (2024) exploring the uptake of Gen AI by business practitioners and views about Gen AI's impact on business skills.
- Generative AI in Real-World WorkplacesReport published by Microsoft synthesizing a number of their studies on Generative AI in workplaces. Note: this is Microsoft funded and produced research and Microsoft has a financial interest in Generative AI tools.
- 2024 Work Trend Index Annual ReportFrom Microsoft and LinkedIn, this report focuses on AI at work based on survey data. It finds that employees want to use AI at work and discusses the impact of AI as reported by workers. Note: The study was commissioned by Microsoft, which has a financial interest in Generative AI tools.
Limitations of Generative AI
Limitations
Generative AI tools do have several limitations. It is important to be aware of these limitations so that you can ensure you get the most out of your interaction with the Gen AI.
Training Data Limitations
Training data for Gen AI LLMs have the following limitations:
- Knowledge cutoff date – the date developers use to indicate the recency of training data
- Cultural bias – negative stereotypes or attitudes on the basis of gender, race, ability, sexual orientation, etc.
- Information quality – the extent to which information is credible
Read more about these limitations below:
- Dated Data: Tracing Knowledge Cutoffs in Large Language ModelsIn this work, we define the notion of an effective cutoff. This is distinct from the LLM designer reported cutoff and applies separately to sub-resources and topics. We propose a simple approach to estimate effective cutoffs on the resource-level temporal alignment of an LLM by probing across versions of the data. Using this analysis, we find that effective cutoffs often differ from reported cutoffs.
- On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?This paper examines the risks of large training data, which includes biases inherent in using all of the data available on the internet.
Hallucinations
Another limitation with LLMs is that they are known to hallucinate or make up factually incorrect information. Check out the hallucination rates for various LLMs:
Also, check out the hallucination example below showing how ChatGPT made up a resource. You can also view the full transcript of the chat interaction.
Navigating Limitations
Despite the limitations of LLMs, you can mitigate the risks involved. The key ingredients to mitigating risks are
- awareness
- strategic use of Gen AI tools
Below are some resources to help. Also, be sure to check out this guide's sections on Prompt Patterns and Gen AI Upskilling, which will help you up your prompting game.
- Beware of BotshitThis article discusses the problem of "botshit" (inaccurate content generated by AI chatbots and uncritically used by humans) and how to avoid it through the use of a guiding framework.
- Nudge Users to Catch Generative AI ErrorsThis article discusses an experimental tool to highlight possible errors from LLMs so that businesses can help employees be more critical users of AI tools.
