IBL News | New York

AI spending surged to $13.8 billion in 2024 from $2.3 billion in 2023 as enterprises embed AI at the core of their business strategies and daily work, according to a study conducted by Menlo Ventures.

This research, titled “2024 State of Generative AI in the Enterprise Report,” done after surveying 600 U.S. enterprise IT decision-makers, points out that we are still in the early stages of a large-scale transformation.

This spending will continue: 72% of decision-makers anticipate broader adoption of generative AI tools soon.

Investments in the LLM foundation model still dominate spending, but the application layer segment to optimize workflows is now growing faster.

These app layer companies—mostly in highly verticalized sectors—leverage LLM’s capabilities across domains to unlock new efficiencies. Enterprise buyers will invest $4.6 billion in generative AI applications in 2024, an 8x increase from the $600 million invested in 2023.

The use cases that deliver the most ROI through enhanced productivity or operational efficiency are:

• Code copilots, such as GitHub Copilot, Cursor, Codeium, Harness, and All Hands.

• Support knowledge-based chatbots for employees, customers, and contact centers. Aisera, Decagon, Sierra, and Observe AI are some of the examples.

• Enterprise search, retrieval, data extraction, and transformation to unlock the knowledge hidden within data silos. Solutions like Glean and Sana connect to emails, messengers, and document stores, enabling unified semantic search across systems.

• Meeting summarization to automate note-taking and takeaways. Examples are Fireflies.ai, Otter.ai, Fathom, and Eleos Health.

AI-powered autonomous agents capable of managing complex, end-to-end workflow processes are emerging and can transform human-led industries. Forge, Sema4, and Clay are some tools.

When deciding to build or buy, 47% of solutions are developed in-house, while 53% are sourced from vendors. Often, organizations discover too late that they have underestimated the difficulty of technical integration, scalability, and ongoing support.

Most customers (64%) prefer buying from established vendors, citing trust.

The leading vertical AI applications are:

• Healthcare, with examples like Abridge, Ambience, Heidi, Eleos Health, Notable, SmarterDx, Codametrix, Adonis, and Rivet.

• Legal, with examples like Everlaw, Harvey, Spellbook, EvenUp, Garden, Manifest, and Eve.

• Financial Services, with examples like Numeric, Klarity, Arkifi, Rogo, Arch, Orby, Sema4, Greenlite, and Norm AI.

• Media and entertainment, with examples like Runway, Captions, Descript, Black Forest Labs, Higgsfield, Ideogram, Midjourney, and Pika.

Rather than relying on a single provider, enterprises have adopted a multi-model approach, typically deploying three or more LLM in their AI stacks, routing to different models depending on the use case or results.

To date, close-source solutions underpin the vast majority of usage, with Meta’s Llama 3 holding at 19%, according to the Menlo Ventures research.

Regarding architectures for building efficient and scalable AI systems, RAG (retrieval-augmented generation) dominates with 51% adoption, while fine-tuning of production molded is only 9%. Agentic architectures, which debuted this year, power 12% of implementations.

Databases and data pipelines are needed to power RAG. Traditional databases like Postgres and MongoDB remain common, while AI-native vector databases like Pinecone gain ground.

Menlo Ventures made three predictions for what lies ahead:

1. Agentic automation will drive the next wave of transformation, tackling complex, multi-step tasks beyond the current systems of content generation and knowledge retrieval. Examples are platforms like Clay and Forge

2. More incumbents will fall. Chegg saw 85% of its market cap vanish, while Stack Overflow’s web traffic halved. IT outsourcing firms like Cognizant, legacy automation players like UiPath, and even software giants like Salesforce and Autodesk will face AI-native challengers.

3. The AI talent drought will intensify. AI-skilled enterprise architects will notably increase their salaries. 

Squint, Typeface…

IBL News | New York

NVIDIA released eight free AI courses this month. Five are hosted at NVIDIA’s Deep Learning Institute (DLI) platform, two on Coursera, and one on YouTube.

1. Generative AI Explained
2. Building A Brain in 10 Minutes
3. Augment your LLM with Retrieval Augmented Generation
4. AI in the Data Center
5. Accelerate Data Science Workflows with Zero Code Changes
6. Mastering Recommender Systems
7. Networking Introduction
8. Building RAG Agents with LLMs

1. Generative AI Explained

Learn learn how to:

• Define Generative AI and explain how Generative AI works
• Describe various Generative AI applications
• Explain the challenges and opportunities in Generative AIhttps://t.co/JTNyqVZY5v pic.twitter.com/7UC7Ac3IWI

— Roni Rahman (@heyronir) March 24, 2024

2. Getting Started with AI on Jetson Nano

Learn how to:

• Set up your Jetson Nano and camera
• Collect image data for classification models
• Annotate image data for regression models
• Train a neural network on your data to create your modelshttps://t.co/LHFbgLIPNQ pic.twitter.com/pjTLnKWyzS

— Roni Rahman (@heyronir) March 24, 2024

3. Building A Brain in 10 Minutes

You’ll learn:

• How neural networks use data to learn
• Understand the math behind a neuronhttps://t.co/E9VEKRKWX1 pic.twitter.com/8VPBXseT15

— Roni Rahman (@heyronir) March 24, 2024

4. Building Video AI Applications on Jetson Nano:

Learn to:
• Create DeepStream pipelines for video processing
• Handle multiple video streams
• Use alternate inference engines like YOLOhttps://t.co/RuBQDoXMOU pic.twitter.com/PAxhAtfVVm

— Roni Rahman (@heyronir) March 24, 2024

5. Augment your LLM Using RAG

• Understand the basics of RAG
• Learn about the RAG retreival process
• Discover NVIDIA AI Foundations and the key components of a RAG model.https://t.co/cG0AbeyxS5 pic.twitter.com/GqjrLjDuuM

— Roni Rahman (@heyronir) March 24, 2024

6. Building RAG Agents with LLMs

Learning Objectives:

• Explore scalable deployment strategies
• Learn about microservices and development
• Experiment with LangChain paradigms for dialog management
• Practice with state-of-the-art modelshttps://t.co/eIqjaBgIU2 pic.twitter.com/MeKd2RGU4r

— Roni Rahman (@heyronir) March 24, 2024

7. Accelerate Data Science Workflows with Zero Code Changes

In this course you will:

• Learn the benefits of unified CPU and GPU workflows
• GPU-accelerate data processing and machine learning
• See faster processing times with GPUhttps://t.co/UJ9RJkJTWh pic.twitter.com/XoQ1goBncB

— Roni Rahman (@heyronir) March 24, 2024

8. Introduction to AI in the Data Center

Learn about AI, machine learning, deep learning, GPU architecture, deep learning frameworks, and deploying AI workloads.

Understand requirements for multi-system AI clusters and infrastructure planning.https://t.co/u96QGB6rVG pic.twitter.com/yLlUWv1e9N

— Roni Rahman (@heyronir) March 24, 2024

[Disclosure: IBL works for NVIDIA by powering its learning platform]

IBL News | New York

After the release of the bot ChatGPT a year ago, the second phase of personalized, autonomous AI agents is emerging.

These agents can perform complex tasks, such as sending emails, scheduling meetings, booking flights or tables in a restaurant, or even complex tasks like buying presents for family members or negotiating a raise.

Personalized chatbots, programmed for specific tasks, that GPT creators will be able to release through the upcoming OpenAI’s GPT Store, are a prelude.

For now, these custom GPTs are easy to build without knowing how to code.

Users just answer a few simple questions about their bot — its name, its purpose, the tone used to respond — and the bot builds itself in just a few seconds. Users can upload PDF documents they want to use as reference material or easily look up Q&A. They can also connect the bot to other apps or edit its instructions.

Although these custom chatbots are far from working perfectly, they can be useful tools for answering repetitive questions in customer service departments.

Some AI safety researchers fear that giving bots more autonomy could lead to disaster, The New York Times reported. The Center for AI Safety, a nonprofit research organization, listed autonomous agents as one of its “catastrophic AI risks” this year, saying that “malicious actors could intentionally create rogue AI with dangerous goals.”

For now, these agents look harmless and limited in their scope.

Its development seems to be dependent on gradual iterative deployment, that is, small improvements at a fast pace rather than a big leap.

In the last OpenAI developer conference, Sam Atman built on stage a “start-up mentor” chatbot to give advice to aspiring founders, based on an uploaded file of a speech he had given years earlier.

The San Francisco-based research lab envisions a world where AI agents will be extensions of us, gathering information and taking action on our behalf.
.

This is really worth your time – a very solid technical introduction to LLMs, great if you’ve not been paying close attention but I picked up quite a few useful details from it too https://t.co/eOfOyWSxC5

— Simon Willison (@simonw) November 23, 2023

The most clearest and crisp explanation, I've ever heard, of how large language models compress and capture a "world-model" in their weights simply by learning to predict the next word accurately.

Furthermore, how the raw power of these base models can then be tamed by teaching… pic.twitter.com/0g7Z5wXOlc

— Zain Hasan (@ZainHasan6) November 21, 2023

IBL News | New York

The output generated by ChatGPT and other LLMs presents legal and compliance risks that every organization has to face or face dire consequences, according to the consultancy firm Gartner, Inc, which has identified six areas.

“Failure to do so could expose enterprises to legal, reputational, and financial consequences,” said Ron Friedmann, Senior Director Analyst at Gartner Legal & Compliance Practice.

• Risk 1: Fabricated and Inaccurate Answers

ChatGPT is also prone to ‘hallucinations,’ including fabricated answers that are wrong, and nonexistent legal or scientific citations,” said Friedmann.

Only accurate training of the robot with limited sources will mitigate this tendency to provide incorrect information.

•  Risk 2. Data Privacy and Confidentiality

Sensitive, proprietary, or confidential information used in prompts may become a part of its training dataset and incorporated into responses for users outside the enterprise if chat history is not disabled,

“Legal and compliance need to establish a compliance framework and clearly prohibit entering sensitive organizational or personal data into public LLM tools,” said Friedmann.

• Risk 3. Model and Output Bias

“Complete elimination of bias is likely impossible, but legal and compliance need to stay on top of laws governing AI bias and make sure their guidance is compliant,” said Friedmann.

“This may involve working with subject matter experts to ensure output is reliable and with audit and technology functions to set data quality controls,” he added.

• Risk 4.  Intellectual Property (IP) and Copyright risks

As ChatGPT is trained on a large amount of internet data that likely includes copyrighted material, its outputs – which do not offer source references – have the potential to violate copyright or IP protection.

“Legal and compliance leaders should keep a keen eye on any changes to copyright law that apply to ChatGPT output and require users to scrutinize any output they generate to ensure it doesn’t infringe on copyright or IP rights.”

• Risk 5. Cyber Fraud Risks

Bad actors are already using ChatGPT to generate false information at scale, like fake reviews, for instance.

Moreover, applications that use LLM models, including ChatGPT, are also susceptible to prompt injection, a hacking technique in which

A hacking technique known as “prompt injection” brings criminals to write malware codes or develop phishing sites that resemble well-known sites.

“Legal and compliance leaders should coordinate with owners of cyber risks to explore whether or when to issue memos to company cybersecurity personnel on this issue,” said Friedmann.

• Risk 6. Consumer Protection Risks

Businesses that fail to disclose that they are using ChatGPT as a customer support chatbot run the risk of being charged with unfair practices under various laws and face the risk of losing their customers’ trust.

For instance, the California chatbot law mandates that in certain consumer interactions, organizations must disclose that a consumer is communicating with a bot.

Legal and compliance leaders need to ensure their organization’s use complies with regulations and laws.
.

IBL News | New York

When choosing and orchestrating an LLM, there are many critical technical factors, such as training data, dataset filtering, fine-tuning process, capabilities, latency, technical requirements, and price.

Experts state that implementing an LLM API, like GPT-4 or others, is not the only option.