Category: Top News

IBL News | New York

Microsoft introduced new agent capabilities with its new Copilot Studio platform this week.

These new agents—which will be available later this year—will allow users to orchestrate tasks and functions.

They can work like virtual employees, for example, monitoring email inboxes and automating data entry, which workers typically have to do manually. They are a kind of chatbot that intelligently performs complex tasks autonomously in a proactive way.

Developers can provide their copilot with a defined task and equip it with the necessary knowledge and actions to run business processes and associated tasks.

Microsoft has launched its Power Platform to orchestrate AI-drive business processes and automate tasks.

The new capabilities allow users to delegate authority to copilots to automate long-running business processes, reason over actions and user input, leverage memory, learn based on user feedback, record exception requests, and ask for help when encountering unfamiliar situations. Copilots can recall past conversations to add relevant context, following tight guardrails.

Here’s how Microsoft describes a potential Copilot for employee onboarding: “Imagine you’re a new hire. A proactive copilot greets you, reasoning over HR data answers your questions, introduces you to your buddy, gives you the training and deadlines, helps you with the forms, and sets up your first week of meetings. Now, HR and the employees can work on their regular tasks without the hassle of administration.”

“We think with Copilot and Copilot Studio, some tasks will be automated completely,” said Microsoft.

A featured example using this Microsoft Power Platform is the Canadian media company Cineplex.

 

IBL News | New York

Accenture (NYSE: ACN) announced it has completed the acquisition of the educational platform Udacity — which was announced on March 5, 2024.

Udacity is now part of Accenture’s technology learning platform and activity business LearnVantage, with over 230 professionals.

One of the main goals of the project is to bridge the gap between online education and the workforce through skill-driven training, with a focus on AI and tech.

IBL News | New York

The technical approach of RAG (Retrieval Augmented Generation) reduces the AI models’ hallucinations, but it doesn’t fully eliminate the problem with today’s transformer-based architectures, writes TechCrunch in an article.

However, a number of generative AI vendors suggest that their techniques result in zero hallucinations.

Given that generative AI models have no real intelligence and are simply predicting words, images, speech, music and other data, sometimes they get it wrong, telling lies.

To date, hallucinations are a big problem for businesses looking to integrate the technology into their operations.

Pioneered by data scientist Patrick Lewis, researcher at Meta and University College London, and lead author of the 2020 paper that coined the term, RAG retrieves documents relevant to a question using what’s essentially a keyword search and then asks the model to generate answers given this additional context.

It’s most effective in “knowledge-intensive” scenarios while getting trickier with “reasoning-intensive” tasks such as coding and math, as it’s hard to retrieve documents based on abstract concepts.

RAG also lets enterprises draw their private documents in a more secure and temporary way, avoiding being used to train a model to allow models.

Currently, there are many ongoing efforts to train models to make better use of RAG-retrieved documents.
.

IBL News | New York

OpenAI unveiled that it’s developing a tool that detects 98% of images generated by its text-to-image generator DALL-E 3 system. The success drops if the images are altered.

The tool, called Media Manager, will be in place by 2025. Currently, the company is working with creators, content owners, and regulators toward a standard.

Media Manager seems to be its response to growing criticism of the approach to developing AI that relies heavily on scraping publicly available data from the web.

“This will require cutting-edge machine learning research to build a first-ever tool of its kind to help us identify copyrighted text, images, audio, and video across multiple sources and reflect creator preferences,” OpenAI wrote in a blog post.

Recently, eight U.S. newspapers, including the Chicago Tribune, sued OpenAI for IP infringement, accusing OpenAI of pilfering articles for training generative AI models that it then commercialized without compensating or crediting the source publications.

OpenAI last year allowed artists to opt out of and remove their work from the data sets that the company uses to train its image-generating models.

The company also lets website owners indicate via the robots.txt standard, which gives instructions about websites to web-crawling bots. OpenAI continues to ink licensing deals with large content owners, including news organizations, stock media libraries, and Q&A sites like Stack Overflow. Some content creators say OpenAI hasn’t gone far enough, however.

A number of third parties have built opt-out tools for generative AI. Startup Spawning AI, whose partners include Stability AI and Hugging Face, offers an app that identifies and tracks bots’ IP addresses to block scraping attempts. Steg.AI and Imatag help creators establish ownership of their images by applying watermarks imperceptible to the human eye. Nightshade, a project from the University of Chicago, poisons image data to render it useless or disruptive to AI model training.
.

IBL News | New York

[Update: August 2024]

The University of California San Diego (UC San Diego) TritonGPT chatbot was upgraded to Llama 3 along with the institution’s vLLM, its CIO announced.

Currently, 17,105 faculty and staff have access to TritonGPT. By the end of May, it will add another 19,502 people for 36,607 with access.

As a platform, TritonGPT uses open-source software and runs on-premise, at low cost, in partnership with the San Diego Supercomputer Center.

TritonGPT, a suite of AI Assistants, handles university-specific questions, creates tailored content, and summarizes documents. It helps students navigate UC San Diego’s policies, procedures, and campus life.

“It’s like having a personal assistant who knows a lot about UC San Diego,” said the university.

Tasks that can be accomplished are, according to the university:

Ask UC San Diego Related Questions: Pose questions like “What is the policy on employee travel reimbursement?” or “What are some good restaurants on campus?” TritonGPT will provide detailed and relevant information.

Content Generation: Need help with content creation? Try commands like “Generate an outline for a presentation slide deck based on <insert topic>” or “Produce an email to thank my employees for <insert what you are thankful for>.”

Document Summarization: Copy and paste documents or articles related to UC San Diego, then ask TritonGPT to summarize the content. It’s a time-saving feature for extracting key information.

Content Editing: Utilize TritonGPT for editing and refining content related to UC San Diego. It’s a valuable tool for polishing emails, reports, or any written material.

Seek feedback and suggestions: TritonGPT can provide feedback and suggestions to help you improve your work processes and procedures. You can ask questions like “What are some ways I can improve my communication skills in the workplace?” or “Are there any suggestions for streamlining our team’s workflow?”

Ask for Recommendations: Seeking recommendations for UC San Diego events, study spots, or local hangouts? TritonGPT has you covered.

TritonGPT consists of the following AI Assistants:

1. UC San Diego Assistant: UC San Diego-related policy, process, and help documentation is spread out over various websites. The UC San Diego Assistant brings it all together by answering your questions directly. It is also great for incorporating UC San Diego’s context in generating new content and brainstorming ideas. Always reference the sources cited when relying upon their answers.
2. Job Description Helper: TritonGPT includes a Job Description Helper that will streamline the job description creation process for hiring managers. Leveraging over 1,300 career tracks job standard templates, it uses a predefined flow that engages hiring managers in a dialogue, capturing the job’s specific requirements. The AI then crafts language that not only complies with established job card standards but also accurately reflects the unique characteristics of the position. This feature reduces the time and effort involved in drafting job descriptions, ensuring they are both precise and tailored to the individual needs of the role.
3. General AI Assistant: This tool expands beyond UC San Diego’s scope, accommodating larger information exchanges. It interacts with a Large Language Model for tasks like document summarization, idea generation, and creating various content such as emails and reports.
4. Fund Manager Coach: Recognizing the crucial role of Fund Managers in overseeing grants and managing departmental finances, this assistant will enhance understanding of UC San Diego’s financial policies and procedures. Fund Manager Coach is trained in the documentation for developing research proposal budgets, advising faculty on contract and grant guidelines, reviewing and approving financial transactions, managing payroll, and ensuring that applicable guidelines are being followed during contract and grant spending.

TritonGPT’s UC San Diego has been trained on extensive public-facing university information, such as:

• Academic Personnel website
• Admissions website
• Blink
• Business Analytics Hub
• Calendar of Events
• Career Center
• Chancellor website
• The Commons
• Course Catalog
• Educational Technology
• Foundation
• Housing and Dining
• Policies (UC San Diego and UCOP)
• ServiceNow Knowledge Base content (public facing)
• Strategic Plan
• Student Financial Solutions
• Transportation
• TritonLink (students.ucsd.edu)
• UC Path website
• UC San Diego Brand
• University Centers
• University Communications
• UC San Diego Today

UC San Diego also has partnered with a UC San Diego and a Y Combinator-funded startup, DanswerAI, to handle the TritonGPT user interface and the under-the-hood RAG management.

Brett Pollak is leading this initiative for the university.

.

[Disclosure: IBL.ai is a partner provider of UC San Diego]

IBL News | New York

The new OpenAI’s GPT-4o app that responds with voice-to-speech commands, images, and video will be available this Monday free of charge for smartphones and desktop computers.

This significant development transforms ChatGPT into a fast, conversational voice assistant with natural dialogue.

Experts see this tool as another setback for Google Assistant, Apple Siri, and Amazon Alexa.

The San Francisco artificial intelligence start-up unveiled GPT-4o this Monday.

Videos showcasing its capabilities were posted on OpenAI’s website.

“We are looking at the future of the interaction between ourselves and machines,” said Mira Murati, OpenAI’s CTO.

OpenAI said it would gradually share the technology with users “over the coming weeks.” This is the first time it has offered ChatGPT as a desktop application.

The new app—which researchers call “multimodal AI”—cannot generate video. But it can still create images that represent the frames of a video.

IBL News | New York

Yesterday, at the Google I/O developer conference, Google unveiled LearnLM, a new family of generative AI models fine-tuned “to make teaching and learning experiences more active, personal and engaging.”

LearnLM is already powering features across Google products, including YouTube, Google’s Gemini apps, Google Search and Google Classroom.

Google also plans to partner with organizations, including Columbia Teachers College, Arizona State University, NYU Tisch, and Khan Academy to see how LearnLM can be extended beyond its own products.

In addition, new to YouTube are AI-generated quizzes to academic videos. This new conversational AI tool allows users to figuratively raise their hands when watching longer educational videos, such as lectures or seminars.

Users can ask the AI to summarize the video or tell them why it’s important. If they want to test their knowledge, they can ask the AI to quiz on the topic. The AI will them give then a multiple-choice question.

These new features are rolling out to select Android users in the U.S.

Google announced it’s adding an open-source framework called Firebase Genkit, that enables developers to build AI-powered applications in JavaScript/TypeScript with Go for content generation, summarization, text translation, and images creation.

Google Keynote (Google I/O ‘24)

More relevant announcements at the Google I/O developer conference:

• Google will be adding a new 27-billion-parameter model to the Gemma 2 model, launching in June.

• Gmail is getting an AI-powered upgrade. From a sidebar in Gmail, users will be able to search, summarize, analyze attachments, like PDFs, and draft their emails using its Gemini AI technology. They will also be able to take action on emails for more complex tasks, like helping users organize receipts or process an e-commerce return by searching their inbox, finding the receipt, and filling out an online form.

• Gemini can now analyze longer documents, codebases, videos, and audio recordings than before. The new Gemini 1.5 Pro can take in up to 2 million tokens, that is, double the next-largest model, Anthropic’s Claude 3, tops out at 1 million tokens.

Two million tokens are equivalent to around 1.4 million words, two hours of video, or 22 hours of audio.

• Google is also building Gemini Nano, the smallest of its AI models, directly into the Chrome desktop client, starting with Chrome 126. This, the company says, will enable developers to use the on-device model to power their own AI features.

• Google announced Imagen 3, the latest in the tech giant’s Image generative AI model family.

• Targeting OpenAI’s Sora, Google unveiled Veo, an experimental AI model that can create minute-long 1080p video clips after a text prompt. For now, Veo remains behind a waitlist on Google Labs, the company’s portal for experimental tech, for the foreseeable future.
.

IBL News | New York

OpenAI debuted yesterday a new generative AI model called GTP-4o, characterized by the ability to handle text, voice, and video, reasoning across (the “o” stands for “omni.”)

“This is incredibly important because we’re looking at the future of interaction between ourselves and machines,” said OpenAI CTO, Mira Murati during a streamed presentation at OpenAI’s offices in San Francisco on Monday. “This is the first time that we are really making a huge step forward when it comes to the ease of use.”

OpenAI trained its most advanced GPT-4 Turbo model to accomplish tasks like extracting text from images or even describing the content of those images, and GPT-4o has added speech to the mix.

• GPT-4o supercharges the voice model of ChatGPT, transforming the chatbot into an advanced assistant.

• Users can ask a question and interrupt the model while it’s answering, enjoying “real time” responsiveness.

• It can even respond generating voices in a range of different emotive styles, including singing.

GPT-4o is more multilingual as well, with enhanced performance in around 50 languages.

These features will evolve further in the future, Murati stated.

The San Francisco–based research lab made GPT-4o available in the free tier of ChatGPT with a rate limit starting yesterday.

Another important announcement was that the GPT Store‘a third-party powered chatbots — which were paywalled to date — are now available to users of ChatGPT’s free tier.

In addition, free users can take advantage of ChatGPT‘s paid memory capability feature — which allows ChatGPT to “remember” preferences for future interactions, file and photo uploading, and web searches for answers to timely questions.

In related news, OpenAI announced that it was releasing yesterday a refreshed ChatGPT UI on the web with a new, “more conversational” home screen and message layout, along with a desktop version of ChatGPT for macOS that lets users ask questions via a keyboard shortcut or take and discuss screenshots.

ChatGPT Plus users will get access to the app first, and a Windows version will arrive later in the year.
.

OpenAI
• Hello GPT-4o
• Introducing GPT-4o and more tools to ChatGPT free users

IBL News | New York

The Foreign Ministry of Ukraine unveiled this week an AI-generated spokesperson who will read official statements.

Dressed in a dark suit, this digital person introduced herself on social media sites as Victoria Shi.

The figure moved her head as she spoke while gesticulating with her hands.

Her appearance and voice were modeled on a singer and former contestant on Ukraine’s version of the reality show The Bachelor.

The statements were written by real people.

Dmytro Kuleba, the Ukrainian foreign minister, said that the main reason for creating her was “saving time and resources for diplomats.”

Shi’s creators are a team called the Game Changers, who have also made virtual-reality content related to the war in Ukraine.

To avoid fakes, the statements will be accompanied by a QR code linking them to text versions on the ministry’s website.
.

IBL News | New York

OpenAI shared a first draft of Model Spec, a new document used to design the model behavior of ChatGPT.

Model behavior is the way that models respond to input from users, encompassing tone, personality, and response length.

This disclosure offers a limited look at the reasoning behind OpenAI’s models’ rules of engagement, whether it’s sticking to brand guidelines or declining to make NSFW content.

The document reflects how The San Francisco–based research lab evaluates tradeoffs when conflicts arise.

The San Francisco–based research lab said that it’s committed to informing on how their AI models behave.