IBL News | New York
NVIDIA started this month to provide free access to its containerized AI NIM microservices for development, testing, and research to over 5 million program members.
The latest downloadable NIM microservices include open-sourced Meta’s Llama 3.1 8B and Mistral AI’s compact Mistral 7B Instruct.
Developer program members can use NIM microservices on up to two nodes or 16 GPUs. Organizations must sign up for a free 90-day NVIDIA AI Enterprise license to use NIM in production.
The ability to use simple APIs to integrate pre-trained LLMs into products has significantly increased developer usage of endpoints and application development frameworks. NVIDIA NIM enables developers to deploy their AI model endpoints rapidly.
In the NVIDIA API Catalog, these developers select a microservice, download a NIM microservice, and get an API key for the container.
Nvidia’s NIM aims to create an ecosystem of AI-ready containers. It includes support for models from NVIDIA, A121, Adept, Cohere, Getty Images, and Shutterstock, as well as open models from Google, Hugging Face, Meta, Microsoft, Mistral AI, and Stability AI.
Nvidia is already working with Amazon, Google, and Microsoft to make these NIM microservices available on SageMaker, Kubernetes Engine, and Azure AI. They’ll also be integrated into frameworks like Deepset, LangChain, and LlamaIndex.
The company plans to add additional capabilities, such as making the Nvidia RAG LLM operator available as an NIM.
[Disclosure: ibl.ai, the parent company of iblnews.org, has NVIDIA as a client]