LLM

A large language model is a type of Artificial Intelligence algorithm that applies neural network techniques with lots of parameters to process and understand human languages or text using self-supervised learning techniques. Tasks like text generation, machine translation, summary writing, image generation from texts, machine coding, chat-bots, or Conversational AI are applications of the Large Language Model. Examples of such LLM models are Chat GPT by open AI, BERT (Bidirectional Encoder Representations from Transformers) by Google, etc.

• Pricing calculator

Tool	Description
Langchain	Building applications with LLMs through composability
Rebuff	LLM Prompt Injection Detector
GPTCache	Semantic cache for LLMs
LLMSherpa	Developer APIs to Accelerate LLM Projects
OpenLLM	Operating LLMs in production
vLLM	Easy, fast, and cheap LLM serving for everyone
LLaVa	LLaVA: Large Language and Vision Assistant
SkyPilot	Run LLMs, AI, and Batch jobs on any cloud
Zep	Fast, scalable building blocks for production LLM apps
Lanarky	Framework to deploy LLM applications in production. Built on top of FastAPI
Text-generation-ui	Web UI for running Large Language
Stable-diffusion-web	Stable Diffusion web UI
Argilla	The open-source data curation platform for LLMs
Text-generation-inference	LLM Text Generation Inference
Peft	PEFT: State-of-the-art Parameter-Efficient Fine-Tuning
Haystack	LLM orchestration framework to build customizable, production-ready LLM applications
FlowiseAI	Drag & drop UI to build your customized LLM flow
Trulens	Evaluation and Tracking for LLM Experiments
DocArray	Pydantic for LLM (Represent, send, store and search multimodal data)
Jina	Build multimodal AI applications with cloud-native stack
Pezzo	Open-source, developer-first LLMOps platform designed
VDP	Integrate AI to process unstructured data in the modern data stack
Leon	Open-source personal assistant
Dify	LLM app development platform
PromptFlow	Build high-quality LLM apps
LangFlow	Langflow is a dynamic graph where each node is an executable unit
ChatbotUI	AI chat for every model
FastChat	An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
UltraChat	Large-scale, Informative, and Diverse Multi-round Chat Data
LiteLLM	Call all LLM APIs using the OpenAI format. (100+ LLMs)
AgentGPT	Assemble, configure, and deploy autonomous AI Agents in your browser
AutoGPT	Accessible AI for everyone, to use and to build on.
AutoGen	A programming framework for agentic AI
OpenWeb-UI	User-friendly WebUI for LLMs
Guidance	A guidance language for controlling LLM
OpenUI	Building UI components with AI
dsPy	The framework for programming—not prompting—foundation models
Semantic Kernel	Integrate cutting-edge LLM technology quickly and easily into your apps
LLMStack	No-code multi-agent framework to build LLM Agents
Vanna	Chat with your SQL database
Deepseek	Closed-Source Models in Code Intelligence
LocalAI	Open Source OpenAI alternative, Self-hosted
LLama-Factory	Unify Efficient Fine-Tuning
Lago	Metering and Usage Based Billing API, Consumption tracking
Lunary	The production toolkit for LLMs
OpenLLMetry	Open-source observability for your LLM application
Evidently	Evaluate and monitor ML models from validation to production
Promptify	Prompt Engineering and Prompt Versioning
Mem0	The memory layer for Personalized AI
Maestro	A framework for Claude Opus to intelligently orchestrate subagents.
Inspect	An open-source framework for large language model evaluations
Anything-llm	The all-in-one Desktop & Docker AI application with full RAG
Gorilla	An API store for LLMs
CrewAI	Framework for orchestrating role-playing, autonomous AI agents.
MLE-agent	Your intelligent companion for seamless AI engineering and research
Promptfoo	Test your prompts, agents, and RAGs
Langwatch	The ultimate LLM Ops platform - Monitoring, Analytics, Evaluations, Datasets and Prompt Optimization
Langfuse	Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets
LM-course	Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks
Chonkie	The no-nonsense RAG chunking library
Crawl4ai	Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper
Exo	Run your own AI cluster at home with everyday devices
Docling	Get your documents ready for gen AI
SuperPrompt	SuperPrompt is an attempt to engineer prompts that might help us understand AI agents
Scrapegraph-ai	Python scraper based on AI
Anything-llm	The all-in-one Desktop & Docker AI application with built-in RAG, AI agents, No-code agent builder, and more.
Data-formulator	Create rich visualizations with AI
FlashLearn	Integrate LLM in any pipeline - fit/predict pattern, JSON driven flows, and built in concurency support.
Pathway AI Pipelines	Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data.
pydantic-ai	Agent Framework / shim to use Pydantic with LLMs
lollms-webui	Lord of Large Language and Multi modal Systems Web
Phoenix	AI Observability & Evaluation
Opik	Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations
deepeval	The LLM Evaluation Framework
oMLX	Local LLM inference server optimized for Apple Silicon Macs with continuous batching and tiered KV caching managed via native menu bar application
Onyx	Open-source AI platform with RAG, web search, code execution, and custom AI agents all in a feature-rich self-hosted interface
Open Notebook	Privacy-focused alternative to Google Notebook LM for AI-powered research

Articles/Talks

• The Practical Guides for Large Language Models
• A Watermark for Large Language Models
• Accelerating Large Language Models with Mixed-Precision Techniques
• Understanding the Fundamental Limitations of Vector-Based Retrieval for Building LLM-Powered Chatbots— Part 1-3
• Advancing AGI for humanity
• Evaluating Large Language Model (LLM) systems - Metrics, challenges, and best practices
• LLM-PowerHouse: A Curated Guide for Large Language Models with Custom Training and Inferencing

RAG

RAG is a technique for augmenting LLM knowledge with additional data.

Vector Database (VDB)

• Do we really need a specialized vector database

Tokenization

Best LLM

• https://blog.abacus.ai/blog/2023/08/10/create-your-custom-chatgpt-pick-the-best-llm-that-works-for-you/
• https://huggingface.co/spaces/openlifescienceai/open_medical_llm_leaderboard
• https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard
• https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard
• https://huggingface.co/collections/open-llm-leaderboard/the-big-benchmarks-collection-64faca6335a7fc7d4ffe974a

Text embeddings

Text Embeddings give you the ability to turn unstructured text data into a structured form. With embeddings, you can compare two or more pieces of text, be it single words, sentences, paragraphs, or even longer documents. And since these are sets of numbers, the ways you can process and extract insights from them are limited only by your imagination.

• Text Embeddings Visually Explained

Embeddings on Multimodal Data

• Embed text, image and video in the same semantic space with the same dimensionality.
• Create the capability for join text with image/video for classification for example.

Nearest Neighbor

Nearest neighbor search (NNS), as a form of proximity search, is the optimization problem of finding the point in a given set that is closest (or most similar) to a given point. Closeness is typically expressed in terms of a dissimilarity function: the less similar the objects, the larger the function values.

ScaNN

• More efficient for a concrete corpus

Question Answering

• Hosted chatbot specifically focused on question answering

Models

• List of Open Sourced Fine-Tuned Large Language Models (LLM)
• Falcon

LLama

• LLama
• LlamaHub - Get your RAG application rolling in no time
• Lit-Llama

Transformers

Fine-Tuning

RLHF

Distillation - Train a small model to mimic the behavior of a large model Adapter Tunning Full Fine Tunning

Parameters

• Frequency_penalty: This parameter is used to discourage the model from repeating the same words or phrases too frequently within the generated text. It is a value that is added to the log-probability of a token each time it occurs in the generated text. A higher frequency_penalty value will result in the model being more conservative in its use of repeated tokens.
• Presence_penalty: This parameter is used to encourage the model to include a diverse range of tokens in the generated text. It is a value that is subtracted from the log-probability of a token each time it is generated. A higher presence_penalty value will result in the model being more likely to generate tokens that have not yet been included in the generated text.

Both of these parameters can be adjusted to influence the overall quality and diversity of the generated text.