Advanced Techniques
Advanced Techniques: 7 artificial-intelligence concepts explained in plain language, each with a short definition and a fuller explanation. Part of the Guide to AI knowledge base.
Guide to AI (ai-know.org) is a free, bilingual (English and Hebrew) knowledge base about artificial intelligence: explained concepts, curated tools, a daily news digest and a browser-side token calculator. It is a static site — no account, no tracking, no paywall.
Concepts in this section
Transfer Learning
Using a model trained on one task as a starting point for another task.
Transfer Learning is a technique where a model trained on one task is used as a starting point for a model on another, related task. This saves time and resources in training and is especially effective when there's little data for the new task. For example, a model trained to identify cats can be used as a base for a model that identifies dogs.
Few-shot Learning
The ability of a model to learn from a small number of examples.
Few-shot Learning refers to the ability of a model to learn a new task from a very small number of labeled examples. This is important in situations where it's difficult to obtain large amounts of labeled data. Few-shot Learning techniques include approaches such as Metric Learning and Meta-Learning.
RAG (Retrieval-Augmented Generation)
A technique that combines information retrieval with text generation to improve the accuracy and relevance of answers.
RAG combines an information retrieval mechanism with a large language model. When the model receives a query: 1. The retrieval mechanism finds relevant documents from a knowledge base. 2. The retrieved information is combined with the original query. 3. The model uses the combined information to generate an answer. This allows the model to provide more accurate and up-to-date answers, relying on external sources of information.
Prompt Engineering
Precise design of instructions for a model to obtain desired results.
Prompt Engineering is the art and science of designing input to a large language model to get the desired output. This includes precise phrasing of the question or instruction, providing relevant examples, and defining the desired format for the answer. Effective Prompt Engineering can significantly improve the performance of a model without the need for retraining.
Fine-tuning
Fine adjustment of an existing model for a specific task.
Fine-tuning is the process of taking a pre-trained model (e.g., BERT or GPT) and adapting it to a specific task through additional training on a smaller, more focused dataset. This allows large and complex models to be adapted to specific tasks efficiently, while preserving the general knowledge the model acquired in its original training.
Explainable AI (XAI)
Approaches to understanding and analyzing decisions of complex models.
Explainable AI focuses on developing methods and techniques for understanding and analyzing the decisions and predictions of complex AI models. XAI goals include: Transparency: Understanding how the model arrives at decisions. Trust: Building trust in AI systems by explaining their decisions. Regulatory Compliance: Meeting legal requirements for transparency in automated decision-making. Model Improvement: Identifying and addressing biases or issues in models. XAI techniques include SHAP (SHapley Additive exPlanations), LIME (Local Interpretable Model-agnostic Explanations), and heat maps for highlighting important areas in images.
AutoML (Automated Machine Learning)
Techniques for automating the process of model selection and hyperparameter tuning.
AutoML refers to a range of techniques and processes aimed at automating machine learning processes, including: Feature selection Model architecture selection Hyperparameter optimization Model evaluation The goal of AutoML is to make machine learning technologies accessible to developers and scientists without deep expertise in the field, and to accelerate the model development process.