Large Language Models and Key Metrics

AI, large language models    |    Beginner
  • 15 videos | 1h 38m
  • Earns a Badge
Large language models (LLMs) are revolutionizing the landscape of artificial intelligence by enhancing how machines understand and generate human language. Models like GPT and BERT can grasp context, meaning, and expression in ways never before possible. Use this course to deepen your understanding of evaluation metrics like HELM, BLEU, ROUGE, and F1 scores, and learn to distinguish between intrinsic and extrinsic evaluation methods. Explore practical examples of evaluating LLMs, reviewing both public and in-house models, and discover their pros and cons. You'll also learn key considerations for selecting an LLM, focusing on scalability and task-specific performance, and aligning evaluation metrics with business needs for effective model selection. Upon completion of this course, you will have a strong foundational understanding of large language models and the key metrics used to evaluate them.

WHAT YOU WILL LEARN

  • Discover the key concepts covered in this course
    Describe the architecture of large language models (llms)
    Define the key concepts of large language models (llms)
    Describe the core principles of large language models (llms)
    Identify the strengths and weaknesses of different llms, including in-house and public models
    Recognize key criteria to consider when selecting an llm, including computational resources, scalability, and task-specific performance
    Identify common llm evaluation metrics such as helm evaluation, eleuther’s test, perplexity, bleu score (bilingual evaluation understudy), rouge score (recall-oriented understudy for gisting evaluation), f1 score, accuracy, loss function, human evaluation, glue and superglue benchmarks, and latency
    Differentiate between intrinsic and extrinsic evaluation methods for llms
  • Outline how evaluation metrics align with different business use cases and criteria for model selection
    Explore a simple llm architecture and its language processing capabilities
    Identify the features, pros, and cons of a public large language model
    Identify the features, pros, and cons of an in-house large language model
    Evaluate a pre-trained llm using the common evaluation metrics
    Use intrinsic metrics (e.g., perplexity) and extrinsic metrics (e.g., task accuracy) using sample tasks
    Summarize the key concepts covered in this course

IN THIS COURSE

  • 1m 41s
    In this video, we will discover the key concepts covered in this course. FREE ACCESS
  • 6m 48s
    After completing this video, you will be able to describe the architecture of large language models (LLMs). FREE ACCESS
  • Locked
    3.  Large Language Model Concepts
    7m 14s
    Upon completion of this video, you will be able to define the key concepts of large language models (LLMs). FREE ACCESS
  • Locked
    4.  Large Language Model Principles
    7m 6s
    After completing this video, you will be able to describe the core principles of large language models (LLMs). FREE ACCESS
  • Locked
    5.  Large Language Model Types
    7m 46s
    Upon completion of this video, you will be able to identify the strengths and weaknesses of different LLMs, including in-house and public models. FREE ACCESS
  • Locked
    6.  Large Language Model Selection Criteria
    7m 3s
    After completing this video, you will be able to recognize key criteria to consider when selecting an LLM, including computational resources, scalability, and task-specific performance. FREE ACCESS
  • Locked
    7.  Large Language Model Metrics
    7m 18s
    In this video, we will identify common LLM evaluation metrics such as HELM evaluation, Eleuther’s test, Perplexity, BLEU Score (Bilingual Evaluation Understudy), ROUGE Score (Recall-Oriented Understudy for Gisting Evaluation), F1 Score, Accuracy, Loss Function, Human Evaluation, GLUE and SuperGLUE Benchmarks, and latency. FREE ACCESS
  • Locked
    8.  Large Language Model Evaluation Methods
    6m 36s
    After completing this video, you will be able to differentiate between intrinsic and extrinsic evaluation methods for LLMs. FREE ACCESS
  • Locked
    9.  Large Language Model Metrics Alignment
    6m 48s
    Upon completion of this video, you will be able to outline how evaluation metrics align with different business use cases and criteria for model selection. FREE ACCESS
  • Locked
    10.  Exploring Large Language Model Design
    9m 51s
    In this video, we will explore a simple LLM architecture and its language processing capabilities. FREE ACCESS
  • Locked
    11.  Public Large Language Models
    6m 14s
    Upon completion of this video, you will be able to identify the features, pros, and cons of a public large language model. FREE ACCESS
  • Locked
    12.  In-House Large Language Models
    7m 12s
    In this video, we will identify the features, pros, and cons of an in-house large language model. FREE ACCESS
  • Locked
    13.  Evaluating Large Language Models
    7m 54s
    In this video, discover how to evaluate a pre-trained LLM using the common evaluation metrics. FREE ACCESS
  • Locked
    14.  Using Large Language Model Intrinsic Metrics
    6m 50s
    In this video, discover how to use intrinsic metrics (e.g., perplexity) and extrinsic metrics (e.g., task accuracy) using sample tasks. FREE ACCESS
  • Locked
    15.  Course Summary
    1m 37s
    In this video, we will summarize the key concepts covered in this course. FREE ACCESS

EARN A DIGITAL BADGE WHEN YOU COMPLETE THIS COURSE

Skillsoft is providing you the opportunity to earn a digital badge upon successful completion on some of our courses, which can be shared on any social network or business platform.

Digital badges are yours to keep, forever.

YOU MIGHT ALSO LIKE

Course Systems Engineering Process: System Analysis & Modeling
Rating 5.0 of 2 users Rating 5.0 of 2 users (2)
Book Knowledge-augmented Methods for Natural Language Processing
Course Generative Modeling Foundations
Rating 4.0 of 1 users Rating 4.0 of 1 users (1)