DeepLearning.AI

Sung Kim

Lucy Park

This course provides a comprehensive understanding of pretraining large language models. You'll learn to prepare data, configure models, and assess performance, with a focus on cost-effective strategies. The course covers creating high-quality datasets, using the Hugging Face library, and evaluating model performance with benchmark tasks.

Pretraining LLMs

An overview of the course and its objectives.

Introduction

Understanding the importance and benefits of pretraining large language models.

Why Pre-training

Learn how to create and clean datasets for effective model pretraining.

Data Preparation

Techniques for preparing your training data for use with the Hugging Face library.

Packaging Data for Pretraining

Explore various options for configuring and initializing models for training.

Model Initialization

Execute a training run and learn to train your own model.

Training in Action

Assess your trained model’s performance and explore common evaluation strategies.

Evaluation

Summarize the key learnings and outcomes of the course.

Conclusion

Test your understanding of the course material.

Quiz

In this course, you'll gain in-depth knowledge of pretraining large language models, from data preparation to model configuration and performance assessment. Learn innovative techniques to reduce training costs and explore various model configurations.

Individuals interested in learning about the process of pretraining large language models.

AI Enthusiasts

Professionals looking to expand their knowledge on model pretraining and performance evaluation.

Data Scientists

Engineers aiming to understand the complete process of pretraining LLMs for better model performance.

Machine Learning Engineers

Artificial Intelligence

Business

Cloud Engineering

Cybersecurity

Data

Design

Devops

Finance

Green Skills

Languages

AI Foundations

AI for Business

AI Automation & Workflows

Agentic AI

AI Coding / Vibe Coding

Agile and Scrum Methodologies

Agile and Scrum Practices 

AI Automation & Orchestration

AI for Data

AI for Design

Access 3,000+ courses tailored to your team's needs, covering AI, Cloud, Cybersecurity, Marketing, Languages, and more.

Mydra's AI Career CoPilot guides you with personalized learning paths tailored to your career goals, education, and experience.

We're more than an upskilling platform — with Mydra, you gain access to:

Mydra makes skill development accessible for your team with flexible payment solutions to fit any budget. Explore our options below.

The Benefits of Choosing Mydra

By canceling, you’ll lose access to all course materials and benefits. If you still wish to proceed, you can confirm below.

Pay now, in full with your personal card.

Credit & Debit Card

Based on your location, these are the available payment options. Click the flag to switch your location.

Which country are you from?

FUNDAE helps you grow professionally with employer-supported training and financial aid, fostering a culture of learning and innovation.

We'll help you design, finance, and manage the ideal training plan for you, maximizing FUNDAE benefits hassle-free.

This programme is eligible for Fundae funding. Your employer may be able to pay for this and reclaim the cost.

You can pause your payments if your monthly income is below {formattedMinThreshold}.

The remaining {remainingPayments} payments scheduled each month based on your income x {percentage}

As an employee, you can reduce your personal taxable income through eligible education deductions, spread over {numberOfInstalments} monthly payments. This combines the tax benefits with more manageable monthly payments from your gross income. To qualify, your employer must offer support for 'Retribución Flexible' and approve your course.

Based on the Retribución Flexible tax savings calculator, you could save {savings} on income tax. This estimate is based on a minimum annual gross salary of {minSalary}

The Tax Savings benefit is received through payroll as a reduction in tax deductions.

As an employee, you can reduce your personal taxable income through eligible education deductions. These tax benefits can be combined with any payment method and may result in significant savings on your annual tax return. However, your employer must offer support for 'Retribución Flexible' and approve your course.

Looks like your {availableLearningBudget} budget is a bit short—you're missing {requiredRemainingAmount} for that {courseCost} course!

Please tell us the date your salary is normally paid to your account so we can schedule your FixPay collection - this will normally be a few days later, but we'll confirm the exact date with you.

We have emailed you a copy for your records. We are now finalising the details and your financial product details will appear here soon.

Thanks for signing the contract!

{classTypes} classes

Once you confirm your course selection, you will receive an email containing the course URL, which will take you directly to the {epName}'s page.

Access your tax benefits or use your employer learning budget to bundle with your payment method.

If you can't find the answer you're looking for, chat with your Mydra advisor!

Frequently Asked Questions

We are currently reviewing the validation of your tax benefits. We will contact you as soon as we have more information.

Split the payments in monthly instalments, or combine it with tax credits or learning budget. To learn more click

Pay your course in full, or combine your upfront payment with tax credits or your employee learning budget. To learn more, click

Don’t know what to learn? We’ll help you discover your next step for success.

Generate your AI-learning path

Learn on your terms. Filter courses based on your skill needs, budget and time.

Tailor your search

Find your course, pay smooth, learn like a pro

Courses Marketplace | Mydra

Name

Please tell us if you have ever received any support from the European Investment Fund

Brief description (tell us about your organization in a few words)

Please add the provider's authorized legal representative to handle updates to the team's legal contract. Without this contact information, the contract cannot be signed, which will delay the sale of the courses. Providing these details promptly will ensure a smooth and timely process.

Please complete this information if you would like to set up the provider's account now, or you can do it later.

A Legal Representative of an Education Provider is authorized to act on the institution's behalf in legal, administrative, and contractual matters. They ensure regulatory compliance, manage agreements, represent the provider in official matters, handle critical decisions, and serve as the primary contact for partnerships and negotiations.

Your request will be forwarded to our partners, and we will ensure the price is made available to you promptly. You will receive an email notification once the price is available. Thank you for your request. Feel free to explore other courses for more exciting learning opportunities here.

Would you like to request the price for this cohort?

Leave it with us - we'll be back in contact as soon as possible

Would you like to find out what dates this course is available?

We will analyze your request, and once these features are available, you will receive an email confirmation. If you want to know more about the benefits of these features, please visit our <Link>Payment Options</Link> page.

Would you like to activate the {paymentFeature, select, learningBudget {Learning Budget} taxBenefits {Tax Benefits} other {this}} feature for this course?

Many companies set aside funds for employee training—let’s check if yours can cover this course!Please provide the contact details of the person in your company who manages L&D budgets. This is usually someone in HR, People, or Learning & Development, but if you're unsure, you can add your manager instead.

<title>What Happens Next?</title> <ul><li>✅ We contact your employer to confirm if they can fund this course for you.</li><li>✅ Once approved, we handle the payment process directly between your employer and the education provider.</li><li>✅ We keep you updated throughout the process.</li><li>📢 Tip: It’s helpful to flag this internally so they know to expect our request—it could speed things up!</li></ul>

Use Your Employer’s Learning & Development Budget for This Course

We will analyze your request, and once this payment option is available, you will receive an email confirmation. If you want to know more about the benefits of this payment option, please visit our <Link>Payment Options</Link> page.

Would you like to request {paymentMethod, select, upfront {Upfront} fix_pay {FixPay} other {}} as your payment option for this course?

We will analyze your request, and once this payment option is available, you will receive an email confirmation.

Please note that your company’s validation is required to use FUNDAE as a payment option. If your company has available FUNDAE credits, they can be used to cover the cost of eligible courses, allowing you to upskill at no additional expense to you.

Would you like to request FUNDAE as your payment option?

Payment Options

Please list the total amount you spend each month on the following three items. If you share your bills with someone else, just put your share.

If you’re already using Retribución Flexible, please input how much you’ve spent this year per category:

While the course fee remains the same, you can use your tax credits to reduce your overall yearly expenses.

Income Share Agreements (ISAs) offer a flexible and affordable solution for professional development. Employees can defer payments until they earn a stable income, reducing financial pressure while advancing their skills. For companies, ISAs ensure a highly skilled workforce without upfront costs, aligning employee growth with business success.

<Chip>{blockedLearningBudget}</Chip> will be deducted once your request is accepted.

As the world evolves, so should you. Upskill to stay ahead.

Check the details of your learning budget.

Join the Mydra community today!

FUNDAE is a crucial resource for companies looking to invest in their employees’ professional growth while benefiting from financial support. It plays an essential role in fostering a culture of learning and innovation in Spain. <ul><li>Reduces the financial burden of employee training.</li><li>Promotes skill development tailored to company needs.</li><li>Enhances competitiveness by upskilling the workforce.</li><li>Increases access to continuous professional development.</li><li>Improves employability and career prospects.</li></ul>

Pay using FUNDAE benefits

Pay via Income Share Agreement

Enhance the value of your training investments. Empower your employees to reach their full potential while aligning professional development with your business goals. Mydra helps you allocate your learning budget efficiently, track progress, and achieve measurable results—ensuring every dollar spent drives growth and success.<ul><li>Take advantage of your company's learning budget</li><li>Combine it with tax benefits or FixPay for maximum savings.</li></ul>

Mydra is dedicated to making skill development accessible for all your employees, seamlessly fitting into any budget. With flexible payment solutions tailored to your financial needs, we maximize learning opportunities for your team. Click below to explore our payment options and their benefits.

At Mydra, we facilitate access to and management of FUNDAE training subsidies for employees, allowing companies to make the most of the funds allocated for professional development.<ul><li>As an organizing company, Mydra handle all administrative and regulatory processes, ensuring compliance with legal requirements, optimizing the use of available training credits, and reducing the operational burden for educational institutions.</li><li>Mydra improves the accessibility of training programs while also increasing demand from companies looking to maximize the return on their investment in workforce development.</li></ul>

We are a FUNDAE organizing company

Mydra offers accessible financing for everyone by covering tuition costs upfront, ensuring that financial barriers don’t prevent learners from pursuing their education. Students only begin making payments after they complete their course, allowing them to focus entirely on their studies. Additionally, Mydra's Income Share Agreements (ISAs) provide exceptional downside protection—payments are paused anytime a learner’s income falls below the minimum threshold of €1,333.33, offering peace of mind and financial security.

Income Share Agreements. Removing financial friction

As a user, you will maximize the value of your training. Reach your full potential while advancing your professional development. Mydra helps users efficiently allocate the learning budget provided by some companies to their employees, track progress, and achieve measurable results, ensuring that every euro invested drives growth and success.<ul><li>Take advantage of your company's learning budget</li><li>Combine it with tax benefits or FixPay for maximum savings.</li></ul>

FixPay lets employees spread their learning costs over time with 0% interest, making professional growth more manageable.How to apply a FixPay:<ul><li>Select a course and 'Pay in installments - FixPay' </li><li>Provide basic information (name, address, salary).</li><li>Identity verification and recent payslip upload.</li><li>Sign a financial agreement.</li></ul>

FixPay

Pay in 0% interest instalments with FixPay

FUNDAE

Pay only when you have an income with ISA

Learning budget

Leverage your Employer Learning Budget

Maximize your training investments with government grants and flexible compensation options. Employees can save up to 30% of their pre-tax salary, making learning more affordable and effective. Compatible with:<Logos><FundaeLogo></FundaeLogo><EdenredLogo></EdenredLogo><CobeeLogo></CobeeLogo><SodexoLogo></SodexoLogo><BetterflyLogo></BetterflyLogo><CoverflexLogo></CoverflexLogo></Logos>

Tax benefits

Access Tax Benefits in two clicks

Smart Payment Options, Zero Hassle

To start, let's fill out this information. It will only take a few seconds.

After sign-up, you’ll able to

Designed for everyone eager to upskill, with flexible payment options available.

Flexible payment options for your learning journey

Easy Smart Payment Solutions Built for you

Spread across 12 monthly payments. Income Flex Protection.

0% Interest Instalments

Accessible for free through your employer. We take care of the admin.

Fundae: Premium courses

Pay a fixed % of your income. Payment Pause Guarantee.

ISA: No up front payment

Seamlessly access your company's training budget.

Employer subsidy

Ensure you're getting the most value out of every euro spent.

Access tax discounts

Take a moment to double-check your personal details and finalize any incomplete steps.

We're reviewing your application, which typically takes just a few hours. Check your summary below—click and click in 'View my Orders' to view details. Track your progress anytime on your dashboard.

You can proceed with the submission of your application by clicking the button below.

Ready to Future-Proof your Career

Whether you’re preparing for your first job, aiming for a promotion, or considering a career change, Mydra’s AI Career CoPilot is here to guide you. This intelligent tool designs a personalized learning path based on your career objectives, education, and work experience.

Tailored Learning Paths for Your Career Goals

Join the top training providers on Mydra today

This helps us personalize your experience. This will take less than a minute.

Which statement best reflects your team's AI experience?

How does your team currently handle repetitive tasks?

What does your company do? Tell us about your products or services.

e.g. We provide financial solutions for students, including tuition financing and flexible repayment plans.

This helps us tailor your experience. It'll take less than a minute.

Which best describes your company?

Select everything that applies — the more you tell us, the more accurate your Work Map will be. Add anything specific in the field below.

Please indicate your team's AI model preference.

Select all that apply — this helps shape your workflows and learning path from the start.

Please indicate your team's level of technical expertise.

This helps us personalize your workflow and recommendations.

Please select your team's primary automation need:

Interested in exploring additional content?

Pretraining LLMs

Overview

Who is this course for?

What will you learn?

Upcoming cohorts