introduction to analysis of algorithms

An Introduction to the Analysis of Algorithms

Other resources., pseudo-app icons..

• Trending Categories

• Selected Reading
• UPSC IAS Exams Notes
• Developer's Best Practices
• Questions and Answers
• Effective Resume Writing
• HR Interview Questions
• Computer Glossary

Introduction to Analysis of Algorithms

In theoretical analysis of algorithms, it is common to estimate their complexity in the asymptotic sense, i.e., to estimate the complexity function for arbitrarily large input. The term "analysis of algorithms" was coined by Donald Knuth.

Algorithm analysis is an important part of computational complexity theory, which provides theoretical estimation for the required resources of an algorithm to solve a specific computational problem. Most algorithms are designed to work with inputs of arbitrary length. Analysis of algorithms is the determination of the amount of time and space resources required to execute it.

Usually, the efficiency or running time of an algorithm is stated as a function relating the input length to the number of steps, known as time complexity , or volume of memory, known as space complexity .

In this Section We are going to cover −

• Algorithms and Complexities
• Asymptotic analysis
• Asymptotic notations
• little o and little omega notations
• Analysis of loops
• Amortized analysis
• Space Complexity
• Pseudo Polynomial Type Algorithm and PATS (Pseudo Polynomial Type Approximation Scheme)
• Heap Building Complexity
• Complexity of Loops with Powers

I love programming (: That's all I know

• Related Articles
• Introduction to Backtracking Algorithms
• Introduction to Searching Algorithms
• Introduction to Greedy Algorithms
• Introduction to Graph Algorithms
• Introduction to Divide & Conquer Algorithms
• Introduction to Pattern Searching Algorithms
• Introduction to Algorithms for Mathematical Problems
• Introduction to Geospatial Data Analysis with R
• Introduction to Theory of Evolution
• What are the types of process scheduling algorithms and which algorithms lead to starvation?
• Introduction to Databases
• Introduction to Backtracking
• Introduction to Iptables
• Introduction to Anime.js
• Introduction to Superphones

Kickstart Your Career

Get certified by completing the course

• Online Degree Explore Bachelor’s & Master’s degrees
• MasterTrack™ Earn credit towards a Master’s degree
• University Certificates Advance your career with graduate-level learning
• Top Courses
• Join for Free

Analysis of Algorithms

Taught in English

Some content may not be translated

81,120 already enrolled

Gain insight into a topic and learn the fundamentals

Instructor: Robert Sedgewick

(289 reviews)

Details to know

Available in English

Subtitles: Kazakh, German, Hindi, Russian, Swedish, Korean, Portuguese (Brazilian), Greek, English, Italian, French, Chinese (Simplified), Spanish, Arabic, Vietnamese, Thai, Ukrainian, Japanese, Indonesian, Polish, Dutch, Turkish

See how employees at top companies are mastering in-demand skills

Earn a career certificate

Add this credential to your LinkedIn profile, resume, or CV

Share it on social media and in your performance review

There are 9 modules in this course

This course teaches a calculus that enables precise quantitative predictions of large combinatorial structures. In addition, this course covers generating functions and real asymptotics and then introduces the symbolic method in the context of applications in the analysis of algorithms and basic structures such as permutations, trees, strings, words, and mappings.

All the features of this course are available for free. It does not offer a certificate upon completion.

We begin by considering historical context and motivation for the scientific study of algorithm performance. Then we consider a classic example that illustrates the key ingredients of the process: the analysis of Quicksort. The lecture concludes with a discussion of some resources that you might find useful during this course.

What's included

4 videos 2 readings 1 quiz 1 discussion prompt

4 videos • Total 75 minutes

• History and Motivation • 11 minutes • Preview module
• A Scientific Approach • 16 minutes
• Example: Quicksort • 30 minutes
• Resources • 17 minutes

2 readings • Total 20 minutes

• Getting Started • 10 minutes
• Exercises from Lecture 1 • 10 minutes

1 quiz • Total 30 minutes

• Analysis of Algorithms • 30 minutes

1 discussion prompt • Total 10 minutes

Recurrences.

We begin this lecture with an overview of recurrence relations, which provides us with a direct mathematical model for the analysis of algorithms. We finish by examining the fascinating oscillatory behavior of the divide-and-conquer recurrence corresponding to the mergesort algorithm and the general "master theorem" for related recurrences.

5 videos 1 reading 3 quizzes 1 discussion prompt

5 videos • Total 71 minutes

• Computing Values • 10 minutes • Preview module
• Telescoping • 15 minutes
• Types of Recurrences • 12 minutes
• Mergesort • 18 minutes
• Master Theorem • 14 minutes

1 reading • Total 10 minutes

• Exercises from Lecture 2 • 10 minutes

3 quizzes • Total 90 minutes

• Recurrences • 30 minutes
• Pop Quiz on Telescoping • 30 minutes
• Pop Quiz on the Master Theorem • 30 minutes

Generating Functions

Since the 17th century, scientists have been using generating functions to solve recurrences, so we continue with an overview of generating functions, emphasizing their utility in solving problems like counting the number of binary trees with N nodes.

5 videos 1 reading 1 quiz 1 discussion prompt

5 videos • Total 84 minutes

• Ordinary Generating Functions • 16 minutes • Preview module
• Counting with Generating Functions • 27 minutes
• Catalan Numbers • 14 minutes
• Solving Recurrences • 18 minutes
• Exponential Generating Functions • 7 minutes
• Exercises from Lecture 3 • 10 minutes
• Generating Functions • 30 minutes

Asymptotics

Exact answers are often cumbersome, so we next consider a scientific approach to developing approximate answers that, again, mathematicians and scientists have used for centuries.

4 videos 1 reading 1 quiz 1 discussion prompt

4 videos • Total 83 minutes

• Standard Scale • 18 minutes • Preview module
• Manipulating Expansions • 19 minutes
• Asymptotics of Finite Sums • 16 minutes
• Bivariate Asymptotics • 28 minutes
• Exercises from Lecture 4 • 10 minutes
• Asymptotics • 30 minutes

Analytic Combinatorics

Analytic Combinatorics. With a basic knowledge of recurrences, generating functions, and asymptotics, you are ready to learn and appreciate the basic features of analytic combinatorics, a systematic approach that avoids much of the detail of the classical methods that we have been considering. We introduce unlabeled and labelled combinatorial classes and motivate our basic approach to studying them, with numerous examples.

4 videos • Total 76 minutes

• The Symbolic Method • 25 minutes • Preview module
• Labelled Objects • 29 minutes
• Coefficient Asymptotics • 12 minutes
• Perspective • 9 minutes
• Errata • 10 minutes
• Exercises from Lecture 5 • 10 minutes
• Analytic Combinatorics • 30 minutes

The quintessential recursive structure, trees of various sorts are ubiquitous in scientific enquiry, and they arise explicitly in countless computing applications. You can find broad coverage in the textbook, but the lecture focuses on the use of analytic combinatorics to enumerate various types of trees and study parameters.

• Trees and Forests • 14 minutes • Preview module
• Binary Search Trees • 22 minutes
• Path Length • 25 minutes
• Other Types of Trees • 13 minutes
• Exercises from Lecture 6 • 10 minutes
• Trees • 30 minutes

Permutations

The study of sorting algorithms is the study of properties of permutations. We introduce analytic-combinatoric approaches to studying permutations in the context of this relationship.

5 videos • Total 91 minutes

• Basics • 17 minutes • Preview module
• Sets of Cycles • 20 minutes
• Left-Right-Minima • 19 minutes
• Other Parameters • 15 minutes
• BGFs and Distributions • 19 minutes
• Exercises from Lecture 7 • 10 minutes
• Permutations • 30 minutes

Strings and Tries

From DNA sequences to web indices, strings (sequences of characters) are ubiquitous in modern computing applications, so we use analytic combinatorics to study their basic properties and then introduce the trie, an essential and fundamental structure not found in classical combinatorics.

5 videos • Total 77 minutes

• Bitstrings with Restrictions • 27 minutes • Preview module
• Languages • 12 minutes
• Tries • 16 minutes
• Trie Parameters • 18 minutes
• Exercises • 2 minutes
• Exercises from Lecture 8 • 10 minutes
• Strings and Tries • 30 minutes

Words and Mappings

We view strings as sets of characters or as functions from [1..N] to [1..M] to study classical occupancy problems and their application to fundamental hashing algorithms. Functions from [1..N] to [1..N] are mappings, which have an interesting and intricate structure that we can study with analytic combinatorics.

6 videos 1 reading 1 quiz 1 discussion prompt

6 videos • Total 72 minutes

• Words • 11 minutes • Preview module
• Birthday Problem • 6 minutes
• Coupon Collector Problem • 13 minutes
• Hash Tables • 13 minutes
• Mappings • 23 minutes
• Exercises from Lecture 9 • 10 minutes
• Strings and Words • 30 minutes

Instructor ratings

We asked all learners to give feedback on our instructors based on the quality of their teaching style.

Princeton University is a private research university located in Princeton, New Jersey, United States. It is one of the eight universities of the Ivy League, and one of the nine Colonial Colleges founded before the American Revolution.

Recommended if you're interested in Algorithms

Princeton University

Computer Science: Algorithms, Theory, and Machines

Coursera Project Network

Unreal para Principiantes: Crea videojuegos

Guided Project

EIT Digital

I/O-efficient algorithms

Algorithms, Part I

Why people choose coursera for their career.

Learner reviews

Showing 3 of 289

289 reviews

Reviewed on Sep 15, 2022

I would highly recommend this course to any developer to understand algorithm analysis.

Reviewed on Apr 8, 2020

Wonderful insights about the study of the algorithm's complexity and combinatoric logic.

Reviewed on Nov 22, 2019

Excellent course, great exercise in combinatorics.

New to Algorithms? Start here.

Open new doors with Coursera Plus

Unlimited access to 7,000+ world-class courses, hands-on projects, and job-ready certificate programs - all included in your subscription

Advance your career with an online degree

Earn a degree from world-class universities - 100% online

Join over 3,400 global companies that choose Coursera for Business

Upskill your employees to excel in the digital economy

Frequently asked questions

Can i earn a certificate in this course.

No. As per Princeton University policy, no certificates, credentials, or reports are awarded in connection with this course.

When will I have access to the lectures and assignments?

Access to lectures and assignments depends on your type of enrollment. If you take a course in audit mode, you will be able to see most course materials for free. To access graded assignments and to earn a Certificate, you will need to purchase the Certificate experience, during or after your audit. If you don't see the audit option:

The course may not offer an audit option. You can try a Free Trial instead, or apply for Financial Aid.

The course may offer 'Full Course, No Certificate' instead. This option lets you see all course materials, submit required assessments, and get a final grade. This also means that you will not be able to purchase a Certificate experience.

More questions

buy the book on Amazon

An Introduction to the Analysis of Algorithms

by Robert Sedgewick and Philippe Flajolet

• Table of Contents
• Online Content

Analysis of Algorithms is a thorough overview of the primary techniques used in the mathematical analysis of algorithms. Drawing from classical mathematical topics (discrete mathematics, elementary real analysis, and combinatorics) and classical computer science topics (algorithms and data structures), it

• Can serve as a textbook for a course in discrete mathematics for computer scientists.
• Is suitable for introducing computer science to students in mathematics.
• Provides access to literature in the field, particularly Knuth’s books and Flajolet’s papers.
• Emphasizes analysis as a basis for a scientific approach to evaluating performance.

Coverage includes recurrence relations, generating functions, asymptotics, and analytic combinatorics, with applications to algorithms and combinatorial structures such as trees, permutations, strings and tries, words, and mappings.

Foreword by D. E. Knuth:

People who analyze algorithms have double happiness. First of all they experience the sheer beauty of elegant mathematical patterns that surround elegant computational procedures. Then they receive a practical payoff when their theories make it possible to get other jobs done more quickly and more economically.

Mathematical models have been a crucial inspiration for all scientific activity, even though they are only approximate idealizations of real-world phenomena. Inside a computer, such models are more relevant than ever before, because computer programs create artificial worlds in which mathematical models often apply precisely. I think that’s why I got hooked on analysis of algorithms when I was a graduate student, and why the subject has been my main life’s work ever since.

The appearance of this long-awaited textbook by Sedgewick and Flajolet is therefore most welcome. Its authors are not only worldwide leaders of the field, they also are masters of exposition. I am sure that every serious computer scientist will find this book rewarding in many ways.

First edition

French translation

Chapter One. Analysis of Algorithms

1.1 Why Analyze an Algorithm? 1.2 Theory of Algorithms 1.3 Analysis of Algorithms 1.4 Average-Case Analysis 1.5 Example: Analysis of Quicksort 1.6 Asymptotic Approximations 1.7 Distributions 1.8 Randomized Algorithms

Chapter Two. Recurrence Relations

2.1 Basic Properties 2.2 First-Order Recurrences 2.3 Nonlinear First-Order Recurrences 2.4 Higher-Order Recurrences 2.5 Methods for Solving Recurrences 2.6 Binary D-and-C Recurrences and Binary Numbers 2.7 General Divide-and-Conquer Recurrences

Chapter Three. Generating Functions 

3.1 Ordinary GFs 3.2 Exponential GFs 3.3 GF Solution of Recurrences 3.4 Expanding GFs 3.5 Transformations with GFs 3.6 Functional Equations on GFs 3.7 Solving the Quicksort Median-of-3 Recurrence 3.8 Counting with GFs 3.9 Probability GFs 3.10 Bivariate GFs 3.11 Special Functions

Chapter Four. Asymptotic Approximations

4.1 Notation for Asymptotic Approximations 4.2 Asymptotic Expansions 4.3 Manipulating Asymptotic Expansions 4.4 Asymptotic Approximations of Finite Sums 4.5 Euler-Maclaurin Summation 4.6 Bivariate Asymptotics 4.7 Laplace Method 4.8 “Normal” Examples from Analysis of Algorithms 4.9 “Poisson” Examples from Analysis of Algorithms

Chapter Five. Analytic Combinatorics

5.1 Formal Basis 5.2 Symbolic Method for Unlabelled Classes 5.3 Symbolic Method for Labelled Classes 5.4 Symbolic Method for Parameters 5.5 Generating Function Coefficient Asymptotics

Chapter Six. Trees

6.1 Binary Trees 6.2 Forests and Trees 6.3 Combinatorial Equivalences 6.4 Properties of Trees 6.5 Examples of Tree Algorithms 6.6 Binary Search Trees 6.7 Average Path Length in Catalan Trees 6.8 Path Length in Binary Search Trees 6.9 Additive Parameters of Random Trees 6.10 Height 6.11 Average-Case Results on Properties of Trees 6.12 Lagrange Inversion 6.13 Rooted Unordered Trees 6.14 Labelled Trees 6.15 Other Types of Trees

Chapter Seven. Permutations

7.1 Basic Properties of Permutations 7.2 Algorithms on Permutations 7.3 Representations of Permutations 7.4 Enumeration Problems 7.5 Analyzing Properties of Permutations 7.6 Inversions and Insertion Sorts 7.7 Left-to-Right Minima and Selection Sort 7.8 Cycles and In Situ Permutation 7.9 Extremal Parameters

Chapter Eight. Strings and Tries

8.1 String Searching 8.2 Combinatorial Properties of Bitstrings 8.3 Regular Expressions 8.4 FSAs and the Knuth-Morris-Pratt Algorithm 8.5 Context-Free Grammars 8.6 Tries 8.7 Trie Algorithms 8.8 Combinatorial Properties of Tries 8.9 Larger Alphabets

Chapter Nine. Words and Mappings

9.1 Hashing with Separate Chaining 9.2 Balls-and-Urns Model and Properties of Words 9.3 Birthday Paradox and Coupon Collector Problem 9.4 Occupancy Restrictions and Extremal Parameters 9.5 Occupancy Distributions 9.6 Open Addressing Hashing 9.7 Mappings 9.8 Integer Factorization and Mappings

T his book is intended to be a thorough overview of the primary techniques used in the mathematical analysis of algorithms. The material covered draws from classical mathematical topics, including discrete mathematics, elementary real analysis, and combinatorics, as well as from classical computer science topics, including algorithms and data structures. The focus is on “average-case” or “probabilistic” analysis, though the basic mathematical tools required for “worst-case” or “complexity” analysis are covered as well.       We assume that the reader has some familiarity with basic concepts in both computer science and real analysis. In a nutshell, the reader should be able to both write programs and prove theorems. Otherwise, the book is intended to be self-contained.       The book is meant to be used as a textbook in an upper-level course on analysis of algorithms. It can also be used in a course in discrete mathematics for computer scientists, since it covers basic techniques in discrete mathematics as well as combinatorics and basic properties of important discrete structures within a familiar context for computer science students. It is traditional to have somewhat broader coverage in such courses, but many instructors may find the approach here to be a useful way to engage students in a substantial portion of the material. The book also can be used to introduce students in mathematics and applied mathematics to principles from computer science related to algorithms and data structures.       Despite the large amount of literature on the mathematical analysis of algorithms, basic information on methods and models in widespread use has not been directly accessible to students and researchers in the field. This book aims to address this situation, bringing together a body of material intended to provide readers with both an appreciation for the challenges of the field and the background needed to learn the advanced tools being developed to meet these challenges. Supplemented by papers from the literature, the book can serve as the basis for an introductory graduate course on the analysis of algorithms, or as a reference or basis for self-study by researchers in mathematics or computer science who want access to the literature in this field.

Preparation. Mathematical maturity equivalent to one or two years’ study at the college level is assumed. Basic courses in combinatorics and discrete mathematics may provide useful background (and may overlap with some material in the book), as would courses in real analysis, numerical methods, or elementary number theory. We draw on all of these areas, but summarize the necessary material here, with reference to standard texts for people who want more information.       Programming experience equivalent to one or two semesters’ study at the college level, including elementary data structures, is assumed. We do not dwell on programming and implementation issues, but algorithms and data structures are the central object of our studies. Again, our treatment is complete in the sense that we summarize basic information, with reference to standard texts and primary sources.

Related books. Related texts include The Art of Computer Programming by Knuth; Algorithms, Fourth Edition, by Sedgewick and Wayne; Introduction to Algorithms by Cormen, Leiserson, Rivest, and Stein; and our own Analytic Combinatorics . This book could be considered supplementary to each of these.       In spirit, this book is closest to the pioneering books by Knuth. Our focus is on mathematical techniques of analysis, though, whereas Knuth’s books are broad and encyclopedic in scope, with properties of algorithms playing a primary role and methods of analysis a secondary role, this book can serve as basic preparation for the advanced results covered and referred to in Knuth’s books. We also cover approaches and results in the analysis of algorithms that have been developed since publication of Knuth’s books.       We also strive to keep the focus on covering algorithms of fundamental importance and interest, such as those described in Sedgewick’s Algorithms (now in its fourth edition, coauthored by K. Wayne). That book surveys classic algorithms for sorting and searching, and for processing graphs and strings. Our emphasis is on mathematics needed to support scientific studies that can serve as the basis of predicting performance of such algorithms and for comparing different algorithms on the basis of performance.       Cormen, Leiserson, Rivest, and Stein’s Introduction to Algorithms has emerged as the standard textbook that provides access to the research literature on algorithm design. The book (and related literature) focuses on design and the theory of algorithms, usually on the basis of worst-case performance bounds. In this book, we complement this approach by focusing on the analysis of algorithms, especially on techniques that can be used as the basis for scientific studies (as opposed to theoretical studies). Chapter 1 is devoted entirely to developing this context.       This book also lays the groundwork for our Analytic Combinatorics , a general treatment that places the material here in a broader perspective and develops advanced methods and models that can serve as the basis for new research, not only in the analysis of algorithms but also in combinatorics and scientific applications more broadly. A higher level of mathematical maturity is assumed for that volume, perhaps at the senior or beginning graduate student level. Of course, careful study of this book is adequate preparation. It certainly has been our goal to make it sufficiently interesting that some readers will be inspired to tackle more advanced material!

How to use this book. Readers of this book are likely to have rather diverse backgrounds in discrete mathematics and computer science. With this in mind, it is useful to be aware of the implicit structure of the book: nine chapters in all, an introductory chapter followed by four chapters emphasizing mathematical methods, then four chapters emphasizing combinatorial structures with applications in the analysis of algorithms, as follows:

INTRODUCTION

1. ANALYSIS OF ALGORITHMS

DISCRETE MATHEMATICAL METHODS

2. RECURRENCE RELATIONS 3. GENERATING FUNCTIONS 4. ASYMPTOTIC APPROXIMATIONS 5. ANALYTIC COMBINATORICS

ALGORITHMS AND COMBINATORIAL STRUCTURES

6. TREES 7. PERMUTATIONS 8. STRINGS AND TRIES 9. WORDS AND MAPPINGS

Chapter 1 puts the material in the book into perspective, and will help all readers understand the basic objectives of the book and the role of the remaining chapters in meeting those objectives. Chapters 2 through 4 cover methods from classical discrete mathematics, with a primary focus on developing basic concepts and techniques. They set the stage for Chapter 5, which is pivotal, as it covers analytic combinatorics, a calculus for the study of large discrete structures that has emerged from these classical methods to help solve the modern problems that now face researchers because of the emergence of computers and computational models. Chapters 6 through 9 move the focus back toward computer science, as they cover properties of combinatorial structures, their relationships to fundamental algorithms, and analytic results.       Though the book is intended to be self-contained, this structure supports differences in emphasis when teaching the material, depending on the background and experience of students and instructor. One approach, more mathematically oriented, would be to emphasize the theorems and proofs in the first part of the book, with applications drawn from Chapters 6 through 9. Another approach, more oriented towards computer science, would be to briefly cover the major mathematical tools in Chapters 2 through 5 and emphasize the algorithmic material in the second half of the book. But our primary intention is that most students should be able to learn new material from both mathematics and computer science in an interesting context by working carefully all the way through the book.       Supplementing the text are lists of references and several hundred exercises, to encourage readers to examine original sources and to consider the material in the text in more depth.       Our experience in teaching this material has shown that there are numerous opportunities for instructors to supplement lecture and reading material with computation-based laboratories and homework assignments. The material covered here is an ideal framework for students to develop expertise in a symbolic manipulation system such as Mathematica, MAPLE, or SAGE. More important, the experience of validating the mathematical studies by comparing them against empirical studies is an opportunity to provide valuable insights for students that should not be missed.

Online lectures. A complete set of curated studio-produced lecture videos that can be used in conjunction with this text are available online. These lectures are fully coordinated with the text, but also include examples and other materials that supplement the text and are intended to bring the subject to life. As with traditional live lectures, their purpose is to inform and inspire, motivating students to study and learn from the text. Our experience is that student engagement with the material is significantly better with videos than with live lectures because of the ability to play the lectures at a chosen speed and to replay and review the lectures at any time.

Booksite. Another important feature of the book is its relationship to our Analysis of Algorithms website , which we refer to as the booksite. This site is freely available and contains supplementary material about the analysis of algorithms, including a complete set of lecture slides and links to related material, including similar sites for Algorithms and Analytic Combinatorics. These resources are suitable both for use by any instructor teaching the material and for self-study.

Acknowledgments. We are very grateful to INRIA, Princeton University, and the National Science Foundation, which provided the primary support for us to work on this book. Other support has been provided by Brown University, European Community (Alcom Project), Institute for Defense Analyses, Ministère de la Recherche et de la Technologie, Stanford University, Université Libre de Bruxelles, and Xerox Palo Alto Research Center. This book has been many years in the making, so a comprehensive list of people and organizations that have contributed support would be prohibitively long, and we apologize for any omissions.

Don Knuth’s influence on our work has been extremely important, as is obvious from the text.

Students in Princeton, Paris, and Providence provided helpful feedback in courses taught from this material over the years, and students and teachers all over the world provided feedback on the first edition. We would like to specifically thank Philippe Dumas, Mordecai Golin, Helmut Prodinger, Michele Soria, Mark Daniel Ward, and Mark Wilson for their help.

Corfu, September 1995      Ph. F. and R. S. Paris, December 2012                        R. S.

An extensive amount of content associated with this book is available online, most of it freely available. The introduction to the COURSES page  and the essay on online learning on this website tell the full story of the types of content, relationships, and how it may best be used by students and teachers of all sorts. The COURSES page for this book gives specific details. Here are some brief descriptions and quick links to this material.

Booksite. Our Analysis of Algorithms website contains hundreds of files, fully coordinated with our textbook and also useful as a standalone resource. It consists of the following elements:

Excerpts. A condensed version of the text narrative, for reference while online. Exercises. Selected exercises from the book and “web exercises” developed since its publication, along with solutions to selected exercises. Assignments. Creative assignments that we have used at Princeton.

This is a living resource that is constantly being improved and updated.

Online lectures. A complete set of curated studio-produced lecture videos that can be used in conjunction with this text are available online via the CUbits platform. This platform applies modern machine-learning models to build searchable transcripts, indexing, and other features that enhance the experience of learning from the videos.

Lecture slides. The lectures are based on hundreds of detailed, carefully prepared slides that are useful for reviewing the lecture content. These are available at the websites just listed, and on the COURSES page for this book , which provides a convenient interface for browsing through them.

MOOC. The Coursera course Analysis of Algorithms is a massive open online course based on the lecture videos that includes exercises, auto-assessed programming assignments, and other features encouraging coordinated study of the material in the book.

Princeton course. You can find details on how we conduct our course at Princeton on our course website . Our students watch lectures online and complete weekly programming assignments. We ensure loose synchronization via the book site and regular e-mail for authoritative communication from faculty to students.

© 2023 Robert Sedgewick.

• Abridged CV
• Curriculum Vitae
• Computer Science
• Analysis of Algorithms
• Analytic Combinatorics
• Quick Links
• Cardinality
• Algorithm Science
• Data Structures
• Online Learning
• Visualization
• 2023 Test of Time
• Quicksort’s 60th
• 21st Century Model
• Cardinality Estimation
• Lasting Legacy of Flajolet
• Science in CS
• Algorithms for the Masses
• LL Red-Black Trees
• Creating Algorithms
• +8 old talks
• Phi Beta Kappa Award
• Flajolet Lecture Prize
• Steele Prize
• Karlstrom Award

CS 4820 Spring 2021

Introduction to analysis of algorithms.

Office Hours: Tuesdays 1–2pm; Wednesdays 10–11am

Zoom link — https://cornell.zoom.us/j/92368128972?pwd=c3pRbXA1YnB4eXpDZlR3ajlJYnR4Zz09 ).

Zoom link — https://cornell.zoom.us/j/96494539853?pwd=YmIxOHlDSEROVW5TektTa2YyU0FXZz09 .

Instructions for phone access here .

Lectures will be given live during this time and you are encouraged to attend. Lectures will also be recorded and available for asynchronous viewing. There will be links to the video+audio and audio-only for each lecture on the schedule page, along with any notes or handouts associated with that lecture. The raw recordings will be available soon after the lecture, to be replaced by edited and closed-captioned versions as soon as they can be processed.

Zoom etiquette If you attend the live lecture, please enable your camera. You may ask questions at any time during the lecture. Type your question in the chat window. A staff member will be monitoring the chat window and will either respond with an answer in the chat or will interrupt the instructor at an appropriate moment.

External links

• Ed Discussions (discussions & announcements)
• CMS (homework submission)
• videonote (recorded lectures)

Enrollment Information

• Salil Desai (ssd73), email
• Sasha Frolov (asf79), email
• Katie Gioioso (krg48), email
• Rishi Penmetcha (rp487), email
• Joy Qi (hq38), email
• Matvey Soloviev (ms2837), email
• Diana Smirnov (zis8), email
• Ruqi Zhang (rz297), email
• Prelim 1: Thursday, March 18, 8:30pm
• Prelim 2: Tuesday, April 13, 8:30pm

Office hour calendar

• DSA for Beginners
• DSA Tutorial
• Data Structures
• Linked List
• Dynamic Programming
• Binary Tree
• Binary Search Tree
• Divide & Conquer
• Mathematical
• Backtracking
• Branch and Bound
• Pattern Searching

• Explore Our Geeks Community
• Algorithms Tutorial

What is Algorithm | Introduction to Algorithms

• Definition, Types, Complexity and Examples of Algorithm
• Algorithms Design Techniques
• Why the Analysis of Algorithm is important?

Analysis of Algorithms

• Asymptotic Notation and Analysis (Based on input size) in Complexity Analysis of Algorithms
• Worst, Average and Best Case Analysis of Algorithms
• Types of Asymptotic Notations in Complexity Analysis of Algorithms
• How to Analyse Loops for Complexity Analysis of Algorithms
• How to analyse Complexity of Recurrence Relation
• Introduction to Amortized Analysis

Types of Algorithms

• Sorting Algorithms
• Searching Algorithms
• Greedy Algorithms
• Backtracking Algorithms
• Divide and Conquer
• Mathematical Algorithms
• Geometric Algorithms
• Bitwise Algorithms
• Graph Data Structure And Algorithms
• Randomized Algorithms
• Branch and Bound Algorithm
• The Role of Algorithms in Computing
• Most important type of Algorithms

Definition of Algorithm

The word Algorithm means ” A set of finite rules or instructions to be followed in calculations or other problem-solving operations ” Or ” A procedure for solving a mathematical problem in a finite number of steps that frequently involves recursive operations” .

Therefore Algorithm refers to a sequence of finite steps to solve a particular problem.

Use of the Algorithms:

Algorithms play a crucial role in various fields and have many applications. Some of the key areas where algorithms are used include:

• Computer Science: Algorithms form the basis of computer programming and are used to solve problems ranging from simple sorting and searching to complex tasks such as artificial intelligence and machine learning.
• Mathematics: Algorithms are used to solve mathematical problems, such as finding the optimal solution to a system of linear equations or finding the shortest path in a graph.
• Operations Research : Algorithms are used to optimize and make decisions in fields such as transportation, logistics, and resource allocation.
• Artificial Intelligence: Algorithms are the foundation of artificial intelligence and machine learning, and are used to develop intelligent systems that can perform tasks such as image recognition, natural language processing, and decision-making.
• Data Science: Algorithms are used to analyze, process, and extract insights from large amounts of data in fields such as marketing, finance, and healthcare.

These are just a few examples of the many applications of algorithms. The use of algorithms is continually expanding as new technologies and fields emerge, making it a vital component of modern society.

Algorithms can be simple and complex depending on what you want to achieve.

It can be understood by taking the example of cooking a new recipe. To cook a new recipe, one reads the instructions and steps and executes them one by one, in the given sequence. The result thus obtained is the new dish is cooked perfectly. Every time you use your phone, computer, laptop, or calculator you are using Algorithms. Similarly, algorithms help to do a task in programming to get the expected output.

The Algorithm designed are language-independent, i.e. they are just plain instructions that can be implemented in any language, and yet the output will be the same, as expected.

What is the need for algorithms?

• Algorithms are necessary for solving complex problems efficiently and effectively. 
• They help to automate processes and make them more reliable, faster, and easier to perform.
• Algorithms also enable computers to perform tasks that would be difficult or impossible for humans to do manually.
• They are used in various fields such as mathematics, computer science, engineering, finance, and many others to optimize processes, analyze data, make predictions, and provide solutions to problems.

What are the Characteristics of an Algorithm?

As one would not follow any written instructions to cook the recipe, but only the standard one. Similarly, not all written instructions for programming are an algorithm. For some instructions to be an algorithm, it must have the following characteristics:

• Clear and Unambiguous : The algorithm should be unambiguous. Each of its steps should be clear in all aspects and must lead to only one meaning.
• Well-Defined Inputs : If an algorithm says to take inputs, it should be well-defined inputs. It may or may not take input.
• Well-Defined Outputs: The algorithm must clearly define what output will be yielded and it should be well-defined as well. It should produce at least 1 output.
• Finite-ness: The algorithm must be finite, i.e. it should terminate after a finite time.
• Feasible: The algorithm must be simple, generic, and practical, such that it can be executed with the available resources. It must not contain some future technology or anything.
• Language Independent: The Algorithm designed must be language-independent, i.e. it must be just plain instructions that can be implemented in any language, and yet the output will be the same, as expected.
• Input : An algorithm has zero or more inputs. Each that contains a fundamental operator must accept zero or more inputs.
•   Output : An algorithm produces at least one output. Every instruction that contains a fundamental operator must accept zero or more inputs.
• Definiteness: All instructions in an algorithm must be unambiguous, precise, and easy to interpret. By referring to any of the instructions in an algorithm one can clearly understand what is to be done. Every fundamental operator in instruction must be defined without any ambiguity.
• Finiteness: An algorithm must terminate after a finite number of steps in all test cases. Every instruction which contains a fundamental operator must be terminated within a finite amount of time. Infinite loops or recursive functions without base conditions do not possess finiteness.
• Effectiveness: An algorithm must be developed by using very basic, simple, and feasible operations so that one can trace it out by using just paper and pencil.

Properties of Algorithm:

• It should terminate after a finite time.
• It should produce at least one output.
• It should take zero or more input.
• It should be deterministic means giving the same output for the same input case.
• Every step in the algorithm must be effective i.e. every step should do some work.

Types of Algorithms:

There are several types of algorithms available. Some important algorithms are:

1. Brute Force Algorithm :

It is the simplest approach to a problem. A brute force algorithm is the first approach that comes to finding when we see a problem.

2. Recursive Algorithm :

A recursive algorithm is based on recursion . In this case, a problem is broken into several sub-parts and called the same function again and again.

3. Backtracking Algorithm :

The backtracking algorithm builds the solution by searching among all possible solutions. Using this algorithm, we keep on building the solution following criteria. Whenever a solution fails we trace back to the failure point build on the next solution and continue this process till we find the solution or all possible solutions are looked after.

4. Searching Algorithm :

Searching algorithms are the ones that are used for searching elements or groups of elements from a particular data structure. They can be of different types based on their approach or the data structure in which the element should be found.

5. Sorting Algorithm :

Sorting is arranging a group of data in a particular manner according to the requirement. The algorithms which help in performing this function are called sorting algorithms. Generally sorting algorithms are used to sort groups of data in an increasing or decreasing manner.

6. Hashing Algorithm :

Hashing algorithms work similarly to the searching algorithm. But they contain an index with a key ID. In hashing, a key is assigned to specific data.

7. Divide and Conquer Algorithm :

This algorithm breaks a problem into sub-problems, solves a single sub-problem, and merges the solutions to get the final solution. It consists of the following three steps:

8. Greedy Algorithm :

In this type of algorithm, the solution is built part by part. The solution for the next part is built based on the immediate benefit of the next part. The one solution that gives the most benefit will be chosen as the solution for the next part.

9. Dynamic Programming Algorithm :

This algorithm uses the concept of using the already found solution to avoid repetitive calculation of the same part of the problem. It divides the problem into smaller overlapping subproblems and solves them.

10. Randomized Algorithm :

In the randomized algorithm, we use a random number so it gives immediate benefit. The random number helps in deciding the expected outcome.

To learn more about the types of algorithms refer to the article about “ Types of Algorithms “.

Advantages of Algorithms:

• It is easy to understand.
• An algorithm is a step-wise representation of a solution to a given problem.
• In an Algorithm the problem is broken down into smaller pieces or steps hence, it is easier for the programmer to convert it into an actual program.

Disadvantages of Algorithms :

• Writing an algorithm takes a long time so it is time-consuming.
• Understanding complex logic through algorithms can be very difficult.
• Branching and Looping statements are difficult to show in Algorithms (imp) .

How to Design an Algorithm?

To write an algorithm, the following things are needed as a pre-requisite: 

• The problem that is to be solved by this algorithm i.e. clear problem definition.
• The constraints of the problem must be considered while solving the problem.
• The input to be taken to solve the problem.
• The output is to be expected when the problem is solved.
• The solution to this problem is within the given constraints.

Then the algorithm is written with the help of the above parameters such that it solves the problem.

Example: Consider the example to add three numbers and print the sum.

Step 1: Fulfilling the pre-requisites  

As discussed above, to write an algorithm, its prerequisites must be fulfilled. 

• The problem that is to be solved by this algorithm : Add 3 numbers and print their sum.
• The constraints of the problem that must be considered while solving the problem : The numbers must contain only digits and no other characters.
• The input to be taken to solve the problem: The three numbers to be added.
• The output to be expected when the problem is solved: The sum of the three numbers taken as the input i.e. a single integer value.
• The solution to this problem, in the given constraints: The solution consists of adding the 3 numbers. It can be done with the help of the ‘+’ operator, or bit-wise, or any other method.

Step 2: Designing the algorithm

Now let’s design the algorithm with the help of the above pre-requisites:

Algorithm to add 3 numbers and print their sum:  

• Declare 3 integer variables num1, num2, and num3.
• Take the three numbers, to be added, as inputs in variables num1, num2, and num3 respectively.
• Declare an integer variable sum to store the resultant sum of the 3 numbers.
• Add the 3 numbers and store the result in the variable sum.
• Print the value of the variable sum

Step 3: Testing the algorithm by implementing it.

To test the algorithm, let’s implement it in C language.

Here is the step-by-step algorithm of the code:

• Declare three variables num1, num2, and num3 to store the three numbers to be added.
• Declare a variable sum to store the sum of the three numbers.
• Use the cout statement to prompt the user to enter the first number.
• Use the cin statement to read the first number and store it in num1.
• Use the cout statement to prompt the user to enter the second number.
• Use the cin statement to read the second number and store it in num2.
• Use the cout statement to prompt the user to enter the third number.
• Use the cin statement to read and store the third number in num3.
• Calculate the sum of the three numbers using the + operator and store it in the sum variable.
• Use the cout statement to print the sum of the three numbers.
• The main function returns 0, which indicates the successful execution of the program.

Time complexity: O(1) Auxiliary Space: O(1) 

One problem, many solutions: The solution to an algorithm can be or cannot be more than one. It means that while implementing the algorithm, there can be more than one method to implement it. For example, in the above problem of adding 3 numbers, the sum can be calculated in many ways:

• Bit-wise operators

How to analyze an Algorithm? 

For a standard algorithm to be good, it must be efficient. Hence the efficiency of an algorithm must be checked and maintained. It can be in two stages:

1. Priori Analysis:

“Priori” means “before”. Hence Priori analysis means checking the algorithm before its implementation. In this, the algorithm is checked when it is written in the form of theoretical steps. This Efficiency of an algorithm is measured by assuming that all other factors, for example, processor speed, are constant and have no effect on the implementation. This is done usually by the algorithm designer. This analysis is independent of the type of hardware and language of the compiler. It gives the approximate answers for the complexity of the program.

2. Posterior Analysis:

“Posterior” means “after”. Hence Posterior analysis means checking the algorithm after its implementation. In this, the algorithm is checked by implementing it in any programming language and executing it. This analysis helps to get the actual and real analysis report about correctness(for every possible input/s if it shows/returns correct output or not), space required, time consumed, etc. That is, it is dependent on the language of the compiler and the type of hardware used.

What is Algorithm complexity and how to find it?

An algorithm is defined as complex based on the amount of Space and Time it consumes. Hence the Complexity of an algorithm refers to the measure of the time that it will need to execute and get the expected output, and the Space it will need to store all the data (input, temporary data, and output). Hence these two factors define the efficiency of an algorithm.  The two factors of Algorithm Complexity are:

• Time Factor : Time is measured by counting the number of key operations such as comparisons in the sorting algorithm.
• Space Factor : Space is measured by counting the maximum memory space required by the algorithm to run/execute.

Therefore the complexity of an algorithm can be divided into two types :

1. Space Complexity : The space complexity of an algorithm refers to the amount of memory required by the algorithm to store the variables and get the result. This can be for inputs, temporary operations, or outputs. 

How to calculate Space Complexity? The space complexity of an algorithm is calculated by determining the following 2 components:   

• Fixed Part: This refers to the space that is required by the algorithm. For example, input variables, output variables, program size, etc.
• Variable Part: This refers to the space that can be different based on the implementation of the algorithm. For example, temporary variables, dynamic memory allocation, recursion stack space, etc. Therefore Space complexity S(P) of any algorithm P is S(P) = C + SP(I) , where C is the fixed part and S(I) is the variable part of the algorithm, which depends on instance characteristic I.

Example: Consider the below algorithm for Linear Search

Step 1: START Step 2: Get n elements of the array in arr and the number to be searched in x Step 3: Start from the leftmost element of arr[] and one by one compare x with each element of arr[] Step 4: If x matches with an element, Print True. Step 5: If x doesn’t match with any of the elements, Print False. Step 6: END Here, There are 2 variables arr[], and x, where the arr[] is the variable part of n elements and x is the fixed part. Hence S(P) = 1+n. So, the space complexity depends on n(number of elements). Now, space depends on data types of given variables and constant types and it will be multiplied accordingly.

2. Time Complexity : The time complexity of an algorithm refers to the amount of time required by the algorithm to execute and get the result. This can be for normal operations, conditional if-else statements, loop statements, etc.

How to Calculate , Time Complexity? The time complexity of an algorithm is also calculated by determining the following 2 components: 

• Constant time part: Any instruction that is executed just once comes in this part. For example, input, output, if-else, switch, arithmetic operations, etc.

Example: In the algorithm of Linear Search above, the time complexity is calculated as follows:

Step 1: –Constant Time Step 2: — Variable Time (Taking n inputs) Step 3: –Variable Time (Till the length of the Array (n) or the index of the found element) Step 4: –Constant Time Step 5: –Constant Time Step 6: –Constant Time Hence, T(P) = 1 + n + n(1 + 1) + 1 = 2 + 3n, which can be said as T(n).

How to express an Algorithm?

• Natural Language:- Here we express the Algorithm in the natural English language. It is too hard to understand the algorithm from it.
• Flow Chart :- Here we express the Algorithm by making a graphical/pictorial representation of it. It is easier to understand than Natural Language.
• Pseudo Code :- Here we express the Algorithm in the form of annotations and informative text written in plain English which is very much similar to the real code but as it has no syntax like any of the programming languages, it can’t be compiled or interpreted by the computer. It is the best way to express an algorithm because it can be understood by even a layman with some school-level knowledge.

Solve DSA problems on GfG Practice.

• DSA in Java
• DSA in Python
• DSA in JavaScript

Please Login to comment...

• RishabhPrabhu
• pushpeshrajdx01
• AnubhavMittal
• kingrishabdugar
• shivanisinghss2110
• itskawal2000
• susobhanakhuli
• shraddhajain3131
• amartyaghoshgfg
• piyushsri20146
• utkarshstm2016
• newsandeepkushwaha
• laxmishinde5t82
• snehalmahasagar

Please write us at contrib[email protected] to report any issue with the above content

Improve your Coding Skills with Practice