An introduction to computational thinking, useful concepts in the field of computer science, and the art of computer programming using Python. Significant emphasis is placed on class meetings and learning to use computers to solve complex, real-world problems. Concepts and techniques are introduced as they are needed to help solve the problems confronting us. Students will learn how to go from an ambiguous problem description to a running solution and will leave the class knowing how to instruct computers to do what they want them to do. Prior experience in computer science or computer programming is not necessary.

This is COMPSCI 50, Harvard University's introduction to the intellectual enterprises of computer science and the art of programming, for concentrators and non-concentrators alike, with or without prior programming experience. (More than half of COMPSCI 50 students have never taken CS before!) This course teaches you how to solve problems, both with and without code, with an emphasis on correctness, design, and style. Topics include computational thinking, abstraction, algorithms, data structures, and computer science more generally. Problem sets inspired by the arts, humanities, social sciences, and sciences. More than teach you how to program in one language, this course teaches you how to program fundamentally and how to teach yourself new languages ultimately. The course starts with a traditional but omnipresent language called C that underlies today's newer languages, via which you'll learn not only about functions, variables, conditionals, loops, and more, but also about how computers themselves work underneath the hood, memory and all. The course then transitions to Python, a higher-level language that you'll understand all the more because of C. Toward term's end, the course introduces SQL, via which you can store data in databases, along with HTML, CSS, and JavaScript, via which you can create web and mobile apps alike. Course culminates in a final project. See https://cs50.harvard.edu/college for advice, FAQs, syllabus, and what's new. Email the course's heads at heads@cs50.harvard.edu with questions.

This is a project-based course emphasizing designing, building, testing, maintaining, and modifying software for scientific computing and data sciences. The class is focusing on a thorough introduction of the Python programming language with discussion of core concepts in object-oriented programming as well as essential data structures useful in most programming tasks. Students will work in groups on a semester long project. Students will further learn how to work with SQL databases and how to integrate them in Python using SQLite3 and Pandas. After completion of this course, students will be able to adapt basic tools and techniques to design complex software systems aimed at solving computational and data processing problems in academic and industrial environments.

Data Science 1 is the first half of a one-year introduction to data science. The course will focus on the analysis of messy, real life data to perform predictions using statistical and machine learning methods. Material covered will integrate the five key facets of an investigation using data: (1) data collection - data wrangling, cleaning, and sampling to get a suitable data set; (2) data management - accessing data quickly and reliably; (3) exploratory data analysis – generating hypotheses and building intuition; (4) prediction or statistical learning; and (5) communication – summarizing results through visualization, stories, and interpretable summaries. Part one of a two part series. The curriculum for this course builds throughout the academic year. Students are strongly encouraged to enroll in both the fall and spring course within the same academic year.

Data Science 2 is the second half of a one-year introduction to data science. Building upon the material in Data Science 1, the course introduces advanced methods for statistical modeling, representation, and prediction. Topics include multiple deep learning architectures such as CNNs, RNNs, transformers, language models, autoencoders, and generative models as well as basic Bayesian methods, and unsupervised learning. Students are strongly encouraged to enroll in both the fall and spring course within the same academic year. Part two of a two-part series.

Fundamental concepts in the design of computer programs, emphasizing the crucial role of abstraction. The goal of the course is to give students insight into the difference between programming and programming well. To emphasize the differing approaches to expressing programming solutions, you will learn to program in a variety of paradigms -- including functional, imperative, and object-oriented. Important ideas from software engineering and models of computation will inform these different views of programming.

Fundamentals of computer systems programming, machine organization, and performance tuning. This course provides a solid background in systems programming and a deep understanding of low-level machine organization and design. Topics include C and assembly language programming, program optimization, memory hierarchy and caching, virtual memory and dynamic memory management, concurrency, threads, and synchronization.

Implementation of efficient interpreters and compilers for programming languages. Associated algorithms and pragmatic issues. Emphasizes practical applications including those outside of programming languages proper. Also shows relationships to programming-language theory and design. Participants build a working compiler including lexical analysis, parsing, type checking, code generation, and register allocation. Exposure to run-time issues and optimization.

We are in the big data era and data systems sit in the critical path of everything we do. We are going through major transformations in businesses, sciences, as well as everyday life - collecting and analyzing data changes everything and data systems provide the means to store and analyze a massive amount of data. This course is a comprehensive introduction to modern data systems. The primary focus of the course is on the modern trends that are shaping the data management industry right now: column-store and hybrid systems, shared nothing architectures, cache conscious algorithms, hardware/software co-design, main-memory systems, adaptive indexing, stream processing, scientific data management, and key-value stores. We also study the history of data systems, traditional and seminal concepts and ideas such as the relational model, row-store database systems, optimization, indexing, concurrency control, recovery and SQL. In this way, we discuss both how and why data systems evolved over the years, as well as how these concepts apply today and how data systems might evolve in the future. We focus on understanding concepts and trends rather than specific techniques that will soon be outdated - as such the class relies largely on recent research material and on a semi-flipped class model with a lot of hands-on interaction in each class.

In this course, students learn critical techniques, concepts, and technologies for building usable interactive systems, alone and in pairs. Assignments provide hands-on experiences with different modern frameworks, platforms, and libraries while conceptual commonalities and distinctions are annotated and explained. Lectures cover relevant basic and advanced topics, such as human cognitive capabilities, iterative prototyping, and human-AI interaction. The final project will require both front-end and back-end development, iterative prototyping with humans, and a final evaluation with target users. Designed for advanced undergraduates.

How could it be that paving a new road might increase congestion for all drivers? Why would a professional sports team ever try not to score in a game that it wants to win? Why would any student rank high schools not in their order of preference when applying? And what are some incentive pitfalls that the designer of a cryptocurrency system should be aware of? In this course, we will examine seemingly strange social phenomena, use mathematical tools to model them and to analyze how and why distorted incentives give rise to them, and explore potential mechanisms to eliminate such phenomena.

This course examines the wide-ranging impact data science has on the world and how to think critically about issues of fairness, privacy, ethics, and bias while building algorithms and predictive models that get deployed in the form of products, policy and scientific research. Topics will include algorithmic accountability and discriminatory algorithms, black box algorithms, data privacy and security, ethical frameworks; and experimental and product design. We will work through case studies in a variety of contexts including media, tech and sharing economy platforms; medicine and public health; data science for social good, and politics. We will look at the underlying machine learning algorithms, statistical models, code and data. Threads of history, philosophy, business models and strategy; and regulatory and policy issues will be woven throughout the course.

This will be a graduate level course on recent advances and open questions in the foundations of machine learning and specifically deep learning. We will review both classical results as well as recent papers in areas including classifiers and generalization gaps, representation learning, generative models, adversarial robustness, out of distribution performance, and more. This is a fast-moving area and it will be a fast-moving course. We will aim to cover both state-of-art results, as well as the intellectual foundations for them, and have a substantive discussion on both the “big picture” and technical details of the papers. In addition to the theoretical lectures, the course will involve a programming component aiming to get students to the point where they can both reproduce results from papers and work on their own research. This component will be largely self-directed and we expect students to be proficient in Python and in picking up technologies and libraries such as pytorch/numpy/etc on their own (aka "Stack Overflow oriented programming").

Introduces essential elements the theory of error-correcting codes. Focuses on the basic results in the area, taught from first principles. Special focus will be given on results of asymptotic or algorithmic significance. Principal topics include: 1. Construction and existence results for error-correcting codes 2. Limitations on the combinatorial performance of error-correcting codes 3. Decoding algorithms 4. Applications to other areas of mathematics and computer science. Lecture notes for this course from previous offerings give further details on the material covered. These may be found at http://madhu.seas.harvard.edu/courses/Spring2020 Notes: Instructor permission needed to enroll. Students interested in the course should write to the instructor explaining what their background is in CS and Math, and explaining why they are interested in the course. Note that the course is an advanced grad course and students are expected to have a strong math background in addition to the CS courses listed in Recommended Prep.

In this course we explore advanced practical data science practices. The course will be divided into three major topics: 1. How to scale a model from a prototype to the cloud. 2. How to use existing models for transfer learning. 3. Introducing intuitive visualization tools for investigating properties and diagnosing issues of models.

Data Science 1 is the first half of a one-year introduction to data science. The course will focus on the analysis of messy, real life data to perform predictions using statistical and machine learning methods. Material covered will integrate the five key facets of an investigation using data: (1) data collection - data wrangling, cleaning, and sampling to get a suitable data set; (2) data management - accessing data quickly and reliably; (3) exploratory data analysis – generating hypotheses and building intuition; (4) prediction or statistical learning; and (5) communication – summarizing results through visualization, stories, and interpretable summaries. Part one of a two part series. The curriculum for this course builds throughout the academic year. Students are strongly encouraged to enroll in both the fall and spring course within the same academic year.

Data Science 2 is the second half of a one-year introduction to data science. Building upon the material in Data Science 1, the course introduces advanced methods for statistical modeling, representation, and prediction. Topics include multiple deep learning architectures such as CNNs, RNNs, transformers, language models, autoencoders, and generative models as well as basic Bayesian methods, and unsupervised learning. Students are strongly encouraged to enroll in both the fall and spring course within the same academic year. Part two of a two-part series.

Algorithms are mathematical objects with real life consequences. How do you say “fairness” and “privacy” in mathematics? How do existing theoretical computer science formulations mesh with legal privacy and nondiscrimination notions? Drawing on key concepts from differential privacy, the theory of algorithmic fairness, and crytography, the course focuses on the analysis and mitigation of privacy loss and unfairness in machine learning and data analysis. Through joint readings and weekly class meetings with the HLS course of the same name, students will develop disciplinary “bilingualism.”

The course examines the interplay between economic thinking and computational thinking as it relates to the design of online platforms and societal decision-making mechanisms. The focus is on fundamental concepts, modeling, and mathematical analysis. Topics covered include game theory, incentive alignment, matching, social choice, fair division and social networks. Special attention is given to ideas that draw on both disciplines, such as worst-case bounds on the inefficiency of equilibria, voting rules that are computationally hard to manipulate, and approximation algorithms that discourage strategic behavior.

An introduction to key design principles and techniques for visualizing data. Covers design practices, data and image models, visual perception, interaction principles, visualization tools, and applications. Introduces programming of web-based interactive visualizations.

This course covers the fundamentals of 3D computer graphics using a modern shader-based version of OpenGL. Main topics include: geometric coordinate systems and transformations, keyframe animation and interpolation, camera simulation, triangle rasterization, material simulation, texture mapping, image sampling and color theory. The course also touches on ray tracing, geometric modeling and simulation-based animation.

In this course, students learn critical techniques, concepts, and technologies for building usable interactive systems, alone and in pairs. Assignments provide hands-on experiences with different modern frameworks, platforms, and libraries while conceptual commonalities and distinctions are annotated and explained. Lectures cover relevant basic and advanced topics, such as human cognitive capabilities, iterative prototyping, and human-AI interaction. The final project will require both front-end and back-end development, iterative prototyping with humans, and a final evaluation with target users. Designed for advanced undergraduates.

Introduction to machine learning, providing a probabilistic view on artificial intelligence and reasoning under uncertainty. Topics include: supervised learning, ensemble methods and boosting, neural networks, support vector machines, kernel methods, clustering and unsupervised learning, maximum likelihood, graphical models, hidden Markov models, inference methods, and computational learning theory. Students should feel comfortable with multivariate calculus, linear algebra, probability theory, and complexity theory. Students will be required to produce non-trivial programs in Python.

Artificial Intelligence (AI) is already making a powerful impact on modern technology, and is expected to be even more transformative in the near future. The course introduces the ideas and techniques underlying this exciting field, with the goal of teaching students to identify effective representations and approaches for a wide variety of computational tasks. Topics covered in this course are broadly divided into search and planning, optimization and games, and uncertainty and learning. Special attention is given to ethical considerations in AI and to applications that benefit society. For more information please see the course website.

Modern AI systems often need the ability to make sequential decisions in an unknown, uncertain, possibly hostile environment, by actively interacting with the environment to collect relevant data. Reinforcement Learning (RL) is a general framework that can capture the interactive learning setting and has been used to design intelligent agents that achieve high-level performance in challenging applications such as Go, computer games, robotic manipulation, health care, and education. This course provides an introduction to reinforcement learning covering a range of problem formulations, algorithms, and theory. The four main themes of the course are (1) Markov decision processes (Bellman equations/optimality, planning, UCB, unknown environments, linear quadratic control, exploration, imitation learning), (2) bandits (epsilon-greedy, UCB, Thompson sampling, contextual bandits, linear bandits, exploration in MDPs), and (3) methods for large-scale systems (policy gradient methods, deep RL, Monte Carlo tree search, Q-learning). There will also be an Embedded Ethics lecture on ethical issues arising in reinforcement learning. The assignments will focus on a mix of algorithmic and statistical principles, along with their programming implementations.

Natural-language-processing applications are ubiquitous: Alexa can set a reminder, or play a particular song, or provide your local weather if you ask; Google Translate can make documents readable across languages; ChatGPT can be prompted to generate convincingly fluent text, which is often even correct. How do such systems work? This course provides an introduction to the field of computational linguistics, the study of human language using the tools and techniques of computer science, with applications to a variety of natural-language-processing problems such as these. You will work with ideas from linguistics, statistical modeling, machine learning, and neural networks, with emphasis on their application, limitations, and implications. The course is lab- and project-based, primarily in small teams, and culminates in the building and testing of a question-answering system.

Learn the practical skills required for applied deep learning work, including hands on experience with method development, model training at scale, and error analysis. You will learn the technical writing skills required for applied AI research, including experience composing different elements of a full research paper. Through structured assignments, you will tackle a scoped-out research project in a small team from conception to co-authoring a manuscript.

The risks to privacy when making human subjects data available for research and how to protect against these risks using the formal framework of differential privacy. Methods for attacking statistical data releases, the mathematics of and software implementations of differential privacy, deployed solutions in industry and government. Assignments will include implementation and experimentation on data science tasks.

Differential Privacy is a mathematically rigorous definition of privacy that has become the de facto standard for statistical analysis of large datasets. Differential privacy provides a concrete measure of privacy loss, and differentially private algorithms are equipped with a parameter for controlling this loss. A signal property of differential privacy is closure under composition, meaning that we can understand and control the cumulative privacy loss as the data are subjected to multiple analyses. In consequence, differential privacy is 'programmable:' one can combine simple differentially private computational primitives in creative ways to obtain privacy-preserving algorithms for complex analytical tasks. The course will cover (1) the basics of differential privacy: the definition and its properties, computational primitives, and composition theorems; (2) selected advanced differentially private algorithms drawn from the literature and a wide range of application areas from industry to the US Census; and (3) lower bounds: limits on what differential privacy – or any privacy technology – can offer.

How can we use tools from statistical learning theory to design better auctions? Can we use cryptography to better implement matching mechanisms? And how should we approach formally proving that welfare in Nash equilibria for many games is not 'much worse' than in the social optimum? This course explores the application of diverse ideas, techniques, and solution aesthetics from theoretical computer science to derive meaningful new insights into classic economic problems. The three main themes are approximation theorems (including bounding the loss in revenue or welfare due to lack of information, to strategic behavior, or to impracticality of the optimal mechanism); various notions of complexity (including computational complexity, communication complexity, and sample complexity); and cryptographic tools (including cryptographic commitments, multiparty computation, and zero-knowledge proofs). Economic applications mostly include analysis of equilibria, pricing, and mechanism design.

The course examines the mathematical and algorithmic foundations of democracy, running the gamut from theory to applications. The goal is to provide students with a rigorous perspective on, and a technical toolbox for, the design of better democratic systems. Topics include computational social choice (identifying optimal voting rules), fairness in political redistricting (avoiding gerrymandering) and apportionment (allocating seats on a representative body), sortition (randomly selecting citizens assemblies), liquid democracy (transitively delegating votes), and weighted voting games (analyzing legislative power through cooperative game theory).

Tiny machine learning (TinyML) is defined as a fast-growing field of machine learning technologies and applications including hardware (dedicated integrated circuits), algorithms and software capable of performing on-device sensor (vision, audio, IMU, biomedical, etc.) data analytics at extremely low power, typically in the mW range and below, and hence enabling a variety of always-on use-cases and targeting battery-operated devices. The pervasiveness of ultra-low-power embedded devices, coupled with the introduction of embedded machine learning frameworks like TensorFlow Lite for Microcontrollers, will enable the mass proliferation of AI-powered IoT devices. The explosive growth in machine learning and the ease of use of platforms like TensorFlow (TF) make it an indispensable topic of study for modern computer science and electrical engineering students.

Advanced statistical machine learning and probabilistic data analysis. Covers discrete and continuous probabilistic modeling and computational inference. Topics include: Bayesian modeling, probabilistic graphical models, latent variables and unsupervised learning, deep learning, time series models, variational inference, and sampling. Requires a final project.

As machine learning models are increasingly being employed to aid decision makers in high-stakes settings such as healthcare, finance, and law, it becomes important to ensure that the decision makers correctly understand and consequent trust the functionality of these models. This graduate level course aims to familiarize students with the recent advances in the emerging field of explainable ML. In this course, we will review seminal position papers of the field, understand the notion of model interpretability from the perspective of various kinds of end users (e.g., doctors, ML researchers/engineers), discuss in detail different classes of interpretable models and model explanations (e.g., case/prototype based approaches, sparse linear models, rule-based techniques, saliency maps, generalized additive models, and counterfactual explanations), and explore the connections between model interpretability and fairness, robustness, and privacy. The course will also emphasize the unique opportunities and challenges that arise when interpreting and explaining the functionality of foundation models (e.g., language models such as GPT-3, diffusion models such as DALLE etc.)The course will comprise of a mix of lectures by the instructor, paper presentations by students, and guest lectures by researchers who have authored seminal papers in this area. This course has a significant research component in the form of a course project that students will be expected to work on throughout the semester. All in all, this course is geared towards those students who are interested in diving deep and conducting research in the field of interpretable and explainable ML.

There has been growing interest in recent years for machine learning systems that are somehow transparent about their inner workings -- whether it be that the entire system is inherently interpretable, or that a single decision can somehow be explained. However, the question of what approach is best for what context remains elusive. In this course, we will focus on computational properties of interpretable machine learning methods, such as faithfulness or stability. Assessing methods with respect to these properties may allow us to rule out poorly-performing approaches without the need for expensive user studies. By categorizing methods by their computational properties, we will also be able to start thinking about which methods might be useful for a specific context. After a few initial assignments, the course will be focused on reading papers, discussion, and a semester-long project.

Vision as an ill-posed inverse problem: image formation, two-dimensional signal processing; feature analysis; image segmentation; color, texture, and shading; multiple-view geometry; object and scene recognition; and applications.

The ability to connect devices over long distances, via the internet, changed our world. The second phase of this revolution, that we are still living in today, came about when these devices became wireless. Now we are at the cusp of a new phase of this evolution where devices are connected, wireless, and controlled – i.e. the robot revolution.Multi-robot systems are becoming more pervasive; from future autonomous vehicle fleets, to drones, to manufacturing robots. As a result, the question of how to control, coordinate, and secure these systems has been a growing topic in the robotics literature in recent years. In this seminar-style course we will do a deep dive into this topic by reviewing classic and recent results in multi-agent planning and control literature. We will cover a wide gamut of applications from control of groups of flying drones, to decision making in autonomous car networks, to space exploring CubeSats.This class will treat both the theory and the practical applications behind multi-robot systems. Students with mathematical inclinations and exposure to graph theory, probability theory, linear algebra, and algorithms will derive the most benefit from this course.

Recent years have seen AI successfully applied to societal challenge problems. Indeed, recognizing the potential of AI for tremendous social impact in the future, 'AI for social impact' is growing as a subdiscipline within AI. In this course, we will discuss successful case studies of use of AI for public health, environmental sustainability, public safety and public welfare. Simultaneously, we will discuss key foundations of the area of AI for social impact. To that end, among other topics, we will focus on challenges in AI for Social Impact, what makes projects successful, how to investigate project impact in the field and ethical considerations for such projects. A key part of this course will be AI4SI projects with non-profits.

Surveys biologically-inspired approaches to designing distributed systems. Focus is on biological models, algorithms, and programming paradigms for self-organization. Topics vary year to year, and usually include: (1) swarm intelligence: social insects and animal groups, with applications to networking and robotics, (2) cellular computing: including cellular automata/amorphous computing, and applications like self-assembling robots and programmable materials, (3) evolutionary computation and its application to optimization and design.

Introduces computational approaches to natural language processing and acquisition by humans and machines, combining symbolic and probabilistic modeling techniques. Covers models such as n-grams, finite state automata, and context-free and mildly context-sensitive grammars, for analyzing phonology, morphology, syntax, semantics, pragmatics, and larger document structure. Applications range from accurate document classification and sentence parsing by machine to modeling human language acquisition and real-time understanding. Covers both theory and contemporary computational tools and datasets. Students taking graduate version complete additional assignments.

Introduces efficient deep learning computing techniques that enable powerful deep learning applications on resource-constrained devices. Topics include model compression, pruning, quantization, neural architecture search, distributed training, data/model parallelism, gradient compression, on-device fine-tuning. It also introduces application-specific acceleration techniques for video recognition, point cloud, and generative AI (diffusion model, LLM). Students will get hands-on experience accelerating deep learning applications with an open-ended design project.

Principles, techniques, and algorithms in machine learning from the point of view of statistical inference; representation, generalization, and model selection; and methods such as linear/additive models, active learning, boosting, support vector machines, non-parametric Bayesian methods, hidden Markov models, Bayesian networks, and convolutional and recurrent neural networks.

Provides an in-depth view of the state-of-the-art learning methods for control and the know-how of applying these techniques. Topics span reinforcement learning, self-supervised learning, imitation learning, model-based learning, and advanced deep learning architectures, and specific machine learning challenges unique to building sensorimotor systems. Discusses how to identify if learning-based control can help solve a particular problem, how to formulate the problem in the learning framework, and what algorithm to use. Applications of algorithms in robotics, logistics, recommendation systems, playing games, and other control domains covered. Instruction involves two lectures a week, practical experience through exercises, discussion of current research directions, and a group project.

Introduces the study of human language from a computational perspective, including syntactic, semantic and discourse processing models. Emphasizes machine learning methods and algorithms. Uses these methods and models in applications such as syntactic parsing, information extraction, statistical machine translation, dialogue systems. Instruction and practice in oral and written communication provided. Students taking graduate version complete additional assignments.

Abstracting the essential components and mechanisms from a natural system to produce a mathematical model, which can be analyzed with a variety of formal mathematical methods, is perhaps the most important, but least understood, task in applied mathematics. This course approaches a number of problems without the prejudice of trying to apply a particular method of solution. Topics drawn from biology, economics, engineering, physical and social sciences.

An introductory course in unsupervised learning with an emphasis on dimensionality reduction and clustering. Topics include principal component analysis, nonnegative matrix factorization, and spectral clustering. In this course we will study these techniques and others with a focus on high-dimensional geometry and insights provided by linear algebra. Numerous data example will be included throughout the course.

An introduction to the mathematical, optical, and computational foundations of computer vision, with a focus on applications in augmented reality and robotic perception. Topics include: camera optics, digital color photography pipelines, multi-camera geometry, image processing and manipulation, simultaneous localization and mapping, lighting and material estimation, and 3D scanning. Emphasis on combining mathematical modeling with robust algorithms for solving ill-posed problems.

How can we use tools from machine learning theory to design better auctions? Can we use cryptography to better implement matching mechanisms? And how should we approach formally proving that welfare in Nash equilibria for many games is not 'much worse' than in the social optimum? This course explores the application of diverse ideas, techniques, and solution aesthetics from theoretical computer science to derive meaningful new insights into classic economic problems. The three main themes are approximation theorems (including bounding the loss in revenue or welfare due to lack of information, to unpriced externalities, or to restricting the functional form of mechanisms); various notions of complexity (including computational complexity, communication complexity, and machine-learning sample complexity); and cryptographic tools (including cryptographic commitments, multiparty computation, and zero-knowledge proofs). Economic applications mostly include analysis of equilibria, pricing, and mechanism design. No prior knowledge in computer science is required.

Analyzes recorded music in digital audio form using advanced signal processing and optimization techniques to understand higher-level musical meaning. Covers fundamental tools like windowing, feature extraction, discrete and short-time Fourier transforms, chromagrams, and onset detection. Addresses analysis methods including dynamic time warping, dynamic programming, self-similarity matrices, and matrix factorization. Explores a variety of applications, such as event classification, audio alignment, chord recognition, structural analysis, tempo and beat tracking, content-based audio retrieval, and audio decomposition. Students taking graduate version complete different assignments.

Fundamentals of computer systems programming, machine organization, and performance tuning. This course provides a solid background in systems programming and a deep understanding of low-level machine organization and design. Topics include C and assembly language programming, program optimization, memory hierarchy and caching, virtual memory and dynamic memory management, concurrency, threads, and synchronization.

This course delves into the design principles and practices of high performance digital computing systems that are cost effectively and reliably manufactured with billions of near atomic scale semiconductor components. Key abstractions and foundational concepts are emphasized as the course covers the basic operation of CMOS transistors and logic gates, combinational and sequential logic including Finite State Machines (FSMs), digital memory subsystems, and machine code culminating with the implementation of a MIPS processor. Lab assignments will focus on the practical aspects of digital hardware design by utilizing Field Programmable Gate Arrays (FPGAs), Verliog (Hardware Description Language) and advanced CAD tools for the design, simulation and verification of digital computing hardware.

Review of the fundamental structures in modern processor design. Topics include computer organization, memory system design, pipelining, and other techniques to exploit parallelism. Discussion of modern topics including GPU architectures, datacenter architecture, mobile/embedded SoC architectures, and machine learning acceleration as time permits. Emphasis on a quantitative evaluation of design alternatives and an understanding of performance and energy consumption issues.

This course studies computer network topics including Layer 2/Layer 3 topology, routing, transport protocols, traffic engineering, network functions, programmable switches, and software-defined networking. Modern networks have grown to large scale (connecting millions of servers) and high speed (terabits per second) to meet the needs of cloud applications in business and society. Thus, in addition to learning the conventional concepts in networking, we will also discuss how to adapt these concepts to large-scale networks. These discussions will hopefully help deepen our understanding of networking technologies. This course includes lectures and system programming projects.

Implementation of efficient interpreters and compilers for programming languages. Associated algorithms and pragmatic issues. Emphasizes practical applications including those outside of programming languages proper. Also shows relationships to programming-language theory and design. Participants build a working compiler including lexical analysis, parsing, type checking, code generation, and register allocation. Exposure to run-time issues and optimization.

This course focuses on the design and implementation of modern operating systems. The course discusses threads, processes, virtual memory, schedulers, and the other fundamental primitives that an OS uses to represent active computations. An exploration of the system call interface explains how applications interact with hardware and other programs which are concurrently executing. Case studies of popular file systems reveal how an OS makes IO efficient and robust in the midst of crashes and unexpected reboots. Students also learn how virtualization allows a physical machine to partition its resources across multiple virtual machines. Class topics are reinforced through a series of intensive programming assignments which use a real operating system.

We are in the big data era and data systems sit in the critical path of everything we do. We are going through major transformations in businesses, sciences, as well as everyday life - collecting and analyzing data changes everything and data systems provide the means to store and analyze a massive amount of data. This course is a comprehensive introduction to modern data systems. The primary focus of the course is on the modern trends that are shaping the data management industry right now: column-store and hybrid systems, shared nothing architectures, cache conscious algorithms, hardware/software co-design, main-memory systems, adaptive indexing, stream processing, scientific data management, and key-value stores. We also study the history of data systems, traditional and seminal concepts and ideas such as the relational model, row-store database systems, optimization, indexing, concurrency control, recovery and SQL. In this way, we discuss both how and why data systems evolved over the years, as well as how these concepts apply today and how data systems might evolve in the future. We focus on understanding concepts and trends rather than specific techniques that will soon be outdated - as such the class relies largely on recent research material and on a semi-flipped class model with a lot of hands-on interaction in each class.

Review of the fundamental structures in modern processor design. Topics include computer organization, memory system design, pipelining, and other techniques to exploit parallelism. Discussion of modern topics including GPU architectures, datacenter architecture, mobile/embedded SoC architectures, and machine learning acceleration as time permits. Emphasis on a quantitative evaluation of design alternatives and an understanding of performance and energy consumption issues.

Specialized AI accelerators enable efficient AI computations for a variety of tasks at various scales using a wide range of parallel, distributed, and embedded computing platforms. For example, in generative AI applications such as ChatGPT and Stable Diffusion, these accelerators allow for (1) distributed model training and low-latency, high-throughput inference serving in the cloud, and (2) efficient private training and inference using local knowledge on resource-constrained edge devices. In this course, students will learn systematic methods for implementing parallel computations for computer vision and language models on numerous computing cores or nodes. They will also learn techniques for co-designing machine learning models, data curation methods, computing algorithms, and system architectures. Upon successful completion of this course, students will be equipped to tackle the challenging tasks of designing and utilizing energy-efficient, high-performance AI accelerators.

This is a graduate-level course on computer networks. This course offers an in-depth exploration of a subset of advanced topics in networked systems. We will discuss the latest developments in the entire networking stack, the interactions between networks and high-level applications, and their connections with other system components such as compute and storage. In this year's edition, we will use machine learning as a prime example to understand its unique requirements and challenges in the context of networking. As machine learning applications increasingly rely on larger models and faster accelerators, the demand for enhanced networking capabilities becomes imperative. Throughout this course, we will study cutting edge networking solutions and principles for co-designing networks with compute and storage, to meet the evolving needs of machine learning applications. The course will include lectures, in-class presentations, paper discussions, and a research project.

Seminar course exploring recent research in computer architecture. Topics vary from year to year and will include subjects such as multi-core architectures, energy-efficient computing, reliable computing, and the interactions of these issues with system software. Students read and present research papers, undertake a research project.

Tiny machine learning (TinyML) is defined as a fast-growing field of machine learning technologies and applications including hardware (dedicated integrated circuits), algorithms and software capable of performing on-device sensor (vision, audio, IMU, biomedical, etc.) data analytics at extremely low power, typically in the mW range and below, and hence enabling a variety of always-on use-cases and targeting battery-operated devices. The pervasiveness of ultra-low-power embedded devices, coupled with the introduction of embedded machine learning frameworks like TensorFlow Lite for Microcontrollers, will enable the mass proliferation of AI-powered IoT devices. The explosive growth in machine learning and the ease of use of platforms like TensorFlow (TF) make it an indispensable topic of study for modern computer science and electrical engineering students.

An introduction to operating systems research. Paper-based seminar course that introduces students to the state of the art in systems research through historical and quantitative lenses. Students will read and discuss research papers and complete a final research project.

An examination of the special problems associated with distributed computing such as partial failure, lack of global knowledge, asynchrony and coordination of time, and protocols that function in the face of these problems. Emphasis on both the theory that grounds thinking about these systems and in the ways to design and build such systems.

This course explores practical attacks on modern computer systems, explaining how those attacks can be mitigated using careful system design and the judicious application of cryptography. The course discusses topics like buffer overflows, web security, information flow control, and anonymous communication mechanisms such as Tor. The course includes several small projects which give students hands-on experience with various offensive and defensive techniques; the final, larger project is open-ended and driven by student interests.

Big data is everywhere. A fundamental goal across numerous modern businesses and sciences is to be able to utilize as many machines as possible, to consume as much information as possible and as fast as possible. The big challenge is how to turn data into useful knowledge. This is a moving target as both the underlying hardware and our ability to collect data evolve. In this class, we discuss how to design data systems, data structures, and algorithms for key data-driven areas, including relational systems, distributed systems, graph systems, noSQL, newSQL, machine learning, and neural networks. We see how they all rely on the same set of very basic concepts and we learn how to synthesize efficient solutions for any problem across these areas using those basic concepts.

Project-based introduction to building efficient, high-performance and scalable software systems. Topics include performance analysis, algorithmic techniques for high performance, instruction-level optimizations, vectorization, cache and memory hierarchy optimization, and parallel programming.

Design and implementation of secure computer systems. Lectures cover attacks that compromise security as well as techniques for achieving security, based on recent research papers. Topics include operating system security, privilege separation, capabilities, language-based security, cryptographic network protocols, trusted hardware, and security in web applications and mobile phones. Labs involve implementing and compromising a web application that sandboxes arbitrary code, and a group final project.

A lab-intensive introduction to digital electronic circuit design. Develops circuit intuition and debugging skills through hands-on lab exercises, each preceded by class discussion, with minimal use of mathematics and physics. After a short introduction to the basics of electronic circuits and MOSFET switches, we move onto digital devices including logic families, Boolean arithmetic, combinatorial and sequential circuits including finite state machines. We continue with analog-digital interfacing, the use of microcontrollers in embedded systems and programmable logic devices (FPGAs). We will also discuss data conversion techniques

An introductory course in theoretical computer science, aimed at giving students the power of using mathematical abstraction and rigorous proof to understand computation. Thus equipped, students will be able to design and use algorithms that apply to a wide variety of computational problems, with confidence about their correctness and efficiency, as well as recognize when a problem may have no algorithmic solution. At the same time, they will gain an appreciation for the beautiful mathematical theory of computation that is independent of (indeed, predates) the technology on which it is implemented.

Computation occurs over a variety of substrates including silicon, neurons, DNA, the stock market, bee colonies and many others. In this course we will study the fundamental capabilities and limitations of computation, including the phenomenon of universality and the duality of code and data. Some of the questions we will touch upon include: Are there functions that cannot be computed? Are there true mathematical statements that can't be proven? Are there encryption schemes that can't be broken? Is randomness ever useful for computing? Can we use the quirks of quantum mechanics to speed up computation?

A quantitative theory of the resources needed for computing and the impediments to efficient computation. The models of computation considered include ones that are finite or infinite, deterministic, randomized, quantum or nondeterministic, discrete or algebraic, sequential or parallel.

Possibilities of and limitations to performing learning by a computational process. Computationally feasible generalization and its limits. Topics include computational models of learning, polynomial time learnability, learning from examples and from queries to oracles. Applications to Boolean functions, languages and geometric functions. Darwinian evolution as learning.

Current research topics in computational complexity theory. Nondeterministic, alternating, probabilistic, and parallel computation models. Boolean circuits. Complexity classes and complete sets. The polynomial-time hierarchy. Interactive proof systems. Relativization. Definitions of randomness. Pseudo-randomness and derandomizations. Interactive proof systems and probabilistically checkable proofs.

An introductory course in theoretical computer science, aimed at giving students the power of using mathematical abstraction and rigorous proof to understand computation. Thus equipped, students will be able to design and use algorithms that apply to a wide variety of computational problems, with confidence about their correctness and efficiency, as well as recognize when a problem may have no algorithmic solution. At the same time, they will gain an appreciation for the beautiful mathematical theory of computation that is independent of (indeed, predates) the technology on which it is implemented.

Design and analysis of efficient algorithms and data structures. Algorithm design methods, graph algorithms, approximation algorithms, and randomized algorithms are covered.

Covers topics related to algorithms for big data, especially related to networks and database systems. Themes include sketch-based data structures, compression, graph and link information, and information theory. Requires a major final research-based project.

Probabilistic techniques and tools for the design and analysis of algorithms. Designed for all first-year graduate students in all areas.

This is a graduate topics class on algorithmic challenges in modern machine learning and data science. We will touch upon a number of domains (generative modeling, deep learning theory, robust statistics, Bayesian inference) and frameworks for algorithm design (spectral/tensor methods, moment methods, message passing, diffusions), focusing on provable guarantees. The theory draws upon a range of techniques from stochastic calculus, harmonic analysis, statistical physics, algebra, and beyond. We will also explore the myriad modeling challenges in building this theory and prominent paradigms (semi-random models, smoothed complexity, oracles) for going beyond traditional worst-case analysis.

First-year graduate subject in algorithms. Emphasizes fundamental algorithms and advanced methods of algorithmic design, analysis, and implementation. Surveys a variety of computational models and the algorithms for them. Data structures, network flows, linear programming, computational geometry, approximation algorithms, online algorithms, parallel algorithms, external memory, streaming algorithms.