Workshop

Memray 101: Demystifying memory profiling in Python

giovedì 23 maggio

11:00 - 13:05
StanzaMozzarella
LinguaInglese
Livello audienceIntermediate
Posti liberi1
    Elevator pitch

    Memray is an open source profiler that tracks memory allocations in Python code, native extension modules, and in the interpreter itself. Let us familiarise you with how Memray is used, the various tools it provides, and how you can utilise them to start analysing and debugging your own code today!

    Descrizione

    Memray is an open source profiler built in-house at and published by Bloomberg. It is used across the software industry, including within companies such as Google, Meta and Nvidia. It tracks memory allocations in Python code, native extension modules, and within the interpreter itself. Join us to learn the basics of how to use it through a series of hands-on exercises.

    Efficient memory usage is an essential component of building performance-optimal applications. As our programs grow, ensuring memory is used efficiently becomes increasingly difficult by hand. Memory profiling tools, such as Memray, can help uncover misused or leaking memory, leading to improvements in application performance.

    In this workshop, you will learn how to use Memray to generate memory profiling reports that trace every function call, so it accurately represents the call stack, unlike sampling profilers. Analyse allocations to help discover the cause of high memory usage, not only in Python but also in native-threads (e.g., C++ threads in native extensions). Become familiar with Memray’s pytest API, and how it can benefit code quality (like preventing future leaks) once it has been integrated into your Continuous Integration pipeline.

    This workshop will be broken down into three main parts, each delving deeper into memory management and various features of Memray. First, we’re going to familiarise ourselves with generating and reading a Memray flamegraph report and learn how to test for memory leaks using the Memray pytest API (Part 1). Then, we will solidify our knowledge by identifying and debugging a couple of sample memory problems, before delving into a more complex example. We’ll talk about Python’s garbage collector, and how it may interact with your code in unexpected ways (Part 2). Finally, we will learn to use Memray to debug issues in native C++ code, and learn how the in-built library ‘functools’ can be misused and cause serious memory issues (Part 3).

    Participants will analyse and understand Memray’s reports and take advantage of its powerful features. After this workshop, they will be empowered to utilise Memray in their own software stacks in order to identify and prevent memory-related issues.

    TagsOpen-Source, Tooling, Performance, Debugging
    Participant

    Gintare Statkute

    After completing her BSc in Computer Science, Gintarė Statkutė joined Bloomberg as a Software Engineer, first in the firm’s London office, and now in Lugano, Switzerland. Having worked on various risk-based products, Gintarė is currently building an ETL pipeline for Equity Data Acquisition at Bloomberg, which is primarily written in Python, and uses open source tools such as Apache Kafka, Apache Airflow, and RabbitMQ. She is also a part of Bloomberg’s internal Python Guild. Her responsibilities relate to company-wide Python training efforts, including regularly running this Memray workshop as a part of the Engineering department’s continuous education offerings.

    Participant

    Jesse Carnaxide

    Jesse Carnaxide is a full-stack developer with more than eight years of experience. He received a BEng and a MEng in Computer Engineering from the State University of New York (SUNY) at Binghamton. He works as a Senior Software Engineer at Bloomberg in New York City. He has worked on various products across the Bloomberg stack, most recently spending the past year on the Enterprise Portfolio Management Engineering team. He is also a leader in the company’s internal Python Guild. One of his favorite aspects of Python is the readability of the language, which is great for maintainability, as well as making it relatively easy to get familiar with.