Scraping the Dungeons. Find your perfect D&D game using Scrapy and Bokeh visualisation

sabato 25 maggio

14:45 - 15:15
Livello audienceIntermediate
Elevator pitch

Together we will explore the new horisons of D&D world. Where the BBEG is not always a lich or a dragon, but a lack of good company and a stable schedule. In this sesstion we will create a simple data-driven app, that will use power of Scrapy to extract the data from open web and Bokeh visualisation


๐Ÿ‰ Introduction to the D&D Gaming Landscape: Embark on a journey through the mystical world of Dungeons & Dragons (D&D). This introduction sets the stage ๐ŸŽญ for understanding the challenge: navigating a sea ๐ŸŒŠ of game data to find the perfect match for each player. Weโ€™ll discuss the need for an automated, user-friendly solution here.

๐Ÿ•ธ๏ธ Scrapy: The Data Harvesting Powerhouse: Dive deep into the capabilities of Scrapy, a leading open-source web crawling framework, perfect for extracting unstructured data. This segment covers the basics of setting up a crawler ๐Ÿค–, navigating through web pages, and extracting key game information. We emphasize ethical scraping practices to responsibly collect data ๐Ÿ›ก๏ธ.

๐Ÿงน Data Cleansing and Transformation: After gathering raw data, we move to cleansing and transforming it. This crucial phase involves techniques for structuring and preparing data for visualization, ensuring that our data is clean and insightful ๐ŸŒŸ.

๐ŸŽจ Bokeh: Illuminating Data with Interactive Visuals: The highlight of our tale is Bokeh, a Python library for creating interactive and engaging visualizations. Watch as we turn the D&D data into captivating, interactive plots and dashboards ๐Ÿ“Š, allowing users to explore and filter through games to find their ideal adventure. This part includes a live demonstration ๐Ÿ–ฅ๏ธ of a Bokeh application, brimming with interactive elements like graphs, sliders, and dropdowns.

๐ŸŒ Real-World Application and Outcomes: Although we do a small amount of data harvesting, enrichment and visualisation - we will see the powerfull and scalable base for a production-ready batteries-included solution for data-oriented businesses

๐Ÿ’ฌ Q&A and Interactive Session: Concluding with a Q&A session, this segment invites audience engagement, offering a chance to discuss and envision future advancements in web scraping in Python.

TagsBokeh, Best Practice, Distributed Systems, Architecture, Scaling

Viktor Zagranovskyy

My name is Vic Zagranowski. I am a seasoned Python developer specialising in developing the web scraping solutions and building an infrastructure around them. I have spearheaded a lot of successfull projects as well as completed multiple optimisations. My technical background and natural inquisitiveness allowed me to successfully reverse-engineer a lot of web communication mechanisms nad benefit buisinesses from it. Outside work, I cherish being a father, instilling values and sharing joys with my family. Iโ€™m also an avid Dungeons & Dragons enthusiast, enjoying the creativity and adventure it brings.