Together we will explore the new horisons of D&D world. Where the BBEG is not always a lich or a dragon, but a lack of good company and a stable schedule. In this sesstion we will create a simple data-driven app, that will use power of Scrapy to extract the data from open web and Bokeh visualisation
๐ Introduction to the D&D Gaming Landscape: Embark on a journey through the mystical world of Dungeons & Dragons (D&D). This introduction sets the stage ๐ญ for understanding the challenge: navigating a sea ๐ of game data to find the perfect match for each player. Weโll discuss the need for an automated, user-friendly solution here.
๐ธ๏ธ Scrapy: The Data Harvesting Powerhouse: Dive deep into the capabilities of Scrapy, a leading open-source web crawling framework, perfect for extracting unstructured data. This segment covers the basics of setting up a crawler ๐ค, navigating through web pages, and extracting key game information. We emphasize ethical scraping practices to responsibly collect data ๐ก๏ธ.
๐งน Data Cleansing and Transformation: After gathering raw data, we move to cleansing and transforming it. This crucial phase involves techniques for structuring and preparing data for visualization, ensuring that our data is clean and insightful ๐.
๐จ Bokeh: Illuminating Data with Interactive Visuals: The highlight of our tale is Bokeh, a Python library for creating interactive and engaging visualizations. Watch as we turn the D&D data into captivating, interactive plots and dashboards ๐, allowing users to explore and filter through games to find their ideal adventure. This part includes a live demonstration ๐ฅ๏ธ of a Bokeh application, brimming with interactive elements like graphs, sliders, and dropdowns.
๐ Real-World Application and Outcomes: Although we do a small amount of data harvesting, enrichment and visualisation - we will see the powerfull and scalable base for a production-ready batteries-included solution for data-oriented businesses
๐ฌ Q&A and Interactive Session: Concluding with a Q&A session, this segment invites audience engagement, offering a chance to discuss and envision future advancements in web scraping in Python.
My name is Vic Zagranowski. I am a seasoned Python developer specialising in developing the web scraping solutions and building an infrastructure around them. I have spearheaded a lot of successfull projects as well as completed multiple optimisations. My technical background and natural inquisitiveness allowed me to successfully reverse-engineer a lot of web communication mechanisms nad benefit buisinesses from it. Outside work, I cherish being a father, instilling values and sharing joys with my family. Iโm also an avid Dungeons & Dragons enthusiast, enjoying the creativity and adventure it brings.