I would like to keep my interest in actually finishing the Spark book. However, I also got an idea to set up a bot to gather real-time data on Porto’s bus locations and compare it to the schedule. This would allow to actually to analyze some hypotheses like: 1. Buses that arrive early and don’t wait 2. Suppressed buses (they just vanish?) 3. Understand where buses take longer and where an optimization of the traffic would improve the timings

This sounds a bit daunting to be sincer but at least would allow me to:

1. Test API knowledge
2. Test streaming pipelines
3. Test analytic skills

At the end of the day could start a conservation on how to improve public transportation.

Readings of the week

• Discover 3 advantages of Podman over Docker | Red Hat Developer
• Even Friendlier SQL with DuckDB - DuckDB
• Becoming A Better Data Engineer - Tips On Translating Business Requirements
• If you want to address tech debt, quantify it first - Stack Overflow Blog
• Python Type Hints - *args and **kwargs - Adam Johnson