Public transportation project | Readings
28 August 2023
1 minutes to read
I would like to keep my interest in actually finishing the Spark book. However, I also got an idea to set up a bot to gather real-time data on Porto’s bus locations and compare it to the schedule. This would allow to actually to analyze some hypotheses like: 1. Buses that arrive early and don’t wait 2. Suppressed buses (they just vanish?) 3. Understand where buses take longer and where an optimization of the traffic would improve the timings
This sounds a bit daunting to be sincer but at least would allow me to:
- Test API knowledge
- Test streaming pipelines
- Test analytic skills
At the end of the day could start a conservation on how to improve public transportation.
Readings of the week
- Discover 3 advantages of Podman over Docker | Red Hat Developer
- Even Friendlier SQL with DuckDB - DuckDB
- Becoming A Better Data Engineer - Tips On Translating Business Requirements
- If you want to address tech debt, quantify it first - Stack Overflow Blog
- Python Type Hints - *args and **kwargs - Adam Johnson
I'm José Cabeda, a data engineer focused on improving data systems and educating on how to use them. I also do a lot of planning and read as much as I can.