First week of the semester | Readings
16 September 2023
2 minutes to read
For this week I can start to do an update status by each subject.
I’ve gotten a bit better working with Jest. I tried using Jest projects but found out there are a couple of options that aren’t available.
Gotten one LeetCode challenge done in both Typescript as well as Python. Hoping to decide on the third language on the next week.
I’ve mainly gotten to work on explore.porto.pt graphql API and found what might be the best endpoint to retrieve the info. To complete the first phase of the project, which it’s to have a deployed system that retrieves the data every x seconds and stores it on a DB.
I may start by writing the graphql reader in python and store it directly in Postgres. After getting this done as a PoC I’m thinking that I could set the realtime streaming by using Postgres CDC and writing again to the DB in a clean state. This because I want to test a complete streaming system with the current project.
No progress on this front. Planning to have a closed presentation on the Webdev for next Friday.
No progress on this front.
Readings of the week
On the reading I’ve been able to get a bunch of articles read. As I’ve switched to omnivore I’ve gotten a new deno script to output the read article as markdown and archive them (github.com/cabeda/pocket). I would like to also add a flag to get the full output with the notes and highlights to help me write down a bit more of my thoughts on each article 🤔
- PostgreSQL: PostgreSQL 16 Released!
- How Not to Die
- Deno vs. Bun vs. Node.js: A Feature Comparison
- Head-to-head comparison of 3 dbt SQL engines | by Niels Claeys | datamindedbe | Sep, 2023 | Medium by Niels Claeys
- You don’t need UUID | Henrique Vicente
- The Top Programming Languages 2023 - IEEE Spectrum by Stephen Cass
- You Want Modules, Not Microservices by Ted Neward
- Dev/Stage/Prod is the Wrong Pattern for Data Pipelines | Enigma by 2023-08-02
- How to Create CI/CD Pipelines for dbt Core | by Paul Fry | Aug, 2023 | Medium by Paul Fry
- Wes McKinney - The Road to Composable Data Systems: Thoughts on the Last 15 Years and the Future by Wes McKinney
I'm José Cabeda, a data engineer focused on improving data systems and educating on how to use them. I also do a lot of planning and read as much as I can.