Lambdas and runs | Readings
23 October 2023
2 minutes to read
The delivery of a big initiative has brought a sense of productiveness. I’ll be grabbing the momentum with the hopes of delivering the next one up until the end of the year. If done with success and with a high degree of quality I’ll be considering this as done.
I’ve completely forgotten about the code challenges. I’ve been focused on the data stream project whenever I have some time and left this on hold. Between reading and the project, frankly this has been my last priority. I guess I could set an hour for doing this at the end of the day while I’m less productive…
I’ve been able to finally set the loader into parquet + S3 as my raws storage. The loader is running as a scheduled lambda which has been prettly reliable (get occasional errors but the lambda ignores them and keeps going on).
For this week I’m focusing down on making sure the data is accessible and start extracting the metrics needed for the dashboard. For the presentation I’m thinking on setting an instance of metabase/redash/superset on top of the duckdb to keep it simple.
I’ve done a small presentation internally but I’ve got 2 topics to talk about. One regarding the usage of duckdb and the other to propose more usage of dbt. I might be able to prepare the presentations this week between the effort to write down an ADR.
I’ve set my end of year goal. Do the traditional 10Km “corrida de São Silvestre” in Porto. For this I’m keeping my weekly training goals:
- 1 soccer game
- 1 run need to reach the 10Km by the end in the next 2 weeks
- 1 swimming training
I still need to find a routine for the latter but should be something feasible if I set my mind to it.
Readings of the week
I’ve increase my to read list from the 78 articles to 106. Not great, haven’t gotten around to read that much. I want to add more highlights
- Deno 1.37: Significant Developer-Focused Enhancements by Agazi Mekonnen
- Ruff v0.1.0
- Terraform 1.6 Makes Testing Framework Generally Available by Matt Campbell
- A student asked how I keep us innovative. I don’t. | nicole@web
- “Making Hard Things Easy” by Julia Evans (Strange Loop 2023) by Strange Loop Conference
- Lesser Known PostgreSQL Features | Haki Benita by Haki Benita
I'm José Cabeda, a data engineer focused on improving data systems and educating on how to use them. I also do a lot of planning and read as much as I can.