When you arrive at a new company, you have that new starter energy.
You want to get your hands dirty, and start making change.
You are sure your knowledge and experience along with your fresh perspective can bring something to the team.
You are not wrong, but I urge you to keep it simple.
Keeping it simple at the start is a powerful tool.
It won’t stop you from bringing in change later, and it gives you time to focus on what is truly important: listening.
Each company is different, shaped by its people and the business context.
When you listen you give yourself a chance to learn.
Maybe they have tried your idea before, maybe you have to follow specific processes, maybe your process is there slightly hidden.
I did not keep it simple when I started, and I did not pay attention to the processes in place. Because of that, I made two mistakes. One where I added complexity, and one where I could not complete a small project, so I added noise.
Both could have been avoided.
On the complexity side, when I onboarded I looked at our coding environment and realized we were using Rye to manage our Python dependencies.
Well, the first thing you see when you try to get Rye is that you should use UV.
So, I made the change to UV, updated some onboarding docs, and tried to push the change through for the projects we were working on - it should have been trivial.
It was not, and it has not happened yet.
In my company, change requires a few steps, very much a different approach to my “startup-y” jobs.
We had a new joiner come in, and now his developer experience is less than ideal.
There are multiple ways to set up your environment - this needs fixing. Lesson learned.
Or at least I thought I had learned it.
I noticed some other inefficiencies in our documentation and making requests to our microservices.
I wanted to bring in some structure to our team, start a repository, host some HTTP requests collection, and then share that with the team.
Again, because I didn’t understand the limitations of the process of “doing new work”, I got stuck with an empty created repository, and no way to commit my collection of requests until I have a ticket assigned.
More noise, no benefit.
I could argue the system is broken, stopping me from innovating and iterating.
But the reality is that I need to learn the system, and then perform change from within.
Get the right allies, prove why this change is valuable, and then work on it.
If I want to simplify the system, I also need to do it from within the existing system.
So yeah, note for future self, listen, learn, change. Keep it simple at the start, and iterate.
The other day I was writing a python test, and I needed a temporary directory and file.
Normally, I make these tests happen with os and pathlib libraries.
It works well enough, and therefore I never had reason to change it.
However, this time I asked Claude for a quick test that I was going to edit after, and it showed me the tempfile
library.
This module creates temporary files and directories. It works on all supported platforms.
TemporaryFile, NamedTemporaryFile, TemporaryDirectory, and SpooledTemporaryFile are high-level interfaces which provide automatic cleanup and can be used as context managers. mkstemp() and mkdtemp() are lower-level functions which require manual cleanup.
You can do things like this:
with tempfile.NamedTemporaryFile(mode='w+', delete=True) as temp_file:
# You can write data to the temporary temp_file
# `data` can just be a string of your liking, like a fake CSV
temp_file.write(data)
print(f"Created temporary file: {temp_file.name}")
# You can then process the file via it's path
# or just process it's content
# ...
So practical! Quick tip of the day! Here’s a basic complete example:
import pandas as pd
import tempfile
import os
import unittest
def read_csv_to_dataframe(file_path):
if not os.path.exists(file_path):
raise FileNotFoundError(f"File not found: {file_path}")
return pd.read_csv(file_path)
class TestCSVReader(unittest.TestCase):
def test_read_csv_to_dataframe(self):
"""Test that CSV data is correctly loaded into a DataFrame."""
# Create sample CSV data
csv_data = """name,age,city
Alice,32,Seattle
Bob,45,Portland
Charlie,28,San Francisco"""
# Create a temporary CSV file
with tempfile.NamedTemporaryFile(mode='w+', suffix='.csv') as temp_file:
# Write the CSV data
temp_file.write(csv_data)
temp_file.flush()
temp_path = temp_file.name
# Test the function
df = read_csv_to_dataframe(temp_path)
# Verify the results
self.assertEqual(len(df), 3)
self.assertEqual(list(df.columns), ['name', 'age', 'city'])
self.assertEqual(df.iloc[0]['name'], 'Alice')
self.assertEqual(df.iloc[1]['age'], 45)
self.assertEqual(df.iloc[2]['city'], 'San Francisco')
if __name__ == "__main__":
unittest.main()
I have recently watched a video on “The 7 Myths of Win-Win Negotiations” by Professor Horacio Falcão. I have a lot to learn.
This lecture was recommended to me by a friend that had the pleasure of doing a workshop based on Prof. Falcão’s method.
Not only was my friend impressed by what he learned, he claimed that the course paid itself back after the first closed
contract.
More …
TL;DR: Docker and DBT starter template
I am currently revising some data modeling concepts, and in the course I am doing they use Pentaho.
However, the lecturer clearly states that Pentaho is just the tool, and the course is generic enough that you
can use any tool you want.
At Cable, we used DBT (data build tool) to build our data models and run them on BigQuery.
I became a big fan of DBT, and really wanted to use it during my course. So, to make my life easier, I created a starter template for Docker and DBT.
You can check out the GitHub repository here.
All files should be self-explanatory, but open an issue if you have any questions.
Motivation
- I did not want to setup a Python environment - I have been using Poetry and it feels clunky to me
- I already had Docker installed on my machine, so I thought that I could have a similar workflow to VSCode’s devcontainers
- I still wanted to use Zed as my editor - so I wanted to make sure that the changes in my host machine were reflected in the container
- I also wanted to be able to generate and access the docs that DBT creates
- And finally I knew I could just use docker-compose to run the dbt container and the Postgres container easily together
I hope this helps you get started with Docker and DBT.
Virtual coffees are a divisive topic. The opinions I have seen around me range from “yuck, no” to “they’re okay”.
Personally, they can be really great! I have had many fulfilling conversations, ranging from learning moments about other areas
of the company to deep, meaningful life conversations. This post serves to advocate for virtual coffee, to both managers and ICs,
and to share a few tips that hopefully will help make your one-on-one moments better.
More …