Posts on Pablo Lopez - Software Engineer & AI Contractor

Lessons Learned from Building Data APIs

Sun, 05 Jul 2026 10:41:15 +0100

Over the last 3 years I’ve spent a lot of time building highly scalable Data APIs, to enable teams to access data from different sources with minimal effort. There have been several mistakes made along the way, and many lessons learned. I believe these lessons will form strong foundations to build performant Data APIs from day one.

Before diving into the lessons, I’ll define Data API.

A Data API is an API service that serves some data to the user. The API backend connects to a database, such as Postgres, Redis or MongoDB, it runs a query based on the API call received, and it serves that data to the end user.

Where is our leisure time now?

Thu, 09 Apr 2026 11:56:05 +0100

Over the past few months I’ve been using AI agents a lot to automate many engineering tasks that otherwise would take me much longer to do.

I’m a firm believer that the era of writing code by hand has finished. With the right workflows these tools can do a lot of the work for us. AI is improving our productivity, and it will continue more and more as the tools mature.

In the Age of coding agents local development is King

Thu, 26 Feb 2026 08:27:51 +0000

Coding agents work best when they have a good feedback loop they can use to verify their changes. Without a good local environment and feedback loop, agentic development quickly becomes a waste of time.

My first experience with software development was javascript, building websites using frameworks like React. I loved my development setup back then. I would always have a local server, automatically refreshing on every change. If anything broke, I would instantly find out and fix the issue right away. I would also have my unit tests running constantly catching bugs in near real time.

What do Postgres Tables Actually look like?

Wed, 11 Feb 2026 19:49:37 +0100

If you were to peek inside Postgres and see how the tables are stored, you’d be surprised they look nothing like the outuput you get when you do a SELECT query.

In this post I look inside Postgres to understand how tables are stored in the filesystem. The goal of this post is nothing more than to understand a small part of the whole Postgres ecosystem to demystify things a bit. It is a step by step process to find out where tables actually live in the filesystem and their structure.

Things I've learned in 2025

Thu, 27 Nov 2025 08:15:55 +0000

This has been one of the years where I’ve learned the most things, probably since I first started learning programming almost 10 years ago. I thought it would be a great exercise to put it into writing to reflect on it all.

The Dvorak keyboard layout

The first thing I set out to learn in 2025 was how to type using the Dvorak keyboard layout. The main reason was simply because it sounded like a fun thing to do. I still remember how much fun it was to learn QWERTY in school, going through all of those timed exercises. This time around it wasn’t as fun, and to be honest, it was very frustrating at times, especially when my brain completely forgot how to QWERTY and I was stuck typing Dvorak full time at very low speeds.

Preparing for the AWS Architect Professional Exam

Wed, 12 Nov 2025 12:06:38 +0000

I have passed the AWS Architect Professional exam with a score of 791/1000!

Preparing for the AWS exam

I have recently completed the AWS Architect Associate Exam, and I thought the Professional would be a nice next step considering all the prep work I had done. I took about three months between passing the Associate and taking the Professional exam.

For the Associate exam I spent a lot of hours each day going through the course content, writing notes and doing the practice exams multiple times, this time I didn’t have the luxury of time. I had to do all of the studying during work hours, where I could only spend about an hour a day (sometimes more, sometimes less), 4 days a week.

Asking the question gives me the answer

Mon, 03 Nov 2025 14:17:51 +0000

Today I wanted to write about something that happens to me at work all the time: the process of asking for technical help, helps me discover the answer to my question.

For example, today I was stuck on some terraform config that wasn’t showing some of the changes I was expecting to see in the diff. I decided to start writing a message in a support channel. As I was writing the message I started thinking about whether I had covered all the bases of things that I could check on before bothering someone to help me on something I could have done myself. At that point, it occured to me that I hadn’t checked a specific file that might have some releveant information. So, I deleted my half typed message, checked the file, and voila! The reason my config wasn’t applying was because I forgot to explicitly add some variables.

Slowly Changing Dimensions

Thu, 30 Oct 2025 08:11:11 +0000

Slowly changing dimensions, what a confusing name. This is the sort of term I’ve been hearing for a while now, but never really bothered to properly look up what it actually means. As it turns out, all it means is how you handle row updates/deletes in your data pipelines. Whether you decide to maintain historical data in your tables by appending new rows when a change happens, or you decide to update in place instead.

Interviewing Makes you More Competent at your Job

Mon, 06 Oct 2025 08:01:22 +0100

Interviewing for other jobs in tech can make you a more competent engineer. My premise for this argument is that when you prepare an interview for a new role you need to prepare to answer questions, technical and non-technical, in a domain that is different from your day to day work. This forces you to step out of your daily work and read and think about different domains and technologies. During the interview process you will get an idea of how well you performed, whether that is via self reflection, or feedback from the emloyer. This gives you a pretty good idea of where you are lacking knowledge and a good direction of where to focus next.

Replicas vs Standby

Sun, 05 Oct 2025 10:11:56 +0100

When working with distributed systems you can encounter terminology such as “replicas” or “standby” intstances.

When I first started learning about distrubuted systems, I assumed “replicas” and “standby” were the same thing, but called differently when talking about scaling or high availability. While they can be the same thing, that is not always the case, so I wanted to clarify the concepts in this post.

Replicas

Replicas are one or more exact copies of the primary instance of your database. Their main purpose is to balance the load.

Don't let Serverless Costs Give You a Heart Attack

Thu, 18 Sep 2025 20:04:11 +0100

Practical strategies to protect your serverless applications from runaway AWS bills, including caching, rate limiting, WAF rules, and DynamoDB safeguards.

From LLM to Agent

Tue, 02 Sep 2025 09:55:49 +0100

After ignoring the whole “agent” hype for a while, I started to get that inevitable feeling of being left behind.

That changed when I came across this blog post explaining the basic flow of an agent. It made me want to try it for myself and see what it actually makes an agent.

This post is my walkthrough of that process: step by step, showing how to turn an LLM into an agent. The code here is mainly illustrative, meant to highlight the concepts rather than provide a fully working solution.

Preparing for the AWS Certified Solutions Architect – Associate Exam

Mon, 01 Sep 2025 08:34:53 +0100

I recently passed the AWS Solutions Architect Associate exam with a score of 848.

This certification covers a wide range of AWS services which has taken a good amount of work and effort to prepare.

In this blog post I wanted to share my experience preparing for the exam, sharing some info about the exam, and what has worked for me.

Preparation

I have about five years of on-and-off AWS experience, mostly with RDS, CloudFormation, Lambda, EC2 and a few other services.

Building a GraphQL API with AWS AppSync and PostgreSQL Aurora

Tue, 05 Aug 2025 14:46:48 +0100

Introduction

This is a guide to myself if I need to deploy an Appsync App in the future. Appsync is a Serverless AWS offering for implementing GraphQL APIs. It integrates nicely with Aurora and DynamoDB. Other integrations are done via Lambdas.

Architecture Overview

Compontents

AWS AppSync: The managed GraphQL service that processes API requests
Aurora PostgreSQL: Serverless relational database for data storage
AWS Secrets Manager: Securely stores database credentials
AWS IAM: Manages permissions between services
AWS CloudWatch: Handles logging and monitoring

The flow of a request is:

Connection Pooling for Postgres using PG Bouncer

Thu, 24 Jul 2025 13:01:19 +0100

I recently had to run load testing for an API that fetches data from Postgres. I was monitoring Postgres when running the first test, and I noticed that hundreds of connections were being opened. I was also seeing runtime errors in Postgres such as “no more connections allowed” or “out of shared memory”.

Clearly the database was not prepared for the load.

So I started to look for ways to improve connection management to the database, and I came across pgBouncer, a connection pooling service for Postgres.

My experience learning the Dvorak keyboard layout

Tue, 15 Jul 2025 12:42:43 +0100

In this post, guest writer Billy interviews Pablo on learning Dvorak. Pablo started to learn Dvorak in January 2025. This is an interview about his experience.

First of all, can you explain what Dvorak is?

Dvorak is a keyboard layout created in the 1930s, proposed as a more efficient alternative to Qwerty. Dvorak groups letters on the keyboard based on how often they are used (in the English language), so the most used ones are located in the “home row” and less used ones are located in locations that require further finger reach.

Making Friends With Postgres: Replication

Mon, 09 Jun 2025 12:00:55 +0100

Replication refers to having copies of your data across different instances. It gives you redundancy when one of your instances goes down so you can still serve data to users from the remaining instances.

Another benefit of a replicated system is being able to load balance your requests across different instances. This can be useful when the load of serving all the requests from a single instance would be too much.

tmux + NeoVim ❤️

Tue, 20 May 2025 12:09:22 +0100

I currently have a basic setup in NeoVim: I use NvChad by default with treesitter to get pretty text highlighting, telescope to navigate files, conform for all my formatting needs and some python utilities such as pyright. This is enough to get me by.

The thing that has been annoying me the most lately is the terminal workflow in NeoVim. To start with, the terminal does not save any history across sessions, which is incredibly annoying.

Building a Reverse ETL Pipeline: Upserting Delta Lake Data into Postgres with Structured Streaming

Thu, 08 May 2025 13:21:30 +0100

In this post, I share how to build a Reverse ETL pipeline to upsert data from Delta (Databricks) into Postgres to provide sub-second response times to our tables.

The goal is to make warehouse data available to downstream systems that require millisecond response times. These systems could be front-end applications that need to consume this data, or online machine learning models that require additional data as input to generate predictions in real time.

We Need to Talk about Partitions

Sun, 16 Mar 2025 14:46:55 +0100

The other day my team came across a peculiar situation with our Postgres database. We had this table partitioned by a primary key, which we’ll call order_id.

We realised one of the queries was taking a very long time to execute, and like good SQL debuggers we looked at the query plan.

Let’s pretend this was the slow query:

EXPLAIN ANALYZE
SELECT * FROM orders WHERE customer_id = 12345;

Upon inspection, we observed that the query plan was scanning at every single partition! And in our case, there were a lot of them.

Load Testing PostgeSQL using Locust

Fri, 10 Jan 2025 15:09:39 +0100

I recently needed to run load tests on a PostgreSQL database. Postgres is no question a reliable and scalable database ready for production use cases, but there might be times when you’ll need to confirm it can handle your specific use case. This might be necessary if you’re using a PostgreSQL database hosted by a third party or if you want to check if your current instance size and specs can manage your existing load.

Test Driven Development

Fri, 13 Dec 2024 15:52:46 +0100

We all know that person that always asks for unit tests on your Pull Request. You curse under your breath wondering why they have nothing better to do than ruining your day.

Unit tests let you know if your codebase is behaving as expected. As long as you’ve written good tests, you can be happy your codebase is healthy if tests are showing green. Any time a test appears in red, you will know that you’ve introduced a bug and you can fix it before it hits production.

Understanding Database Indexes

Sun, 08 Dec 2024 16:32:43 +0100

Indexes are like going to the airport and looking at the signs to see where your gate is located. The signs tell you how to get to your gate quickly. The alternative to not having signs is walking the whole airport randomly until you come across your gate. So by having the signs you can reach your gate a lot faster. Makes sense right? This is the idea behind indexes.

Preparing for the Kubernetes Application Developer Exam

Tue, 10 Sep 2024 15:37:39 +0100

Learning

To prepare for the exam I took the Udemy Kubernetes Certified Application Developer (CKAD) with Tests which I could access for free through my company. The course covers all the topics required for the exam with labs to practice each section which I found super helpful. The labs are hosted on KodeKloud and they are free to access as part of the course.

I had some previous experience interacting with kubernetes as part of my job, but I lacked a good understanding of the basic concepts.

Career Transition into AI

Sat, 16 Dec 2023 14:59:43 +0100

This is my story of how I went from non-tech into becoming a Machine Learning Engineer. I must have told this story multiple times in many different forms, but this will be the first time it goes into writing. Enjoy.

I have a bachelor’s degree in Commerce. I found the last few years of school so exhausting that the thought of doing an engineering degree put me off. Four + years of no life? Oft, no thanks. I also thought I wasn’t smart enough to do engineering anyway so instead I chose Commerce - which “coincidentally” was the same degree my brother was doing.