few days ago I did something that I never thought I’d do again, and started putting some of our more critical logs in Postgres.

Never, ever do this, unless:

• You’re draining only canonical log lines which are a summarized digest of everything that happened during a single API call, rather than a flood of voluminous and low quality logging data.

• You’re putting them in an ephemeral database, so they can be shedded as necessary and won’t interfere with higher fidelity data like in the case where recovery from backup is necessary.

• You’re using a partitioned table which makes dropping old data easy and fast.

And even then, it’s a technique that’s probably going to have trouble scaling to enormous sizes.

Fast as a Service by brandur.org favicon