Problem 1

Problem #1: How much will the storage of logs cost for a standard, monolithic 100,000 RPS web application?

Assumptions

Let’s estimate annualized cost of logs.

Say each request generates 1KB of logs.

Say we can put them in blob storage:

9 TB / day * 30 days / month * $0.02 / GB / month = $5400 / month = $64800 / year

But we don’t usually need to keep logs that long. Let’s say we only keep last three days

Ah, but we are also charged by writes to blob. Let’s start with 1 write per request.

Napkin math technique: let’s batch writes by 1000. Then our answer just changes by a factor of 1000.

Another thousand: $15.768 / year

Alternatively, logs pricing is $0.5 / 1 GB. For one year:

For three days:

I didn’t think to estimate how much we fits into 1KB.

1 character per byte, or 1000 characters.
wc p1.md (before this bullet) is about 2KB, so 1KB seems right order of magnitude for a log.

I also like computing with scientific notation. Let’s walk through it:

Solution also used disk storage pricing at $0.01 GB / mont. Possibly don’t have to pay for writes at this level?

Writes get pricey:

Write every 1000 requests:

10^3 bytes / request * 10^3 requests / batch = 10^6 bytes / batch (1 GB / batch)
10^5 requests / s * 1 batch / 10^3 requests = 10^2 batches / s
10^2 writes / s * $5 / 10^6 writes = $0.0005 / s
$0.0005 / s * 9 * 10^4 seconds / day * 30 days = $1350 / month