Do You Need Cloudflare for a Small Docker Compose App?

My reader portal was using about 1.3 GB of memory on a server with 8 GB. I was paying for the other 6.7 GB to sit there and look reassuring.

At the same time, everyone I asked about bots said the same thing: put Cloudflare in front of it. So I set myself two questions. How small can this server go? And do I actually need a protective layer, or do I just feel that I should?

I answered the first by moving to a 4 GB server. I answered the second by reading my own logs. This post is about both, and about the three bugs that only showed up after the move. If you run a small Docker Compose app and wonder whether you are being naive without a CDN in front, this one is for you.

How Much RAM Does a Small Flask and Celery App Need?

The portal is a Flask app (a Python web framework) with a Celery worker (a separate process that runs background tasks), a scheduler, Postgres and Redis, all in Docker Compose behind Nginx. I described how to set up a stack like this in From Localhost to Live, and how to deploy it from GitHub Actions in Docker Compose CI/CD with GitHub Actions.

I measured before deciding anything. After the move, the whole stack used about 1.3 GiB of the 3.3 GiB the new server has available. That is roughly 40 per cent, which leaves room for a busy day, a backup and an unexpected spike.

I chose a 4 GB plan with two dedicated CPUs. The old server cost about $55 a month. The new one is $40 plus $8 for the provider’s automatic backups. Honestly, the saving is small. The real gain is that the server now matches the app, and I understand why it has the size it has.

What Do Nginx Access Logs Show About Bot Traffic on a Small Site?

Before adding any protection, I wanted to know what I was protecting against. Nginx records every request in its access log, a plain text file with one line per request, so I counted.

# Status codes, most common first (Nginx's predefined `combined` log format: field 9 is the status)
awk '{print $9}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head

# Obvious scanner probes
grep -E '\.env|wp-login|xmlrpc|\.git/|phpmyadmin' /var/log/nginx/access.log | wc -l

Rotated logs are compressed, so use zcat for those. You may also need sudo, because Nginx logs are readable by an admin group only.

Here is what my fifteen days looked like:

Read that last row again. Fifteen days, and a few real people. The rest was robots knocking on doors that do not exist. A scanner looking for wp-login on a Flask app gets a 404 and moves on. That 404 costs me one log line.

Scanner traffic of this kind is noise, not an attack. The distinction matters, because the cure for noise is patience and the cure for an attack is a different set of tools.

Why Cloudflare Challenge Pages Break Fetch Calls Between a Blog and a Portal

I am not against Cloudflare. It is a good product, and I may well use it one day. I skipped it for a specific technical reason, and I think this reason is worth sharing because it is easy to miss.

My blog lives on GitHub Pages at daehnhardt.com. My portal lives at app.daehnhardt.com. Pages on the blog call the portal from the visitor’s browser with fetch, using credentialed CORS requests, to show things like bookmarks and gated content.

The blog and portal addresses are cross-origin (different hostnames) but same-site (the same registrable domain), a distinction explained well on web.dev. The same-site detail does not change the argument, so I will not dwell on it. What matters is what a bot challenge does to a fetch call.

Cloudflare’s challenge pages work by returning a full HTML page that the visitor’s browser renders and solves. A Cloudflare challenge page is an HTML interstitial that a browser must render and solve, so a fetch or XHR call cannot complete it. A fetch call from my blog cannot show that page to anyone. The fetch call simply receives HTML where it expected JSON, and my bookmarks stop working. Cloudflare’s own documentation describes a cf-mitigated: challenge header you can use to detect this in fetch and XHR code. Cloudflare documents it because it expects the situation to happen.

Bot Fight Mode, the simplest switch, makes it harder still. Cloudflare states that it may challenge API or mobile app traffic, and that you cannot bypass it with WAF custom rules or Page Rules. You could configure around all of this, but I would be configuring around it for a problem my logs say I do not have.

The three Cloudflare options compare like this for a blog that calls a portal through fetch:

There is a lighter option I may use later: Turnstile on the sign-up and contact forms. Turnstile runs a small challenge in the form, and your server then validates the token with Cloudflare. The token lasts five minutes and works once. This protects the forms without touching the API calls.

For now I rely on rate limits inside the app and a firewall that allows only SSH, HTTP and HTTPS. I also run fail2ban for SSH only. I considered a fail2ban jail for Nginx, but it would mostly tidy noise that is already harmless.

When I would change my mind: if the logs turn hostile. A sudden jump in login attempts, sign-ups from nowhere, or a rise in server errors would all send me back to Cloudflare, probably starting with Turnstile.

How to Move a Docker Compose App to a New VPS With Little Downtime

I wanted the move to be dull, and it was, thanks to a written runbook and a rehearsal. The order was:

1. Build the new server and lock it down first. Key-only SSH, root login off, firewall on, before any app was installed.
2. Rehearse the data move. A script dumped the old database, copied it, restored it on the new server and compared the row count of every table. I ran it in rehearsal mode first and only went live once the counts matched exactly.
3. Get the HTTPS certificate before switching DNS. I used Let’s Encrypt’s DNS challenge, which proves you control the domain with a temporary DNS record, so the new server had a valid certificate before any visitor reached it.
4. Lower the risk of stale DNS. My DNS records had a 600-second lifetime, so a switch takes effect within about ten minutes.
5. Cut over in a short window. Stop the old app, do a final sync, start the new one, switch DNS, then test from a private browser window.
6. Keep the old server for a week. The old server stays as a fallback until I am sure nothing was missed.

One lesson from step 5 deserves its own box.

Retire the old host with docker compose down, not docker compose stop.

A restart policy is a Docker setting that decides whether a container returns after it exits or the server reboots. My containers use restart: always. Docker’s documentation says a manually stopped container is restarted when the Docker daemon restarts, so a stopped container comes back after a server reboot. If my old scheduler had come back, I would have had two schedulers on two servers sending every scheduled email twice. down removes the containers, so there is nothing to come back.

Three Bugs After Moving a Docker Compose App to a New Server

The move went well. Things that had been quietly wrong for months did not.

1. Celery Worker Missing a Volume Mount: “master file missing”

My web container and my Celery worker share one image but not one set of mounts. I had mounted the backup folder only where I remembered it.

The nightly database dump runs in the worker, so for a while it wrote dumps inside the container, where a redeploy erased them. Later I added a feature where a background task attaches a protected PDF to an email. The web container could see the PDFs. The worker could not, so the task failed with “master file missing” after the page had already told the reader the email was on its way.

The fix was one extra line in the Compose file (the second volume below), plus a test so it cannot quietly regress:

celery:
  volumes:
    - ./backups:/backups
    - protected_files:/app/protected_files:ro

The lesson: a task runs in the worker’s container, so the worker needs every file the task touches. A missing mount shows up in the worker log, not as a web error, and nobody sees the log until they go looking. Now I check it after each deploy with docker compose exec celery ls on the folder.

2. Alert Emails Bounced Because the Recipient Was the Default Sender

I had built a small resource monitor that emails me when memory, disk or backups look unhealthy. The first alerts arrived as bounce messages. I had not set an admin recipient, so the app fell back to its default sender address, noreply@ on my own domain, and mailed that. My mail provider rejects mail to that address with a 550 error.

I set the recipients explicitly and sent a test. Fixing the recipients took five minutes. The lesson is that the first test of any alert should be a real one, delivered to a real inbox, before you trust it.

3. Rate-Limit IP Blocks That Never Expire

My app blocks an IP address after a few rate-limit breaches. The blocks were permanent. Quietly, one of them locked me out of my own admin area. When I dug in, I also found that signed-out visitors’ bookmark checks, which the blog fires on every page view, were counting towards the limit. Those bookmark checks are ordinary page-view traffic, not abuse.

Now blocks last an hour, a block I place by hand is never overwritten, the same expiry applies to the account flag that comes with the block, and that polling no longer counts. Automatic defences should forgive, because the person they catch is usually not an attacker.

Cloudflare as a Doorman: a Mental Model

Think of the server as a flat and of Cloudflare as a doorman.

A doorman is excellent if your building gets a stream of strangers wandering in. But he also stops the postman, the delivery driver and your own cousin with a key, and you have to teach him each of their faces. If your building mostly gets people who ring the right bell, the first thing to do is look at who actually rings. Mine mostly got people ringing doorbells that did not exist.

Checklist: Right-Sizing a Small Server and Skipping a CDN

1. Measure real memory use before choosing a server size.
2. Read your access logs for a couple of weeks before buying protection.
3. List every cross-origin fetch your front end makes, and ask what a challenge page would do to each.
4. Rehearse the data move and compare row counts per table.
5. Get the certificate ready before switching DNS.
6. Retire the old host with docker compose down, never stop.
7. Give workers the same mounts as the web container, and check them after each deploy.
8. Send a real test alert to a real inbox.
9. Make automatic blocks expire and exempt yourself.
10. Write down when you would change your mind about the protection you skipped.

Want these commands on paper? I put the ones I used into a two-page cheat sheet: reading Nginx and container logs, routine server maintenance, and the database copy, certificate and DNS steps of a move. I ran the log and database commands before they went on the page. It is free with a free membership, along with the other cheat sheets. Print it and keep it next to the keyboard.

Final Thoughts: Cloudflare Not Yet, With a Trigger to Revisit

I did not decide against Cloudflare. I decided not yet, with reasons I can write down and a trigger for reconsidering. That feels more honest than either installing it because everyone does, or ignoring it because I was in a hurry.

The server move was the easy part. The surprises came from small gaps that a bigger, emptier server had been hiding. If you are right-sizing your own setup, expect the same, and expect to learn something useful.

Have you looked at your own access logs recently? What did they say?

References

• Cloudflare: Bot Fight Mode, including the note that it may challenge API traffic and cannot be skipped with WAF custom rules.
• Cloudflare: Detect a challenge page response, the cf-mitigated header for fetch and XHR.
• Cloudflare: Turnstile server-side validation.
• web.dev: “Same-site” and “same-origin”.
• Let’s Encrypt: Challenge types, the DNS-01 challenge.
• Docker: Start containers automatically, restart policies.
• My earlier posts: From Localhost to Live and Docker Compose CI/CD with GitHub Actions.