Docker, php-fpm and nginx

tl;dr

I followed examples found on more than one site, but my php scripts were downloading as source. The problem turned out to be that I was creating a site.conf file for my PHP configuration in the default site’s location, but another file in the Nginx container called default.conf had another configuration that overrode mine. I changed my site.conf to default.conf and everything worked well. This post is mostly about the process that led me to the solution.

How I got Here

My main production environment at work uses Ubuntu 16.04 LTS with PHP 7.0. I’d like to run Symfony 4.x, but that will require PHP 7.1. The move from 14.04 and PHP 5.x was more difficult than I’d hoped, so I’m not keen to jump to 18.04 with its PHP 7.2 support right away.

I’ve been wanting to avoid these production environment version issues by moving to Docker, and I spent a good solid week or two learning about Docker, and dockerizing Symfony applications. I loved how Docker containers worked together, and how well Docker dovetailed into my preference for 12 Factor Apps.

At the end of the day I ran into two problems. Symfony’s performance in dev mode in a container was much worse than native. A factor of 10x worse. If that wasn’t enough to put me off, I was also experiencing a problem with segfaults in Apache in about 15% of page loads. I spent a lot of time trying to resolve these issues but had no luck. I’m pretty sure Apache was running out of memory, but I couldn’t seem to give it more.

That was two or three years ago, and Docker has been improving in leaps and bounds, so I tried again a little over a year ago but ran into the same issues.

Third Time’s the Charm?

After reading about and studying CQRS for a couple of years, I’ve been given the go ahead to do a project with it! I hope to be able to use Prooph for this, but like Symfony 4 Prooph requires PHP 7.1. Time to take another run at Docker. I’d love to have my dev and production environments more in sync. Optimism is a key survival trait for developers, and mine is as healthy as ever. If this doesn’t work I’m going to have to find some other way to implement CQRS, and I’ll be damned before I roll my own.

I run through the latest Docker beginner’s tutorial and it comes back to me like I’d never abandoned it. Then I do the docker-compose section. I remember docker-compose as a new thing the last time I looked at it. Now its mature, and beautiful. I finish the tutorial and try to set up a good environment for Symfony 4 and Prooph.

I remember that Alpine based containers were prefered for their small size, and I want to build my new environment on that, but there are no PHP containers that provide Apache on Alpine. I’m not willing to bloat things with Debian as much as I love Debian in other use cases, so I decide to run with php-fpm and Nginx.

Run with What and Who Now?

I’ve been working with PHP since it freshly hit version 3.0, and Apache was still a joke name because it was “a patchy server” based on NCSA httpd. So, I have sympathy for anyone who isn’t familiar with php-fpm or Nginx. If this is old hat for you feel free to skip to the next H2 heading. On the other hand, you might enjoy some of the history presented in this section from someone who has lived through it.

FastCGI

The first web apps that didn’t involve writing your own HTTP server took advantage of something called CGI, or the Common Gateway Interface. This simply ran an executable, providing the request on the standard input, and reading the response on standard output. I have many less than fond memories of writing these in C, and later C++ before PHP swept me off my feet.

The overhead of loading the CGI executable for each page load became a big bottleneck, so PHP was packaged as an Apache module. This allowed PHP to become part of the web server so that it never had to be reloaded unless the web server was reloaded. Only the php script would have to be reloaded – a problem that was later solved by opcode caching.

This was a huge boost, but it was limited to the Apache server and didn’t help things on Windows servers or other HTTP servers. About the same time another technology called FastCGI was emerging as a response to Netscape’s NSAPI for their own HTTP server. FastCGI allowed the CGI process to stay resident in memory, and communicate with the web server using named pipes, UNIX sockets or TCP instead of stdin/stdout.

FPM is PHP’s FastCGI Process Manager, which provides the stuff to allow your PHP scripts to play nicely and quickly with web servers that use FastCGI.

Nginx

Old school UNIX servers used a technology called “forking” which is different from the forking done today on GitHub – but they’re not completely different, so let’s start there. You find a piece of software you like on GitHub and you want to play with it and make your own modifications. So, you fork a copy into your own repo and get busy.

This makes an identical copy, but you can modify this copy without changing the original. If you know a bit about how git works, you know that you haven’t really in fact even made a copy, but you’ve made a new reference to the same stuff. When you start committing changes, those references are updated to point to your new work so that your fork can become something different.

The simplest way to write a network server such as Apache is to do the same sort of thing with your server process. It sits there waiting for connections, and when one comes in it forks itself. The original copy continues to wait for new connections while the fork, which started out as a duplicate, instead handles the request. For anyone who hasn’t played with this it’s pretty cool. Your software basically says, “clone me”, and the original can say “I’m just gonna chill out and wait for more work” while the clone has to do that work. Oh how I wish I had my own fork() method!

As fantastic as this is, it has some performance problems. Even when you only copy references to a process, and only create copies of memory blocks when they differentiate like files in your git commits, this is still slower than it needs to be. The first solution to this problem was to introduce threading. Threading solves the problem by cloning some of the process, but not all of the process. This meant that code had to be thread safe, and it was often buggy and introduced its own bottlenecks in how it dealt with those bugs.

Then in 2004 Nginx was introduced, and at least in terms of speed it blew away every other HTTP server that came before it. You see even if you have the most lightweight of threads, and you’ve eliminated all of the resultant bugs, and the problems with semaphores and mutexes, which is how we forced threads to get along with each other, you still had one insurmountable problem. Every time a CPU core had to switch from one thread to another, all of the register values also had to be swapped out – these are like the short-term memory of a CPU. This context switching had an overhead which wasn’t well enough appreciated before.

Nginx took what was at the time a very counter intuitive approach. Instead of forking off new processes or even spawning new threads, Nginx was very good at dealing with a list of clients a few bytes at a time. So instead of saying that this process / thread would send the whole response and get interrupted by the operating system’s task scheduler as needed so that another process / thread could send some other response, Nginx managed all that on its own. It would send some data to this client, and then to that client, and so on. Single tasking like this seemed like a crazy idea, but by avoiding the overhead of the context switches to the CPU Nginx turned out to be the fastest thing going for serving web traffic.

Those of you with an appreciation for JavaScript asynchronous code execution and the incredible performance of Node.js will recognise their roots here and appreciate why Nginx and Node.js work so well together.

Php-fpm and Nginx are a great combination, and I was determined to keep their advantages as well as the size advantages of Alpine in my Docker solution.

My First Kick at the Can

It should have been easy. I started with a simple docker-compose.yml file that specified just basic Nginx and php-fpm containers:

version: '3'

services:
    php:
        image: php:7.1-fpm-alpine
        volumes:
            - ./code:/usr/share/nginx/html
    web:
        image: nginx:1.15.4-alpine
        ports:
            - 80:80
        volumes:
            - ./code:/usr/share/nginx/html
            - ./nginx/site.conf:/etc/nginx/conf.d/site.conf:ro

Nice and tight, with no Dockerfiles required. I lifted the site.conf from an excellent online tutorial, but when I tried to load my index.php it just downloaded the source code. I should point out that I didn’t lift the file verbatim… I didn’t want to deal with setting up a .local domain, and so I removed some things that I didn’t need. Looking back, I’m pretty sure it would have worked if I hadn’t trimmed it down, but this seemed harmless at the time.

How I Fixed It

Finding a concise solution on StackOverflow is sweet, and I really appreciate the countless hours spent by the volunteers there who make my life easier. But as the proverb goes, “give a person a fish and you feed them for a day; teach a person to fish and you feed them for a lifetime”.

I tried to find the gift of a fish on StackOverflow, but I was not so lucky. I wasn’t in the mood to fish, so I tried some more, but still I had no luck in finding a canned answer that just worked. Thankfully, when push comes to shove I know how to fish.

I started by switching my Docker images to Debian versions: php:7.1-fpm-stretch and nginx:1.15.4-stretch. I wanted to make sure that fpm was running on port 9000.

$ docker exec -it prooph_php_1 bash
[email protected]:/var/www/html# telnet localhost 9000
bash: telnet: command not found

As expected. No big deal…

[email protected]:/var/www/html# apt update; apt install -y telnet
... clipped for brevity ...
[email protected]:/var/www/html# telnet localhost 9000
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
^]
telnet> quit
Connection closed.
[email protected]:/var/www/html#

Ok, so something is listening on my php container for port 9000. Can I connect to it from my Nginx container?

$ docker exec -it prooph_web_1 bash
[email protected]:/# apt update; apt install -y telnet
... yada yada yada ...
[email protected]:/# telnet php 9000
Trying 172.21.0.2...
Connected to php.
Escape character is '^]'.
^]
telnet> quit
Connection closed.
[email protected]:/#

So far so good. If my Nginx container can connect to FastCGI on the php-fpm container, then the problem is with the Nginx config. I re-read it and it looks good. I read the documentation on every directive, and they all look good. What the hell?

I go back to the basics. I’m pretty sure that the problem is with the Nginx configuration. What does the actual nginx.conf look like? It all looks pretty vanilla, but I pause on this line:

include /etc/nginx/conf.d/*.conf;

That would load my config. I verify that it is where I expect it to be:

[email protected]:/etc# cat /etc/nginx/conf.d/site.conf
server {
    index index.php index.html;
    root /usr/share/nginx/html;

    location ~ \.php$ {
        try_files $uri =404;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_pass php:9000;
        fastcgi_index index.php;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param PATH_INFO $fastcgi_path_info;
    }
}

Then as I’m staring at the location line it hits me… What else is in the conf.d folder? Could there be other competing location entries?

[email protected]:/# ls /etc/nginx/conf.d/
default.conf site.conf

There it is. I rename my site.conf to default.conf and adjust my volume entry in docker-compose.yml. I restart docker-compose and BOOM, there it as, a set of glorious phpinfo() output in my browser.

I’d spent a lot of time searching for a canned answer, but I hadn’t come across this one. I thought it would be worth writing up so that others who find themselves with the same problem might find a solution.

Next Steps

Next, I’ll consider how to run composer in my php container so that I can set up Symfony and Prooph. I’ll add a database and see if I can get xdebug working with PhpStorm. With any luck I’ll be actually using Prooph before the week is out.

Advertisements

MyOath – MySQL Promises for Node

I have mixed feelings about ORMs, and prefer to err on the side of the more lightweight solution when I’m not sure which tool will be most appropriate to the problem I’m trying to solve. I recently found myself re-inventing the CRUD wheel once again for a node app I’m writing, and tried to find a library with a little more abstraction than the basic MySQL library without going full ORM – but turned up nothing that was appealing.

Over the last few months I’ve really embraced the promises pattern in my JavaScript, and I wanted something that would let me execute SQL queries asynchronously with promises. I also wanted something that would be DRY for the REST API’s I’m often writing.

I have to say that I’m pretty happy with the result. I’ve been using the Q Promise Library, but I’ve heard good things about some other libraries so I wrote this to support a range of promise libraries so that you can use your preferred library natively with this library. Its methods return promises so they can be easily chained with other promises. The library exposes methods which correspond to HTTP methods for GET, POST, PUT and DELETE so that you can spend more time on your app instead of writing yet another set of CRUD methods.

You can find the docs on the project’s github page. I’d appreciate all constructive feedback whether it is positive or negative.

Slim RedBeanPHP and Knockout

Last night I gave a talk about building websites with Knockout.JS, which I’ve been happily using for a long time.  I demonstrated using Slim as an application framework, and RedBeanPHP as an ORM on the back-end. I’ve used Slim on a few projects in the past, but this was my first time playing with RedBeanPHP. I’m looking forward to learning more about this cool little ORM.

The main message of the talk was that choosing simple frameworks can mean writing and maintaining less code, and can help to overcome obstacles to adopting TDD.

At the same time, I’m a big fan of Angular, Symfony and Doctrine.  For projects that can benefit from those more elaborate architectures they should be used. For smaller projects, or to make getting traction with TDD easier you should consider smaller frameworks like those touched on here.

If I were doing this talk again I would break it into three parts to that I could spend more time on each library, and on the unit testing. I was barely able to finish in an hour and a half, and I feel terrible for putting the group through such a long and code-heavy talk (again).

The slides are posted on slideshare, and the code is posted on github.

The Intense Role Playing Game

Today I released a very simple role-playing game which I developed for use at Scout camps. It is ideal for playing around a camp fire, but works just fine around a table too. The rule book is only 12 pages long, and only 2 of them are really about the rules. Most of it is help for the person who’s running the game such as story ideas.

I hope that it will be approachable by non-nerds since the rules are so simple. I’ve only played it with youth aged about 10 – 14 but I expect adults would have fun with it too.

Check it out at http://intenserpg.vicmetcalfe.com/.

MeteorJS: First Impressions

I saw some buzz today about Meteor, a new set of technologies for building JavaScript applications which blurs the lines between server code and client code.  I’ve had a notion like this bouncing around in my head for the last year or so, and so I was keen to jump in and try it.  I watched the screencast, installed the app, and installed the leaderboard example.  The installation went flawlessly, and I was able to start the example app and bring it up in my browser without any trouble.  I was off to a great start!

Read more…

SOLID PHP & Code Smell Wrap-Up

Here are the slides from the SOLID wrap-up I presented at GTA-PHP last night (Google DocsPPTX, PDF).  I’ve also made a gist for the LSP example referenced on slide 8 that I couldn’t find at the time.  I covered the Liskov Substitution Principal (LSP), the Dependency Inversion Principal (DIP) and the Interface Segregation Principal (ISP) in PHP.  I’ve covered SRP and OCP in earlier posts.  I mentioned that I wasn’t particularly fond of applying the ISP in PHP, and Ilia Alshanetsky pointed out that PHP’s performance can suffer if you use too many small interfaces in your PHP.  We discussed the notion of putting constructors in interfaces, and Guilherme Blanco pointed out that since you can’t instantiate an interface an abstract class would be a more appropriate way to require a specific constructor.  Peter Meth and Dan (sorry Dan, I don’t know your last name!) asked where to find a good summary of the SOLID principals, and I suggested the Wikipedia entry, which has a nice table for jogging your memory on the acronym’s meaning.  Thanks to everyone who showed up!

JS301 Week 4: Knockout and Objects

This week we will look at how to use knockout.js to keep our in-memory models and user interface in sync, and we’ll look at how objects are used in JavaScript.  We’ll finish up with some suggestions on how you might continue to expand your JavaScript horizons.  The slides are available online and for download (pptpdf).  Update: I’ve had stability problems with jsfiddle.net today, so just in case here’s the source for the OO examples.  The knockout examples are from their web site, so we’re safe there.