Django / Nginx — Making SSL Work on Django Behind a Reverse Proxy & HTTP Only Apache

Update: django 1.4 has a new feature that allows you to specify which HTTP header implies SSL via a new setting SECURE_PROXY_SSL_HEADER, but it comes with dire warnings that your front end proxy MUST strip this header if not secure so that the user can not set the header manually to imply an SSL secure connection.

I didn’t think of this, and it’s a great point… my example did not explicitly set the flag OFF if not on port 443.

https://docs.djangoproject.com/en/1.4/ref/settings/#secure-proxy-ssl-header

How to make SSL work on Django when the certification is done on the front-end proxy server & apache is only listening on http.

I have nginx working as a reverse proxy for Apache / mod_python serving Django, somewhat according to the instructions here:

http://lethain.com/entry/2007/jul/17/dreamier-dream-server-nginx/

Basically, Nginx sits in the front, and forwards all requests but /media/ to apache.

My problem arose when I wanted to implement this SSL Redirect middleware: http://www.djangosnippets.org/snippets/880/

The setup, and why the middleware loops infinitely

I have nginx dealing with the SSL, forwarding requests to the same apache vhost as regular http, so apache is blissfully unaware of SSL.

This means request.is_secure() always returns False, which means the SSLRedirect middleware endlessly loops around a redirect because is_secure keeps returning False.

The fix

Add some extra header to the request in Nginx that specifies an HTTPS connection

To fix the problem I just set up my nginx to add a header “HTTP_X_FORWARDED_PROTOCOL” = “https”

Next, I replaced the request.is_secure() in the SSLRedirect middleware with a check to first see if the above mentioned header is in request.META, and then if its value is ‘https’. Return true if that is the case, and we successfully get a redirect.

Edit the SSLRedirectMiddleware

Edit the middleware to check for our secure header.

You must edit the above mentioned snippet somewhere in the _is_secure(self, request): definition.

Modify to:

def _is_secure(self, request):
     if request.is_secure():
         return True

     if 'HTTP_X_FORWARDED_PROTOCOL' in request.META:
         return True

Edit nginx conf

Edit nginx.conf (mine lives in /etc/nginx/nginx.conf) wherever you have your nginx listening on port 443.

Add a the custom header to your configuration where you have your other headers set (right inside the location brackets).

    proxy_set_header X-Forwarded-Protocol https;

Restart and you’re done!

Restart nginx and apache and you are good to go.

Django — Insert default data on syncdb. One easy command.

Order now and save 40%! But wait! Call in the next 15 minutes and we’ll double your offer, FOR FREE!

So this is the safe way to provide initial SQL data on a syncdb, in the form a .sql file in your app:

http://www.djangoproject.com/documentation/model-api/#providing-initial-sql-data

But I find it a pain in the ass.

I use the initial_data fixture to load data on syncdb. I find it extremely easy to do and requires no setup. You could literally set this up in 3 seconds. one command sounds too good to be true.

Its not very documented so I’ll post it here. I see little references to initial_data and dumpdata here and there, but yeah:

<code>python manage.py dumpdata > initial_data.json</code>

dumpdata dumps all of your models (using your default manager) in json format. check the dumpdata help for other args like –indent=Num to make it readable.

Anyways, that single command will dump the data to initial_data.json and it will be ready to go.

The only problem with this approach is if you modify your tables to have new non nullable fields this will throw an error. But if you are done with an app and it has basic categories, by all means, python manage.py dumpdata app > initial_data.json.

Linux — Edit files on your server directly from your desktop with SSHFS

Fantastic. I migrated our project to a production server environment and started to lose my workflow as I was forced to use many terminals at once.

At home, I tend to have a combination of 4 terminals up and a few GUI editors as well. Some things are better done with one, and some with the other. I use the terminals for directory surfing and making small changes. While I use a GUI app like Kate that splits into 6-8 pieces to work on bits of code.

Anyways, my solution right now is to mount my server onto my local filesystem.

If you are on Ubuntu, just apt-get install sshfs, and mount a directory just by typing:
ssfhs USERNAME@IP:/PATH/TO/DIR /MOUNT/PATH You might have to specify a port with -p if you are using a non standard ssh port (you should).

So, I’ve got my server mounted on my desktop. Fantastic.

Unmount with
fusermount -u /MOUNT/PATH

Apache — No space left on device. Failed to create global mutex

Turns out this is a bug. I saw tons of help pointing in other directions, but the end problem was that apache keeps forgetting to shut down one of its processes.

If your apache has more than one server (ie, port 80 and 442), etc/init.d/apache2 reload fails to close one of them.

So.. try running a pgrep apache and see how many processes show up!

It still hasn’t been fixed for mod_python for gutsy, so I’m upgrading to hardy.

PostgreSQL — could not bind ipv4 socket: cannot assign requested address

Jebus. This one wasted 4 hours and was a terrible terrible experience. Absolute frustration.

If I can understand all of this one day, I will be extremely proud. People who work on these systems are ridiculous geniuses.

I was getting ready to move our software to the live server but for some reason the postgresql server died on it. Checking up on logs shows that the server lost available memory and oom-killer, well, killed stuff.

So, upon restarting, postgresql would not start, giving me this error: could not bind apv4 socket: cannot assign requested address

Is postmaster listening on 5432? no its not lsof -i :5432 shows nothing. So whats up?

After reading posts after posts (most people suggest “you are probably already running postmaster”) I came upon an obscure reference to localhost not being resolvable.

ping localhost… nothing happens. There’s our problem.

Turns out its been running fine for months w/o a restart, and I had a wrong settings in /etc/network/interfaces

I had my iptables rules not auto-loading because I spelled the script ‘itables’ instead of ‘iptables’.

So.. bottom line? check to see if you can ping localhost.

Django — Easily save your model instances with dumpdata and loaddata

So I’ve always wanted to know how to write the XML fixtures for automatically loading my project db with initial data, but I always thought I’d have to learn something so I stayed away and simply entered test data by hand.

Boy was I wrong.

I had to manually write shipping tables for several shipping categories / states and state tax rates so I wanted them in a fixture I could simply load into django anywhere else.

Turns out it is amazingly easy.

Note: I previously used format=xml in this example but I’ve since switched to json (the default) because I was getting parser errors when I tried to loaddata the xml files

python manage.py dumpdata --indent=4 > shipping_fixture.json

python manage.py dumpdata
dumps all of your apps. add your app afterwards to dump only one piece.

Notice the command simply has a long output and no options to save it to a file somewhere. That’s where you use > to specify a file to write the data to.

–indent=4 just adds indentation because the default writes all data at the beginning of the line.

python manage.py loaddata shipping_fixture.json will load your data. You can also add a default fixtures setting in settings.py to have commands like manage.py flush delete the db & repopulate with your fixture.

Great stuff.. who knew it would be so freaking easy.

Photoshop — Utterly annoying “save as” bug. DPI setting won’t change

These are the kinds of problems that make me want to harm something fuzzy.

I am doing label registrations for our wines, and the person who sends them into the ttb verifies the labels by literally printing them and measuring them. Well, she only takes files that are 150DPI.

DPI is Dots per Inch (pixels) which is only used to calculate printing size. An image has a set amount of pixels, your ‘megapixels’ rating on a camera means your pictures have that many pixels in them.

So: ‘saving as’ in the most recently updated Photoshop CS3 fails to save DPI information to the JPEGs. It utterly corupts the file.

Example:

I save as JPEG, and the file resolution is read at something like 381 DPI vert/horizontal. I’ve tested this and it is dependant on how big the file is you are trying to save (pixel size). Even after opening it up, photoshop reads it as the correct resolution (150DPI in the image settings menu) but will NOT save the DPI metadata, no matter what.

So I am forced to “save for web” which saves at 92DPI, then open it separately, convert it to 150DPI, and save it again.

Ridiculous.

Fix your ‘save as’ feature.

Django — django.contrib.auth Login Issues: Site._meta.installed / Site matching query does not exist

Well, I wondered what happened here and googling was futile.

Was getting errors on django.contrb.auth.views.login

If Site._meta.installed:
current_site = Site.objects.get_current()

After reading into what that does, it pulls SITE_ID out of your settings to check what db to look into. Well, my django project had a SITE_ID of 2 from many months ago.

Simply changed that to one and problem fixed.

Ubuntu — Running Two Firefox Intances on Dual Screen (Seprate X Screen — not LargeDesktop or TwinView)

There are benefits and downsides to using separate X screens for dual screen setups vs a “large desktop” which just means the computer thinks there is one desktop, with two monitors showing the same desktop.

Separate X screens means you have an entirely different screen on each monitor. This is useful to me because I can have two SETS of workspaces on each screen. My smaller monitor to the left has 3 workspaces, the large 30″ on the right has 6 screens. I keep my browser, chat, music on the left monitor, and it doesn’t move no matter how much I switch around on my main screen.

With a large desktop or TwinView, you would switch both screens by switching workspaces. So having that second screen doesn’t help THAT much if you already have a large main screen as you’d still need to switch around just to look on your second screen.

Anyways, FireFox complains that you an instance is already running if you try to run firefox on each screen. To get around the problem, start firefox with -P, which runs the profile selector.

Create a new profile and there you go, you have a new firefox instance running on your other screen.

From now on, just start firefox by running firefox -P.