A few days ago I decided to upgrade to Python 2.7. I’m running Windows 7 64-bit — pretty sweet as far as Windows goes. ;) So, I’m thinking to myself, “I’m running a 64-bit OS, why was I running a 32-bit Python?”
While the core Python distribution is available in 64-bit, many many packages that I depend on only supply precompiled binaries for the 32-bit Python distribution. Why? I have no idea. There are two things you can do.
- Use this site. There are a bunch of packages available with 64-bit in mind that aren’t available from the package’s maintainers. MySQL-Python, for instance.
- Compile them yourself. The unofficial repository doesn’t have all packages on PyPI compiled for Windows. gevent is one I’ve come to depend on a lot, and it’s not available — so I had to find a way to build extensions myself. Here’s how…
Install Microsoft Visual C++ 2008
Don’t bother with MinGW. Let me say it again — DO NOT USE MINGW FOR THIS! For one, the standard mingw distro is 32-bit. I found a gcc toolchain for 64-bit Windows, but I couldn’t get it to work. The Python import lib is made for Visual Studio. There are apparently ways to convert the file to something compatible, but I spent 4-5 hours trying to get this to work to absolutely no avail. Save yourself the trouble.
Additionally, you can’t use Visual C++ 2010. Python’s distutils lib is not set up to handle it. Visual C++ express works, as long as it’s 2008.
Note that if you have Visual Studio 2008 Professional, Team Studio or whatever, you should be able to stop here. The Express editions, however, don’t have the 64-bit environment, so we need to do more stuff.
Install the Windows 7 Platform SDK
Now just called the Windows SDK. You can get it here. It’s pretty large, so be prepared. Obviously, make sure you install the 64-bit environment.
Trick distutils
distutils looks for a file called vcvarsall.bat, runs it, and gets the include and lib directories that the batch file sets up. The batch file sets up the environment based on what platform you supply to it — in this case, amd64. Unfortunately, Visual C++ Express does not have the proper files for 64-bit compilation, but you can set it up pretty easily.
vcvarsall.bat should be in a directory like: C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC
You need to create:
- The directory — C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\amd64\
- The file — C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\amd64\vcvarsamd64.bat
The Windows SDK comes with a fully working 64-bit environment, so we just need to point vcvarsamd64.bat to the new SDK — which distutils doesn’t recognize.
So in vcvarsamd64.bat put:
call "C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\SetEnv.cmd" /x64 /Release
Assuming you let the Windows SDK install in the default location.
Still not done
We have to patch distutils now. Unfortunately, the new linker doesn’t generate .manifest files by default, but distutils tries to embed a manifest file in the dll (pyd) that it just built, and *will fail* if it is unable to do so.
To fix this, add the follow line to distutils\msvc9compiler.py after line 648:
ld_args.append('/MANIFEST')
That’s it!
You should now be able build your own extensions for 64-bit Python in Windows 7! You can have PyCrypto, gevent, ZODB, and so on.
Side Note
If you’re having trouble with pip or easy_install opening up a separate console window, it’s an easy fix. It’s not necessarily a problem, but it’s annoying — the console window disappears as soon as the operation is done, whether or not it fails or completes.
The issue is that setuptools is running a 32-bit application, and Windows 7 (smartly) runs 32-bit applications in a separate process.
The fix is to uninstall setuptools and pip, and reinstall setuptools from source. Do not use ez_setup.py. I don’t know if you need to be able to build extensions before you can build setuptools, but that’s what I did. After that, you can easy_install pip, and pip will now run in a 64-bit environment too. Yay!