Brave New World - Goodbye Scons + Mingw

Today, I finally had to bid farewell to the build system that's been serving me so well for the past 10 years (well, apart from that brief stint where I had to switch over to msvc for a few weeks as the build was broken). Yesterday, the scons build system was finally removed from Blender, leaving just cmake.  *sob*



As I've written about before, I'm really not a fan of the cmake approach (multi-step builds, the need to manually rebuild the cache from time to time, manually adding files to the buildsystem,etc.), and especially about what it means for the development workflow. However, what's done is done, and given that most devs have now moved over to favouring using the "native" buildscripts of whatever compilers they were using (with all the unsightly warts that those have), it's understandable that scons was going to lose out. Sadly, gone along with it is the possibility to use mingw/gcc on Windows: there's just so much going wrong here (besides the ugliness of makefiles) that it looks like I'm going to have to learn to put up with msvc.

There are several reasons why I don't really like the cmake + msvc route:
  1) Size - The libs folder for msvc is almost 10 GB, while the mingw one is about 400mb tops. That's on top of the size of the compiler itself. This is especially problematic in a country where large data transfers can still be quite sluggish (i.e. you really don't want to go over 100mb for any file still, or else it's a pain, and poses a distinct risk of blowing your monthly allowance if you do it too many times)

  2) Separate compile steps (and the need to go through the IDE to compile) - Fortunately, Campbell has set up a rather awesome batch-script which bundles these steps together, making it a smoother experience for everyone. I had a few troubles getting it to work initially (due to various configuration troubles - more in a moment), but it now works fine

  3) MSVC's errors are not as nice or useful as GCC's - This was at least true the last time that I did a comparison of the two, though it has to be said that GCC's have gotten worse over time (become more verbose and cluttered, like the useless ones from msvc). Basically, MSVC didn't use to warn about many/any of the things I'd usually care about, but would spend tons of time spewing lots of incomprehensible garbage about stuff I didn't want to know about. And then, on the off chance it complained about something that was important, the warning was usually not as clear or direct about what exactly had gone wrong (compared to the comparable gcc warning)

  4) MSVC's debugger, while much lauded, has always been incomprehensible to me - I hate  the fact that with MSVC, you need to have a debug build to stand a chance at tracking down a segfault (while with gcc, even the crumbs it leaves behind on a normal build are usually sufficient to pin down and fix the bug without recompiling), and that to track down a bug using that debugger, you've got to deal with the laggy IDE (e.g. waiting for it to come up when the program segfaults, waiting for it to get it's shit together, then waiting for it to present to you where it's crashed in the middle of execution but only if you manage to answer it's confusing 5-choice stupid dialog). Then, when it finally shows you where the crash happened, it's utterly confusing to figure out what's going on, as it'll usually have unwrapped all the way down to the Assembler in the internals of Windows... when you finally make it back up to your code, you're then left with the confusing challenge of figuring out where the heck you are, and what state is being shown (i.e. is this the "current state" after this line was executed, or is it the "previous state" before this line gets executed, or if it is the previous state but the current line has already been executed, ...) Things only get worse if you try to step the code (when on a breakpoint), and watch as garbage appears and disappears from cells, never quite corresponding to the before/after states of any statement being executed....  Grrr....!   Just give me the simplicity of a gdb terminal!  "bt + p"  are sufficient tools for debugging work anyday!


----

To get things running today, I faced a few challenges.

1) To resolve the size problem (namely the challenge of downloading all the libs), I ended up doing a partial checkout of the libs folder via TortoiseSVN.
   (Tip: In the Checkout dialog, click on the "Choose items..." button to pick and choose which parts of the repository you grab. Note that if a folder has a checkmark beside it, all it's children will get included, UNLESS you start clicking on its children. Then, only the selected children get included. Be warned about this, as I ended up wasting time redownloading a few of these - "RMB -> Update to Revision -> Choose Items..." to fix problems from missing files)

  Despite avoiding the OpenCollada folder like the plague (following discussion with another dev who had grabbed those files), and also skipping a few of the other folders I knew I wasn't ever going to need, the libs folder still weighed in at over 1 GB.

  Another note about size is the size of binaries/build files: Although I'm now compiling a heavily reduced version of Blender (knocking out a bunch of stuff I don't actually need/use, especially when developing animation stuff), the build dir + binaries weigh in at around 830mb, while the same for mingw was 520mb (for more!)

For me, the size of this stuff is an issue, as I had just 3GB free space left on my C-drive, where all the Blender dev stuff is stored, so I preferred having some way to minimise the amount of space needed (so that I can fit a few more working tree branches around instead)!

2) mingw didn't work anymore. It was a bizzare error about missing files:
C:\blenderdev\master2\blender>

mingw32-make -f ..\build\Makefile
process_begin: CreateProcess(NULL, \C\Apps\CMake\bin\cmake.exe -H\C\blenderdev\master2\blender -B\C\blenderdev\master2\build --check-build-system CMakeFiles\Makefile.cmake 0, ...) failed.
make (e=2): The system cannot find the file specified.
mingw32-make: *** [cmake_check_build_system] Error 2

3) libmv can't be disabled in cmake - Doing so will result in an error like this:

Error    1    error LNK1104: cannot open file '..\..\lib\Debug\bf_intern_libmv.lib'    C:\blenderdev\master2\build_windows_core\source\creator\LINK    blender

4) HP are truly a bunch of boneheaded morons. Not only do they have pigheaded hardware designers (who create buggered hinges, colour skewed monitors, nasty keyboards, and unusably annoying trackpads), but they also have a habit of setting some really bizzare environment variables:

Namely, "Platform" is set to "MCD" instead of "x64" or "Win64".  WTF is "MCD"?!

So, if you ever get the following error on an HP when compiling with MSBuild:

Microsoft (R) Build Engine version 12.0.31101.0
[Microsoft .NET Framework, version 4.0.30319.34209]
Copyright (C) Microsoft Corporation. All rights reserved.

C:\blenderdev\master2\build_windows_lite\Blender.sln.metaproj : error MSB4126: The specified solution configuration "Release|MCD" is invalid. Please specify a valid solution configuration using the Configuration and Platform properties (e.g. MSBuild.exe Solution.sln /p:Configuration=Debug /p:Platform="Any CPU") or leave those properties blank to use the default solution configuration. [C:\blenderdev\master2\build_windows_lite\Blender.sln]

"Build Failed"

You'll need to go into your environment variables and do some fixing up work. (Oh, and remember to restart your console window to make the changes take hold!)