Response to the "Action Fake User" issue

or Answer to "Where did my Actions go 2.62?" ...

Wow. I've just gotten back to checking various things afterseveral busy days of being swamped with work and deadlines, and it appears that a massive firestorm has been erupting in multiple places over the past few days. So I'll try to take some time to respond and hopefully "diffuse" the situation a bit without inciting vitriol (fingers crossed).

Firstly, let's get a few things out of the way first:
1) I understand very well that loosing data is not a nice experience. It's happened to me before, and yes, I was quite grumpy and angry about it at the time.
2) This may or may not come as a surprise, but I'll freely admit that the Game Engine and related workflows have never really been a priority for me, and that I am strongly biased towards optimising things for character animation in film. What this means in practice is that I don't actively try to make life terrible for people working on game animations; rather, I won't focus too much on making any improvements for those workflows.
3) As usual on the internet, things seem to have been blown out of proportion a bit. IMO, once again, there is an undercurrent of, "let's throw the baby out with the bath water" just because one tiny aspect is deemed offending. It's not the first time I've seen this, and I'm very sure it won't be the last.
4) "Muscle memory" (or muscle-memory-like effects) is an absolute PITA. I'm getting increasingly annoyed and frustrated about problems related to this, and the plague-like gentility effects this has on getting anything done. At this rate, we should really all still be using command-lines or programming using punch cards!
5) You can't please them all.
</end rant>

Reasons for the "offending" change
Here are some of the factors which influenced this change. Some of these have featured in other debates on this over the past few days.

1) Over the years, I've seen countless complains about problems to delete actions, hundreds of otherwise unused and empty actions lurking around in files making them balloon out to several hundred meg, and mutterings about the convoluted systems in place to clear fake users. Indeed, there have even been vocal pushes and movements to have this changed over the years (note that this is in the plural).

2) When I publicly solicited feedback on this, there were NO OBJECTIONS at that point. In fact, I didn't hear of any objections to this for quite a while after leading up to the first release that this featured in. IIRC, there weren't even many (if any) calls for the behaviour of adding fake users by default to be kept in any cases where the issue of fake users popped up in the past. Heck, there weren't really any major complaints that this was bad even post-release; though for fairness, there were finally people who noticed that this had changed and wanted to know what changed.

But IMO, this is really symptomatic of some greater issues we've been having in recent times, which are not really isolated to this case.

3) The fact that Actions were created with Fake Users was inconsistent with the rest of Blender. No other datatype was created with Fake Users.

4) Some may wonder why they were made this way in the first place; "Surely there must have been a valid reason!"

AFAIK, the original actions system arose in the "Not a Number" days as part of the NLA system. This was very much inspired by Hash's AnimationMaster, and Blender at the time (early 2000's, dotcom bubble era) was trying to position itself as an integrated end-to-end tool for game design. There was also a mentality that animators could create libraries of reusable actions, that could then be chucked in the NLA or in the game engine and just used as-is. In such a system, it's clear to see why making Actions get Fake Users by default makes sense.

However, there are several things that have changed since then that means that circumstances are not the same as they were back then.
- For one thing, the Actions of today are really the IPO-blocks of yesterday, with some additional capabilities on top. Now, as we know, the IPO's of old didn't have Fake Users on by default :)
- Years of experience have shown that it is near impossible to really have truly reusable libraries of actions that could be blended together in a NLA. For that reason, the 2.5 NLA system really tried to put a greater emphasis on NLA-like systems for performing "layered animation", whereby we recognise the fact that people need to be able to apply corrections on top of rough action libraries, or even just animate in a layered (refinement passes) workflow. So the idea of creating action libraries becomes much less important.
- With a more "just keyframe it" attitude to animating, it makes sense that the actions that do get created are only useful while they are referenced. For the other, smaller use cases where you want to use them elsewhere, the tools in those cases "should" (*) either make sure they reference things and/or users should mark the actions as needing special attention/should be kept. This is the same as pinning, starring, flagging, or labelling items in other application domains.
  (*) As things currently stand, the tools don't currently do this. This is something that needs some tweaks.
- The last factor leads into the next point...

5) Game animators do not seem to be such a major use case for Blender anymore. Or, put another way, the game making community in Blender is a very small minority. Maybe it's just my bias creeping through here though ;)

As such, the importance of explicitly marking data as being "important" didn't seem like such a massive thing for the majority of use cases faced by Blender users. Sure, a few users might have to do it, but they're explicitly creating these libraries already, so it's not like they aren't putting in any extra effort by not specifying their actions as being note worthy. [NOTE: check on some of the proposed solutions before complaining].

6) Removing Fake-Users was actually a hang-over from the original Animato design in 2.5, that, for various reasons (probably it was a combination of laziness of going through making sure this was done properly, and other more pressing things to get implemented again to make 2.5 usable) never really got done. Over the course of the 2.5 series, the change couldn't be made either, since we were moving towards feature stability and removal of bugs.

However, the change to a 2.6 series release provided an opportunity to resolve this issue. There is a generally accepted convention that on "major" version number increments (please don't bring up that old debate about Blender's version numbering scheme; this is not the time and place for it), that some relatively major changes may take place from the previous versions. This is what happened here.

Communication of the change
Some people have commented that the change was not communicated clearly. To a certain degree, I agree, and apologise for some of the mistakes here that apply to things done from my end. However, once again, there are a few more sides to the story here.

1) I personally noted this change in at least 3 places: in the commit log, in a standalone post here noting the importance of this change (which IIRC was and has been broadcast in a few other places), and in the list of important changes made in the GSoC11 Pepper branch. Now, this was only announced in these places as this was before the changes had been merged back to trunk. Mentioning this in other places before this had happened would have just lead to further confusion. I do not see any alternatives here yet.

2) The GSoC11 Pepper branch page was NOT originally meant act as (part of) the release notes for that release directly. That was not my intention at any stage. It was assumed at the time of writing that, as in the past, this would form the basis for a revised set of notes that would be made as part of the official release notes. Such an official page would have outlined any such issues much much clearer, and made sure that the most important issues were properly noted in a clearly identifiable place (i.e. at the very start).

3) The GSoC11 Pepper branch page was just a list of items in animation system categories, not ranked in terms of what may affect game engine users. In retrospect, perhaps it might've paid off to note the potential of data loss here as well. Will note this for next time.

Catch-22: The silence before the post-hoc flameballing
This problem has come up before, but it's still happening now, and has lead to this whole debacle.

For the past few years, we've had a situation where people are all too eager to jump on a flameballing bandwagon after a release, complaining to high heavens about a "worst bug evah", calling for heads to roll, and otherwise saying that everything is unusable/broken. On the other hand, leading up to these releases, we provide ample opportunity for people to test the software, find bugs or problems with it, and complain/tell us about it so that we can hopefully resolve or at least warn people about them. But few people ever complain or bother to test. And an even smaller number of those people even come anywhere close to picking up or mentioning the problems that a vocal few pick up later on.

The fact is, we cannot fix the problems which we cannot see. Sure, we usually test these things for their intended use cases, sometimes for a few other corner cases, and sometimes stumble across things in actual usage. Yes, some developers do actually use the software, just not as often as most artists do, otherwise, who does the coding work!

However, it must be noted that we cannot also "perform usability testing" as one commentator casually remarked, as if it were just like walking down to the dairy and grabbing a bottle of milk, and that it's a sin if you don't. To do so, you need a willing pool of users, readily available to perform your testing on. We simply don't really have that. Sure, we have a user community willing, even begging at times, to test fancy new stuff, but the smaller workflow stuff just falls off the bottom. IIRC, I've tried doing this before, and failed.

Also, the geo-distributed nature of the development model really shows up here. If there were situations where the statement "usability participants are a precious resource" really rings true, then this would would be it. We simply cannot just walk next door and find experienced Blender users (the exception not the rule is when the developer in question happens to be on a stint in the Blender Institute during an Open Movie Project, with a room full of "stressed" artists next door).

Data Loss Prevention 
A few people have mentioned the general principle that applications should not lose data. Indeed, this is often a good argument.

However, I believe that with the modern state-of-the-art version control systems that there should really be no excuse for anyone not to save backup versions of important projects they are working on anymore. (Granted, there was a project recently this week where I did not do this, but I actually came to regret that decision halfway through). Recently, I've been increasingly putting every single important project under version control (at one stage, I even considered that unversionable file systems should really be phased out, and that distributed version control technology should become a standard feature or even replacement of all file systems) and committing after each significant/semantically-linked block of changes. The advantage is that there is no longer the need to fear accidentally bludging your file, as you can roll back to an earlier version you worked on, without filling up your working directory with dozens of different versions.

It also gives you the chance of taking/saving out multiple versions of the file, and then manually stitching together the nice parts of each to get a new hybrid. That would have saved many of the people who lost whole sets of actions in this case, as they'd have been able to fetch out the different .blend files committed after each action was done, and merge these actions into the final .blend safely.

However, if we just rely on Blender's autosaved old versions, we're limited to a granularity limited by what the system deemed important (i.e. between file saves, or at a fixed time interval), which actually limits the usefulness of these backups for restoring good versions after significant semantic-editing sessions. Also, this system quickly fills up directories with many copies of the file, which may actually not be too useful either.

What we can do about the Actions problem
1) Revert the change wholesale.
This seems to be the desired outcome for some people for obvious reasons (satisficing, resistance to change, muscle memory, etc.)

However, I do not support this approach. It's not solving the problem, but just moving it back to where it was.

I also liken it to the trigger-happy, just-demolish-them-all mentality that CERA has increasingly taken to buildings in Christchurch over the past few months.

2) Fix the datablock deletion policy and asset management system
This has been discussed to death already and would be a "nice to have" or ideal solution, but it is a large and ambitious project in itself. As such, this is unlikely to occur in the short term, and as such, probably doesn't solve our problems.

3) Active notification of data loss potential
Option 1 - One option would be to actually warn the user when they try to create a new action. For example, if a user tries to add a new Action for an object, and there's already an Action there but with just a single user (but no Fake User), then there could be a prompt warning about the potential for data loss. I'm not sure on the best wording for this yet, but this is definitely an relatively well-contained option we can pursue, perhaps regardless of any other changes that may or may not take place.

Option 2 - Another option would be to have a two part system for notifying users of what data will be lost everywhere.
  - The first would be included as part of the Status info in the Info Bar, and would state the number of datablocks that will be lost on the next save. Clicking on this could bring up the actual information (open new window?), or at worst, just show a tooltip redirecting users to check on this.
  - The second would be a new view in the Outliner. This would list, by Datablock type, the datablocks that have zero users, and which are going to be lost. There should be readily visible tools/widgets beside each to add a fake user or so to rectify the situation.

4) Apply Patches - "Game Animation Improvements"
Thinking further beyond the raft of solutions proposed already, this is perhaps my preferred option currently.

I've been reviewing a patch by Tom Edwards (IIRC; go search the patch tracker yourself for details) which includes some tools which may actually solve this problem at its root cause instead of trying to defer it for another day. While I had been waiting for some tweaks to be finalised before re-reviewing it, in light of this debate, I've reevaluated its worth and importance for inclusion in trunk.


The proposed solution here solves the problem that with Action Libraries, we have no way of associating them with the data that they can be applied to. If we can explicitly link this up, with some additional incentives of facilitating easier previewing (such as allowing some previewing which loops through different actions to test the transitions) and/or updating on bone renaming, then we really have no problem with lurking zero-user actions getting "lost". This is what a major part of this patch does.