Wednesday, October 19, 2011

Moodle is a crappy bugfarm!

... or perhaps just the current version rolled out on the servers I've used it on was (though I wouldn't be surprised if most of the bugs still held)...

Like many other educational institutions around NZ, UC has been getting in on the whole "online learning" thing with a Moodle-based CMS (aptly named after a naming contest yielded the "best entry" by some random unknown as "Learn"). As diligent COSC student, and also dedicated UI design (semi-)professional, paying attention to the small details of how computer systems work and present themselves to users is what we do, and is also where we can aim to score brownie points over our competitors ;)


Having used "Learn" quite a bit over the past few years (in particular over the last year), we've noticed some things about the system which are quite dodgy and in some ways IMO fundamental failures in UX Design and/or implementation (perhaps issues that shouldn't be too alarming, considering the codebase is a muddle of PHP, Javascript, and/or other unknown evils we shall not talk of hacked together as a student project at a tertiary institution somewhere in the world).

1) Submitting assignments: the "Send for Marking" button
As I've mentioned before in an earlier post, sometimes adding an extra button that you think will clarify things actually ends up making matters worse. Well, this is one of those.

Any student who's ever had to submit an assignment electronically using the "Assignment" submission system will be aware that the thing has a stupid "Send for Marking" button at the bottom of the assignment submission page (unless of course, the lecturer apparently set it to the other option, which we shall call "instant death"). Officially, the page calls an uploaded submission as "draft" until you click the "Send for Marking" button, but after you click that button, it prompts about "finalizing" the submission and hence not being able to edit the submission again.

However, AFAIK, this button is completely and wholly redundant while being sneakily misleading, all the while offering no actual beneficial functions to the end user (aka the student in this case). The simple fact is that no student in the right frame of mind with any sanity left (a rarity during the dying stages of assignment crunch, but still not a sign of complete insanity to click) would (willingly want to) click on that button. What if 5 minutes later I suddenly realised I'd made a mistake, and left out a few files when creating the final bundle, or even submitted the WRONG bundle? What if for some reason I get some time later (even rarer, but it can happen) and end up overhauling part of the assignment?

After coming across this ambiguity several times and having asked a few lecturers about it, it turns out that even if students don't click this button, the lecturers can extract the submissions come marking time anyway.

Although it's possible to see what the devs were thinking ("ah let's see... we have a 'student can submit assignment' use case, where they'll do x,y,z, steps so we'll need a,b,c buttons... Brilliant! All implemented. We can tick this one off now"), there appears to have been a fundamental failure to think or even evaluate the UI from the perspective of someone actually using it would.

UX EPIC FAIL!


2) Quizzes : Submitting answers to individual questions vs "Submit All and Finish"
Somewhat paradoxically to the previous case, when answering quizzes, a similar thing can't seem to happen for partially completed quizzes.

With most quizzes, there are "Submit" buttons per question, which when clicked will tell you whether you got the question right. At the bottom of the page, there are three buttons "Save without submitting", "Submit All", and "Submit All and Finish". While quite a bit can be said about having these three options, I'm not going to bite into these here.

Now the issue is this: if a student goes through the quiz, answering each question and pressing submit after doing each one, then forgets to press "Submit All and Finish" (even if they successfully completed every question in the quiz) and the deadline passes, the system registers a 0 mark for that quiz. Ahem!

IMO, this system should really be taking the answers of the last "attempt" (even an "attempt in progress") on the due date, assigning a 0 mark to any question not attempted, and using that as the score for the quiz. This means that users who may have been caught out by finishing all the questions in the quiz, having their system crash before being able to press SAAF, and then subsequently forgetting about this as a result to other stuff coming up while the system reboots are not penalised.

The simple lesson of this story is: Relying on users to "finalise" transactions in situations where they're prone to suffer from premature closure error, as in their opinion they've finished performing the task already, but penalising them from not "appeasing the machine" is a bad strategy.

The second sub-lesson (probably) is: if you go about designing your UI's from the perspective of a "datahead", the resulting software is going to have the telltale and disgusting signs of "YAFDPOS".  At times, I have to wonder whether the design of this system falls into the whole "becomes" problem from OO inheritence, and ends up having one DB table for "completed quiz attempt" and another for "WIP quiz attempt". Same goes for issue #1.
(*1 - "datahead" = my derogatory term for anyone who goes out thinking about things in terms of database selects, joins, inserts, deletes, and updates, and trying to bastardise and regurgitate any and every problem into a stock staid CRUD representation that strictly appeases these demons. There are many of these people out there.)
(*2 - "YAFDPOS" = "Yet Another Freakin' Database Piece of Shit". Yes, I have a thing against databases, though I will admit they have their roles in some domains)
3) "Workshops" : framed in broken nostalgia
And now we get on to one of the buggiest parts of "Learn"...

Not since the early days of when I've played with the web, dating back over a decade now, have I seen stuff any modern web site/app that uses frames for anything. Javadoc might be an exception, but then again, that was done along with Java back in the 90's, being of a similar vintage to the cobweb-laden Swing, so that's probably not a fair case.

And it's not without any fair reason that most of the world abandoned these in droves like the plague, or just like how the use of the blasted "blink" HTML tag is frowned upon. It turns out that getting navigation to work in any predictable and/or sane manner when dealing with frame-heavy pages turns out to be a nightmare.

Yet, here is "Learn", a relatively new web app, which goes out and uses this broken concept for its "Workshops" feature, specifically for the part where students perform peer assessment of work submitted by others. How this works is that when you click on an "assess" link for a submission, you're taken to a page which is divided into two frames: the submission on the bottom, and an assessment form at the top.

When you submit your assessment (in the top half), you'd expect to be taken back to the page with the list of assessments, right? Well, it only gets you partway there. The implementation doesn't appear to have totally grasped the intricacies of frames programming, as just the top half navigates to the links page. The bottom frame stays put, displaying the last submission's content still.

Now, what happens when you click on another "assess" link? If you expected everything to work like the first time, you'd only be half right again. You see, it seems that the implementation totally forgets that there is an old frame lurking at the bottom of the page, so it goes ahead and divides the top-half view in half again! So now we have "assess" frame, "submission 2" frame, and "submission 1" frame stacked on top of each other. It only gets worse as you click on more and more submissions!

4) "Workshops" : cryptic numbers, premature submission finalisation, etc.
The horrors of the "Workshops" component doesn't end there. Far from that... it'd be letting you off far too easily for that to be the case!

In a further act of violating the principles of least surprise and internal consistency, the Workshop module's policy towards editing/continuing at a later date are in direct contradiction to anything else seen in the rest of the system.

It operates on a "can be edited until an arbitrary timeout or until session ends" basis. So, if you were typing up a submission in the browser (something you learn not to do in this case, but how would you know!), if for whatever reason you need to leave that session to work on something else for a while or whatever, then coming back, you could be left with a partially finished submission which you cannot edit anymore. Or perhaps you've gone through and given numerical scores for the submissions for your classmates, but wanted to come back later when you had more time to fit in comments. Oops... you cannot do that... you can only do that within the same session that you submitted an assessment.

This policy directly violates the behaviour seen with quizzes (with their "Continue Last Attempt" buttons), and can be a source of frustration and assessment issues.

It should also be noted that when all the submissions have been assessed, the system shows in curly braces a series of numbers beside each "score". No-one has been able to make any sense of these cryptic numbers, and the system's attempts at providing a glossary/legend to interpreting these only makes things worse. Gah!

5) "Wikis" : Crappymatic HTML Web-Editor Style
Adding insult to injury, the so-called "Wiki" capabilities of Moodle are a complete and utter joke. They're like a watered down, let's strap some wiki-like features on top of our crappymatic HTML web text editor and call it a wiki. Oh, and let's make creating new pages and/or linking to them successfully a complete and utter confusing misery (after 6 months of tinkering with it, I can still not say with any certainty whether an attempt at creating some pages will result in anything happening or going to the right places).

Not that their web-editor is anything to crow home about either. It's all to easy to end up with formatting inconsistencies, etc. But at least it lets you directly edit a long, unformatted blob of raw HTML code for the content if you so desire... Yipee! </end_sarcasm>

6) Random login troubles
Increasingly, as the university has been making increased usage of the system, we've been seeing and experience many more cases where login-troubles surface. Stuff like trying to log in an continually getting bombarded with "Session error" or repeated enter your password again boxes. All of these usually require closing the current tab and starting a new one, but the frequency with which this is occurring (3 times within the past day here) is concerning.

7) PDF Files
PDF's are a common format that are indispensable as a document interchange format which preserves formatting from end to end, reducing the plethora of charset and/or missing font problems that used to plague people sending documents around in the past.

Not so helpful though, is the display of PDF's in browser-embedded viewer plugins (or for that matter, any other content viewer which tries to open itself within a browser as a plugin, taking over the menu bar and the content area). Namely, the notoriously crappy Adobe browser plugin, with its combined tendencies to:
1) take ages to load everytime (minimum 2 seconds, maximum I've seen was somewhere in the 10 second range, at which point it would usually take the browser with it)
2) have immense troubles trying to render a page of content without lagging for a while and/or flickering when needing to update
3) potentially taking down the entire browser, hard, if when you try to close most tabs with an embedded viewer, especially if it was busy doing something

This situation has gotten so bad that I've had to spend ages disabling and removing every trace of PDF = load Adobe plugin, replacing where I can with "suggest to save to disk". (Fortunately, it seems that Chrome have said: let's do away with external vendor crap, and let's just implement our own PDF viewer using some combination of HTML, CSS, and Javascript, just like practically everything else in this browser - Out of interest, does anyone know if this is based on that HTML5 PDF rendering script that was doing the rounds a few months ago, lead by some FF guys?)

Anyways, it seems that Moodle has the bad habit of trying to enforce the display of embedded PDF's in popup windows if you normal-click on a PDF link. However, with all the aforementioned PDF-embedding disabling, it turns out that this just results in a blank window popping up. Why? Well it seems that that window proceeds to call some Javascript to try and launch a blasted embedded viewer to show the content in that popup window. Yaarrrggghhh!

In most cases, the workaround I've found is to open the link in a secondary tab, then click on the direct link there to save off the file. But it still sucks that for a few files, the popup window appears to be the only method of access :/

---------

So, in conclusion...
If you're still reading this an you're a Moodle developer:
What're you doing still sitting their like a deer in foglights?! Get cracking on fixing these bugs, and checking out whether there are any more where they came from!
elif you're some institution's IT adviser:
Get in touch with whoever your consultants advising you on selecting this option are, and demand that they get this sorted between them and the Moodle devs before you're willing to finalise a contract on this. 
else:
Beware of these pitfalls with Moodle. If your handy with web-tech, you may want to consider giving them a hand solving these problems. ;)

No comments:

Post a Comment