Techniques to adopt for Smooth Usability in Spatial Immersive XR Design
Whether you’ve tried a few immersive experiences or a large number of them in your time, one thing becomes apparent quite quickly – not all immersive experiences are created equal. Some apps and games feel slick, polished, and intuitive. Others can feel awkward, clunky, and unintuitive, falling short of their potential. Functional yet flawed. Not a showstopper, but something not you'd want in the spotlight either.
In game development, a fantastically evocative term has grown around these failings and shortcomings, and has now become a widely used and well-understood descriptor: ‘Janky’.
And what's fascinating is that VR, AR and MR have become the new showcase for jank of all types. It's so frequent and widespread that expecting some janky aspects has just become the norm for immersive spatial experiences. XR users and developers seem to have accepted that it's something that is practically inevitable. And yet it's rarely viewed within development as something that can be factored into development plans or that should be considered during initial ideation and design.
That's not entirely true, and just like any risk that could derail your plans, there are early steps and decisions you can make to enure sure your Immersive App or Game can present a fantastic and jank-free user experience.
First of all, what precisely is Jank? The word emerged from rap slang in the 1990's, and was used to describe something run down, poor quality, unreliable or undesirable. Janky has since become a common term when critiquing technology and media, with it finding a particularly fertile and natural home as a descriptor for all the unintended outputs that arise from the complex and unpredictable systems of videogames. Audiences are now as likely to use 'jankiness' as a measure of a game's worth as commonly as playing time, accessibility considerations, it's gameplay, or the overall entertainment value it offers. The difference berween "It's a bit janky" and "it's pretty janky" can easily make all the difference when a user is considering picking up your app.
It’s a term that has a range of subtly different meanings summed up well by Ryan Cooper in his 2018 essay ‘The joy of Jank’
“Jank is ... a mixture of bugginess, minor glitches, strange animations, bizarre control schemes and any other number of possible occurrences or abnormalities... Much like porn, you know it when you see it.”
Jank is a kind of catch-all phrase for certain kinds of weirdness and lumpiness that’s eminently noticeable by the user, but that stops short of making a videogame unplayable, or an app unusable. Jank doesn’t break your experience outright, but it’s clearly something that is not up to the standard the user, and often the developer themselves, is hoping for.
That can arise from many different causes – bugs, technical and performance issues, or rough and unrefined design or implementation – and often these are caused by a lack of time, money, skills or resources: factors every production has to confront in some capacity. So while Jank is never game-breaking, it can have a critical influence on the perceived quality of your experience, and that can impact your title's reputation and success.
Sometimes, it can even become a feature, or showcased as part of the identity of the game itself – a whole genre of games like Surgeon Simulator, Goat Simulator , Manual Samuel and QWOP were born which capitalize on the unpredictable fun and charm that can result from a purposely obtuse and flawed implementation of their physics and control systems. The term ‘Jank’, then, is really describing the disconnect between the user’s expectations about how the elements of an app or game (interface, rules, mechanics, visuals, etc.) ought to behave, and how they actually do behave. Turns out that playing within that disconnect space, and never quite being able to predict the outputs from your inputs can be both hilarious and frustrating in equal measure.
Embracing the Jank is one way to accommodate it
Clockwise from Top-left : QWOP, Goat Simulator, Manual Samuel and Surgeon Simulator in action
With Surgeon Simulator, the disparity between the player’s control inputs and the game’s subsequent outputs are core to the fun. Goat Simulator was born from a broken mess that was silly and fun because of it's problems. The jank brings the fun to an otherwise predictable experience by subverting your expectations and surprising you with unpredictable outcomes. Without having to adjust to the janky, vague and disconnected control of your Surgeons hands, there wouldn’t be much fun. If the athlete in QWOP or the poor soul in Manual Samuel were as direct to control as other games, the core entertainment factor of them would disappear. Like challenging someone to build a house of cards while wearing rubber gloves*, the fun lies in the effort and adjustment needed in trying to overcome the disparity and still complete the task.
These are all 2D pancake examples, of course. In XR experiences, jank feels endemic to the experience itself. At this point in VR's history, some degree of jank is almost always expected at this point. Read almost any review of a mid-budget VR game, MR experience or AR app and the reviewer will often dispassionately state that ‘there are expected levels of jank’, its presence accepted and dismissed as rote. This isn’t surprising of course – our expectations of how things will behave is based on real life, and as we all know there are often large discrepancies in the quality of how things look, feel and behave in extended/synthetic realities when compared to their real-world analogues. And in terms of software maturity, most desktop or smartphone applications feel more refined, more fully developed, offer a more satisfying interaction experience, and are generally just more jank-free than their VR equivalents. Not to mention that the demands of wearing the equipment add another layer of consideration for the user at the heart of your experience.
If you’re developing a VR, AR or MR app, then, Jank can be pretty hard to avoid, and is casually-but-reluctantly accepted by your users as just being a regrettable but expected ‘cost’ of those mediums. We do our best to avoid it, but we know that sometimes a little stutter here, some wacky physics there, a model pop, a loading hiccup, or some unfortunate design choice will rear it's janky head and leave a sour taste in your user's mouth. No matter if the other 97% of your application is flawless, all it takes is a few too many moments of jank, and the user's impression of your experience goes quickly south. In the last days of development it's not uncommon for a game or app to feel lumpy, inelegant and in bad need of refinement, and we're faced with releasing a janky game and hoping it survives the initial reviews until we can get the worst of it patched.
It can be a nightmare of triage and questioning whether you made some fundamentally wrong choices way back when - and wondering what you can do to fix them now.
But it doesn’t have to be so.
*(I shamefully admit, I cite this particular example since this was the test that eliminated me from my tryouts as a fresh-faced 21 year old Law student for the TV show The Crystal Maze back in 1990 (after one of my classmates got onto Blind Date), something I’m still bitter about as none of the challenges on the show itself ever really requested that the contestants try to overcome a similar layer of interface-jank to succeed, and my failure demoted me to the 'reserve squad' who never got a chance. My career as a popular TV celebrity was killed before it ever started, so on this timeline Hollywood never called. Now I dedicate a fair portion of my client work hunting and eliminating jank, and it is of course to avenge that lost opportunity).
What is smooth usability and why is it important
Half Life: Alyx is still VR's high watermark for smoothly usable interaction and navigation systems
I recently heard a non-designer friend describe the seminal Half Life Alyx – arguably VR’s most accomplished and jank-free gaming experience - as having ‘smooth usability’, which is a perfectly brilliant way to describe the opposite of jank.
While ‘Zero Jank’ might be the desired aim for any immersive experience, it’s a pipe dream that you can't really plan and build for. Things can always be smoother, snappier, shinier. ‘Smooth Usability’, however, feels quite measurable, quite quantifiable, and fairly straightforward to plan for as part of development.
Working to implement smooth usability won’t eliminate all the jank, of course, but it will address the causes of many of the causes that commonly lead to janky elements in your finished experience.
We're looking to narrow the number of places and ways that jank can manifest in your final product, by observing good design and planning practices during the earlier stages of development. Here's what you need to know.
What to know about achieving 'Smooth Usability' in XR
1. It’s all about user expectations
Like almost every aspect of spatial immersive XR design, user expectations are absolutely the key consideration that should shape how you proceed. If Jank results from the disconnect between the user’s expectations about how elements “should” behave, and how they actually do, then the most important element to be thinking about is what the user’s expectations actually are at any given point in the experience. It’s crucial to remember that your design choices need to be grounded in presentations and interactions that you can be confident will behave as the user expects them to.
Rather than trying to figure out how to close the gap between expectation and reality for any given aspect you have included, and thus minimize or eliminate the jank, it can be much easier to simply choose to use interactions and presentations that avoid jank altogether. This may sound obvious and even trite, but in my experience helping clients, smooth usability is something they often attempt to massage into the mechanics of the experience after they’ve built it.
But the reality of XR development is that you make choices that are one-way, and that can affect neighboring mechanics and systems significantly. By the time you are reviewing the systems you've implemented with an eye for quality instead of just functionality, you might realise that you committed to a fundamental design choice early on where some degree of jank was always going to result, and you'll have to accept that you'll never be able to make that interaction or moment feel as natural, intuitive and polished as you were hoping for.
In most cases, hindsight can at this point identify another route that would have been easier and more straightforward to implement, and that would have more completely satisfied the user’s expectations and avoided much of the jank. Invariably, this is a direction that needed to be committed to during early planning and prototyping but sometimes the implications of the choice do not manifest until much later. I have helped many clients to avoid this situation, and worked to rescue others that have realised their mistakes too late. But the closer your project gets to the finish line, the more you will realise that these janky moments are here to stay, and that there is often no way to eliminate the cause of these janky elements without being turning back the development clock.
Let’s quickly look at the seminal Half Life: Alyx for an example to illustrate this. If there is one weapon that the Half Life series is best known for, it’s the crowbar, which is traditionally the protagonist’s signature melee weapon, always the first weapon the player acquires, and an aspect that dictates and informs large parts of the game's design. Much of the combat, exploration and puzzling the series is famous for is designed around, and enabled by, the interactions and behaviors of the crowbar. It's absolutely core to the title's DNA.
But to bring the series successfully into VR, the developers dropped the crowbar completely. As developer Robin Walker explained, the team “lost, like, a year and a half of our lives trying to get a crowbar in VR.” Not only is melee combat hard to implement with smooth usability in VR (even the best examples of melee implementation today, such as Blades and Sorcery, Battle Talent, and Until You Fall, have janky elements and unintuitive mechanical aspects which seek to overcome the lack of physical contact and resistance) but it actively encouraged players to behave and think in ways that didn’t fit with their new protagonist, Alyx herself, and the embodiment in that character that they wanted players to feel.
Despite achieving a highly competent VR implementation compared to competing melee combat titles, the crowbar added plenty of janky elements when carried, but not being used, by the player. As Robin explains, “It certainly unlocked a lot of gameplay we liked…but it had lots of problems”. Ditching the weapon altogether removed those problems and their associated jank, but at a high cost, losing access to gameplay opportunities they had fully committed to, and necessitating a soft reboot to replace many of the core mechanisms 18 months into development, and with it forcing much of the neighboring design and planning into redundancy.
But it was unarguably worth it – removing a janky element, no matter how significant to the IP and beloved by the fans, removed an inevitably janky element that would never satisfy the player’s expectations, and never enable the smooth, deeply immersive experience that ultimately emerged.
The challenge with this approach, of course, comes with being able to prototype and test these elements early enough to identify and remove any unfixable jank-providers before moving into the more expensive phases of production. Experience can count for a lot here, so working with a consultant (such as myself!) can help to identify un-fixable and unwanted janky elements that will arise from your user's expectations meeting your planned features, and the earlier in pre-production you can catch them, the less chance they will have organically grown their tendrils into your wider design in a way that's hard to reverse.
2. The benefits of Intuitiveness
The second thing to aim for if you want to deliver a best-in-class, jank-free and smoothly usable experience at the end of production is to focus hard on choosing and creating intuitive mechanics. Implementing interactions that we can do on autopilot is the key here.
We rarely think about the mechanics of how to interact with familiar things in real life, and when we interact with the virtual world, we subconsciously expect the same level of intuitiveness.
When an interaction behaves outside of expectations, our 'auto pilot' brains have no choice but to instead switch into a higher priority processing mode, to dedicate time and effort to actively process what we’re experiencing and figuring out how to instead make it behave to produce the output effect we are expecting. This is a model originally proposed by the psychologists Keith Stanovich and Richard West, splitting our brains into an 'autopilot' brain 1, and an 'active processing' brain 2, and it's a perfect model for understanding the thresholds for an intuitive interaction.
Any interaction we perform in an immersive medium where we don't ever think about it is a successful one - for the magic tricks of immersion and believability, it's important to keep the user's active processing focused on the diegetic matters inside the virtual world, without needing to think about the trick itself, or why it's not working. As soon as we have to intuit what those missing steps in the process are, our brains are having to deal head-on with this intruding bit of jank and it increases the cognitive load. At that point, the mental process is firmly in brain 2, and is now the very opposite of intuitive.
There can be a lot of those situations that crop up in XR. That breaks the magic of our immersion, and tasks us with thinking about the mechanics of the operation itself. Jank in immersive mediums can be very much characterized as moments that remind you that none of this is real. Sustained believability is the key.
Many interactions that feel intuitive in the real world can work the same way in virtual mediums, but many of them fundamentally can't, because of all the differences in interface, fidelity, and feedback that can highlight their falsity to the user - something we discuss on this site often. The designer’s skills at identifying the right mechanics and interactions to use here is of paramount importance to the perceived quality of the end result. We always have a choice when building immersive experiences about how interactions should work, and my advice is to always figure out what works well and feels intuitive and then rely on those interactions as much as possible, rather than trying to bridge the disparities between what you communicate is going to happen, what the user intuitively expects, and what the tools at our disposal can deliver.
A great example of this is doors in VR. When was the last time you had to think about how to open a door in real life? We’re all very familiar with the mechanics of how a door swings on it’s hinges, how turning a handle enables us to open some doors, and how it behaves when we push or pull it. Probably the only times we ever notice the mechanical interaction we have with a door IRL is when we push instead of pull, or when it’s heavier to swing than expected, or when it’s stiff because it doesn’t fit correctly in its frame – and even then, it’s rarely more than a single moment of cognitive recognition and acknowledgement of how to modify our behavior to get the result we want.
Virtual doors in VR though are notably janky. They can take an otherwise smooth, polished, and immersive experience and in a moment turn it into a clown show. The weighting is wrong and they flap about unrealistically, the way we interact with handles can be sticky and make it hard to ‘let go’ of the handle like we would IRL, and they often open both ways as a compromise to usability. Almost every door in VR experiences can and will wrench the user out of their immersion, and tuning their behavior to match our expectations can be a long and tricky task. Dungeons of Eternity, an otherwise deeply immersive, polished and smoothly usable Meta Quest 3 title, fails hard at this particular hurdle - and it really shatters the sustained immersion the user has been enjoying when it happens.
Most of the doors in this game have been replaced with ‘force fields’ you pass through, which are mechanically non-problematic, and thematically justified by a 'dungeons and dragons but a bit cyber' theming. But the expectations of a classic dungeon-crawling experience meant the developers wanted to feature locked rooms with loot inside (knowing what their audience will expect) and while the key-unlocking is mechanically sound, the unlocked doors are simply awful to use in their execution, flapping and banging uncontrollably with the slightest touch, and adding an unnecessary layer of jank to an otherwise stellar VR implementation.
The developers at Othergate have admirably aimed to implement doors that can be open and closed at will, under natural physical control, because it should be a great, immersive and intuitive interaction. It rarely works well, however, and even though it is trying to do something that seems both necessary and achievable, it trips over it's own ambitions and requires active concentration and focus to navigate what should be a natural, instinctive interaction. Go Go Brain 2! It's an unnecessary indulgence, because gameplay-wise there’s no real requirement to ever close these doors once opened. Rather than a complex physics model seeking to recreate the behavior of a real door, the developers could have the doors animate opening when you first push them, and then keep them forever open. This would have removed this singularly noticeable janky element in the game, and the door’s behavior, although not as flexible as a real life door, would have worked fine for the utility it needs to possess in the game.
In summary, if you can’t make something behave intuitively, think hard about replacing it with a different implementation where the jank is less likely to surface. VR mechanics are one of those things in life that succeed when you don’t notice them, and fail if you do.
3. Bury the mechanics of your interaction deep, and deliver the expected outcome.
And with (2) above in mind, it makes sense to avoid interactions in which the mechanics get showcased and put under repeated scrutiny. If it’s not intuitive, the user has to start actively figuring out what they need to do, and that means them deconstructing the live experience and debugging the problem for themselves. Asking the user to look too closely at the interactions they’re undertaking will just highlight all the aspects that don’t behave as expected, and in VR, MR or AR there will always be unexpected behaviors.
Instead, figure out the user’s intent, understand the outcome they expect, and deliver that outcome.
Returning to our Dungeon Door example above, the user’s ultimate intent when interacting with the door is simply to have it open and no longer block them from entering the room. We open doors on autopilot in real life, and we don’t have to think about it. Some kind of 'True-Door®' behavior modelling is a nice-to-have feature, but only if it works intuitively - and that means the user isn’t consciously noticing the behaviors or the mechanics enabling them. It’s a lot of work that, if it’s done right, the user won’t ever consciously notice, but if it’s done wrong will actively hurt the immersive qualities. The user doesn’t give you bonus credit for trying hard, but they will look poorly on something that doesn’t behave as expected.
Instead, the developers might have just replaced it with another kind of force-field door that requires some kind of gem/key etc to remove the force-field, and then stays forever open - which would have fit perfectly with the logic of their other doors in the game. Or perhaps rather than cleverly modelling the physics of the door so it’s fully interactive and controllable, they could just create a believable animation that opens the door when the user unlocks it or pushes it, and leaves it open. This could be modulated to play faster or slower based on the user’s input forces, and give the user even more of an illusion that the door is reacting to their interaction as expected. Either of these approaches, where the expected outcome is delivered but there’s a much lower chance of noticeable surreality in the way the door behaves, is probably more likely to deliver a smoothly usable feel to the interaction, and be a lot less work for the developers (and less frustrating for players).
In a synthetic reality, emulating the expected outcomes to evoke them successfully is nearly always better than building a complex simulation of them, the challenge of VR being that you can rarely (if ever) guarantee a 1:1 match with reality. As long as Brain 1 can tick off the things it's expecting, and doesn't encounter anything it's not expecting, you've a much better chance of keeping the user immersed, and the interaction feeling intuitive enough that they don't notice the missing elements that can't be convincingly expressed.
4. Put the effort where it will be encountered, and choose your battles wisely
Accepting that some jank will still make it in despite your best efforts is just being a realist. Doing what you can to make sure the fewest number of your users will encounter it is the next logical step. Of course I would always encourage developers and creators to aim for 100% jank removal wherever possible, but it rarely IS possible. The quest for perfection sooner or later needs to take a back seat to business realities.
One such reality is this: people don’t always finish games (a 2019 Steam survey suggests only 14% of players tend to finish games they buy) and it’s safe to assume they don’t all explore the furthest reaches and most advanced features of apps, either. Apposite to the sentiment that games are about the journey more than the destination (and opposite to our narrative instincts), the truth is your App's most advanced features or your game’s narrative ending probably matters less than almost every preceding part of the player’s journey, at least in terms of how it affects the perceived quality, simply because they'll see the least amount of footfall.
Front-loading your development focus and testing effort to concentrate on making a great first impression can make a lot of sense from this point of view. The content and features that your user will definitely encounter in their first session are going to dictate the impression they take away, and that's going to inform how soon they want to come back and revisit, and how enthused they are going to be to talk about their initial impressions with others. And since XR apps are still sold almost exclusively on word-of-mouth recommendations and in-store user reviews, first impressions can be disproportionately important in how your app or game is perceived by potential customers.
Initially that might strike you as being somehow underhanded or tricksy, but it's normal to concentrate on making a great first impression - spend enough time with any person, product or service and you'll eventually be able to form a fuller opinion, but there's got to be an attractive enough reason for you to stick around in the first place. You're in first date territory with your users, so present your most polished and likeable version of yourself. Plenty enough users will post store reviews, Reddit thoughts or fire off Xweets to give their first impressions, so this is obviously a key high effort area. Good reviews will give you the best chance for buoyant sales, and that buoyancy may mean your customers provide the funds to go back in and try to raise the bar in those sections you weren't able to address before launch. But if there's too much mention of 'Jank', you'll be fighting to raise the title's reputation and engaging in rounds of fixes and polish which will become increasingly hard to attract customers with and don't make for good marketing stories. More polish should always be a positive, but past a certain point it starts to look like you're perennially fixing up your app, and people wonder how bad it can be to still be needing repair work after months in the wild.
As well as that first-contact experience, there are multiple other high-value areas to focus on;
key areas that you can be confident will benefit from special attention. Many of them are the things that every user of your software will encounter while using it, and so these are high value areas for you to focus on, ripe for imbuing a sense of quality, finish and that feeling of smooth usability into your game or app.
A few of them are areas that are sometimes treated as an afterthought by developers, but that can actually be constant touch points encountered by your users during their time in your virtual world. Don't forget that users aren’t aggregating and totalizing an overview the jank in the app the same way that someone with a developer’s purview might; instead, they’re running into that jank whenever they run into it, every time it happens, for as long as they're exposed to them. The jank they don't see doesn't have a negative effect.
So it makes good sense to figure out where your users are really spending their time, and make sure the portions of your app that they encounter most frequently and for the longest periods are high on your agenda. Those contact points are worth serious consideration as places to spend extra time and effort if you’re looking to raise the overall sense of quality, usability, and care that your experience communicates to your users.
Jank, Witcher-style | Image CDProjectRed
Join us next time in Part 2 to find out what these key contact points are, how you can make best use of them, as we continue exploring tips and strategies to help you make sure your jank isn't showing.
In Part 2 of this article, we'll be identifying unexpected areas of VR, MR and AR experiences where your jank might be showing, and why they can be of benefit you as priority targets for polish. Don't touch that dial!