Musings on the Current State of Kotlin Multiplatform (Q3 ’19)

Musings on the Current State of Kotlin Multiplatform (Q3 ’19)

I’ve spent a lot of time chatting in Slack and on Twitter about the state of Kotlin Multiplatform. These are usually piecemeal conversations but often center around common themes. To spread the word better and help streamline these conversations, I’m going to write periodic posts about the current state of the KMP ecosystem.

Catching Up

I think we can start by revisiting some predictions I had last year around KMP in 2019.

 

Mainstream Adoption

The overall prediction was that we’d be seeing recognizable names diving into KMP and releasing serious apps. Not exactly a wild guess. It is safe to say that was correct, although with a caveat. We’ve been having lots of conversations with teams evaluating and shipping KMP, but these are mostly private. Similarly, if you installed an app using KMP on iOS, you wouldn’t know it, which is the point. I expected tech companies to be a little more forward about the interesting tech they’re using, as I think it really helps with recruiting, but it turns out that not everybody thinks that way.

It is still early days, and you still need to know what you’re doing, but production code is certainly shipping with KMP in Q3.

Multithreaded Coroutines

Not yet. I was *way* off on that. Will discuss later.

Compiler Plugins

This is also “not yet”, although if you want something very specific you could persuade the Kotlin Native team to let you bake something in (if you’re looking for something serious to work on, mockk native). The Kotlin compiler in general is going through big changes. There’s a consolidation happening that will be good long term, but we’ll be waiting for official plugin support in the short.

Other Libraries 

I mean, sure.

Paid License Debugger 

This prediction is interesting, and meta for me. We released an Xcode pluginthat lets you debug Kotlin. Jetbrains has also released a version of App Code that can be used for debugging, and recently a native debugger for Intellij, although all of these tools will need some future polish.

Reactive Library

Yes, ish. I was thinking that would happen after Multithreaded coroutines. There is Reaktive. It is more of a “traditional” Rx-like library. I think most apps at this point use something built in-house.

Q3 2019

Rather than just a big rundown of where things are at, I’m going to create sort of a composite character and consolidate multiple conversations I’ve had recently.

Q: Is anybody currently using KMP in their apps? There have been a lot of blog posts from earlier in the year, but the situation seemed flakey. Has there been much progress since then?

A: Yes! People are using KMP in apps. KMP is relatively new, generating lots of excitement, and moving very fast. That means there’s been a lot of aging content. If getting started or looking for updated info, it’s best to check out the Kotlin Slack, check out some tutorials, and see more detailed docs. If you’re looking for a super quick intro, check out our getting started series. We should have a few more coming out soon. They’re quick, and we’re going to keep them updated with each release, so they don’t age too fast.

Q: We are exploring the possibility of using Kotlin MPP for business logic and architecture (storage, networking, etc). Is that feasible and how much logic can be shared?

A: That is really what KMP is designed to do, so it’s definitely feasible. It will not be easy to build UI’s, certainly not shared ones (although there will certainly be shared UI options in the future). What’s really missing at this point are some libraries you might want to use, like date/time support, or local file access. The good news is that 1) library work is accelerating as the community grows, and 2) Kotlin has an optional interop design, so it’s easy to plug in whatever you want from the native platform to implement a missing feature. If you can’t find a library for something you want to do, it’s easy to just code it “native”. Date formatting, for example.

Q: I’ve heard coroutines aren’t supported on iOS, and they’re needed for concurrency. How does that work?

A: Interesting topic. Coroutines in native (and JS) are single-threaded. That’s not the same as unsupported. You cannot, however, do all the stuff you would on JVM/Android. That has definitely put off some developers, but this is really multiple questions. Coroutines are not needed for concurrency. In the context of native mobile development, we usually want to do something off the main thread, then return to the main thread. You can do this, and it’s straightforward to write. We use suspend functions in the Droidcon app where the main thread suspends, a background task is run, then execution resumes in the main thread. Flow is more complex, but we’re using it in a Firestone SDK wrapper currently (not public yet). Ktor uses suspend functions instead of callbacks for network calls.

However, the whole community expects multithreaded coroutines will significantly increase adoption. There are no target dates, but there is ongoing work and I’d bet something to appear around Kotlinconf.

Understanding native concurrency helps the discussion. I have a couple of blog posts (and that’s what I’m talking about at Kotlinconf):

Q: What’s the Swift and general iOS interop situation like?

A: There are the mechanical interop details, then there’s what I’d call the Swift developer interop, which is related, but different. Swift interop with Kotlin is through an Objective-C interface generated by the Kotlin compiler. That means whatever issues a Swift developer will encounter with Swift to Objc interop will be present. On the other hand, Apple has spent considerable time trying to make that interop as painless as possible. That means Swift naming overrides, generics handling, etc. The Kotlin Objc output is constantly improving, and currently supports many of those features.

There are no details yet, but I have it on good authority that direct Swift interop may be a priority in the not-too-distant future.

The politics of shared code and Kotlin on iOS are a different story. The goal is to steadily improve the tooling experience for everybody, including iOS developers. That is certainly a huge goal of ours. That should help, but there’s going to need to be some selling on the idea too. Need to start seeing Kotlin talks as iOS meetups!

Q: Our iOS team is really excited about SwiftUI and said we should hold off on Kotlin until we figure that out.

A: Not in the form of a question, but I’ll rephrase: “Can you use Kotlin with SwiftUI?” Yes. Not the actual “UI” part, but the logic and architecture, which is what KMP is really for, sure. SwiftUI doesn’t change the equation much. We’re all pretty excited about it too.

You cannot have structs coming out of Kotlin. That is true. The data you use in SwiftUI does not need to be a struct. Struct support from Kotlin is an ongoing discussion, and we may implement something in the future that extends the interop, and/or down the road there may be formal support in Kotlin. For now, though, if the lack of structs is a show-stopper, then you’ll have an issue.

What’s Happening at Touchlab?

Touchlab is hiring full time and looking for contractors. Mostly Android, some multiplatform in the future: http://go.touchlab.co/trdw

We recently launched The Touchlab Refactory; an incremental refactoring service with KMP which focuses on a single feature at a time with no disruption to your current development workflow. Learn more here: http://go.touchlab.co/refactory

 
How does Kotlin Multiplatform stack up against other solutions? Our scorecard, explained.

How does Kotlin Multiplatform stack up against other solutions? Our scorecard, explained.

In my last article, I explained the reasons why you should try Kotlin Multiplatform — not because it can do everything, but precisely because it can’t.

It doesn’t do UI, or even attempt to. It does, however, do one thing really well: code business logic for iOs or Android. You code once, and deploy to either platform with virtually no hiccups because you keep the UI separate and native to its respective platform.

Naturally, introducing a relatively new way of coding immediately draws comparisons to other popular multiplatform solutions. The scorecard with which I concluded the article is meant to be a quick, point-by-point comparative glimpse at the three predominant solutions alongside KMP. Here, I wanted to provide some specifics on why we scored each the way we did.

 UI layer

 

 

KMP

Why rate KMP so high on UI when it doesn’t even have a UI layer? Because it doesn’t get in the way of the UI at all. Flutter, RN, and Xamarin all say sharing UI is a benefit to buying into their respective ecosystems — but each have gaps between expectations and reality. When using KMP, you code UI separately, in native, giving you greater control of the outcome for the best user experience possible. 

Flutter

Flutter doesn’t enable you to easily use native components to code UI. Instead, it takes control of all rendering, providing either a transparent view that shows the native Android code underneath, or uses PlatformViews which requires fairly clumsy native interop. Either way, it essentially co-opts the entire UI on its own canvas. And if there are bugs, you have to wait for the updates from Flutter. 

React Native

While you can create custom native UI widgets with React Native, Javascript performance is still an issue. Often, for example, developers will say it promises 60fps, but it gets in the way of doing good UI. It also hinders development efficiency because its UI widget libraries are not maintained by React Native. 

Xamarin

If you have a soft spot for .NET, Xamarin will be usable to you. But Xamarin.Forms has a lot of limitations, and Xamarin itself does not really create that ecosystem of great UIs. You’ll have to make separate UIs in Xamarin, Android as well as Xamarin.iOS, which is a lot more hassle than doing them natively. Similarly, this is why we don’t recommend writing UIKit code in Kotlin Native (even though you can).

Business logic

KMP

For Android, KMP is already as native as it can be. For iOS, it still has some issues with sharing business logic with idiomatic Swift code, but overall, it’s super-fast — and  the native interop means you don’t have bridge channels to deal with. 

Flutter

At present, Flutter does not share business logic with native code adequately. The Flutter team is working at integrating it into native apps, but at this point it is still experimental. When you need to integrate native code, you need to communicate across channels, otherwise you have to code it all in Dart — a language that is not widely used or supported. And while the Dart VM is fairly fast, it’s not as fast as a fully native solution. Basically, if you don’t want to deal with interoperability issues, it’s easier to work entirely within Flutter. 

React Native

To interop with native, it has to go through the JavaScript interpreter [update: Facebook knows this is a problem and recently released their own JS engine specifically for RN apps]:, so it’s always going to lag behind more native solutions. Also, it’s costly using JavaScript to bridge to native — more costly than Xamarin or Flutter.

Xamarin

Essentially, Xamarin presents the same problem as Flutter does: no direct interoperability with native code. You have to use C++, JNI, Java, Kotlin, or Objective-C Bindings. Currently not possible to interop with Swift, you can’t share business logic with non-Xamarin apps at all, and Microsoft has no plans to do this.

Developer efficiency

KMP

Debugging is straightforward with KMP because the debugging tools you’re already using are compatible. Android coders can debug in the Kotlin IDE, and iOS coders can debug in Xcode. In the meantime, as the libraries for KMP continue to grow, you’re not locked in — just use the tool that makes sense for that ecosystem. 

Flutter

With Flutter, you need a wrapper for its libraries to communicate with native code.You can’t use Xcode for iOS development. And while you can use Android Studio, you’re not really building an Android app with Flutter, you are building a Flutter app and using the Flutter plugins. Essentially, you’re in foreign territory.

React Native

Facebook has put a lot of effort into React Native, enabling you to choose from many great debugging tools and IDEs with good integration. It has a large and growing community that stems from the web development ecosystem. Even so, you’re unable to use the native libraries as easily as you can with KMP, so you still need to put work into libraries that haven’t yet been ported and integration with existing apps. And if you aren’t already a web shop, you need to learn the whole ecosystem.

Xamarin

To integrate existing native libraries, Xamarin.iOS only supports binding to ObjC Frameworks, and Xamarin.Android requires Java Bindings or the JNI. Sometimes, you can’t bring any new frameworks from native iOS or Android into Xamarin Projects. Additionally, you need to use LLDB or GDB for debugging native code.

Making the most of multiplatform

KMP

With KMP, Android is already fully native, so a bridge is not necessary. And, on iOS, KMP compiles a native framework with Objective-C headers, so, again, a bridge is not necessary. You can easily take a mostly business logic Android library and make it work as a KMP library (even easier if it’s already written in Kotlin). And while the UI has to be separate, it’s not a problem because the UI can then be a very thin layer that’s not dependent on the business logic layer. 

– Common architecture practices like MVP, MVVM, and CLEAN already encourage separating business logic and thinner UIs.  It’s only a slight modification to accommodate the shared module layer.

Flutter

Taking an existing Android library and making it a Flutter library is a whole lot harder than it is when using KMP. Flutter is really designed to be its own self-contained ecosystem. It’s difficult to share with non-Flutter apps. And while sharing the UI is better than it is in React Native or Xamarin, you have to build your own libraries, or write in Dart.

React Native

React Native is not ideal for larger apps or larger teams, and even though the developer community is large, it’s mostly made up of web developers, and therefore not necessarily attuned to mobile development fully. There are significant differences — screen sizes, UI expectations, variable device performance due to CPU, RAM, and graphics chips variability, offline and low or unreliable network connectivity, and platform-specific innovations that have been driven by significant engineering investment from Apple, Google, and Samsung.

Xamarin

Xamarin has its own ecosystem, with a relatively small developer community, and very limited sharing with other ecosystems. You have to build the UIs in Xamarin.Forms, or you have to build them separately using C# in Xamarin.Android and Xamarin.iOS.

Potential for code sharing

KMP

Even though KMP does not share the UI, which limits how much you can share, you are able to architect the app so that the UI layer is as thin as possible, which in fact is good architecture practice anyway. And it’s also good practice to make the best UI for users, which requires UI divergence between platforms anyway. Using KMP, you won’t run into obstacles when the UIs do need to diverge. And you will more easily be able to share non-UI code with less of a performance cost.

Flutter

Flutter provides everything except platform-specific features, which require a channel to native code. So if a library you need already exists in the Flutter ecosystem, great. But if not, you need to create it yourself which essentially requires writing code for three platforms — Android, iOS, and the Flutter Package API to communicate with the platform-specific implementations.

React Native

React Native provides everything but platform-specific features or high-performance logic (making it less desirable than Flutter), each of which requires a bridge to native code. If a library you need already exists in its ecosystem, you’re set. If not, like Flutter, you need to create it yourself, which requires writing code for three platforms — Android, iOS, and the React Native Bridge Module.

Xamarin

For simple apps, Xamarin.Forms allows you to share close to 100%. But it’s not ideal for more complex apps, where a lot of UI and any platform-specific features need to be separated into Xamarin.iOS and Xamarin.Android projects.

Maximum app quality 

KMP

When coding with KMP, you get fully native UI and business logic, with the same app quality as fully native apps in iOS or Android. 

Flutter

While the UI is performant (“buttery smooth” as the Flutter dev-rel people like to say), it’s easy to end up with something that’s perceived as lower quality to seasoned users. Also, Dart VM is fairly fast — much faster than Javascript — but there are still channel overhead issues you must deal with. As well, support for older devices is not robust. For Flutter in particular, the UIs may look modern on older devices, but the performance may be worse than native UI and implementation.

React Native

On the positive side, React Native architecture encourages native views and widgets, so you don’t fall behind platform UX updates. (Users expect these; otherwise, apps start to look old to them.) There is, however, a higher risk of lower performance because of Javascript (especially true on Android devices where standard Javascript is about as performance as an iPhone 6s and Facebook recently releases their own engine specifically for RN apps on Android).

Xamarin

For Xamarin, UI is really an afterthought. Xamarin.Forms has been getting more attention recently, but even when incorporating native widgets, you end up writing a full app in Xamarin.iOS and Xamarin.Android before getting something that modern users would generally accept as good quality.As for performance, it is true, the MonoVM is fairly fast, but there are still bridging overhead issues.

Kotlin Multiplatform Can’t Do It All. Which is Exactly Why You Should Try It!

Kotlin Multiplatform Can’t Do It All. Which is Exactly Why You Should Try It!

Before I begin, I want to be clear that all tools are created to solve problems, but none of them solve all problems. If your development team is in a hurry, or high-quality UI is not a major priority, go ahead and choose an ‘end-to-end’ development platform like Xamarin or Flutter or React Native to code your app for Android and iOS. You’ll get the job done 🎉!

Now, if you already have UIs for your apps, or if you want optimized UI for each platform, you’ll want to use Kotlin Multiplatform. The reason? It doesn’t do UI at all (not yet, anyway). But what it does do — and really well — is business logic for Android and iOS apps. 

Though the other multiplatform solutions aspire to support all application layers, they can’t adequately cover them all.

And sharing UI code across platforms is not necessarily desirable anyway. Very often when this is done, there will need to be multiple iterations to make the UI look and behave more natively. This will burn through development cycles, putting more pressure on your dev teams to deliver on time. And normally, the business is pushing for more features over UI quality. Sharing UI is risky, and it’s rarely good for morale, or for business.

 

Unlike Xamarin, Flutter or React Native, Kotlin Multiplatform does not live within its own ecosystem. Instead, it is very much like a “choose your own adventure” book, which is what makes it so powerful.

While KMP currently has limited libraries (though the number is growing), it does enable you to use all existing libraries and tools on iOS and Android, so there’s no need to wait for libraries or implement hacks and workarounds. You can’t do that with Flutter or React Native without running into sizable obstacles.

The output from Kotlin Multiplatform is just another package on Android and framework on iOS. That can save a significant amount of time and headaches because there is far less time spent writing bridge code or fully re-writing the things that are missing from other solutions. 

To code business logic in Flutter, your team first has to code shared logic in a language that is not widely used – Dart – in a new ecosystem, with a smaller community, and difficulty bridging it with existing code. In React Native, your mobile team would need to immerse themselves in the web ecosystem of JavaScript’s new IDEs and other tools . And in Xamarin, they’d have to code in C# , use Visual Studio, and a smaller, less active community. To make matters worse, whichever of these platforms your team uses, they’ll need a communication bridge between native and non-native code. 

In Kotlin Multiplatform, however, your team can code the platform-specific business logic, with direct communication with the native platform, with no need to wait for libraries or implement hacks or workarounds.

(You can if you want, that’s part of the adventure you choose.) And even if there are any issues, optional sharing with Kotlin Multiplatform means you only need to revert the code directly related to the issue — no ripping out the whole engine because of a bad spark plug. So, you always have choices. 

This is important of course, because it’s the business logic that determines how all features within an app will work. Because you’re writing the native code once for this layer, you accelerate development time and help ensure a solid code base. Plus, writing one set of native code is a highly effective way of future-proofing the code for later releases.

Kotlin Multiplatform, in other words, provides your dev teams with greater flexibility.

The other multiplatform solutions are essentially proprietary, resulting in vendor lock-in. It also results in the need to manage, in effect, a third platform because the ecosystems are too different from the native platforms and, because they try to solve everything but they can’t solve everything, you will need to write more platform specific code than advertised.

Unlike Xamarin and React Native, Kotlin Multiplatform doesn’t require a VM. Flutter doesn’t require a VM in production, but it does put you in a non-native ecosystem writing in a non-native language unlike Kotlin Multiplatform which respects the native languages and ecosystems of each platform. Kotlin Multiplatform is the most native multiplatform solution your team can use today. 

KMP doesn’t hide the fact that you’re dealing with multiple platforms because it already compiles to a native library for iOS or Android.

There are no intermediate layers to deal with, virtually eliminating any interop bottlenecks. And since Kotlin Multiplatform works with the native platform ecosystems rather than becoming its own, devs can use the tools and libraries they’ve always used including new platform innovations like SwiftUI and Jetpack Compose. Limitations you do encounter are not dead ends because you can always code around them with Kotlin, Swift, or whatever language lets you solve the issue with the least risk.

In sum, here’s how we measure the world of multiplatform solutions. Next week, we’ll share more details about the specifics of our rankings. In the meantime, if you’re interested in speaking to us about Kotlin Multiplatform, please contact us.

 

Measure multiple cross-platform development solutions

7 Questions about The Touchlab Refactory (incremental refactoring with Kotlin Multiplatform)

7 Questions about The Touchlab Refactory (incremental refactoring with Kotlin Multiplatform)

Transcript and vide below

Question 1. What is The Touchlab Refactory? 

Kevin: Touchlab Refactory is an incremental refactoring service that we provide for our clients. Over the years at Touchlab we have done this for a number of clients where we come in and we refactor architecture we reduced tech debt. We improve testing all those kinds of things and now we’re specifically focused on using Kotlin Multiplatform to also consolidate your code bases to reduce duplicate code reduce duplicate logic and architecture which will help prepare your code base for the future, reduce errors all kinds of stuff so you can focus on delivering products and new features well while we. clean up the code base and make things run smoother.

——

Question 2. Why do I need it? 

Kevin: Pretty much every team on the planet would love to go back to refactor code, fix their architecture, improve test coverage but you have ongoing priorities and deadlines, which prevent you from doing so. We can come in and do that simultaneously while you’re delivering your ongoing work.

——

Question 3. How does it work?

Kevin: We come in for a fixed engagement, generally 8 to 12 weeks with our independent code logistics team. We’ll do an architectural review of your codebase to identify features best consolidated with Kotlin Multiplatform and begin that refactoring process. At the end we’ll leave you with a detailed roadmap for your Kotlin Multiplatform future.

——

Question 4. How much of our time will it take?

Kevin: There’s a minimal impact on your time because we’re coming in to refactor your existing code. We don’t need to sit down with users, we don’t need to iterate with the business, we don’t need to do feature development. We are taking existing code and making it better, so while you’re working on new features our workflow happens concurrently but independently and doesn’t impact your feature delivery schedule.

——

Question 5. Do we risk missing a deadline?

Kevin: So do you risk missing a deadline??? Easy answer is NO. Kotlin Multiplatform by its nature is optional and incremental, so we can take things feature by feature and you don’t have to do the whole thing at once. This code exists already in your code base so you can take this refactored code and put it into and deploy it on your platforms when you’re ready. You don’t have to do it right away there’s no external deadline for this to happen.

——

Question 6. What about the cost?

Kevin: It’s fixed cost and time engagement and with the predefined scope of work. Our approach is to take one feature at a time of your product and refactor it and improve it and our goal is to show that this is a sound strategy this is something that’ll work it’s low risk for you and at the end you’re going to have something measurable that you can use.

——

Question 7. So what’s the best reason for partnering with Touchlab?

Kevin: Well in the short term it’s really important to improve code quality and consolidate your codes so you can move faster so you can iterate faster deliver better product deliver safer products that has less issues less problems for both your internal teams and your users but it’s also about the future the future is not just iOS and Android the future is maybe the web the future is maybe voice the futures maybe other platforms. The future is hard to see and we are preparing for the future, a post-platform future by reducing risk and having an adaptable codebase and adaptable team and skills and we think that that’s really important and we think Kotlin Multiplatform is going to deliver that.

——

A Multiplatform Case Study for an Inclusive Virtual Try-on Experience

A Multiplatform Case Study for an Inclusive Virtual Try-on Experience

Getting Started:

At Touchlab, it is typical for us to take on leads and projects that are new to us. We’ve worked on hardware, watch faces- all kinds of things- with no previous experience; we embrace the challenge and rise to it. It is atypical, however, for client research to blossom into a philosophical, platform agnostic, week-long design sprint.

But, we are nerds.

It all started as a simple conversation with people in the retail cosmetics industry. First, we noticed how fun it was to take virtual selfies with funky eyeshadows. Then, we noticed that some virtual try-on apps were waaay better than others. Finally, we got feisty (and immensely curious) when two makeup apps failed to recognize Jeff’s face correctly.

Not quite. 

We saw value in spending time to develop a deeper understanding of the virtual try-on world- really to foster our own awareness of what it takes to accomplish inclusive and feasible design in this space.

Our goal? To design a user-centric, e-commerce experience for a single makeup brand in one week.

Cultivating Empathy and Acknowledging Context:

When it comes to people who wear makeup, you cannot generalize into one or two flattened personas; anyone can wear makeup and the reasons behind applying even a simple black eyeliner are fluid and highly subjective.

Matching eyebrows to lids: the things we notice now. More from Ziggy here:

As Nielsen puts it in the 2018 Future of Beauty Report, “There is no one beauty shopper.” It is also important to note that wearing makeup — be it skin tone or neon green — is less related to confidence, vanity, conformity or belonging than some might think.

For most, it is a form a self-care and for many, a creative and even artistic expression of Self. This is how we are approaching makeup during this design sprint.

(By the way, if you didn’t already know, the biggest makeup trend in 2019 is gender fluidity.)

This idea of makeup as a form of expression is by no means new. But culturally and technologically, the beauty industry must embrace/is embracing the “no rules” approach as social boundaries are broken, social media influences the path-to-purchase and buyers across industries begin to trust and expect VR offerings.

Our Discovery Process:

Armed with what we feel is a grounding contextual view of the modernizing makeup industry, Touchlab’s design team kicked off a more hands-on discovery process by conducting a contextual inquiry and investigation through many of Manhattan’s makeup stores.

Why? In order to get off of our laptops and challenge any preconceived assumptions about makeup shoppers IRL. 🕵🏻‍♀️ 🕵🏻

 Left: Sephora’s virtual try on installation. Right: An example of MAC’s in-store product organization.
As we made our way around the different retailers, four interactions stick out:
  1. Quiet 20-somethings experiment with every bright-blue lipstick in the store. The makeup artists were kind and attentive, but let everyone play around with bold blues and shiny reds unbothered. Everyone was rubbing makeup on, wiping it off, checking themselves out, asking questions. Allowing and encouraging customers to play creates a positive retail environment and a magnetic vibe.
  2. A seasoned makeup artist explains their goals for their popular youtube channel; to target people who are “going somewhere” and do memorable street makeovers to match expectations around that experience.
  3. An older woman’s experience at an in-store virtual installation is somewhat marred by a technical gaffe; her virtual lipstick is overtly offset from her actual lips, seemingly because of her wrinkles. Despite this error, she expressed how fun it was and stayed for over 15 minutes.
  4. Two girls hesitate at the door of the store that seems a bit, well, “stuffy.” They leave almost immediately.

Amongst the perfumed chaos of the wildly different retail environments, we do detect an overarching trend: all kinds of people want to play and experiment with fun makeup, no matter their intention to buy.

Trying on different shades on your hands is commonplace in stores. Virtual try-on certainly solves this “pain point.” (Is it a pain point?)

User Testing and Product Audits

We moved into user testing and product audits of the more popular virtual makeup apps, the goal: to test not only the usability of these virtual experiences but also people’s reactions to e-commerce-oriented flows versus more playful user flows.

Examples of our testing sessions during which we asked users to complete 3 tasks. Users were consistent in having trouble completing retail transactions and finding the virtual try-on feature.

We tested the most popular apps in both the App Store and Google Play. The reactions were emotionally loaded. Most reacted negatively to commerce-oriented user flows that sent them back and forth between traditional, scrollable product grids and the virtual try on camera. Some had trouble even finding the virtual try-on feature on many apps.

Memorable quotes include:

“I don’t even know what this picture means. How do I even get to the try-on? I feel like it’s trying to just sell me *things*.”

“Where are my lipstick colors even going after I try them?”

“Wait! I wanted to see the whole look!”

However, we observed that users had the most fun when playing with colors as a form of self-expression:

“Dang, I look good. 👩‍🎤

“I’m going to go with… this color — “Dangerous” — because that’s how I feel.”

 

Danger is a pretty cool middle name.

Defining our Scope:

Surfacing from our deep immersion into virtual-try on make-up apps proved rather difficult for us designers; we are both now addicted to makeup apps and were getting pretty good at selfies. 🤳🤳🤳👩‍🎨 👨‍🎨

Touchlab Designers conducting product audits WHILE looking really, really good. If we do say so ourselves.

It was time to start taking our data and synthesizing it into actionable points of departure. Harnessing the passion of the makeup-wearers we interviewed and observed, we constructed a persona spectrum.

“Instead of defining one character, persona spectrums focus our attention on a range of customer motivations, contexts, abilities, and circumstances.” — Margeret P. for Microsoft Design (More here)

We know from our process of empathetic understanding that customer motivations for trying on makeup virtually vary greatly, but an overarching motivation might best be called “play”- with an underlying asterisk: ***”play” without the blight of glaring technical virtual misunderstandings for which a modern user has little tolerance.

Left: virtual try on of lipstick. Right: IRL photo of that same lipstick. I would (and did) buy this in-store, but not online. Not all virtual makeup was as inaccurate. 

We are moving forward with the assumption that users are less likely to buy a makeup product (in-app or in-store) if they do not trust the accuracy of the application of the virtual makeup. We define “Accuracy” here as the belief that the product will look, in real life, just as it does on camera. Accuracy, then, is relative to the proper recognition of the user’s facial features, skin tone, race, age, etc.


 Brainstorming at the office. #nomakeup #nofilter

From user testing sessions, we gathered that users of virtual try-on experiences want:

  • A balance of “Play” and “Pay” (Allowing product exploration and experimentation to be the natural point of sale, similar to the in-store experience. )
  • A means for creative expression with makeup
  • A variety of products to try
  • Accuracy in the virtual try-on experience. (Where product accuracy cannot be achieved, at least a positive, fun experience can funnel users in-store.)

Users of virtual try-on experiences need:

  • Easy control of the camera view with one hand/thumb
  • Proper architecture and timing of e-commerce product areas during the play to pay funnel.
  • A well-lit camera view, flexible enough for a myriad of environments
  • A natural multi-platform user experience
  • Facial Recognition for all genders, ages, and races*

*Feasibility-wise, this is an idealistic thought summarily strangled by the inequity in facial recognition we’ve witnessed thus far in our research. The feasibility of speedy and comprehensive machine learning for all faces will come from developers out there; as designers, we feel it is important for us to engage in the conversation and point out the inequity. Idealistic? Yes. Nonetheless, an underlying need.

With our user-centric motivations in mind, as well as recognition of the strengths and shortcomings of existing products- we arrived at our goal:

Create a modern, e-commerce, camera-focused experience that feels seamless and authentic.

Ideation meets its frenemy- Feasibility:

With our guiding principles in mind, we asked ourselves: How do we harness existing powerful API’s and a cultural affinity towards dramatic and playful makeup to create a delightful and inclusive e-commerce experience?

 ~Low Fidelity Ideation Techniques including sketches and thumb maps~ AKA Paper!

For the ideation process, we considered both constraint and context, respectively restricting and potentially accrediting features down the road. Constraints become especially important here when dealing with a camera/virtual experience:

For instance, design-wise, we must consider the one-handed “thumb zone.” and commonplace user experiences and, related, expectations around popular camera-oriented apps.

More about Scott Hurff’s thumb zone here. Note that these photos reflect right-handed thumb zones.

Feasibility-wise, we must remember:

  1. Our sprint time constraint 👮🏼‍♀️
  2. The importance of an e-commerce funnel
  3. The availability and robustness of cosmetic API’s
  4. The machine learning curve of accurate makeup application for the whole spectrum of race, gender, and age.

Some of the contextual circumstances that we considered include:

  • Physical limitations, temporary or permanent. (We are, after all, asking the user to take a selfie and therefore should absolutely keep the one-handed experience in mind.)
  • Users with the inability to deeply focus on the task at hand- be it temporarily due to various forms of distraction, or permanently, perhaps due to disability.
  • A range of environmental lighting and how that might influence virtual quality and accuracy

Scoping in, “Must Have”, feasible features include:

  • Ability to save looks/products for later
  • A predominantly camera view
  • Cart / Wearing list
  • Guided process
  • Color picker
  • Control over makeup intensity
  • Adding to bag
  • Seeing Makeup Details
  • Easily Changing/Removing/Adding Products

Features that we should or could have:

  • Automatic camera light correction
  • The ability to report or fix incorrect virtual makeup application
  • Saved “Looks” that apply all at once
  • The ability to share

How exactly those retail elements are presented was guided by user reaction and also screen space available, considering that we don’t want to cover the users’ face with text during their selfie session.

Design Progression:

During low-fidelity prototyping, we noted and observed users’ expectations.

Because of the e-commerce element of this project, we needed to test when, where and how to show the user the product information sooner rather than later. We started off by letting them play first, then show them their choices in a cart-like setting after the fact.

Even though the process to get to the cart did not take long, users rejected being in the dark; they wanted to know exactly what they are wearing when they were wearing it.

 Mid-fidelity prototypes did not show the user the e-commerce details until they saw their cart. Both the copy and the timing were problematic.

Mid-fidelity prototyping identified weak spots in both visual hierarchy and the timing of e-commerce introduction as we strove to balance UI and branding with the selfie-camera view.

During our iterative process, we determined what we think is the most natural way to introduce e-commerce elements into a playful virtual experience using thoughtful choice architecture:

For lipstick application, for instance, you first narrow the product areas by color-which is easily changed-, then choose the preferred finish (and thereby the exact product line and color), and then finalize with manipulation of the intensity of the makeup application.

For brows, however, you should first choose what type of application method you prefer, then choose between the few colors that are offered (thereby the exact product line and color), and then select the “style” (which is where things get fun again, for the user).

Hi-fi prototyping with re-architecture of e-commerce details

In our “final” prototype, we wanted to reflect a selfie-focused experience that encourages the user play with products without feeling forced to buy those products. 

In order to do so, we introduced the product description as a minimalistic, clickable overlay at what users deemed an appropriate time- and added the ability to change the products that the user is wearing from a “now wearing” list.

This person has opened a virtual makeup app, they want to try on makeup, they want to have fun, they want to see the price of the makeup, and ultimately, they just might buy something that they like.

The takeaway from designing this sprint for multiplatform is the importance of maintaining a human-centric approach so that when we pass off to developers, they understand our thoughts on tackling feasibility in this space.

“Final” prototype for both iPhone 8 and Pixel 3

A note on prototyping:

Because of our one-week time limit, we decided against framer and instead chose to work with still images to imitate a live camera view. A natural next best option would have been to use Sketch, but keeping in mind that early concepts need iteration to happen fast we decided to use Adobe XD

XD allowed us to change and test things between and across the team quickly and efficiently. Additionally, the lack of an app for previewing Sketch files on Android was a deal breaker.

XD’s auto-animate unlocked effortless micro-interactions on the final prototype that gave it an extra layer of realism; this was absolutely key in our virtual + selfie experience.

Ending Thoughts:

If you have read anything about Touchlab recently (see this post about our recent announcement with Square!) you have probably picked up that we are proponents of multiplatform design and development.

In this sprint, we forego assumptions about the differences between iOS and Android users and instead design for our spectrum.

Does it matter if iOS users “spend more money on makeup products” or if Android users “take fewer selfies”? Defining a person (or persona) by the phone that they buy seems silly and too blanketed.

As the industry of mobile development moves to a future where iOS and Android Engineers are becoming Mobile Engineers, our design team becomes Mobile Designers.

We focus on expressing our client’s brands across platforms the right way, instead of focusing on items on a list.

Frances Biedenharn & Nelmer De La Cruz

Our philosophy is to step back from “I’m an iOS designer. I’m an Android designer” and instead advocate for experience designers that understand the possibilities and constraints of each platform.

Accelerating Development with Kotlin Multiplatform

Accelerating Development with Kotlin Multiplatform

When leading a team of iOS and Android developers and creating the same application for both platforms, you are bound to face one (or all) of these issues:

1. PRODUCT: Tech debt and architecture issues are slowing development
Your team is having difficulty adding new features because too many parts of your codebase are tangled and interconnected. As a result, new bugs emerge whenever you add new features or fix existing bugs. Your team is constantly putting out fires when they could be innovating on the product.

2. PEOPLE: Your iOS and Android teams are siloed and lack collaboration
Android and iOS engineers should be able to deliver on both platforms. However, modern teams often lack a structure for leveling up engineers to deliver on both platforms. Furthermore, with little or no collaboration, this makes it particularly difficult to achieve feature parity between the two apps. It’s just too labor-intensive and requires too much management time.

3. PROCESS: Your engineering approach could be better defined
It could be that your iOS and Android engineers work at different speeds, making it tough to keep feature releases in sync. At the same time, independent backlogs can make some things slip between the cracks. And if some features are deliberately delivered on one platform before the other, it’s more difficult to predict when the second will be available because the code isn’t being shared. In effect, separate apps and separate teams result in divergent apps and user experiences.

 

 

The cause of these setbacks across people, product and process is a lack of common ground between developers and too much dependency and coupling with each platform. The solution, then, is to converge these development efforts and divorce it from the platforms.

Code sharing, specifically with a native framework like Kotlin Multiplatform, encourages better architecture by pushing the developer to decouple the view and approach the logic from a more modular perspective. By isolating the presentation layer from the business logic layer, you not only streamline the architecture, but also simplify bug testing, maintenance, and upgrades. All these gains come just from moving towards a Multiplatform friendly architecture, before even addressing the obvious benefit of only having to write the code once for both platforms.

 

(diagrams by Sam Hill, designed by Nelmer De La Cruz)

 

Compared to alternative multiplatform solutions like React Native or Flutter, KMP focuses on maximizing shared business logic. We believe this provides an advantage over the loathed “cross-platform” model by providing consistent platform experiences for users while sharing as much code as possible. For a deeper dive into why we prefer KMP over alternatives, you can view our webinar here.

 

How to get started

So where do you start with Kotlin Multiplatform (KMP) — In a word, small.

 

PRODUCT: Start small — and avoid big risks

Start by isolating a single layer or component or feature. For example, you can isolate the persistence layer, in effect creating a single database implementation for both platforms.

Isolating the persistence layer enables it to function for the entire app across both platforms. In other words, every time the app calls the database, it behaves the same; it’s not platform-specific. Rather than using CoreData for iOS or Room for Android, you can use a single implementation of SQLDelight for both.

As you start isolating layers such as persistence and improving the architecture in KMP it becomes easier to start refactoring other layers as well. For example, using KMP, a networking component can move from AlamoFire on iOS and likewise from Retrofit on Android to a shared layer, perhaps using KTOR.

 

PEOPLE: Create platform-agnostic teams

With KMP, your development teams no longer have to be siloed. By expanding the common ground and simplifying platform code, you gain an opportunity to easily level up your platform engineers — both iOS and Android — to become mobile engineers. A few low-risk ways to do this?

– Introduce flex time for Android engineers to learn iOS, and vice-versa.

– Insist the same platform-specific developer tackle the same user story on both platforms. (This is understandably a slower process but it has higher long-term ROI.)

– Perform shared code reviews. (For example, have an Android developer write a pull request, and get an iOS developer to review it.)

– As tasks start becoming more multiplatform, code reviews start giving developers opportunities to see complete stories and draw parallels between platforms

 

PROCESS: Promote D.R.Y. (Don’t. Repeat. Yourself)

Creating a platform-agnostic environment extends beyond just development. It’s equally important to consolidate the processes that drive development. Tracking a multiplatform feature in a single place makes it simple to maintain platform alignment. Having all of you multiplatform developers participating in the same planning processes improves communication. Consolidated defect tracking makes sure bugs don’t fall through the cracks on a platform.

 

 

So, if you are looking for opportunities to accelerate and optimize development, KMP is a great way to begin streamlining and future-proofing your productpeople, and process:

– Product: KMP focuses on sharing code at the business logic layer (where most architecture issues occur) and enables you to address the issue in a piecemeal fashion — no big decisions. You start small at your own pace and see the results before continuing.

– People: Using KMP, your engineers will be better prepared for the future, increasing the value of your department to the wider organization because they can deliver on multiple platforms.

– Process: With KMP at the center, you introduce better engineering processes that promote code sharing and multiplatform development.

If you’re interested in learning how KMP can accelerate mobile innovation at your organization, please contact us.