The Future of Cross-Platform is Native

The Future of Cross-Platform is Native

Cross-Platform Baggage

Cross-platform development is not held in high regard these days, largely because the apps that purport to provide cross-platform support have never really done the job effectively. But I believe the time has come for us to reconsider.

The arguments in favor of cross-platform development are the same as they’ve always been (D.R.Y., Risk Mitigation and Feature Parity)

 

Don’t. Repeat. Yourself (D.R.Y.)

There’s the argument for gaining efficiencies and cost savings through streamlining the development process. Program once to create cohesive code that can be deployed simultaneously to iOS and Android.

 

Risk Mitigation

There’s also the argument for minimizing risk regarding how UI will be developed. The biggest risk is that the UI won’t meet user expectations on either platform. This is a major reason why development teams opt for programming apps independently from one another. At the same time, business logic and backend development are at higher exposure because both determine how all features within an app will work. Ideally, in a cross-platform development scenario, an organization could take the time to focus on the nuances of backend and logic development, while putting less strain on UI development.

 

Feature Parity

The other argument is for feature parity and inclusivity, that is, fewer differences in functionality, whether iOS or Android. The benefit – you treat all users equally because they are using essentially the same program, whichever platform they choose.

 

Native Multiplatform Development

However, what was missing from “cross-platform” is native multiplatform development: Native CPU, Native UX, and Native developer experience and tools for iOS and Android. Cross-platform programming has the potential to thrive if the focus shifts to native coding, which is a more direct approach to produce the same functionality across platforms and devices. To better understand this, here’s a quick look at the most popular cross-platforms solutions and their native limitations.

 

Xamarin

Xamarin was one of the first to focus on a native approach to programming across iOS, Android, and Windows, starting with shared business logic and working its way toward shared UI with Xamarin Forms. However, its native elements are limited because it lives within its own ecosystem and uses C# (a language not native to iOS or Android development) and Microsoft Visual Studio instead of Android Studio or Xcode.

Xamarin Mobile Architecture

 

React Native

React Native (RN) represented a leap forward in terms of how developers thought about cross-platform because it empowered them to apply their web development knowledge to build native iOS and Android apps. But, like Xamarin, it too, lives within its own ecosystem, using Javascript and non-standard editors . And like Xamarin, it needs to wrap native controls and view hierarchy from its own interop, making it necessary to construct the UI with its own language.

React Native Mobile Architecture

 

Flutter

Flutter, a newer addition to cross-platform programming, uses its language Dart to create iOS and Android apps. Flutter also makes use of rich widgets to provide remarkable native experiences on Android and iOS platform – but the widgets are not native. Flutter also employs a shared UI platform that only works on mobile with a language (Dart) that isn’t widely used.

Flutter Mobile Architecture

These are just a few examples of why cross-platform programming has traditionally been a serious challenge to manage. It’s one of the reasons I believe we should move away from the term “cross-platform.”  A more apt term is “multiplatform”, because the goal is that any code you share maximizes what each platform offers.

 

Kotlin Multiplatform

That brings us to Kotlin Multiplatform. It is the rising star in the multiplatform space, and is, in fact, more native than Xamarin, RN or Flutter. Currently the dominant Android language, Kotlin has a strong, enthusiastic base of developers worldwide, and is praised by the community for providing a superior developer experience. Kotlin Multiplatform enables developers to write once, and test once, then use the same code across iOS, Android, and Web apps.

While not the first multiplatform tool to split business logic and UI (for instance, Xamarin for logic/libraries and Xamarin Forms for UI), it’s more native than Xamarin, RN, and Flutter because it uses shared logic and libraries below the UI layer, which developers can interact with in the native developer environments — Xcode, Swift, and Objective-C for iOS; Android Studio and Kotlin for Android; JavaScript for the web — and it outputs native code for each platforms.

As a language, Kotlin enables developers to produce applications more cohesively. It’s a modern language that dovetails with native platforms on Android, iOS, Java, and the web, allowing development teams to build on what’s already been coded. And because it’s essentially an extension of Java, it’s relatively easy for Java developers to get started. Kotlin isn’t too much of a departure from Swift either and at Touchlab we currently have iOS developers coding in Kotlin.

Kotlin Multiplatform Mobile Architecture

Equally important, Kotlin is a way to future-proof applications developed today. It’s a sound technology investment because the code works on all platforms without vendor lock-in like Xamarin or React Native. So, whether the dominant platform in the future is web or mobile doesn’t really matter, because the code ports to either environment.

At the same time, development teams no longer have to be siloed, and instead can work together as a cohesive whole. There is no longer a need to have dedicated iOS and Android teams.  This unified mobile approach simply makes more sense operationally, technically, culturally, and financially.

Is Kotlin the safest bet? Everything indicates that it, in fact, is. In a world where we’re always looking for greater efficiencies in development time and costs, Kotlin has proven itself to be a very viable player – one that has already shown itself to be a truly multiplatform programming option.

Stately, a Kotlin Multiplatform Library

Stately, a Kotlin Multiplatform Library

This started as a monster single post, now split in 2. Part 1, Saner Concurrency, is about what Kotlin is doing with concurrency.

During my talk at KotlinConf, I promised a part 2 of Stranger Threads to better explain threading and state in Kotlin/Native. I built a library instead.

Update!!! Video and Slides from my Droidcon UK talk!

What is Stately

Stately is a collection of structures and utilities designed to facilitate Kotlin/Native and multiplatform concurrency. As of today, it is a set of expect/actual definitions that most apps will wind up needing, and a set of frozen, sharable collection classes that allow you to maintain mutable collections across threads.

Why does it exist?

Kotlin/Native, and hopefully soon, all of Kotlin, will be implementing Saner Concurrency. Native, today, has runtime rules, and notably a lack of “standard” concurrency primitives, that help ensure runtime concurrency sanity. The 2 basic rules are:

  1. All mutable state is in one thread. It can be transferred, but is “owned” by one thread at a time
  2. All shared state is immutable (frozen)

These are good rules, because they will make concurrent code safer.

However, there are times where being able to access shared, mutable state is pretty useful. That may change as we adapt architectural thinking, but at least for today, there have been a few tricky situations to arise without that available. Global service objects and caches, for example.

Kotlin/Native has a set of Atomic classes that allow you to mutate state inside of an immutable (frozen) object. The Stately collections are constructed with atomics.

For the most part, I’d expect Stately to be used sparingly, if at all, but it’s one of those things that you’ll really miss if you need it and don’t have it.

What you shouldn’t use it for

Kotlin/Native has rules that help encourage safer concurrency. Changing frozen state is, on some level, enabling manually managed concurrency. There are practical reasons to have these collections available, but if you’re using them a lot, it might be better to try for some architectural changes.

If you’re new to Native and running into mutability exceptions, you might be tempted to make everything atomic. This is equivalent to marking everything synchronized in Java.

It’s important to understand how Native’s threading and state work, and why you need a concurrent collection. But, you know, no judgements.

Basic Principles

The collections mostly act like their mutable counterparts. You designate generic types, and get/set/remove data entries. One key thing to note.

Anything you put into the collection will get frozen.

That is very important to understand. The collection itself is “mutable” in the sense that you can add and remove values, but the collection and values it holds are all frozen.

For data objects, this is generally OK, but for callbacks, this may be somewhat mind bending to understand. That’s not a Stately problem so much as a Kotlin/Native problem.

I’ve talked to several people who struggle with this reality. All I can say is it seems weird at first, but is not as big of a deal as you think. You just really need to be aware of what that means.

For example, in the Droidcon app, we’re using LiveData to build a reactive architecture. To avoid freezing everything in the Presenter/UI layer, we simply keep all of our callbacks thread local to the UI thread, which means they don’t need to be frozen. The lower level callbacks all exist in Sqldelight, and are frozen, but they exist just to push data back to the main thread and don’t capture UI-layer state.

Summary, it’s different, but not that bad. The details would turn this blog post back into a monster, so I’ll just push out my promised “Stranger Threads” deadline out by a couple weeks.

A lot of how we think about architecture will change as coroutines mature on Native, and as the community has some time to think about the implications. I suspect there will be less use for shared collections as that happens, but for today, they’re pretty useful.

Available Collections

CopyOnWriteList

Similar to Java’s CopyOnWriteArrayList. In fact, the JVM implementation is Java’s CopyOnWriteArrayList. Useful in cases with infrequent changes and critical read stability. Registering callback listeners, for example. If you’re changing the list often, or it’s large, each edit requires a full copy. Something to keep in mind.

SharedLinkedList

This is a mutable list that will have reasonable performance. All edits lock, just fyi, but individual edits themselves are pretty small, so locks are quick. You can hold onto a node reference that will allow you to swap or remove values without traversing the list, which is important in some cases.

There are 2 basic flavors. One has unstable but performant iterations, the other I’d call CopyOnIterateList. It’s similar to CopyOnWriteList, except the copy happens when you call iterate. This may prove more generally useful than COWAL, as it should handle frequent changes better, but if you’re editing often and iterating often, remember you’re dealing with an aggressively locking data structure.

The other flavor will let you iterate without copying or locks, but edits to the list will be reflected while you’re iterating. The edits are atomic, so you won’t get errors, but you’ll also wind up with potentially fuzzy iterations. If that isn’t an issue, this should be a better choice.

I am aware that there are lockless implementations of linked list, but I didn’t attempt one. Going for simple.

SharedHashMap

Implements a pretty basic version of our favorite structure HashMap. Performance should have similar characteristics to what we’re used to from Java🌶, although obviously locking will produce different absolute numbers. In short, same big O.

I probably don’t need to go into where a hash map would be useful, but in the service object context, it’s probably caches, which leads to…

SharedLruCache

Shared version of a least recently used cache. If you’re not familiar, there’s a cap on the number of values, and the “oldest” get bounced first. “Oldest” being defined as the least recently accessed.

Status

This is pretty new. As Kotlin 1.3 matures, we should have a more stable deployment, and may add some other features. I’ll try to add some issues for “help-wanted” in case anybody wants to contribute.

Also, as of today (Friday 10/26) the JS code has an implementation but tests need to be wired in. That means don’t use the JS until that happens.

Notes
The only supported Native implementation is for mac and iOS. Other Native targets should work, except we’ll need to find a lock implementation. Pthread_mutex is fine, except you need to destroy it explicitly, and K/N has no destructor. That means a ‘close’ method on the collection, which I’d rather avoid. Right now there’s a spin lock, but not sure if that’s a great idea.

The collection implementations could be described as minimal. My main goal was to start replacing some of the custom C++ code we’d put into earlier K/N implementations, which generally existed because of a need for shared state. If there’s a real need for something else, open an issue to discuss.

🌶 Similar to Java 7 and below. That is, if your bucket size doesn’t increase, as your entry size increases, or you have a bad hash, you start to approach N, because the bucket list means a lot of scanning. In Java 8, after the list gets to size 8(ish), it’ll store those values as a tree, so worst case gets capped. Our hash map resizes like Java’s, so you should only find yourself in “worst case” if your hash is bad, or if you have horrible settings on your map. You probably don’t need to know this, but Java 8’s optimization is kind of cool, and I didn’t bother trying to implement it 🙂

Saner Concurrency (and the cost of change)

Saner Concurrency (and the cost of change)

I’ve been talking about Kotlin/Native for a little while now, and one of the big things I tell people is that threading will be very different than what they’re used to. It’s just something that you need to learn and understand.

Kotlinconf was a couple weeks ago. Progress on Multiplatform was definitely a big topic. A more general discussion surfaced around how Jetbrains and the Kotlin team are thinking about concurrency. Andrey said in his Keynote that future Kotlin will focus on Saner Concurrency. I think we should unpack this a bit.

It is generally accepted that managing concurrency by using mutexes, synchronized blocks, etc (AKA. “manually” or “status quo”), is difficult and error prone. Concurrency in Java, with Java’s concurrency primitives, is difficult.

Some languages attempt to improve upon this situation using various approaches. Another point from the keynote I thought was great is that being unique is not important. Assimilating the best ideas of other languages is important (2:40).

Saner Concurrency

Kotlin/Native has baked into it some ideas about concurrency that make managing it safer. Andrey alluded to the idea that Kotlin, across variants, will start attempting to tackle “Saner Concurrency”. This is good to watch (37:00) to get a sense of where Kotlin is going, which makes me hopeful that not only will the various incarnations of Kotlin share the same concurrency concepts, but that those concepts will start to emerge as first class parts of the language at compile time rather than strictly runtime (Rust-like?).

By “various incarnations” I mean Kotlin-JVM will hopefully start getting some of the “new stuff” Kotlin/Native is getting🚧.

One of the consequences of change is that things will be different. We’ll need to learn new ideas, build different libraries, and adopt different ways of coding. That inevitably means resistance and time.

I wound up on the closing panel of the conference (thanks Hadi!), and one of the questions I gave an answer to addresses this issue. It wasn’t actually a question so much as expressing concern that the different threading model will slow down adoption (at 18:00)

I jumped in because, frankly, I had the same concerns earlier on. A few months ago I drafted up a blog post talking about how such a departure will impact adoption and may ultimately derail the effort. Thankfully, like most of my blog posts, I never finished it, and I don’t think that way anymore.

Considering it was on stage, at the end of a conference, I think I expressed my thoughts OK, but let me expand a bit here.

At the beginning of my talk, I told everybody about how I got there. I’d spent a couple years working on an architectural framework based on J2objc called Doppl. In short, compatible Android structures, lots of familiar libraries, and a gradle tool. Replicating testable mobile architecture across Android and iOS, in a familiar and practical manner.

How practical? The day after Google announced room, I got Room working on iOS, from the java source jars, in about 90 minutes.

Here’s the thing, though. Nobody cared. I bet more people read this post in the first week than read that post after a year and a half.

Obviously these aren’t direct parallels. The Android community wants Kotlin and iOS wants Swift, so pitching a Java/Objc thing isn’t fun (trust me). Community interest is critical, no matter how practical your pitch may be.

However, I think it’s also important to have a vision. Kotlin wants to adopt better, safer concurrency practices. They’re making the language available on many platforms, including Webassembly, and making “tough love” choices for a better future platform.

I think building something practical but somewhat compromised will make it easier to learn, and definitely easier to develop the library ecosystem necessary to support what currently exists, but it’s also uninspiring. It does not encourage change.

In short, yes, I think “giving up” on change would make adoption quicker for an individual iOS developer, but I don’t think it would be appealing to the iOS developer community. Change and new ideas, although they will take more work, are appealing and inspiring, and will ultimately be the best bet at gaining adoption.

When somebody says, “Kotlin is just Swift for Android”, this is meaningful pushback. It’s not simply “another language”.

In any case, it doesn’t matter. They’re doing it ✈️

This post started as one post, but turned into a monster. Part 2 will be released after my Droidcon UK talk Friday where we announce a new Kotlin Multiplatform state management library…

🚧 I didn’t want to derail up top, but one major issue for right now is that JVM and Native do some runtime things fairly differently. That is likely to create confusion early on, especially with teams that have engineers fully on one platform or the other. It would be great if we could run JVM with the same runtime checking as Native, even if just in a debug configuration.

Future of Shared Code = Kotlin Multiplatform

Future of Shared Code = Kotlin Multiplatform

If you’ve been following our development of Doppl for the last couple years, we’ve built a Gradle toolset for J2objc, an architectural structure with an Android compatible API, and published a number of popular libraries for that ecosystem. In other words, a full toolset extension of J2objc which allows sharing logic and architecture from Android to iOS.

J2objc is a mature, stable technology. If you have an iPhone, you’re more likely to have some J2objc in there than all the other non-HTML frameworks put together. J2objc makes sense not because it lets Java developers publish iOS code. It makes sense because it allows a platform native language produce platform native libraries, with optional code sharing, without any significant barrier or translation layer.

We use this framework with current clients, and would absolutely take new projects with it if we have the bandwidth to do so.

However, we’re officially out of the Doppl business.

The future of shared code and architecture, as far as we’re concerned, is Kotlin Multiplatform.

Some of the reasons we like it are the same as why we like J2objc. Native language, optional sharing, platform compatibility. As of the last few months, all R&D and open source development has been focused on KMP. As of KotlinConf this year (or shortly after) the tooling and libraries will be ready for production work. It is time to burn the ships.

To be better open source citizens, and to avoid internal distractions, we’ll be unpublishing Doppl and related libraries. If you’re one of the handful of orgs who use the library, please reach out and we’ll make sure you can access the repos.

I want to be clear that this doesn’t mean I think J2objc is bad tech. I still think it’s amazing technology. If you wanted to maximize your shared mobile architecture today, it would be significantly more productive than other options.

However, the community excitement around Kotlin Multiplatform is intense.

Over the next several months there will be a number of library announcements. It’s also not just for Android/iOS logic, but back end and front end architecture, including native mobile, web with JS, and webassembly. There is a lot of work to be done, but there’s a much wider scope of problems to solve.

We are, of course, looking for products and orgs interested in getting started. Also, I’m starting the search for our first Multiplatform specific engineer (besides me). Reach out.

Kotlin/Native (Stranger) Threads

Kotlin/Native (Stranger) Threads

Chapter 1: Getting Started

This is going to be a multi-part series on threading in Kotlin Native. To build a reasonably functional mutltiplatform implementation, understanding threads is critical. The Kotlin/Native design and the way threads are intended to be used is quite different than what we’re used to in Java land. I’ll be learning this as we go, but as many have pointed out, if you really want to understand something, try explaining it to others.

Why Stranger Things theme? KN is opinionated about threading and state. To keep you out of trouble, it creates a world where different threads mostly can’t see each other’s data, and have very little interaction. However, underneath that world is a similar looking, but far more powerful (and potentially dangerous) world where you have access to everything. It’s the Upside Down, basically¹. We won’t be talking about that much right now, but we’ll definitely need to poke around and make some new structures to enable productive app development.

Tools

You’ll need some kind of test environment to run things in. For now I’d suggest downloading CLion. Read this on installing Kotlin/Native support. As mentioned in the beginning, I’m just getting started with this. The same is true of CLion. I hope we’ll be able to do KN in Intellij directly, but CLion appears to be the tool for the bulk of KN and C++ development. That means learning CLANG, at a minimum. Not right away, though. We can poke around for a while without understanding CLANG.

To verify everything is working, create a new KN project and use the HelloWorld sample.

CLion New Project Dialog

Hit the play button in the action panel and you should see “Hello, Native World!” printed to console.

Source

We’ll be trying out some stuff. To run the source, clone this:

Docs

This is probably a good time to step back and say if you don’t have a pretty decent understanding of how threads work in general, you can run through the samples, but you may be lost often. The intro stuff will probably be useful, but its safe to say we’ll also wind up in the weeds periodically. Will update with good intro resources if I come across them.

Anyway, back to KN. Read the CONCURRENCY doc in the KN repo. It explains a lot. Today we’ll go over the basics of Worker, how to pass “hot” data, and what data freezing is.

Worker

Workers are the core unit of concurrency provided for out of the box with KN. I’ll start by saying this:

Each thread, including the main thread and each worker, has it’s own copy of state.

I’ll repeat this a few times in this post, so if there’s an important takeaway, it’s that.

You create workers and pass them jobs to complete. I would imagine there are implementations where this doesn’t correspond to a thread, but right now, for iOS at least, that will boil down to a thread. A worker is a thread with a job queue. Also, remember, it’s own copy of state.

Simple Example

Let’s make a worker and have it do something.

Call ‘startWoker’ to start a worker. This will create the thread and make it available for jobs.

You create jobs with “schedule”. That method takes a TransferMode param, a producer lambda, and the actual job, also as a lambda.

Inside the job lambda is the work you actually do.

Scheduling returns a Future. You can use this to get the job result inside the calling thread. We call ‘consume’ simply to wait until the job is done.

If you’re manually creating workers and leaving that context, shut them down properly with ‘requestTermination’.

Let’s dig into each piece in more detail.

Worker Lifecycle

The worker is a managed resource that you create, and if you’re going to be forgetting about it, you probably want to shut down. That corresponds to two methods: startWorker() and requestTermination().

There’s not much to say about ‘startWorker’, other than it creates a worker queue. ‘requestTermination’ is a little more interesting. By default it’ll finish all jobs in its queue, but you can pass in a boolean ‘false’ to tell it to shut down asap.

If you’re creating workers, you should shut them down. If you’re creating a set of workers that are intended to stick around with your app at all times, don’t worry about it. If your process gets killed, so with your threads.

Consume

The schedule call returns a Future. On that you can check status and get results of the job. ‘consume’ will force your thread to wait for the job to finish, and pass the result of the job into the lambda. In our simple case, we return nothing, so the lambda accepts nothing.

Scheduling Work

Now that the simple stuff is out of the way, it’s time to look at scheduling work. This is where we need to get very precise.

The schedule method takes 3 parameters.

  1. TransferMode — Checked or Unchecked. The “safe” option is Checked.
  2. Producer lambda — This should return the data that will be passed into your job.
  3. Job lambda — This is the “work” that will be run on the different thread. It has a single input param, which is the output of the producer lambda. You can return a value, which can be read by your calling process.

The primary complication around all of these parameters is that your data needs to cross between threads. As mentioned, each thread has it’s own state, and isn’t allowed to see others’ state. How can that happen? To stick with the post theme, you’re sending something through the Upside Down and back again. To do that safely, there are some considerations.

The Lambdas

The goal of how workers interact is to protect state. KN will do some magic to check that the state you’re passing isn’t referenced anywhere else OR that your state is frozen. Frozen is a runtime immutable state that we’ll discuss later.

In other words, KN wants to make sure your state is safe. Either nobody else can see it, or nobody can change it (including you).

How does it check? That’s an Upside Down topic for a different chapter.

Producer Lambda

This lambda returns an instance of data that is passed into the job lambda. It’s run in the same thread that calls it.

Why a lambda you ask? The runtime wants to make super sure you’re not dragging data around. The KN runtime is serious about references. So much so that passing hot data into a worker isn’t necessarily an easy operation.

This will fail because there’s a local reference:

fun failedReference(){
    val worker = startWorker()

    val dat = SomeData("asdf")
    worker.schedule(TransferMode.CHECKED, {dat}){

Even if you don’t plan on doing anything to ‘dat’, the KN runtime will get upset. More curious, this works:

var glob = SomeData("asdf")
fun getAndClearGlob():SomeData{
    val temp = glob
    glob = SomeData("qwert")
    return temp
}

fun globalReference(){
    val worker = startWorker()

    worker.schedule(TransferMode.CHECKED, {getAndClearGlob()}){

After calling ‘getAndClearGlob’, nothing is pointing at the object originally at ‘glob’. That seems logical! How about this?

var glob = SomeData("asdf")
fun getAndClearGlob():SomeData{
    val temp = glob
    glob = SomeData("qwert")
    return temp
}

fun globalReference(){
    val worker = startWorker()

    glob = SomeData("inside")
    
    worker.schedule(TransferMode.CHECKED, {getAndClearGlob()}){

That will fail. You and I know you’re not doing anything with the reference created inside the method, but KN’s worried about it.

All Threads Have Their Own State

I’m reminding you of this again here so I can point out an issue with the sample above. Keeping data at the “global” scope just so you can pass it to a worker is probably not a great design, but because all threads get their own state, the worker will have it’s own copy of ‘glob’ that does nothing but take up space and init/management cycles.

If you really want to ruin your day, try this:

val worker = startWorker()

fun doNotDoThis(){
    worker.schedule(TransferMode.CHECKED, {}){
        for(i in 0..1000000)
        {
            //Let's kill time
            val b = (i+i).toDouble()/1000.toDouble()
        }
    }.consume {  }

    worker.requestTermination()
}

The top level ‘worker’ can be called from multiple methods to start jobs. Nice! Right? Well, ‘startWorker’ creates the worker, which gets its own copy of state, which creates a worker, which gets its own copy of state…

That’s super bad. Got bit by that right out of the gate.

We’ll probably have a much longer discussion of how to structure “hot” data to get it over the wall to other threads. Here are some other strategies.

Freeze

If you’re following general best practices, the data being passed through threads should be immutable anyway. KN has a special method called ‘freeze’ that makes your data very seriously immutable at a runtime level.

fun freezeLocal(){
    val worker = startWorker()

    val localData = SomeData("asdf")
    localData.freeze()

    worker.schedule(TransferMode.CHECKED, {localData}){
        println("In thread ${it.a}")
    }.consume {  }

    println("In main ${localData.a}")

    worker.requestTermination()
}

Freeze actually modifies metadata flags on runtime objects which tell the runtime that seriously nothing will touch the data. It’s more than just having immutable values. You can now pass that frozen object to multiple threads and reference it from your calling thread. No problem!

Well, some problems.

Common Kotlin has no concept of “freeze”. That’s a minor issue. We should be able to do something with extension functions to get a noop method into common and the JVM. More pressing is that KN and everybody else is going to be different at runtime. Could the concept of freeze be ported to the JVM? Would that be valuable? Questions for another time, and certainly somebody else 😛.

A more immediate issue is class design. The SQLDelight demo, for example. SQLiteDatabase and the Helper objects involved all have state that is protected by Java threading constructs. That design is going to need to be rethought.

Or you can just do this:

fun freezeLocal(){
    val worker = startWorker()

    val localData = SomeData("asdf")

    worker.schedule(TransferMode.UNCHECKED, {localData}){
        println("In thread ${it.a}")
    }.consume {  }

    println("In main ${localData.a}")

    worker.requestTermination()
}

No freeze. Just pass in as TransferMode.UNCHECKED. In this case, probably not a huge deal. The String in SomeData is a val. Nothing is changing anyway. In other, more complex cases, we’ll need to be more careful.

Workers?

I’m looking at this through the lens of shared mobile architecture. That means interacting with the host system. You can’t have a worker for the home thread, so out of the box, you’d need to implement some kind of regular Future checking mechanism. That doesn’t sound great.

iOS has thread event queues. That’s how you communicate back to the main thread. In future posts we’ll explore the idea of leveraging the same checks and concepts involved in Worker “transfer” to ensure the data safety and memory management of KN, but actually perform the threading with iOS’s dispatch queues, and be able to post messages back to the main thread. This will almost certainly involve a new library and possibly some PR’s to the KN repo. See how it goes!

For multiplatform, we’ll need some Worker-ike interface that can resolve back to Looper/Handler on Android. In JVM, we won’t have the data checking and freeze stuff, so testing is going to have to be robust enough to run likely scenarios and catch data integrity check issues on the iOS side.

Or just pass everything unchecked and watch out for this guy.

Demogorgon, obv

Future Chapters

Some possible future chapters:

  1. Wrapping iOS dispatch queues instead of “Worker”.
  2. Rethinking SQLite and (possibly) SQLDelight design. Listeners watch queries, which pass data base to the main thread (ultimately). In JVM, that data can exist across threads, but not in KN (out of the box).
  3. Exploring global state options. You can, but that’s like hanging out near the portal.
  4. Enter the Upside Down and poke around a bit.

Nice suits

Touchlab

Is looking for orgs that want to get started with Kotlin multiplatform, and hiring mobile developers interested in it. Reach out.

¹ The demogorgon is the memory manager? Probably best not to overthink the analogy. Also, Netflix legal team. Love the show! Please don’t get mad about the images. XOXO

SQLite/SQLDelight ❤ Kotlin Multiplatform

SQLite/SQLDelight ❤ Kotlin Multiplatform

A few weeks back I posted a piece about Mobile Oriented Architecture and how I think coding products for screens will evolve. At the end, I said I was keeping an eye out for the Kotlin/Native multiplatform commit.

I was. Really. Read the commit emails every morning. Yet somehow I missed the fact that multiplatform was possible.

You can build multiplatform projects using Kotlin/Native. I won’t say it’s super productive yet, but you can do it. I gave a talk about it last night, and have some code to share.

Update, Video!

TL;DR

If you want to run SQLDelight on iOS, skip to “One more thing” below. Anyway…

Kotlin Mobile Today

Kotlin multiplatform for mobile consists of common Kotlin, Kotlin/JVM, and Kotlin/Native. The slide above represents roughly how building multiplatform would look today, from a “how much code are you sharing” perspective. Obviously, the JVM side has a ton of stuff available. On the iOS side, the Native team has wrapped a bunch of resident libraries with the interop, but that’s kind of it. The middle represents what you could theoretically share. Just logic, and to a limited degree (size of that area varies widely according to how much logic, obviously).

To be useful, we need libraries. The essential libraries. We don’t need everything currently available on the JVM side, just the critical stuff. Android is a mature ecosystem, which means you have a lot of special purpose stuff. If you open a production Android app and look at the library list, it’ll probably be huge, but if you really looked at it, the list of things critically necessary for shared architecture would be pretty short.

  • Networking
  • Threading
  • Something reactive
  • Properties (like shared preferences. Typed storage for when sql is too much)
  • SQL (or at least a “boxy” database. Something with a schema)
  • Testing, logging, utilities, yada yada

 

The lesser used H&H image

All of these need fully Kotlin-ized rewrites. It sounds like a lot of work, but a few things to keep in mind.

Copying is easier than creating

One of the reasons why the discussion around “cross platform” is broken is we think 2 apps means 2x work. It does not.

Product development involves iteration, which involves throwing away a lot of work. You shouldn’t need to do the same on your second platform. Similarly, “smartphone” mobile has a decade of iterating on libraries and best practice ideas. To build a new ecosystem, you can just take the current result of that work and build analogs. It’s non-trivial, but it’s not starting over either.

Ecosystem interest is intense

To say a lot of developers like Kotlin would be an understatement. There is intense interest in creating things for Kotlin. Some communication and coordination will be useful, so 10 groups don’t build the same thing (although a little competition doesn’t hurt either). However, I’m predicting as more examples of multiplatform emerge and the tools stabilize, we’ll see a flood of multiplatform libraries emerge. This will pick up roughly end of summer.

Wide open

If you’ve come into Android fairly recently and wanted to do some open source work, you’ll notice it’s pretty crowded. Getting attention can be difficult. This particular field is wide open but very likely to be very active very soon. It is a great time to get involved.

A bit of my own background. The last several years have been largely focused on the non-technical side of running a mobile development shop. As such, there has been a bit of open source, but nothing super committed. During early Android, however, there were few libraries and I had a lot more time on my hands.

I did not write ORMLite. I did, however, work with the author to make the db interface generic, then implement the Android-ORMLite bridge. It is *very* likely that ORMLite would never have existed on Android, or come much later if that didn’t happen. A lot of apps used it, because, for better or worse, there wasn’t much else available. As long as you don’t make a bad library, you can have a big impact in an early ecosystem, as going from nothing to something moves the needle a lot more than writing something that’s different, but competing with similar things.

What do I think of ORMLite? That’s a different post. Summary, it was generally more productive than straight SQL, and we learned a lot 🙂

Anyway, I’m a database nerd.

Doppl

Doppl is a set of libraries built on top of J2objc, to facilitate sharing logic and architecture between Android and iOS. It works, but pitching a Java/Objective-C thing in the time of Kotlin/Swift is difficult. I also think Kotlin multiplatform has a more meaningful future (webassembly FTW).

We are using it when the situation dictates, but are internally very much focused on Kotlin. On the plus side, “Doppl” was essentially an implementation of Android architecture on iOS. A lot of similar skills, concepts, and in some cases code, can be used.

SQLite on Kotlin

I think Kotlin Multiplatform needs a “driver level” SQLite implementation. By that I mean something that is at least similar to the Android SQLite implementation, assuming there’s no performance impact. There has been a debate about that implementation since the beginning of Android, and a desire to “fix” it, but I suspect that hasn’t happened yet because it turned out to not be a big enough problem. So I’m not going to try that now.

In theory, we have an implementation of Android’s SQLite stack in Doppl/J2objc that can simply be added and typealiased. Once that’s stable and has some solid testing around it, reimplement the J2objc layer in Kotlin.

SQLite on Android

SQLite on Android

All SQLite code ultimately talks to the shared library ‘sqlite’ that exists pretty much everywhere. On top of that, AOSP has C++ that provides methods to push/pull data in and out of the sqlite library. On top of that is the Java interface everybody is familiar with, that talks to C++ with JNI. It’s a fair amount of stuff, but conceptually very simple.

SQLite with Doppl

SQLite with Doppl

To replicate this functionality on iOS, we took the ArOSP C++ and Java code. There was some necessary trimming. For C++, it referenced a lot of libraries available to the broader system, which we truncated to reduce package size. For Java, structures that aren’t relevant for iOS were removed (anything for ContentProvider, basically).

Wishful Thinking Diagram

This is conceptually what Kotlin Mobile Multiplatform SQLite looks like.

J2objc?

I can tell you from experience that most developers are not excited about Java that gets morphed into Objective-C. I will say this: It’s super stable, and as far as numerical distribution, is probably on more phones than the rest of the “code sharing” solutions combined. However, I find it really hard to get past the weirdness in people’s minds.

Beyond that, there are actual mechanical issues. Just adding it means 5–10m extra download size. If you’re using it for lots of your app, that’s acceptable. For a library, not so much. Also, while it comes from Java and in its own ecosystem feels like Java, I would call it the uncanny valley of types. The interface between Kotlin and J2objc requires a lot of little adapters.

Wishful Thinking Diagram

Early on my thinking about how to quickly get Kotlin multiplatform working was to use J2objc as the stand-in for the JVM. It is, quite literally, a Java JRE implemented in Objective-C. However, don’t. Calling into specific APIs is totally doable, but as a generic solution, it really wouldn’t go well.

Anyway, the “Wishful Thinking” diagram actually looks like this.

It’s complicated

This diagram isn’t to scale. The majority of the heavy lifting is in J2objc-land, but compared to the simplicity of the Android side, there’s a lot of adapters and delegates going on.

Over the next few weeks, the plan is to reduce that down by writing Kotlin that talks directly to the C++. Something like this.

For today, though, the implementation we have works. I don’t think there’s a significant performance impact, if only because we’re ultimately pushing something to disk, so a couple extra function calls aren’t going to do much.

One more thing…

The big talk “reveal” is that I wasn’t going to actually demo SQLite. I’ve been chatting with Alec Strong about SQLDelight and it’s Kotlinization. As it turns out, he’s been rewriting the output to generate Kotlin, and not just JVM Kotlin. Common Kotlin. They don’t have a Sqlite interface to run it on, and that’s what I’m working on.

So, the “demo” is actually…

Kotlin Multiplatform SQLDelight

The Kotlin output on SQLDelight is pretty amazing, and the separation of a driver interface will allow SQLDelight to be run in non-Android contexts pretty easily. On the SQLDelight side, I only had to change one thing. SqlPreparedStatement bindLong and bindDouble use ‘long’ and ‘double’ as their parameter names, which blew up in Swift. That’s a Kotlin/Native framework generator fix (either escape or fail upstream), but I just did a workaround to get through the day.

The Kotlin/Native compiler, AFAIK, doesn’t have a package manager and needs source to build, so I just grabbed the runtime code for SQLDelight and threw it into the driver build. This works, but SQLDelight with Kotlin is in SNAPSHOT mode and under heavy development. I blew up the demo 2 hours before start by doing a clean build. So, there’s a good chance this won’t compile when you try it. I’ll add a version that doesn’t depend on remote code soon, but flying out to I/O in the morning, so unlikely till the following week.

The code

Steps in README. Sample was taken from Kotlin/Native calculator.

Future

The demo is open source now. We want to do some more work on the driver code before opening that. Kotlin/Native is *very* different with regards to threads, and the SQLite Android code does some thread management internally. That needs a think. It’s not that bad, but you can’t just take the same logic and move it over. Also, a more complete testing strategy needs to be implemented. It’s first priority after I/O.

Whether *this* Sqlite library is what ultimately gets used for SQLDelight on iOS, or if somebody releases something more purpose-built, is less clear, but see up top about a little competition.

Predictions

I’m making some predictions, which is always a bad idea, but here goes.

Late-summer: Several good examples and usable libraries. My goal is to have Droidcon NYC’s app be fully Kotlin shared architecture.

KotlinConf 2018: I’ll just assume JetBrains is going to push hard to have something really solid by then. If only a psychological milestone.

Mid-2019: Mainstream. I have no idea of actual adoption, but if you want to share a bunch of code on mobile, the tools, and at least the critical libraries will be available.

Say Hello!

We (Touchlab) are very much looking for orgs that want to get started asap with Multiplatform. We also build apps without the fancy shared code. If interested please reach out.

We are also hiring. Mostly right now that’s standard app work, but if you are interested in Kotlin and Multiplatform, this is a great place to be.