SQLite/SQLDelight ❤ Kotlin Multiplatform

A few weeks back I posted a piece about Mobile Oriented Architecture and how I think coding products for screens will evolve. At the end, I said I was keeping an eye out for the Kotlin/Native multiplatform commit.

I was. Really. Read the commit emails every morning. Yet somehow I missed the fact that multiplatform was possible.

You can build multiplatform projects using Kotlin/Native. I won’t say it’s super productive yet, but you can do it. I gave a talk about it last night, and have some code to share.

Update, Video!

TL;DR

If you want to run SQLDelight on iOS, skip to “One more thing” below. Anyway…

Kotlin Mobile Today

Kotlin multiplatform for mobile consists of common Kotlin, Kotlin/JVM, and Kotlin/Native. The slide above represents roughly how building multiplatform would look today, from a “how much code are you sharing” perspective. Obviously, the JVM side has a ton of stuff available. On the iOS side, the Native team has wrapped a bunch of resident libraries with the interop, but that’s kind of it. The middle represents what you could theoretically share. Just logic, and to a limited degree (size of that area varies widely according to how much logic, obviously).

To be useful, we need libraries. The essential libraries. We don’t need everything currently available on the JVM side, just the critical stuff. Android is a mature ecosystem, which means you have a lot of special purpose stuff. If you open a production Android app and look at the library list, it’ll probably be huge, but if you really looked at it, the list of things critically necessary for shared architecture would be pretty short.

• Networking
• Threading
• Something reactive
• Properties (like shared preferences. Typed storage for when sql is too much)
• SQL (or at least a “boxy” database. Something with a schema)
• Testing, logging, utilities, yada yada

The lesser used H&H image

All of these need fully Kotlin-ized rewrites. It sounds like a lot of work, but a few things to keep in mind.

Copying is easier than creating

One of the reasons why the discussion around “cross platform” is broken is we think 2 apps means 2x work. It does not.

Product development involves iteration, which involves throwing away a lot of work. You shouldn’t need to do the same on your second platform. Similarly, “smartphone” mobile has a decade of iterating on libraries and best practice ideas. To build a new ecosystem, you can just take the current result of that work and build analogs. It’s non-trivial, but it’s not starting over either.

Ecosystem interest is intense

To say a lot of developers like Kotlin would be an understatement. There is intense interest in creating things for Kotlin. Some communication and coordination will be useful, so 10 groups don’t build the same thing (although a little competition doesn’t hurt either). However, I’m predicting as more examples of multiplatform emerge and the tools stabilize, we’ll see a flood of multiplatform libraries emerge. This will pick up roughly end of summer.

Wide open

If you’ve come into Android fairly recently and wanted to do some open source work, you’ll notice it’s pretty crowded. Getting attention can be difficult. This particular field is wide open but very likely to be very active very soon. It is a great time to get involved.

A bit of my own background. The last several years have been largely focused on the non-technical side of running a mobile development shop. As such, there has been a bit of open source, but nothing super committed. During early Android, however, there were few libraries and I had a lot more time on my hands.

I did not write ORMLite. I did, however, work with the author to make the db interface generic, then implement the Android-ORMLite bridge. It is *very* likely that ORMLite would never have existed on Android, or come much later if that didn’t happen. A lot of apps used it, because, for better or worse, there wasn’t much else available. As long as you don’t make a bad library, you can have a big impact in an early ecosystem, as going from nothing to something moves the needle a lot more than writing something that’s different, but competing with similar things.

What do I think of ORMLite? That’s a different post. Summary, it was generally more productive than straight SQL, and we learned a lot 🙂

Anyway, I’m a database nerd.

Doppl

Doppl is a set of libraries built on top of J2objc, to facilitate sharing logic and architecture between Android and iOS. It works, but pitching a Java/Objective-C thing in the time of Kotlin/Swift is difficult. I also think Kotlin multiplatform has a more meaningful future (webassembly FTW).

We are using it when the situation dictates, but are internally very much focused on Kotlin. On the plus side, “Doppl” was essentially an implementation of Android architecture on iOS. A lot of similar skills, concepts, and in some cases code, can be used.

SQLite on Kotlin

I think Kotlin Multiplatform needs a “driver level” SQLite implementation. By that I mean something that is at least similar to the Android SQLite implementation, assuming there’s no performance impact. There has been a debate about that implementation since the beginning of Android, and a desire to “fix” it, but I suspect that hasn’t happened yet because it turned out to not be a big enough problem. So I’m not going to try that now.

In theory, we have an implementation of Android’s SQLite stack in Doppl/J2objc that can simply be added and typealiased. Once that’s stable and has some solid testing around it, reimplement the J2objc layer in Kotlin.

SQLite on Android

SQLite on Android

All SQLite code ultimately talks to the shared library ‘sqlite’ that exists pretty much everywhere. On top of that, AOSP has C++ that provides methods to push/pull data in and out of the sqlite library. On top of that is the Java interface everybody is familiar with, that talks to C++ with JNI. It’s a fair amount of stuff, but conceptually very simple.

SQLite with Doppl

SQLite with Doppl

To replicate this functionality on iOS, we took the ArOSP C++ and Java code. There was some necessary trimming. For C++, it referenced a lot of libraries available to the broader system, which we truncated to reduce package size. For Java, structures that aren’t relevant for iOS were removed (anything for ContentProvider, basically).

Wishful Thinking Diagram

This is conceptually what Kotlin Mobile Multiplatform SQLite looks like.

J2objc?

I can tell you from experience that most developers are not excited about Java that gets morphed into Objective-C. I will say this: It’s super stable, and as far as numerical distribution, is probably on more phones than the rest of the “code sharing” solutions combined. However, I find it really hard to get past the weirdness in people’s minds.

Beyond that, there are actual mechanical issues. Just adding it means 5–10m extra download size. If you’re using it for lots of your app, that’s acceptable. For a library, not so much. Also, while it comes from Java and in its own ecosystem feels like Java, I would call it the uncanny valley of types. The interface between Kotlin and J2objc requires a lot of little adapters.

Wishful Thinking Diagram

Early on my thinking about how to quickly get Kotlin multiplatform working was to use J2objc as the stand-in for the JVM. It is, quite literally, a Java JRE implemented in Objective-C. However, don’t. Calling into specific APIs is totally doable, but as a generic solution, it really wouldn’t go well.

Anyway, the “Wishful Thinking” diagram actually looks like this.

It’s complicated

This diagram isn’t to scale. The majority of the heavy lifting is in J2objc-land, but compared to the simplicity of the Android side, there’s a lot of adapters and delegates going on.

Over the next few weeks, the plan is to reduce that down by writing Kotlin that talks directly to the C++. Something like this.

For today, though, the implementation we have works. I don’t think there’s a significant performance impact, if only because we’re ultimately pushing something to disk, so a couple extra function calls aren’t going to do much.

One more thing…

The big talk “reveal” is that I wasn’t going to actually demo SQLite. I’ve been chatting with Alec Strong about SQLDelight and it’s Kotlinization. As it turns out, he’s been rewriting the output to generate Kotlin, and not just JVM Kotlin. Common Kotlin. They don’t have a Sqlite interface to run it on, and that’s what I’m working on.

So, the “demo” is actually…

Kotlin Multiplatform SQLDelight

The Kotlin output on SQLDelight is pretty amazing, and the separation of a driver interface will allow SQLDelight to be run in non-Android contexts pretty easily. On the SQLDelight side, I only had to change one thing. SqlPreparedStatement bindLong and bindDouble use ‘long’ and ‘double’ as their parameter names, which blew up in Swift. That’s a Kotlin/Native framework generator fix (either escape or fail upstream), but I just did a workaround to get through the day.

The Kotlin/Native compiler, AFAIK, doesn’t have a package manager and needs source to build, so I just grabbed the runtime code for SQLDelight and threw it into the driver build. This works, but SQLDelight with Kotlin is in SNAPSHOT mode and under heavy development. I blew up the demo 2 hours before start by doing a clean build. So, there’s a good chance this won’t compile when you try it. I’ll add a version that doesn’t depend on remote code soon, but flying out to I/O in the morning, so unlikely till the following week.

The code

touchlab/kotlinmultiios

Steps in README. Sample was taken from Kotlin/Native calculator.

Future

The demo is open source now. We want to do some more work on the driver code before opening that. Kotlin/Native is *very* different with regards to threads, and the SQLite Android code does some thread management internally. That needs a think. It’s not that bad, but you can’t just take the same logic and move it over. Also, a more complete testing strategy needs to be implemented. It’s first priority after I/O.

Whether *this* Sqlite library is what ultimately gets used for SQLDelight on iOS, or if somebody releases something more purpose-built, is less clear, but see up top about a little competition.

Predictions

I’m making some predictions, which is always a bad idea, but here goes.

Late-summer: Several good examples and usable libraries. My goal is to have Droidcon NYC’s app be fully Kotlin shared architecture.

KotlinConf 2018: I’ll just assume JetBrains is going to push hard to have something really solid by then. If only a psychological milestone.

Mid-2019: Mainstream. I have no idea of actual adoption, but if you want to share a bunch of code on mobile, the tools, and at least the critical libraries will be available.

Say Hello!

We (Touchlab) are very much looking for orgs that want to get started asap with Multiplatform. We also build apps without the fancy shared code. If interested please reach out.

We are also hiring. Mostly right now that’s standard app work, but if you are interested in Kotlin and Multiplatform, this is a great place to be.