SQLite on Kotlin/Native

Tada! (Matty Matheson, Viceland, Giphy)

Today we’re releasing an early-ish version of Sqlite for K/N. More specifically, its a version of the Android Sqlite stack for K/N, that can be used in multiplatform projects.

touchlab/knarch.db
_knarch.db – K(otlin)N(ative)A(rchitecture) Database_github.com

Back in early May we demoed SQLite and SQLDelight running on iOS with Kotlin/Native and multiplatform. To implement the iOS side, I wrapped J2objc/Doppl code, which worked, but wasn’t exactly pretty under the hood.

In that post I said the goal was to get to this:

Pretty diagram

Well, that’s this.

I made a promise at the NYC meetups to release something before Droidcon Berlin. Flying out tomorrow, so here it is!

This is definitely an early release. Over the summer we’ll be updating the architecture, particularly around the multiplatform portion of it. Also, because K/N’s memory and threading model are significantly different than the JVM’s, some features were disabled and architecture simplified to be able to release something stable. We’ll be revisiting those decisions and adjusting the architecture as we go.

Details of the process and design will have to wait for follow on blog posts and/or talks. For now, here’s an overview of what’s available.

Main Project

The repo has 4 main folders.

cpp

This contains the code that directly talks to sqlite3. Much of this code is ported from AOSP and Android’s SQLite stack, but types are changed from JNI to what K/N uses. There’s also some code to facilitate state management and threading in K/N.

kotlin

The really interesting folder is kotlin-ios. This is the implementation of SQLite on iOS. You’ll notice a familiar set of classes (SQLiteDatabase, etc). The majority of these originate from AOSP, but have been updated to Kotlin and to live in an environment without the JVM or directly managed multithreading.

Much of the test folder is ported from CTS. This area of the code base will need a lot more TLC in the future, but the CTS tests provide a nice base to work from.

If you’re curious (and on a mac), run ‘./gradlew build’ on the kotlin folder, then look under kotlin-ios/build for the generated kexe file. This will run the test suite.

sample-notepad and sample-notepad-sqldelight

Two basic sample apps. One just SQL, one running SQLDelight’s multiplatform. Spoilers, the SQLDelight one is more interesting.

***Update!!!***
I totally forgot to mention that the android side of the samples isn’t done. Sqlite working on android isn’t exactly interesting. Soon, though.

SQLDelight

SQLDelight’s Kotlin generation and multiplatform support is currently in alpha. I have a fork with a driver implementation that allows it to function on iOS. I would not try to put this into production today, but definitely kick the tires, and something more official will certainly be coming along soon.

touchlab/sqldelight
_sqldelight – Generates Java models from CREATE TABLE statements._github.com

Usage

Currently there isn’t a dependency mechanism for K/N, as far as I know, so using the library means copying around the compiled artifacts. See the sample apps on how this is done. This gets super painful fast, so some kind of dependency publishing and management is high on the wish list.

Threading?

Threading on K/N is very different than what you’re probably used to in the JVM. The SQLite stack on Android does a lot of thread management internally. SQLiteOpenHelper, SQLiteDatabase, and all of its associated parts can (and need to be) frozen to be passed around to multiple threads. They all use the same sqlite connection under the hood. Internal state does change, but is kept in C++ structures in a way that is safe for K/N.

I tried to get clever with K/N and keep some non-frozen data that got passed around between threads. That ate a couple weeks and was sad. Would not recommend.

Nothing in programming feels more hopeless than a non-repeatable race condition. Well, if we ignore anything people related I guess. Just talking about code here.

— Kevin Galligan 🎸🐶🚀 (@kpgalligan) June 6, 2018

Cursor, I feel inclined to point out, is not thread safe, but Cursor isn’t really thread safe in the JVM either. I think it would be useful if K/N had a way to mark a class that specifically can’t, or shouldn’t, be frozen, such that it’ll throw at runtime if you try, but I guess you’ll find out soon enough. Anyway…

I’m way overdue for another Stranger Threads post. In it I’ll talk more about state and threads (and Atomics!).

KNarch?

K(otlin)N(ative)Arch(itecture). Sounds like narc. Names are hard. Suggestions welcome.

Want to help?

There’s a bunch of work that can be done here. Pick through issues, “needs hands” in particular. It’s a good opportunity to understand K/N at an architectural level. Video explaining how to set up the build environment coming after Droidcon Berlin.

Speaking of …

Other than Berlin, expect to hear more at Droidcon NYC, (probably) Droidcon London, and (very excited about this) KotlinConf! Also probably every Android NYC meetup for the foreseeable future. Sorry in advance, local friends.

Jobs!

Touchlab is currently working on projects with Doppl, and aggressively hunting down early Kotlin/Native adoption clients. We are very committed to delivering client work and open source code on this platform. If this is what you’d like to do, get in touch. We’re also remote friendly. Just FYI.

Projects!

If you’re part of an org that wants to start sharing architecture and be part of the cool crowd, we definitely want to talk to you.

Now, starting slides for the Berlin talk… (update: taking longer than planned)