Kevin Galligan, Author at Touchlab | We're the Kotlin Multiplatform Experts

SQLite/SQLDelight ❤ Kotlin Multiplatform

by Kevin Galligan | May 4, 2018 | Future of mobile, Thoughts

A few weeks back I posted a piece about Mobile Oriented Architecture and how I think coding products for screens will evolve. At the end, I said I was keeping an eye out for the Kotlin/Native multiplatform commit.

I was. Really. Read the commit emails every morning. Yet somehow I missed the fact that multiplatform was possible.

You can build multiplatform projects using Kotlin/Native. I won’t say it’s super productive yet, but you can do it. I gave a talk about it last night, and have some code to share.

Update, Video!

TL;DR

If you want to run SQLDelight on iOS, skip to “One more thing” below. Anyway…

Kotlin multiplatform for mobile consists of common Kotlin, Kotlin/JVM, and Kotlin/Native. The slide above represents roughly how building multiplatform would look today, from a “how much code are you sharing” perspective. Obviously, the JVM side has a ton of stuff available. On the iOS side, the Native team has wrapped a bunch of resident libraries with the interop, but that’s kind of it. The middle represents what you could theoretically share. Just logic, and to a limited degree (size of that area varies widely according to how much logic, obviously).

To be useful, we need libraries. The essential libraries. We don’t need everything currently available on the JVM side, just the critical stuff. Android is a mature ecosystem, which means you have a lot of special purpose stuff. If you open a production Android app and look at the library list, it’ll probably be huge, but if you really looked at it, the list of things critically necessary for shared architecture would be pretty short.

Networking
Threading
Something reactive
Properties (like shared preferences. Typed storage for when sql is too much)
SQL (or at least a “boxy” database. Something with a schema)
Testing, logging, utilities, yada yada

All of these need fully Kotlin-ized rewrites. It sounds like a lot of work, but a few things to keep in mind.

Copying is easier than creating

One of the reasons why the discussion around “cross platform” is broken is we think 2 apps means 2x work. It does not.

Product development involves iteration, which involves throwing away a lot of work. You shouldn’t need to do the same on your second platform. Similarly, “smartphone” mobile has a decade of iterating on libraries and best practice ideas. To build a new ecosystem, you can just take the current result of that work and build analogs. It’s non-trivial, but it’s not starting over either.

Ecosystem interest is intense

To say a lot of developers like Kotlin would be an understatement. There is intense interest in creating things for Kotlin. Some communication and coordination will be useful, so 10 groups don’t build the same thing (although a little competition doesn’t hurt either). However, I’m predicting as more examples of multiplatform emerge and the tools stabilize, we’ll see a flood of multiplatform libraries emerge. This will pick up roughly end of summer.

Wide open

If you’ve come into Android fairly recently and wanted to do some open source work, you’ll notice it’s pretty crowded. Getting attention can be difficult. This particular field is wide open but very likely to be very active very soon. It is a great time to get involved.

A bit of my own background. The last several years have been largely focused on the non-technical side of running a mobile development shop. As such, there has been a bit of open source, but nothing super committed. During early Android, however, there were few libraries and I had a lot more time on my hands.

I did not write ORMLite. I did, however, work with the author to make the db interface generic, then implement the Android-ORMLite bridge. It is *very* likely that ORMLite would never have existed on Android, or come much later if that didn’t happen. A lot of apps used it, because, for better or worse, there wasn’t much else available. As long as you don’t make a bad library, you can have a big impact in an early ecosystem, as going from nothing to something moves the needle a lot more than writing something that’s different, but competing with similar things.

What do I think of ORMLite? That’s a different post. Summary, it was generally more productive than straight SQL, and we learned a lot 🙂

Anyway, I’m a database nerd.

Doppl

Doppl is a set of libraries built on top of J2objc, to facilitate sharing logic and architecture between Android and iOS. It works, but pitching a Java/Objective-C thing in the time of Kotlin/Swift is difficult. I also think Kotlin multiplatform has a more meaningful future (webassembly FTW).

We are using it when the situation dictates, but are internally very much focused on Kotlin. On the plus side, “Doppl” was essentially an implementation of Android architecture on iOS. A lot of similar skills, concepts, and in some cases code, can be used.

SQLite on Kotlin

I think Kotlin Multiplatform needs a “driver level” SQLite implementation. By that I mean something that is at least similar to the Android SQLite implementation, assuming there’s no performance impact. There has been a debate about that implementation since the beginning of Android, and a desire to “fix” it, but I suspect that hasn’t happened yet because it turned out to not be a big enough problem. So I’m not going to try that now.

In theory, we have an implementation of Android’s SQLite stack in Doppl/J2objc that can simply be added and typealiased. Once that’s stable and has some solid testing around it, reimplement the J2objc layer in Kotlin.

SQLite on Android

All SQLite code ultimately talks to the shared library ‘sqlite’ that exists pretty much everywhere. On top of that, AOSP has C++ that provides methods to push/pull data in and out of the sqlite library. On top of that is the Java interface everybody is familiar with, that talks to C++ with JNI. It’s a fair amount of stuff, but conceptually very simple.

SQLite with Doppl

This is conceptually what Kotlin Mobile Multiplatform SQLite looks like.

J2objc?

I can tell you from experience that most developers are not excited about Java that gets morphed into Objective-C. I will say this: It’s super stable, and as far as numerical distribution, is probably on more phones than the rest of the “code sharing” solutions combined. However, I find it really hard to get past the weirdness in people’s minds.

Beyond that, there are actual mechanical issues. Just adding it means 5–10m extra download size. If you’re using it for lots of your app, that’s acceptable. For a library, not so much. Also, while it comes from Java and in its own ecosystem feels like Java, I would call it the uncanny valley of types. The interface between Kotlin and J2objc requires a lot of little adapters.

Early on my thinking about how to quickly get Kotlin multiplatform working was to use J2objc as the stand-in for the JVM. It is, quite literally, a Java JRE implemented in Objective-C. However, don’t. Calling into specific APIs is totally doable, but as a generic solution, it really wouldn’t go well.

Anyway, the “Wishful Thinking” diagram actually looks like this.

This diagram isn’t to scale. The majority of the heavy lifting is in J2objc-land, but compared to the simplicity of the Android side, there’s a lot of adapters and delegates going on.

Over the next few weeks, the plan is to reduce that down by writing Kotlin that talks directly to the C++. Something like this.

For today, though, the implementation we have works. I don’t think there’s a significant performance impact, if only because we’re ultimately pushing something to disk, so a couple extra function calls aren’t going to do much.

One more thing…

The big talk “reveal” is that I wasn’t going to actually demo SQLite. I’ve been chatting with Alec Strong about SQLDelight and it’s Kotlinization. As it turns out, he’s been rewriting the output to generate Kotlin, and not just JVM Kotlin. Common Kotlin. They don’t have a Sqlite interface to run it on, and that’s what I’m working on.

So, the “demo” is actually…

Kotlin Multiplatform SQLDelight

The Kotlin output on SQLDelight is pretty amazing, and the separation of a driver interface will allow SQLDelight to be run in non-Android contexts pretty easily. On the SQLDelight side, I only had to change one thing. SqlPreparedStatement bindLong and bindDouble use ‘long’ and ‘double’ as their parameter names, which blew up in Swift. That’s a Kotlin/Native framework generator fix (either escape or fail upstream), but I just did a workaround to get through the day.

The Kotlin/Native compiler, AFAIK, doesn’t have a package manager and needs source to build, so I just grabbed the runtime code for SQLDelight and threw it into the driver build. This works, but SQLDelight with Kotlin is in SNAPSHOT mode and under heavy development. I blew up the demo 2 hours before start by doing a clean build. So, there’s a good chance this won’t compile when you try it. I’ll add a version that doesn’t depend on remote code soon, but flying out to I/O in the morning, so unlikely till the following week.

The code

touchlab/kotlinmultiios

Steps in README. Sample was taken from Kotlin/Native calculator.

Future

The demo is open source now. We want to do some more work on the driver code before opening that. Kotlin/Native is *very* different with regards to threads, and the SQLite Android code does some thread management internally. That needs a think. It’s not that bad, but you can’t just take the same logic and move it over. Also, a more complete testing strategy needs to be implemented. It’s first priority after I/O.

Whether *this* Sqlite library is what ultimately gets used for SQLDelight on iOS, or if somebody releases something more purpose-built, is less clear, but see up top about a little competition.

Predictions

I’m making some predictions, which is always a bad idea, but here goes.

Late-summer: Several good examples and usable libraries. My goal is to have Droidcon NYC’s app be fully Kotlin shared architecture.

KotlinConf 2018: I’ll just assume JetBrains is going to push hard to have something really solid by then. If only a psychological milestone.

Mid-2019: Mainstream. I have no idea of actual adoption, but if you want to share a bunch of code on mobile, the tools, and at least the critical libraries will be available.

Say Hello!

We (Touchlab) are very much looking for orgs that want to get started asap with Multiplatform. We also build apps without the fancy shared code. If interested please reach out.

We are also hiring. Mostly right now that’s standard app work, but if you are interested in Kotlin and Multiplatform, this is a great place to be.

Mobile Platform Convergence

by Kevin Galligan | Apr 4, 2018 | Current, Future of mobile, Thoughts

Our last post about Mobile Oriented Architecture talked about Android, iOS, and the web beginning to look similar at an architectural level. That is a future which is a little far out, and if I’m being honest, for the web end, certainly speculative. Several things will have to come together, and regardless of how well they do, mobile and web are going to have some differences. Knowledgeable practitioners will be able to deal with these differences, but mobile and web will always be more like cousins than siblings.

However, if we think a lot closer to current reality, it’s not all bad news. The MOA diagram I posted looked like this.

Everything’s the same!

Most projects don’t really look like that, though. What the web is doing and what mobile is doing tend to differ somewhat (or significantly). What iOS and Android are doing, however, tend to be the same. The diagram should really look like this.

Mobile is usually pretty different.

Why your product’s feature set should be the same on Android and iOS should be obvious. They’re essentially the same thing. A mobile device. Unfortunately how apps are built for each is quite different. Different languages, libraries, ecosystems, etc.

The funny story here, though, is how not different they actually are. Again, architecturally speaking. They’re both unix-y systems, with files, threads, networking. Both ship with sqlite (and you can link to it). The raw components you need to actually construct the logic of your apps exists in both places, and is basically the same.

Doppl?

That is the general idea behind Doppl. Take some AOSP code, the architectural bits that you’re likely to use in an app, and implement them on iOS using J2objc. You construct your UI the same way you had previously, but both Android and iOS can call into a shared, tested architecture.

There’s a lot to unpack behind it. The longer blog post discussing concepts can be found here, as well as a much updated video from Droidcon SF.

That work is ongoing. We recently pulled the gradle plugin out of the Doppl project because it builds against stock J2objc now (if you tried the original j2objc gradle plugin, they’re similar but different).

Awesome! Nailed it, right?!

It is an interesting time to be pushing a Java thing in the Android community. After Google announced Kotlin support, pretty much nobody wanted to hear about Java. I very much understand this position, but the reality is, although not as sexy, J2objc is quite stable and being used by some very interesting products in production.

On the other hand, some folks are also sorta/kinda using it, but in a much more limited capacity, for a number of reasons. If you’re talking about product development in large teams, Ray’s talk is very interesting. Some shops are simply not going to try to do this kind of stuff. They have the resources to build everything separate, and have the testing and architectural discipline to do so effectively.

There’s also going to be a political issue with existing teams. Using J2objc/Doppl, your development efforts start to look like this.

It’s an Android world

The iOS team is not going to be happy about this situation. There’s no real way to spin it otherwise. Dev becomes Android focused.

This is not bad if you aren’t dealing with an established internal iOS team. In fact, it becomes a lot easier to outsource the 2nd platform, assuming you want to build Android-first. That’s another blog post in the making, but replicating something that exists is a lot easier than making a new thing.

Kotlin Multiplatform

The other tech we’re really excited about is Kotlin Multiplatform. Rather than having all the tech Android focused, there’s a real opportunity for the Android and iOS communities to come together and have a shared ecosystem. As Ray says (see the sorta/kinda video again), when using code sharing, somebody is inevitably working in a foreign environment. Kotlin is obviously Android friendly, but it is also very similar to Swift. The libraries that will need to be built are going to need solid iOS and llvm understanding. There is a real opportunity for collaboration.

There’s still a lot of work ahead of us, though. I am sure there are going to be a number of groups adding different libraries. Sometime in the near future I think we’re going to co-opt the Doppl AOSP sqlite code and see if we can make a Kotlin/Native sqlite access layer that is still source compatible with Android’s sqlite stack (support library sqlite stack, anyway).

If everything goes right, though, the diagram can look like this.

Mobile teams working together!

Technically it could look like this, but meh.

iOS first, still using Kotlin for logic and architecture

But why not ___?!

I’ll get around to the various “Why not ___?” posts at some point. The short version is, it’s not that I think those things are bad. There will be people that use them, and they will be successful. We just think that for the type of work we do, and the types of orgs we work with, these types of tech are your best bet. All of our reasoning will come down the development efficiency and risk. Again, Droidcon SF video explains it best.

Mobile Oriented Architecture (MOA)

by Kevin Galligan | Mar 31, 2018 | Current

Hi! I’ve had a bit of radio silence for the past few months, but there’s been quite a bit going on. New website, etc. Expect to see a few progress updates in the near future. Thought we’d start with what’s fresh on the mind.

I gave an impromptu overview of this concept yesterday to the Touchlab team. It was a bit of a whiteboard disaster, and probably a little late in the day for sweeping industry concepts. Later, at the Kotlin Meetup, I talked to two different people who kind of immediately got what I was talking about. Context is important I guess.

One of the issues while explaining to the team was, “if this happens … whenever they get around to that”. There are timelines, agendas, and moving parts involved. We have to think forward a bit.

Time Traveling

I’m just going to suspend reality here, pretend it’s the early 2020’s, and everything I want the Kotlin team, the mobile community, and the broader industry to do has happened.

Mobile First

For the most part, “cross-platform” means taking web ideas and putting them onto mobile. That’s the wrong direction. Sharing code and architecture allows what we’re calling the Mobile Oriented Architecture.

MOA. Yes. Another acronym. You’re welcome.

Just a heads up. We’re talking architecture. Not UI. Not yet, anyway. UI and the features that make native native (notifications, force touch, “the notch”) are important but not part of this discussion¹.

At its most basic, MOA means applying the same code base, using mobile-tested architectural techniques, to deliver whatever client you’d like. Doing this for Android and iOS is trivial. For the web, WebAssembly (and to some degree JS) makes this feasible.

Android and iOS is trivial?

Trivial is a strong word, but the process is straightforward at least. Under the hood, they are basically the same. This was the conclusion of our J2objc/Doppl odyssey (It was also the premise, but I digress).

They are very similar systems: both are unix-y, have threads, sqlite, files, networking, etc. The Android community prefers MV* as an architectural construct. iOS apparently likes VIPER. The point of both is to make code more reliable, testable, and maintainable. The choice of either is community preference and has nothing to do with platform.

Android and iOS can be thought of as simply the “mobile platform” as far as architecture is concerned. That was true way, way back in 2018, assuming you were OK with Java (it’s the 2020’s, remember).

There are, of course, several stacks with which you could implement MOA, but the Kotlin platform is distinct for a number of reasons:

It is a modern language with excellent tooling.
On the Android side, it’s fully embraced and “native”, adding zero platform risk or impacting development efficiency.
The interface between “native” and “shared” is not an exception but a normal and expected part of development (aka. Smooth interoperability). For example, you don’t need to wait for somebody to wrap a library around some new iOS things. You have Swift readily available.
We’ve managed to convince the iOS community that embracing Kotlin/Native and multiplatform for architecture sharing is a great way to be “mobile first”, as opposed to React Native or whatever. (Again, to highlight, this is the future. Selling iOS folks is on the todo list)
That same iOS community has done a lot of work creating the truly multiplatform libraries needed to have efficient development.

The iOS community is, I think, one of the critical pieces that made Kotlin MOA feasible. Swift and Kotlin are very similar. Kotlin is obviously “Android leaning”, but not so much as is Java. Because Kotlin/Native doesn’t have the JVM, for better or worse, it forces a more balanced bridge between the two environments. Summary: it’s not just the “Android language”².

Back End?

Besides just sharing code, there are some interesting consequences of MOA.
When all of your code is in different languages and implemented separately, you tend to have “back end” engineers, and teams focused around the different client platforms. If you‘re implementing lots of client logic with a single code base, the “back end” engineers are going to get involved in the client code that consumes the service they’re building.

Rather than looking at the remote client as this “other” thing, it’s just a remote part of a connected system. For example, imagine a microservice style system with multiple clients. It might look something like this.

But it can look like this.

That’s not just the api call implementations. You could generate them from Swagger or similar. It’s logic, storage, offline, whatever, and all of the associated testing. Doing this in an environment without significant trade-offs with regards to development efficiency, runtime bloat, or risk is the goal.

Sounds like SOA

Sure. The mobile part is how you architect the client side and where web fits in. You’re always offline until your call succeeds. 5G is “better”, but didn’t fix the fact that you’re mobile any more than 4G fixed it. Taking what we as a community have learned from mobile and architecture, putting that together with the teams building the server side to produce mobile-focused api’s, and producing a better functioning unit is what MOA is about. The web inherits the mobile architecture⁴.

But it’s still 2018

A number of things need to happen between now and then for the Kotlin version of all this to be real.

Kotlin/Native needs to be a fully baked part of the Kotlin multiplatform ecosystem. Around KotlinConf last year we were thinking this would be early 2018, but it’s not quite there yet.

If you follow progress on github, there’s some really cool stuff happening. Read the concurrency doc, for example. If you’re enough of a language nerd to know why Rust’s concurrency is cool, you’ll notice the same concepts. On the downside, reading the doc gives off the vibe that there’s going to be a lot of stuff that only functions in Kotlin/Native. If your thread sync needs to be handled in platform-specific code (and it obviously would be for JS), it’s not the end of the world, but still. Hopefully Kotlin JVM and Native aren’t too dissimilar.

We’ll need Kotlin/Native iOS-friendly libraries. I think there’s a strong pitch to make to the iOS community about why this is something they should get into, but that’ll be a whole different blog post.

WebAssembly?

We’ll have to see about WebAssembly’s progress as a standard. That’s a much bigger “if/when”. Kotlin/Native already has surprisingly good support (sample) for the current Wasm implementation, but Wasm needs a lot of the things on their post-MVP roadmap to materialize in order to be useful as a general purpose web app architecture. Threads, gc, and solid DOM access for starters. This is the part where the industry as a whole is going to have to deliver. For the suggestion box, it would also be great if we could reconsider Web SQL and make file storage universal. Thanks, industry!

But can’t you do this already with JS?!

Yeah, kind of. There will be multiple platforms from which you can deliver mobile and web. C++ all the things. Ruby for everybody. Dart, I guess. It’s more about the organization of everything. JS frameworks (React, NativeScript, whatever) are generally going to be web-focused. Flutter in its own way is a reaction to React, but this isn’t a JS or Flutter rant, so we’ll move on.

In a bit.

Am I anti-Javascript? Yes. Kind of. It’s a single-threaded, typeless, no-consequences world because of decisions a handful of people made in the mid-90s. I also like it sometimes too. What I really dislike is the default position that everything needs to be Javascript. It doesn’t. Assuming WebAssembly works out, it’ll be interesting to see what impact that has⁵.

We (Touchlab) just see this next wave of technologies as a better answer for a lot of scenarios. Especially if you’re a large org doing a lot of Java (and there are a lot of those). Outside of just this stack, the more similar web and mobile become, the less different everything needs to be (architecturally speaking, of course).

For now, you can definitely do Android and iOS. Those implementations will be able to talk to Kotlin/Native. See here. Everything kind of works together.

So, anyway. Keeping an eye out for that multiplatform commit…

¹ So, my UI rant. The short version is, most of the cross-platform stuff is trying to be an “everything” solution. Architecture and UI. Most include some way of interacting with “native” bits, but they’re generally treated very much as an exception. That means you need to make Big Decisions about platform choice. I like little decisions. What to do about more common UI is something we should look at, but the foundations should be poured first.

²There are some concepts to unpack here. J2objc is not something most iOS devs have been excited about largely because it feels kind of like a hand-me-down from the Android side. React Native and everything else that’s JS-based is a hand-me-down from the web. Xamarin, well, you’re either an MS shop or you’re not. I think the iOS community would embrace Kotlin more, largely because there’s going to be a need for iOS-leaning libraries, and until somebody builds a viable Swift for Android, it’s the best opportunity to *not* be in a hand-me-down situation. Mobile teams need a response when management says “what about ____?”, and this is a good one.

³ Don’t get too tripped up on the details. You could write a book on that. I don’t think the “server devs” should ship packages sdk’s to the “front end devs”. There’s a lot of history about how you should and should not do this. If you’re old enough, you’ll remember when Java Enterprise Bean spec tried to dictate separate roles for different parts of your team. Guess they still do? Amazing. Would not recommend. A good write up of implementing code sharing with different team focus can be found in the “Developing & Debugging” section of this blog post from Slack.

⁴ You could make it one big monolithic back-end service, but the mobile part still needs to get done right. Server architecture recommendations today tend to like a lot of little services, so that’s what we put in there, but you don’t *need* to do it that way.

⁵ i.e. What happens to JavaScript if you don’t need it for the web?

Droidcon NYC, on iOS, with Kotlin Native

by Kevin Galligan | Oct 31, 2017 | Droidcon NYC, Thoughts

(Crosspost from medium.com/@kpgalligan)

I don’t want to lead you on too much about the status of coding Kotlin on iOS. I’m very excited about where that’s going, but it’ll be a little while.

The Kotlin team is making tons of progress, though. It’s coming and it’ll be pretty great.

One of the things that will be needed are Kotlin libraries that don’t need access to the JRE. A lot of stuff is in the JRE, so many libraries will have some issues.

I’m not sure running Doppl/J2objc under Kotlin native will be how a lot of apps are built, but I was curious to see if it would work. After all, J2objc is basically a JRE running on iOS. With some smoothing, I think the two will go together quite well.

The Doppl Droidcon app runs RxJava 2, Retrofit 2, Dagger, Android Architecture components (RC1), with room, all on iOS, with Swift as a front end.

Here’s a Kotlin native version, calling into the same Doppl code underneath (RxJava 2, etc). It’s only the main schedule page. There are technical hurdles on the Kotlin side that I won’t go into, and I think they’re a mix of things the Kotlin team is fixing and/or me not fully understanding how to interact with the framework (and not wanting to spend more than a few hours on a proof of concept).

Touchlab/Kotlin-native-1

Get the Droidconsample branch. You’ll need to run through the Kotlin Native setup instructions. You’ll also need Doppl functional. Run through the new and improved Tutorial.

Replace the Doppl dist dir in a couple places:

samples/uikit/src/main/c_interop/objc.def

and…

samples/uikit/build.gradle

Then run

./gradlew uikit:build

and finally

./gradlew -p uikit compileKonanApp

That’ll take a while.

Then open the ‘konan.xcodeproj’ project and run. You’ll need an actual device, not a simulator.

If you do everything right, you’ll get an unstyled schedule list app for an old conference, but still, it works!

See you at KotlinConf and Droidcon SF!

Doppl Updates

by Kevin Galligan | Oct 20, 2017 | Thoughts

(Crosspost from medium.com/@kpgalligan)

We’ve been super busy updating lots of stuff after Droidcon, and since nothing is ever really “done” I wanted to share a few big things:

The Droidcon 2017 app was approved by Apple without any issues this time. Changed dogs.

The code behind that app can be found at: Doppllib/DroidconDopplExample

There are a few important points related to that project. It’s constructed with a reactive architecture using RxJava 2 and Android Architecture components, running on iOS. Yesterday, we updated the Android Architecture components to RC1, which was recently released.

The Xcode side of Doppl’s build system has been greatly simplified. Your code and dependencies are all wrapped in an Xcode Framework that Swift can call. No bridging header or manual management.

See the Party Clicker Sample for a tutorialon building apps. That brings us to another piece of news. To help document and make the framework easier to use, we’re getting some expert assistance from Mark Murphy, author of The Busy Coder’s Guide to Android Development.

Keep an eye on the Doppl website over the coming weeks, or sign up for the newsletter and we’ll keep you posted.

« Older Entries

Next Entries »