Kotlin Native Stranger Threads Ep 2

Episode 2 — Two Rules

http://makeitstranger.com/

This is part 2 of my Kotlin Native threading series. In part 1 we got some test code running and introduced some of the basic concurrency concepts. In part 2 we’re going a bit deeper on those concepts and peeking a bit under the hoot.

Some reminders:

1. KN means Kotlin Native.
2. Emoji usually means a footnote 🐾.
3. Sample code found here:

touchlab/StrangerThreadsSample
_Contribute to touchlab/StrangerThreadsSample development by creating an account on GitHub._github.com

Look in the nativeTest folder: src/nativeTest/kotlin/sample

Two Rules

K/N defines two basic rules about threads and state:

1. Live state belongs to one thread
2. Frozen state can be shared

These rules exist to make concurrency simpler and safer.

Predictable Mutability

Rule #1 is easy to understand. For all state accessible from a single thread, you simply can’t have concurrency issues. That’s why Javascript is single threaded 😤.

If you generally understand thread confinement, the KN runtime enforces an aggressive form on all mutable state. Feel free to skip to “Seriously Immutable”

As an example that you’re likely familiar with, think of the android and iOS UI systems. Have you ever thought about why there’s a main thread? Imagine you’re implementing that system. How would you architect concurrency?

The UI system is either currently rendering, or waiting for the next render. While rendering, you wouldn’t want anybody modifying the UI state. While waiting, you wouldn’t care.

You could enforce some complex synchronization rules, such that all components’ data locks during rendering, and other threads could call into synchronized methods to update data, and wait while rendering is happening. I won’t even get into how complex that would probably be. The UI system really gets nothing out of it, and synchronizing data has significant costs related to memory consistency and compiler optimization.

Rather than letting other threads modify data, the UI system schedules tasks on a single thread. While the UI system is rendering, it is occupying that thread, effectively “locking” that state. When the UI is done, whatever you’ve scheduled to run can modify that state.

To enforce this scheme, UI components check the calling thread when a method is called, and throw an exception if it’s the wrong thread. While manual, it’s a cheap and simple way to make things “thread safe”.

That’s effectively what’s happening here. Mutable state is restricted to one thread. Some differences are:

1. In the case of UI, you’ve got a “main” thread and “everything else”. In KN, every thread is it’s own context. Mutable state exists wherever it is created or moved to.
2. UI (and other system) components need explicitly guard for proper thread access. KN bakes this into the compiler and runtime. It’s technically possible to work around this, but the runtime is pretty good at preventing it 🧨.

Maintaining coherent multithreaded code is one of those rabbit holes that keeps delivering fresh horrors the deeper you go. Makes for good reading, though.

Seriously Immutable

Rule #2 is also pretty easy to understand. More generally stated, the rule is immutable state can be shared. If something can’t be changed, there are no concurrency issues, even if multiple threads are looking at it.

That’s great in principle, but how do you verify that a piece of state is immutable? It would be possible to check at runtime that some state is comprised entirely of vals or whatever, but that introduces a number of practical issues. Instead, KN introduces the concept of frozen state. It’s a runtime designation that enforces immutability, at run time, and quickly lets the KN runtime know state can be shared.

As far as the KN runtime is concerned, all non-frozen state is possibly mutable, and restricted to one thread.

Freeze

Freeze is a process specific to KN. There’s a function defined on all objects.

public fun \<T\> T.freeze(): T

To freeze some state, call that method. The runtime then recursively freezes everything that state touches.

someState.freeze()

Once frozen, the object runtime metadata is modified to reflect it’s new status. When handing state to another thread, the runtime checks that it’s safe to do so. That (basically) means either there are no external references to it and ownership can be given to another thread, or that it’s frozen and can be shared 🤔.

Freezing is a one way process. You can’t un-freeze.

Everything Is Frozen

The freeze process will capture everything in the object graph that’s being referenced by the target object, and recursively apply freeze to everything referenced.

data class MoreState(val someState: SomeState, val a:String)

@Test
fun recursiveFreeze(){
  val moreState = MoreState(SomeState("child"), "me")
  moreState._freeze_()
  assertTrue(moreState.someState.isFrozen)
}

In general, freezing state on data objects is pretty simple. Where this can get tricky is with things like lambdas. We’ll talk more about this when we discuss concurrency in the context of application development, but here’s a quick example I give in talks.

val worker = Worker.start()

@Test
fun lambdaFail(){
  var count = 0

  val job: () -> Int = {
    for (i in 0 until 10) {
      count++
    }
    count
  }

  val future = worker.execute(TransferMode.SAFE, { job.freeze() }){
    it()
  }

  assertNull_(future.result)
  assertEquals_(0, count)
  assertTrue(count.isFrozen)
}

There’s a bit to unpack in the code above. The KN compiler tries to prevent you from getting into trouble, so you have to work pretty hard to get the count var into the lambda and throw it over the wall to execute.

Any function you pass to another thread, just like any other state, needs to follow the two rules. In this case we’re freezing the lambda job before handing it to the worker. The lambda job captures count. Freezing job freezes count ⚛️.

The lambda actually throws an exception, but exceptions aren’t automatically bubbled up. You need to check Future for that, which will become easier soon.

Freeze Fail

Freezing is a process. It can fail. If you want to make sure a particular object is never frozen recursively, you can call ensureNeverFrozen on it. This will tell the runtime to throw an exception if anybody tries to freeze it.

@Test
fun ensureNeverFrozen()
{
  val noFreeze = SomeData("Warm")
  noFreeze._ensureNeverFrozen_()
  assertFails {
    noFreeze.freeze()
  }
}

Remember that freezing acts recursively, so if something is getting frozen unintentionally, ensureNeverFrozen can help debug.

Global State

Kotlin lets you define global state. You can have global vars and objects. KN has some special state rules for global state.

var and val defined globally are only available in the main thread, and are mutable 🤯. If you try to access them on another thread, you’ll get an exception.

val globalState = SomeData("Hi")

data class SomeData(val s:String)

//In test class...
@Test
fun globalVal(){
  assertFalse(globalState.isFrozen)
  worker.execute(TransferMode.SAFE,{}){
    assertFails {
      println(globalState.s)
    }
  }.result
}

object definitions are frozen on init by default. It’s a convenient place to put global service objects.

object GlobalObject{
  val someData = SomeData("arst")
}

//In test class...
@Test
fun globalObject(){
  assertTrue(GlobalObject.isFrozen)
  val wval = worker.execute(TransferMode.SAFE, {}){
    GlobalObject
  }.result
  assertSame(GlobalObject, wval)
}

You can override these defaults with annotations ThreadLocal and SharedImmutable. ThreadLocal means every thread gets it’s own copy. SharedImmutable means that state is frozen and shared between threads (the default for global objects).

@ThreadLocal
val _thGlobalState_ = SomeData("Hi")

@Test
fun thGlobalVal(){
  assertFalse(thGlobalState.isFrozen)
  val wval = worker.execute(TransferMode.SAFE,{}){
    thGlobalState.freeze()
  }.result

  assertNotSame(thGlobalState, wval)
}

@SharedImmutable
val sharedGlobalState = SomeData("Hi")

@Test
fun sharedGlobalVal(){
  assertTrue(sharedGlobalState.isFrozen)
  val wval = worker.execute(TransferMode.SAFE,{}){
    sharedGlobalState.freeze()
  }.result

  assertSame(sharedGlobalState, wval)
}

Atomics

Frozen state is mostly immutable. The K/N runtime defines a set of atomic classes that let you modify their contents, yet to the runtime they’re still frozen. There’s AtomicInt and AtomicLong, which let you do simple math, and the more interesting class, AtomicReference.

AtomicReference holds an object reference, that must itself be a frozen object, but which you can change. This isn’t super useful for regular data objects, but can be very handy for things like global service objects (database connections, etc).

It’ll also be useful in the early days as we rethink best practices and architectural patterns. KN’s state model is significantly different compared to the JVM and Swift/ObjC. Best practices won’t emerge overnight. It will take some time to try things, write blog posts, copy ideas from other ecosystems, etc.

Here’s some basic sample code:

data class SomeData(val s:String)


@Test
fun atomicReference(){
  val someData = SomeData("Hello").freeze()
  val ref = AtomicReference(someData)
  assertEquals("Hello", ref.value.s)
}

Once created, you reference the data in the AtomicReference by the ‘value’ property.

Note: The value you’re passing into the AtomicReference must itself be frozen.

@Test
fun notFrozen(){
  val someData = SomeData("Hello")
  assertFails {
    AtomicReference(someData)
  }
}

So far that’s not doing a whole lot. Here’s where it gets interesting.

class Wrapper(someData: SomeData){
  val reference = AtomicReference(someData)
}

@Test
fun swapReference(){
  val initVal = SomeData("First").freeze()
  val wrapper = Wrapper(initVal).freeze()
  assertTrue(wrapper.isFrozen)
  assertEquals(wrapper.reference.value.s, "First")
  wrapper.reference.value = SomeData("Second").freeze()
  assertEquals(wrapper.reference.value.s, "Second")
}

The Wrapper instance is initialized and frozen, then we change the value it references. Again, it’s very important to remember the values going into the AtomicReference need to be themselves frozen, but this allows us to change shared state.

Should You?

The concepts introduced in KN around concurrency are there for safety reasons. On some level, using atomics to share mutable state is working around those rules. I was tempted to abuse them a bit early on, and they can be very useful in certain situations, but try not to go crazy.

AtomicReference is a synchronized structure. Compared to “normal” state, it’s access and modification performance will be slow. That is something to keep in mind when making architecture decisions.

A small side note. According to the docs you should clear out AtomicReference instances because they can leak memory. They won’t always leak memory. As far as I can tell, leaks may happen if the value you’re storing has cyclical references. Otherwise, data should clear fine when your done with it. Also, in general you’re storing global long lived data in AtomicReference, so it’s often hanging out “forever” anyway.

Transferring Data

According to rule #1, mutable state can only exist in one thread, but it can be detached and given to another thread. This allows mutable state to remain mutable.

Detaching state actually removes it from the memory management system and while detached, you can think of it as being in it’s own little limbo world. If for some reason you lose the detached reference before it’s reattached, it’ll just hang out there taking up space 🛰.

Syntactically, detaching is very similar to what we experienced with the Worker producer. That makes sense, because Worker.execute is detaching whatever is returned from producer (assuming it’s not frozen). You need to make sure there are no external references to the data you’re trying to detach. The same syntactically complex situations apply.

You detach state by creating an instance of DetachedObjectGraph, which takes a lambda argument: the producer.

data class SomeData(val s:String)


@Test
fun detach(){
  val ptr = DetachedObjectGraph {SomeData("Hi")}.asCPointer()
  assertEquals("Hi", DetachedObjectGraph<SomeData>(ptr).attach().s)
}

Like Worker’s producer, non-zero reference counts will fail the detach process.

@Test
fun detachFails(){
  val data = SomeData("Nope")
  assertFails { DetachedObjectGraph {data} }
}

The detach process visits each object in the target’s graph when performing the operation. That is something that can be a performance consideration in some cases 🐢.

I personally don’t use detach often directly, although I’ll go over some techniques we use in app architectures in a future post.

TransferMode

We mentioned TransferMode a lot back in the last post. Now that we have some more background, we can dig into it a bit more.

Both Worker.execute and DetachedObjectGraph take a TransferMode parameter. DetachedObjectGraph simply defaults to SAFE, while Worker.execute requires it explicitly. I’m not sure why Worker.execute doesn’t also default. If you want a smart sounding question to ask at KotlinConf next year, there you go.

What do they do? Essentially, UNSAFE let’s you bypass the safety checks on state. I’m not 100% sure why you’d want to do that, but you can.

It’s possible that you’re very sure your data would be safe to share across threads, but the purpose of the KN threading rules is to enable the runtime to verify that safety. Simply passing data with UNSAFE would defeat that purpose, but these’s a far more important reason not to do that.

We mentioned back in post #1 that memory is reference counted. There’s actually a field in the object header that keeps count of how many references exist to an object, and when that number reaches zero, the memory is reclaimed.

That ref count itself is a piece of state. It is subject to the same concurrency rules that any other state is. The runtime assumes that non-frozen state is local to the current thread and, as a result, reference counts don’t need to be atomic. As reference counting happens often and is not avoidable, being able to do local math vs atomic should presumably have a significant performance difference 🚀.

Even if the state you’re sharing is immutable but not frozen, if two threads can see the same non-frozen state you can wind up with memory management race conditions. That means leaks (bad) or trying to reference objects that have been freed (way, way worse). It you want to see that in action, uncomment the following block at the bottom of WorkerTest.kt:

@Test
fun unsafe(){
  for (i in 0 until 1000){
    unsafeLoop()
    println("loop run $i")
  }
}

private fun unsafeLoop() {
  val args = Array(1000) { i ->
    JobArg("arg $i")
  }

  val f = worker.execute(TransferMode.UNSAFE, { args }) {
    it.forEach {
      it
    }
  }

  args.forEach {
    it
  }

  f.result
}

I won’t claim to know everything in life, but I do know you don’t want memory management race conditions.

Also, as hinted with 🤔, the shared type throws a new special case wrinkle into all of this, but that’s out of scope for today, and only used with one class for now.

Maybe This Is All Going Away

Now that we’ve covered what these concurrency rules are, I’d like to address a more existential topic.

I started really digging into KN about a year ago, and I wasn’t a fan of the threading model at first. There’s a lot to learn here, and the more difficult this is to pick up, the less likely we’ll see adoption, right?!

I think having Native and JVM with different models can be confusing, although you do get used to it. Better libraries, better debug output, being able to debug in the IDE at all: these things will help.

There is some indication that these rules may change. There has been little clarification as to what that means. Reading through those comments, and from my community conversations, there is definitely some desire for the Kotlin team to “give up” and have “normal” threading. However, that’s not a universal opinion, it’s pretty unlikely to happen (100% anyway) , and I think it would be bad if it did. If anything, I’d rather see the ability to run the JVM in some sort of compatibility mode, which applies the same rules.

What I think we could all agree on is the uncertainty won’t help adoption. If there’s likely to be significant change to how Native works in the near term, going through the effort of learning this stuff isn’t worth it. I’m really hoping for some clarification soon.

Up Next

We’ve covered the basics of state and threading in KN. That’s important to understand, but presumably you’re looking to share logic using multiplatform. JVM and JS have different state rules. Future posts will discuss how we create shared code while respecting these different systems, and how we implement concurrency in a shared context. TL;DR we’re making apps!

---

🐾 I’m prone to tangents. Footnotes let you decide to go off with me.

😤 Yes, workers. All the state in the worker is still local to a single thread, and the vast majority of JS simply lives in the single thread world.

🧨 In more recent versions of KN, the runtime seems to catch instances of code accessing state on the wrong thread due to native OS interop code, but I’m not sure how comprehensive that is. In earlier versions it didn’t really check much, and it was pretty easy to blow up your code. If you want to experience that for yourself, try passing some non-frozen state around with TransferMode.UNSAFE.

🤔 There’s a recent addition to the runtime that defines another state called shared. It’s effectively allowing shared non-frozen state. You can’t call it and create shared objects 🔓. It’s an internal method. Currently you can create a shared byte array using MutableData, and that’s about it. I’d be surprised if that’s the only thing “share” is used for. We’ll see.

🔓 That’s not 100% true. I’m pretty sure you could call it, but I’m not saying how 🙂

⚛️ You can make this example work with an atomic val, although I’m guessing you wouldn’t actually write it this way. But…

@Test
fun atomicCount(){
  val count = AtomicInt(0)

  val job: () -> Int = {
    for (i in 0 until 10) {
      count.increment()
    }
    count.value
  }

  val future = worker.execute(TransferMode.SAFE, { job.freeze() }){
    it()
  }

  assertEquals(10, future.result)
  assertEquals(10, count.value)
  assertTrue(count.isFrozen)
}

🤯 This wasn’t true early on. Each worker would get it’s own copy on init. That’s a problem if you do something like define a global worker val (which gets created on init, and defines a worker val, which gets created on init…).

🛰 I’ve made a horrible Star Trek analogy. It’s like beaming. It can live on one side or the other, but can do nothing in transit. If it results in the same entity in two places you have a big problem, and it can be lost in transit forever.

🚀 I’m definitely not suggesting you do any premature optimization, but if you were constructing some performance critical code and had a big pile of state hanging around, it’s maybe something to consider. Non-frozen state wouldn’t have the same accounting cost. Now forget you read this.