SQLite on Kotlin/Native

SQLite on Kotlin/Native

Tada! (Matty Matheson, Viceland, Giphy)

Today we’re releasing an early-ish version of Sqlite for K/N. More specifically, its a version of the Android Sqlite stack for K/N, that can be used in multiplatform projects.

 

Back in early May we demoed SQLite and SQLDelight running on iOS with Kotlin/Native and multiplatform. To implement the iOS side, I wrapped J2objc/Doppl code, which worked, but wasn’t exactly pretty under the hood.

In that post I said the goal was to get to this:

Pretty diagram

Well, that’s this.

I made a promise at the NYC meetups to release something before Droidcon Berlin. Flying out tomorrow, so here it is!

This is definitely an early release. Over the summer we’ll be updating the architecture, particularly around the multiplatform portion of it. Also, because K/N’s memory and threading model are significantly different than the JVM’s, some features were disabled and architecture simplified to be able to release something stable. We’ll be revisiting those decisions and adjusting the architecture as we go.

Details of the process and design will have to wait for follow on blog posts and/or talks. For now, here’s an overview of what’s available.

Main Project

The repo has 4 main folders.

cpp

This contains the code that directly talks to sqlite3. Much of this code is ported from AOSP and Android’s SQLite stack, but types are changed from JNI to what K/N uses. There’s also some code to facilitate state management and threading in K/N.

kotlin

The really interesting folder is kotlin-ios. This is the implementation of SQLite on iOS. You’ll notice a familiar set of classes (SQLiteDatabase, etc). The majority of these originate from AOSP, but have been updated to Kotlin and to live in an environment without the JVM or directly managed multithreading.

Much of the test folder is ported from CTS. This area of the code base will need a lot more TLC in the future, but the CTS tests provide a nice base to work from.

If you’re curious (and on a mac), run ‘./gradlew build’ on the kotlin folder, then look under kotlin-ios/build for the generated kexe file. This will run the test suite.

sample-notepad and sample-notepad-sqldelight

Two basic sample apps. One just SQL, one running SQLDelight’s multiplatform. Spoilers, the SQLDelight one is more interesting.

***Update!!!***
I totally forgot to mention that the android side of the samples isn’t done. Sqlite working on android isn’t exactly interesting. Soon, though.

SQLDelight

SQLDelight’s Kotlin generation and multiplatform support is currently in alpha. I have a fork with a driver implementation that allows it to function on iOS. I would not try to put this into production today, but definitely kick the tires, and something more official will certainly be coming along soon.

Usage

Currently there isn’t a dependency mechanism for K/N, as far as I know, so using the library means copying around the compiled artifacts. See the sample apps on how this is done. This gets super painful fast, so some kind of dependency publishing and management is high on the wish list.

Threading?

Threading on K/N is very different than what you’re probably used to in the JVM. The SQLite stack on Android does a lot of thread management internally. SQLiteOpenHelper, SQLiteDatabase, and all of its associated parts can (and need to be) frozen to be passed around to multiple threads. They all use the same sqlite connection under the hood. Internal state does change, but is kept in C++ structures in a way that is safe for K/N.

I tried to get clever with K/N and keep some non-frozen data that got passed around between threads. That ate a couple weeks and was sad. Would not recommend.

Cursor, I feel inclined to point out, is not thread safe, but Cursor isn’t really thread safe in the JVM either. I think it would be useful if K/N had a way to mark a class that specifically can’t, or shouldn’t, be frozen, such that it’ll throw at runtime if you try, but I guess you’ll find out soon enough. Anyway…

I’m way overdue for another Stranger Threads post. In it I’ll talk more about state and threads (and Atomics!).

KNarch?

K(otlin)N(ative)Arch(itecture). Sounds like narc. Names are hard. Suggestions welcome.

Want to help?

There’s a bunch of work that can be done here. Pick through issues, “needs hands” in particular. It’s a good opportunity to understand K/N at an architectural level. Video explaining how to set up the build environment coming after Droidcon Berlin.

Speaking of …

Other than Berlin, expect to hear more at Droidcon NYC, (probably) Droidcon London, and (very excited about this) KotlinConf! Also probably every Android NYC meetup for the foreseeable future. Sorry in advance, local friends.

Jobs!

Touchlab is currently working on projects with Doppl, and aggressively hunting down early Kotlin/Native adoption clients. We are very committed to delivering client work and open source code on this platform. If this is what you’d like to do, get in touch. We’re also remote friendly. Just FYI.

Projects!

If you’re part of an org that wants to start sharing architecture and be part of the cool crowd, we definitely want to talk to you.

Now, starting slides for the Berlin talk… (update: taking longer than planned)

Tick tock

Blindsided: The Hidden Cost in Mobile App Development

Blindsided: The Hidden Cost in Mobile App Development

“I just wanted the world to see I was real with it | Wanted a deal, got it, and couldn’t deal with it” – Joe Budden

Previously, I answered the most popular question ever regarding mobile app development: How Much Does It Cost To Build A Mobile App?.  You’ve since read that post, done your homework, found a great service provider (still looking, hit me up), answered all the questions about pre-engagement cost levers, and have a price that’s on budget.

You are good to go, right?

WRONG!

“But if the devil’s in the details, then I’m Satanic” – Drake

Don’t be blindsided by a good deal. (All respect due to Stevie Wonder … he’s a national if not global treasure!)

We only discussed the pre-engagement factors that affect the cost of developing a mobile application. However, there’s a major, hidden, shady A.F. way that costs can increase during the engagement, and it is all in the contractual statement of work.

RISK MITIGATION IN CONTRACTS

Typically, engagements will either be of one of two contractual formats:

  • Fixed bid: the product requirements, deliverables, necessary work effort and resultant price are set in the statement of work contract – ostensibly a low-to-no risk engagement
  • Time & materials: given a degree of uncertainty, client pays the contractor for the work effort exhausted toward achieving the statement of work, understanding that the desired deliverable may or may not be achieved within the contracted work effort

One can debate the merits of either format from a budgetary or resource planning perspective, however, in our experience, this is immaterial due to an unethical trick in practice.

FIXED BID TRICKS

Clients inexperienced with contractors may opt for fixed bid given the attractive benefit of a known price – it is an easier internal sale. Some unscrupulous contractors leverage this to win the engagement via a low fixed bid, while also withholding some of the unknowns necessary to complete the deliverables that have not been detailed in the contract.

What then happens is a slippery slope. The engagement kicks off, everything is great, work is getting done, but then, the unknowns are revealed and a work order or change order is presented as it deviates from the contract.

Uh, oh .. your fixed bid price is no longer fixed!

At this point, the client is stuck with the contractor, so there is no competitive bid on the work/change order. Whenever the scope of work creeps, so does the price. Now you have to go back and ask for more budget after you promised fixed costs. Ouch.

“But what I’m doing is not a good look | I never did it by the good book as a lifetime crook” – The Roots

In the world of mobile apps, where the hardware and software technology is constantly being updated, there are a host of unknowns that make fixed bid historically difficult if not impossible to navigate. Repeat after me:

Fixed bid mobile development is an aggressive business tactic that never ends well.

TIME & MATERIALS TRANSPARENCY

In a time & materials contract, where there is clear definition of price-for-level-of-effort rates, the client understands implicitly and explicitly how scope creep affects the pricing. We find that this clarity aligns both the client and contractor:

  1. We both know directionally where we want to end up.
  2. We both understand that there might be some known unknowns and unknown unknowns along the way.
  3. We agree on how the pricing structure will work in advance of those eventualities.

Though it takes more time to explain the value of time & materials, there’s a beauty in being transparent, resulting in better mutual guidance, fewer legal rotations, happier clients, and repeat business or referrals. 👍🏽

BOTTOM LINE

I pride myself on running an ethical mobile advisory and development business, where we reveal every risk, every cost, and every solution path before signature.

If you’re looking for a trusted, ethical mobile services partner, please consider Touchlab. We’re ready to earn your business. My name is Jeff. I help build businesses.

August meetup: Realm, and Smoke & Mirrors in Android

August meetup: Realm, and Smoke & Mirrors in Android

For August, we returned to the lovely, very cushiony auditorium of our August sponsor, The American Express Company. It’s a bit of a walk over to the west side, but that didn’t stop the 116+ members who came out. (Did it help that the World Trade Center is a lured PokemonGo stop??)

Kevin started the night off with news about livestreaming at Droidcon NYC, sponsored and made by possible by JW Player. Grab a conference ticket before it’s too late – we’re serious!

Across the street from AMEX headquarters. Across the street from AMEX headquarters.

Donn Felker took the stage first with a talk on Realm. He successfully squeezed 50+ slides into his informative 20-minute talk, quickly running through the benefits and features of the mobile object database. His awesome deck can be found here. Check it out for sample code and links to useful resources.

Our first speaker, Donn Felker. Our first speaker, Donn Felker.

Israel! Israel!

Our second speaker, Israel Camacho, received a warm welcome from the group. He recently moved to New York from California, and we’re thrilled to have him join our Android community! Kevin also surprised him with a birthday cupcake as he turned one year older the day before. Israel walked the group through neat UI tricks in Google Photos, such as transitions and image loading. Check out his slides here and his sample repo here. After the meetup, we watched the Olympics at Lilly O’Brien’s (Israel got his birthday drink!) and cheered as Phelps and Ledecky both took gold for USA.

We hope to see everyone at our “The Road to Droidcon” meetup at Squarespace – final date TBD, but it’ll be the week before Droidcon 9/29-9/30 🙂 See you then!

Summer bash at Button’s rooftop

Summer bash at Button’s rooftop

A few hours before our meetup’s summer rooftop bash, it started pouring torrentially. We were worried – meetup members even commented on our page and asked, “Sooo about that rooftop…” Luckily, the Android gods smiled down on us and cleared the skies just in time for our July meetup at Button. Our members were treated to sliders, copious amounts of wine and beer, and pizza from the awesome Button team. We also had Android tech talks from Uber, Instagram and LinkNYC.

Meetup organizer and touchlab President Kevin Galligan greeted the group with his usual finesse and started the night with chuckles: “It’s a bit crowded in this room so nobody move. We won’t sweat this way.”

Kevin captivates the crowd with his welcome. Kevin captivates the crowd with his welcome. Kevin and Sveinung with Button give the host welcome. Scroll to the bottom if you Kevin and Sveinung with Button give the host welcome. Scroll to the bottom if you’d like to enter their contest, you could win an Amazon Echo!

Jeff Hu with the Uber Android team was our first speaker. He introduced Uber’s internal Octopus automated test framework and dove into details of UberRUSH’s unique end-to-end testing. His pro tip for the group: “‘Make Magic’, one of the Uber culture values. Uber is built on the magic of pressing a button and having a car appear. As a mobile engineer, let’s develop forward-thinking ideas, and produce high quality work that raises the bar. #alwaysbetesting.” Thank you Jeff for speaking, even with the 9pm flight he had to catch right after!

Jeff Hu, Uber Jeff Hu, Uber

Blake Arnold and Justin Taylor took the floor after and walked us through the LinkNYC project. They described the history of the wifi kiosks and how Android played a critical part in development. They also gave a special shout out to our very own Izzy Oji of touchlab, for her part in the project since the very beginning. For Android beginners, Blake and Justin advised, “Get a solid foundation in computer science, whether from formal schooling or on your own. Make a couple applications in your spare time, open source them, and be open to learning other platforms like web dev. That’ll get your foot in the door at most companies. The rest can be learned on the job.”

Izzy (touchlab), Blake and Justin (Intersection) Izzy (touchlab), Blake and Justin (Intersection)

Kang Zhang with Instagram finished off the night with a talk on Android Client Architecture. He shared the team kept the app’s performance reliable and lean as their client infrastructure evolved over the past few years. Kang’s #1 tip for our group? “Do simple thing first.” He pointed us to “Instagram + Android: Four Years Later” if you’d like to read more about Instagram’s recent mobile development.

Kang Zhang, Instagram Kang Zhang, Instagram

Our July meetup was a huge success – thanks to Button for generously providing booze and food. From our host: “Reminder to submit your Button for your Android app by 7/21 to enter to win an Amazon Echo for the best Button implementation! Email screenshots or an APK, even if it’s a test build, to android@usebutton.com to apply! Button makes it easier for app developers to monetize by helping users discover their next action. By connecting the apps where users spend time, with the apps they spend money, Button provides a monetization strategy where everyone wins.”

Thanks to everyone who made it out, and to the Button team (Stephanie, Sophia, Kevin & Sveinung) for coordinating.
See you at our August 9th meetup with Israel Comacho and Donn Felker at AMEX!

Designing ResearchStack’s open source UX framework

Designing ResearchStack’s open source UX framework

In the fall of 2015, touchlab partnered with Cornell Tech and Open mHealth to bring precise medical research apps to Android with an open source sdk and ux framework called ResearchStack.

Executing this project would be unlike most projects – rather than designing one experience and interface for a single product, we would be designing and building an extensible framework, with designs covering many potential bits and pieces researchers and developers might need to carry out medical research on Android.

Beyond that, we would be designing and building an Android version of Mole Mapper, an existing ResearchKit™  app for iOS that helps users document and keep track of moles across their body, allowing them to share that information with their doctor and promote earlier detection of skin cancer.

So, how would we approach the UX framework? One of the primary goals (besides IRB approval) was to create an experience that was as easy and straightforward as possible but which also ensured that the user is – and remains – completely aware of how their data is managed and what information the apps collect.

The other major piece was creating something users would want to use to keep track of their health and participate in studies. Anecdotally, we learned that some researchers creating apps for ResearchKit had challenges with user retention. The real value of mobile research studies is in accessibility – if users can join a study and track their information easily, studies could operate with unprecedented sample sizes. That’s why we decided to tackle this problem head-on.

So with both of those goals in mind, the UIUX framework behind ResearchStack was broken into three main pieces – onboarding, dashboard and tasks, and data management and visualization. These three pieces would get the users smoothly into the app/study, make completing research and documentation tasks easy (and maybe even enjoyable), and make data clear for them and their care providers.

Onboarding

Pre-roll onboarding (or onboarding that has to happen before the user can open the app) is something that’s easy to get wrong, creating an experience that annoys the user or makes it feel like the barrier to entry is too high.

That said, a medical research app requires the user to be informed before joining, so the challenge here was to build out a process that felt light even though there could be as many as sixteen actions or more between the user and officially being part of the study. If the user doesn’t want to be part of the study, there is an affordance for simply using the app to keep track of their own information.
The onboarding is itself broken into three pieces.

First, eligibility. Is the user actually eligible to join the study? This step uses general qualifying questions to determine if a user is eligible to participate in the study. Like the rest of the onboarding process, the interface is minimal by design. Question layouts and list controls are predictable and uniform, with a high contrast color palette of just white, gray, and blue to visibly highlight questions, selections, and progressive actions.

Next is consent. The user needs to have a clear picture of what the study entails, the purpose of the app, and the fact that they can leave the study at any time. Besides providing this information to the user with illustrations, videos, or links to further reading, the consent process can ensure the user is truly informed by requiring a brief quiz covering the preceding information.

The quiz steps maintain the predictable question layouts from before, but are more dynamic. After a user submits their answer, a message will tell the user whether they are correct or not, highlighting both the user answer and the correct answer, and providing an explanation when needed.

The bottom action bar in this case is also dynamic – if the user needs to read an explanation about the answer that extends off-canvas, the “NEXT” button transforms to “MORE,” and – when touched – scrolls the text progressively until the end, when the user can finally touch “NEXT.”

The same mechanic is used at the end of the quiz when the user finds out whether they passed the quiz. Answers are explained one more time, and the user can go back to give it another try.

Once the user is fully informed and ready to join the study, they need to register and create  an account. We designed a flow that allows the user to sign up, enter personal info, and create a secure passcode to encrypt their data, even if they exit the app during signup and then come back.

Dashboard & tasks

Inside a research app, we wanted to create some UX definitions for at least two primary goals – study participation through “tasks,” and the visualization of data for the user and their care providers.

Most basic tasks come in the form of surveys or measurements. These kinds of tasks rely on the same predictable question layouts mentioned above, but tasks can contain various types of questions and inputs, either on the same screen or on multiple sequential screens. So we came up with designs that allow for this kind of modular assembly of tasks.

Apps like Mole Mapper make use of these patterns too, even while simultaneously including tasks requiring more unique interactions like using the camera and visually measuring objects.

 

Tasks, in the framework design, are managed on one half of the dashboard interface. They appear as a list of items that can show visual categorization with colored indicators. In most cases, completed tasks will slide to the bottom of the list for a given day, and previous days’ responses can be found in a chronological timeline below that.

But this dynamic can change, even within the visual definitions we provided. For example, Mole Mapper’s tasks are simply divided into two subheads – to-do and done. Since Mole Mapper works on a monthly cycle and previous mole measurements are stored in an interface that visually connects them to the moles, there’s no reason to keep duplicate entries in a chronological timeline on the main interface.

As explained before, keeping users engaged and participating is tough. Part of this is ensuring that participants complete their tasks to create full data sets. To try and encourage participation, we made creative use of visual cues to get users back into the task list until all the tasks are done.

In the tab bar, the task icon or text label will be badged with a small circle to indicate some unfinished business. If the user is still ignoring that and looks into their data dashboard, visual cues will prompt them to “FINISH” tasks, with the button simply leading back again to the task list.

Data management & visualization

Designing charts and graphs is and was a pleasure. Data visualization is a rewarding exercise, offering the chance to design something that looks deceptively smooth for how rigid the underlying data are.

For the initial release of the open source framework, we designed several kinds of visualizations, from a basic pie chart to interactive line graphs and multivariate bar charts.

The same reduced palette as before allowed us a lot of freedom to clearly communicate information and add new colors to the palette for multiple variables as needed. Keeping things minimal meant we had plenty of room to add new touches of information.

And as always, one of the goals of the design was to create something predictable for the user that would easily and quickly build a mental model of how data visualizations worked and what they meant. Consistent “expand” icons allow users to see larger views of their information, cards can be arranged in any order and still appear familiar, and clear headers and labeling explain everything.

On the management side of data, we wanted to make sure the user always had options to manage their data and participation status. Persistent action icons in the app’s toolbar provide access to information about the study and settings around data sharing, consent, security, and study participation. If the user wants to learn more, stop sharing, or even leave the study, it’s only a tap away.

Conclusion

Extensible design is a recurring theme in our work, and nowhere does it better shine than an open-source UX framework.

The design of ResearchStack’s framework is one that allows for easy customization and organization without losing the hallmarks that make it easy for users to get started and stay engaged.

Simple palettes, clear and predictable layouts, and a visual hierarchy that doesn’t get in the way of unique and diverse interactions come together to lay a foundation for designing precise medical research apps for Android.

Bringing Mutative Design to NYC Apps at IDEO

Bringing Mutative Design to NYC Apps at IDEO

IDEO welcomes NYC Apps with beer and pizza!  Photos courtesy of NYC Apps.  IDEO welcomes NYC Apps with beer and pizza!  Photos courtesy of NYC Apps.

It’s small world, but did you know that the NYC tech world is even smaller? I recently met up with an old colleague, Leah Taylor, who told me about an upcoming NYC Apps meetup focused on design. As the go-to shop for Android design & development, I thought it’d be great if touchlab got involved somehow. Lo’ and behold, Kevin happened to know the meetup’s organizer, Serko Artinian, from the good ol’ coworking days. A few weeks later, our lead designer, Liam Spradlin, presented his latest project at the meetup, hosted by IDEO.

While everyone was chowing down on Saluggi’s pizza, Serko shared a fun Lyft ad to jumpstart the evening. He also welcomed the group with recent tech news before giving the floor to the speakers.

Liam gave the group an intro to Project Phoebe, his latest project focused on “mutative design” — a design methodology for interfaces that can adapt automatically to users. He gave examples of how a contacts app could enhance touch target sizes and highlight important actions for a child, and how the same app could enhance contrast for a user with low vision. There was a flurry of questions after the talk from fascinated designers. One person wondered how long a mutative interface would take to adapt. If he had an argument with his girlfriend and was angrily sending text messages with errors, would the interface mutate that day but switch back after the fight was over? (The answer: Mutations probably wouldn’t immediately react to sudden temporary temporary behavior changes like these.) If you’d like to learn more about Project Phoebe, you can read about it here and find the open source repo here.

Liam Spradlin introducing Project Phoebe. Photos courtesy of NYC Apps. Liam Spradlin introducing Project Phoebe. Photos courtesy of NYC Apps.

Simon Kirk, the director of business development at Invision, walked the group through the latest upcoming features that’ll improve collaboration between designers and developers. IDEO’s team talked about conversational interfaces. They showed how SMS enables conversational UI, and how it can be used for quick testing of conversational interfaces.

The IDEO The IDEO’s team presentation on conversational interfaces. Photos courtesy of NYC Apps.

A big congrats to Serko for putting on another amazing meetup. Thanks so much for having the touchlab team!
Also, 2 epropz to IDEO for hosting and ordering Saluggi’s!

A group shot of the evening A group shot of the evening’s hosts and speakers with their teams.  Photos courtesy of NYC Apps.