Android Sync Bus

On several projects we’ve needed the ability to sync stuff offline.  We’ve tried a number of approaches, and eventually wound up building a call bus service.

The basic idea behind the sync call bus is you post a command to the service, it adds it to a queue, and attempts to process the command.  The bus keeps commands in order, processes them one at a time, and will persist the command to be run later if the system can’t handle the command now, generally because you don’t have a network connection.

Integration

1. You need to grab our jar file and add it to your project.
2. Create a service class that extends from either SuperbusService or LocalFileBusService
3. Add your service to the AndroidManifest.
4. Create command instances and add them to the bus in code.
5. Profit.

Currently you’ll also need to add android.permission.ACCESS\_NETWORK\_STATE permission, although I’m thinking we should remove the network dependencies from the core service and make the network stuff optional (but today we’re talking about today).

SuperbusService or LocalFileBusService

The original library was just SuperbusService.  You’d need to implement your own storage mechanism.  On the projects we’ve done, we use Sqlite.  The issue here is each command needs a different table schema, or you need something sort of hacky, which is what we did (an id field and a string field, and the string would have data in your own format).  The positive side of using Sqlite is the strict transactions, so not much worry about having weird save issues, etc.

Recently I’ve added LocalFileBusService.  The idea is we create a special directory in the local file store, and in there we put one file per command.  The benefit here is you can structure the data any way you want, and assuming we nail the file saving and removing, things should work well.  Again, though, this is new.  Were I starting now, though, I’d use the local file store.  Simpler implementation.

Command Instances

The Command object is where the work gets done.  The core Command object has several abstract methods that need to be implemented:

String logSummary() : Return a string that will be logged out for this Command (you’ll want this if you don’t want to be confused during dev and testing)

boolean same(Command command) : Sort of like “equals”.  If you don’t want duplicate commands added, say for a screen refresh, implement this method to model that.  If you add a command, an there’s an existing one that is the “same”, the new one will be ignored.

void callCommand(Context context)throws TransientException, PermanentException : This is where you do your stuff.  Its recommended that you capture all Exceptions and throw the appropriate one (explained below).

There are 3 error handling methods you can override: onTransientError, onPermanentError, and onUnknownError.  If you want to log something, or tell the user something (With Notifications, not an alert box. We’re in a Service!), do it here.

Errors

There are 2 basic types of errors defined for the Command. Transient and permanent.  Their names pretty much describe what they are.  Transient errors are situational and will go away on their own.  Permanent are system errors that you can’t recover from.

When something breaks, but is situational and will likely resolve itself, throw a TransientException.  Examples are losing your network connection, or possibly not having the SD card mounted.  TransientExceptions cause the queue and bus to stop processing temporarily, but the Command is not removed.  This is critical if the order of commands matters (in our projects is often does).

When something breaks, and its a hard error, throw a PermanentException.  Examples would be a SqlException, or a NullPointerException.  When this happens, your Command instance is removed, and you’ll need to deal with that.  If Command instances after the failing one depend on the failed one, you may have issues.  A concrete example, if you’re inserting something remotely, you’ll expect an ID to come back.  If you don’t have that ID, and the other Commands expect it, those calls will fail.  Just something to think about.

Using Local File

To use the local file store, extend LocalFileBusService, and create instances of LocalFileCommand.  You’ll need to implement methods to read and write your command state, which is a little cumbersome.  Another option is to use JsonFileCommand.  You can add and pull fields from a JsonObject instace.  The system will then handle saving and inflating the Command instances.  Pretty sweet.

Why not Java Serialization?  Ehh, I feel like version upgrades will be weird.  Could use it.  Just need to be careful.  I feel like json is simpler and easier to diagnose visually.  Situation is fluid at this point.  Could also use something like protocol buffers, which sort of misses the point (network communication), but lets you define serializable data objects that allow for upgrading in a formal way.

But I digress.

Source Code

Grab the bus source here:

https://github.com/touchlab/Superbus

I added an EXTREMELY simple example app here:

https://github.com/touchlab/Superbus-Example

Random Thoughts

The bus processes in order, which may not really be what you want.  For example, maybe parts of your app load new data and don’t want to wait for long writes or uploads.  I haven’t really tried this yet, but it should be really easy to create multiple bus services.

For the local file store, one of the things I need to investigate more is partial writes, and running out of space.  When adding a file, it writes the full file, then renames it, so in theory, you shouldn’t have a partially written file, but its new code, so who knows?  Also, I need to review the processor.  If a stored file fails loading, it should be removed completely.  If not, it’ll sit around forever.

One thing to be careful of, if you’re always throwing TransientException, but the command can’t process, these data files may just build up, and the Service will just keep stalling.  Be careful.

The update schedule is pretty basic.  If the service gets put to sleep, it doesn’t wake up till it gets another notification.  What would make sense would be for the service to listen for reconnect notifications to check its store.  I don’t want to burn the battery when not necessary, though.  That’s on the todo list.

Anyway, hit me up with questions/issues.

-Kevin