The word vocabulary has a special place in my memory…
I recall as a small boy being totally unable to pronounce the word - I could get as far as the “vocab” part and would then stumble hopelessly on the second half of the word, and circumlocution was difficult because there are no good synonyms (that I’m aware of) for its everyday meaning.
The only other word I recall having difficulty with (excepting my almost total rhotacism) was “abominable”. Until one day someone — I can’t recall who — suggested I say “a bomb in a bull” instead. My impasse was broken immediately and I wasted no time applying the technique to vocabulary which I imagined to be a bizarre cuss: “Vocab you Larry!”
To this day I still pronounce the word with a slight overemphasis on the first two syllables.
I was reminded of this by an online vocabulary test that several people have shared recently. I scored well (by which I mean I beat my wife’s score, which is surely the truest test of a man’s vocabulary) and this prompted me to relate a little of my history with vocabulary and to make the following observation.
So much of our time as programmers is occupied with the creation of our own enduring abstractions that an excellent vocabulary is a powerful secret weapon. The ability to find les mots justes with which to name constants, classes and methods and everything else one must name in an API, enhances understanding and sidesteps confusion.
Great abstractions deserve the best names and to identify either is greatly satisfying.
There’s been lots of commentary about Apple’s unveiling of iCloud at the WWDC. I haven’t had any time to contribute my own views, but I have read a number of posts about iCloud (they’ve been difficult to avoid). One thing that has really jumped out at me is the apparent confusion that permeates the discussion of native applications as contrasted to web applications.
For the purpose of this post, I define a web application to be an application composed of HTML pages running in a browser and define a native application as any application implemented otherwise. The “native/web” distinction seems to be poorly understood, here’s my take.
A web application that does not use local storage/caching will probably be rendered inaccessible by a total network failure. A native application that doesn’t use web storage will probably be rendered unusable by a disk failure that corrupts the user’s data partition. The first is more likely, the second is more catastrophic.
Advancing this scenario to reflect more modern implementations, the web application can use local storage to mitigate network failure, and the native application can use web storage to mitigate disk failure. For modern web applications running in modern browsers, the differentiator has little to do with network availability.
Whether it’s true or not, the perception that native applications are inherently more tolerant of network problems than web applications may come to be seen as mostly irrelevant; it may not always be the case that computing devices frequently lose Internet connectivity. And even if reliable connectivity remains elusive, the list of characteristics that are uniquely associated with native applications, including the ability to run offline, can be expected to shorten significantly: Offline availability, Local storage, Geolocation, Media capture Multitouch, Background processes, 3D are all coming (or have come) to HTML, and the list could go on.
I think the issue that will remain least adequately addressed by web applications is that of performance, particularly performance on mobile devices. Putting aside the use of Canvas for producing graphics, animations and games, application UIs that are both responsive and complex remain beyond the comfortable reach of the processors in mobile devices. And unless a revolution in battery technology occurs this is likely to remain the case.
Even accounting for the recent performance improvements in browsers on all platforms (JS in particular has seen spectacular gains), performance lags native applications. This can be masked by more powerful computers, and often by clever application design, but the gap is real and readily detected by users.
In my opinion, there are two essential benefits to web applications over native applications.
Linking
The URL/URI is an incredibly powerful tool. It allows any web resource(*) to reference any other web resource. This is an advantage that native applications lack, almost entirely. Android does expose Intents but these lack a persistent serialized form (and in any case serve a different purpose), nothing else comes close, except perhaps sharing a URL at which the native application’s executable can be downloaded, but this is a weak simulacrum(†).
Anyone can discover a web application, share it’s URL, and in principle invite everyone else in the world with a browser and a connection to the Internet to use it too. No native application is shared in the same way.
Graceful degradation
Though the Web is cross platform, it’s well appreciated that supporting the huge number of browsers that users can throw at a website comes at a marked price in its development; the time spent adapting perfectly good HTML and JS to accommodate bugs in browsers can be colossal. This may precede attempts to progressively refine the web application to take advantage of new browser technologies which could be available.
For extremely complex web applications which are also required to serve very large numbers of users, I don’t think it’s at all certain that the cost of developing a good web-based UI is necessarily less than that of writing a multitude of native applications that could span the same number of users. But where Web development wins incontestably is that the user experience is elastic and though it may be that only a small proportion of users have the best experience, every user can have some experience assuming their device has a browser. Native applications on the other hand lead to a situation where some people simply can’t access the application and never will unless a suitable native application is developed.
And that’s it — there’s no answer to be drawn from all this, certainly no simple answer. The choice one faces between developing a native application and a web application is deeply nuanced. Personally, I love developing both.
(*) More specifically any resource that is complex enough to embed an ASCII character sequence.
(†) Though I do have ideas about how some classes of native application could be seamless linked in this way.
I recently implemented OAuth for the Yoke service I’ve been developing. Yoke sign-in currently uses OAuth 1.0 for Twitter integration (via twitter4J) and a homespun OAuth 2.0 flow for Facebook and is used within my Android applications via a WebView. The regular Android approach would be to use the AccountManager API, but there are a number reasons I chose not to use it:
At the time, these all seemed like pretty compelling reasons to choose OAuth for my applications, and they still do. But now I think that it could be a risk for users to register via OAuth within mobile apps, especially Android apps. Why? It’s simply this:
The security of OAuth is predicated on the use of an honest browser client, within mobile apps this cannot be assumed.
Nothing prevents a malicious application developer from bundling, into their app, a browser component that mimics the behaviour of the native one. On Android, this is made easier because the source is open, and there isn’t any pre-vetting of apps published to the Market.
A compromised browser could readily skim off your password and log it to a server and it would be difficult, if not impossible, for you to discover that your credentials had been stolen in this way. The problem is that we are all conditioned to look past the browser, and to recognize the sign-in pages of the online services we use (Google, Facebook, Twitter etc.) and to give them our trust, unconsciously.
I will continue to provide authentication within Yoke apps using OAuth since I’ve invested time implementing it, but in the future I may be a little more cautious.
A few months ago I discovered that the Google Analytics Java library for Android can interfere with the performance of your application because it accesses a database on the main thread.
Not wanting to abandon analytics, I’ve been using this class ever since. It’s a useful wrapper around the GoogleAnalyticsTracker class which I thought I’d share because it’s simply very useful. It:
Using it is very simple, here’s an example:
And here’s the class which you’re welcome to use in any of your own projects.
In my first technical post about Yoke and I thought I’d describe how I tackled building a scalable system for linking (yoking) users and items using App Engine.
Links between Users and Items are called Yokes, they are first class objects and are stored in the datastore as entities. This allows them to contain indexable information about how items relate to users.
In a SQL database this would be modelled naturally with two one-to-many relationships from Users to Yokes, and from Items to Yokes. The equivalent data model in App Engine involves adding two key properties to Yoke entities that associate each of them with a User and an Item. In App Engine there is the added consideration of entity groups. Entity groups are the unit of transactionality in App Engine and have a heavily constrained write rate as a result.
Since any given Item or User may be yoked extremely frequently, this has the consequence that Yoke entities must be root entities and cannot therefore partake in a transaction that also modifies the properties of an Item or User. But why would a datastore transaction which, say, creates a Yoke need to modify properties on the User or the Item or some other entity?
App Engine lacks the familiar SQL aggregation operations such as COUNT, SUM and AVG, so answering queries such as “how many times has this item been yoked?” requires that the aggregation be performed by the application by maintaining a counter in the datastore, and this necessarily involves at least one other entity. To also answer queries like “how many times has this user been yoked?” naturally requires two counters.
On what entities should we store our counters? Naively, we might think that the User and Item entities would be good choices, but this is not the case. To see why, it’s necessary to look at the distribution of yokes. Specifically, some items/users will be extremely popular and will be yoked very frequently. This is a problem if we store counters as part of the User/Item entities because those entities will become heavily contended.
To overcome this we need a new kind of entity, which the Yoke platform calls a Standing. Standings aggregate information about Yokes, for both Users and Items. So when the Yoke platform records a new Yoke it actually needs to create or update three different entities: the Yoke, the User’s Standing and the Item’s Standing. The process by which this occurs is necessarily quite complicated on App Engine because:
To help meet these requirements, Yoke transactionally enqueues a task when it creates (or deletes) a Yoke entity. The task wraps-up the work of updating the Standings. This means that the Yoke is created (or deleted) immediately, but the work of reconciling the Standings is deferred (usually only briefly), can take as long as necessary (up to 30mins), and will be automatically retried in the event of an error. These are standard characteristics of tasks in App Engine, unfortunately tasks also introduce a couple more complications:
To handle this, each task that adjusts a Standing is allocated an ID. At each step in the process of adjusting the Standing (including the first step which involves adding or deleting the Yoke) a temporary Marker is created containing the same ID. Before the task starts its work, it attempts to get each possible Marker from the datastore and, based on which Markers are present, it resumes its work at a particular step. After the task has finished its work, all the Marker entities are deleted.
Since it’s a possibility (though only a remote one) that a task may fail to clean up all of its Markers, a cron job sweeps through periodically and deletes any Markers it identifies as being stale. The only loose-end this leaves is the possibility that two tasks will ‘race’ to create the same Standing entity (creation of Standings is deferred since many Items and Users will never be yoked). This is handled by a separate mechanism that I may describe in a later post.
The absence of unrestricted transactions in the App Engine datastore does significantly increase the work needed to perform operations that one would consider to be basic in SQL databases or their equivalents, but any given approach (there is more than possible approach here) can be generalized and reused. So perhaps its best to think of App Engine as providing a lower-level persistence service that can then be tailored to meet specific needs.
Though this is well known by Java veterans, a brief recap can’t hurt: There several ways that Java’s autoboxing can bite you on the ass when you combine it with the ternary operator.
Here’s the classic ternary/autoboxing interaction that programmers don’t expect:
Integer n = null;
// we only want to increment n if it's not null
n = n == null ? n : n + 1;
Bang, this will raise a NullPointerException when a is null. Why? Because adding 1 to n will result in a primitive int requiring the JVM to coerce the other side of the ternary operator into a primitive (which fails because it’s null).
If the above convinces you that autoboxing doesn’t play nicely with the ternary operator, you can try avoiding it:
// okay so autoboxing is tricky, don't use it then
Object d = true ? new Integer(1) : new Double(1.0);
Here d gets assigned an instance of Double equal to 1.0. Maybe not what you expected; autoboxing even interferes here.
Here’s a final one, which isn’t actually a side-effect of autoboxing, though it’s more likely to occur in scenarios where autoboxing is used. It begins with a simple error:
Integer n = 1;
//... n was actually a counter retrieved from a map ...
n = n == null ? 1 : n++;
The resulting value of n? 1 of course! At first glance, this looks like regular code for simply initializing/maintaining an Integer counter. Then you realize that the post-increment operator is being used instead of a basic addition. The nasty thing here is that the code will compile and run but will never count beyond 1. This is because the value returned from the ++ operator is the value of the variable before it’s applied, and the assignment it performs is overwritten by the result of the ternary operator.
It’s remarkable that after this many years of computer science we still have mainstream computer languages that obfuscate counting.
I’m pretty sure that I’ve seen a few questions asked about stuttering framerates within Android games that were be traced to the garbage collection of String objects generated by Integer.toString().
Avoiding GC cycles is important in animations and games. The Android Canvas class helpfully gives you a drawText method that takes a char[] rather than a String for just this reason.
All you need is some code that will write your integer into a character array. Here’s a small class just for that:
As promised in an earlier post about intents I’m sharing a tip on how to return dynamically generated resources from a ContentProvider — though what’s written here applies equally well to an Activity returning a result.
The basic scenario is this: You have implemented a ContentProvider and you want to use it to return something that is too big to fit into a cursor, say an image. It’s clear from the relevant Android platform javadocs, though perhaps not from the general ContentProvider documentation, that this is done by implementing openFile and returning a ParcelFileDescriptor.
But now suppose that the image resource is being rendered on the fly. How do you implement openFile?
As per the contact of openFile you need to return a ParceFileDescriptor which means that the resource must be returned via an open file (or possibly a socket, which we’ll ignore). This means that the image must first be written to a file, which is then opened as a ParceFileDescriptor. The interesting question here is when do you delete the file?
You can’t wait until the calling process has finished reading the file because (a) you won’t get notified when that occurs and (b) it may never exhaust the stream anyway. You can’t rely on the calling process to delete the file for you either, also, the whole idea behind wrapping the resource in this way is to guard it from direct access by other processes. You could delete the file after a fixed time period — assuming that the caller will have finished reading the file by that time. But this is unnecessarily complex, because the answer turns out to be very simple:
You delete the file immediately after you’ve opened the ParcelFileDescriptor but before you return it from the openFile method. Here are the relevant lines of the PlantProvider class from my Daisy Garden application; the try/finally block is there to ensure the file gets deleted no matter what — the last thing we want to do is leak precious storage space.
The reason this works is down to the way that Linux filesystems operate: directories maintain links to files, when a process opens a file a new link is created, closing a file or removing it from a directory removes a link. When there are no links to a file, the file is deleted.
So by opening the file in our application, we create a link to it. Then ‘deleting’ the file actually unlinks it, but the file won’t really be deleted until the file descriptor is discarded (ie. the last link is removed). This will happen automatically at some point after the calling application has finished accessing the file and any associated ParcelFileDescriptor objects have become garbage.
So it’s a very simple solution, but only if you know something about how the filesystem can be expected to work on Android devices.
One of my favourite things about the Android platform is how easily application can share data using Intents and ContentProviders. They provide a universal basis for Android applications to interoperate and I try to expose them whenever it’s practical, even for something as singular as my Daisy Garden application.
I’ve just published a new version of Daisy Garden and with it, Intents and ContentProviders that enable the users of other Android applications to choose flowers they have grown in Daisy Garden — assuming that the developers choose to support it. And the great thing is that supporting third-party intents is very easy to do (and even if no other developers use them, they will be heavily used to integrate my own applications).
The Java code embedded below is a self-contained Activity that responds to a tap on the screen by inviting the user to pick a flower and then displaying the flower as a bitmap. There are two basic parts: forming an Intent for the user to pick a flower and forming a Uri to the flower’s bitmap. Both of these aspects can be customized; messaging to the user, and the bitmap size for example.
It’s very simple — more than half the code is boilerplate and comments. And this is what’s really good: in a well structured Android application this functionality is simple to expose too — I’d go as far as saying it’s almost automatic, so there is very little reason not to share (non-sensitive) data between applications.
Having said all this, there is an interesting technical detail here that meant that I had to give some thought to exposing the bitmaps. Daisy Garden is generating the flower bitmaps “on-the-fly” when they are requested from the ContentProvider and it’s not obvious how to share these unless you know a little about how Linux filesystems work. I’ll try to find some time to outline the technique in a subsequent post.
Java code follows:
After my previous post comparing Java’s regular random number generator to its cryptographic SHA1PNG relation, Benjamin Manes suggested using double-hashing has a way of generating multiple and provided an implementation for me to test.
The initial results appear to be excellent. It yielded by far the shortest test time (approx. 4s compared to 18s for the standard Java random number generator) and in this limited test exhibited a distribution that is almost indistinguishable from that of SHA1PNG.
