Oz Weather usually provides you with all the latest Australian weather information, but today is a little different. Oz Weather has itself been experiencing some app store weather – and its been a bit rough. 😦
Background
Oz Weather has so far been a very successful iPhone app in the Australian app store. You can read more about it here.
But this particular story started with an app update to Oz Weather.
Because I wanted to add many new features, I had been thinking about creating a new app altogether, and perhaps calling it Oz Weather Pro and of course charge a bit more for the extra features and work that had gone into it. But the problem with doing this is that Apple offers no upgrade route between apps. Consequently existing users of Oz Weather would have had to pay just the same as any new users, and that to receive a partial replication of the features they already owned and had paid for before. I thought that was hardly fair to all those customers who had so enthusiastically supported the app since its launch last year, and other developers who had done that were panned for it.
So I carefully devised a plan which would ensure that no existing user would be disadvantaged at all. And what’s more I thought I would attempt a little tried but oh-so-frequently requested feature – a fully working, but time-limited trial of the new features before you decide whether or not you want to pay for them via in-app purchase.
I was encouraged by the fact that some others had indeed already attempted this, and actually had their apps approved by Apple (eg. the Palettes app). All I had to do was implement it all.
This did turn out to be a bit trickier than I first thought, but I did manage to solve it all, and I may write a little more about that in a future post.
Planning the Release
Before releasing the app, of course I carefully reviewed what I had done in light of Apple’s approval process. I was keen to get the app approved before Christmas, so didn’t want to risk a rejection after two weeks, which then leads to at least another two week delay after a re-submission. Once thing that particularly concerned me was that one of the new features used the camera overlay API call, which was introduced in OS3.1. My concern was realistic, as it was based on the fact that the Sun Seeker app was initially rejected precisely due to this API, as a result of the fact that I had provided backward compatibility for OS3.0 users. Even though doing this is within Apple’s guidelines, and is even recommended by them, it was rejected anyway, supposedly for “forbidden” use of camera overlay. I also took a quick poll about this issue on twitter, and the general consensus of other developers was that it would be risky to do this, and even though perfectly legitimate in practice, was quite possibly still going to be rejected.
From previous usage statistics, I estimated that about 25% of existing Oz Weather users would be excluded from an update that worked only with OS3.1+. However, I reasoned that there was nothing to stop me submitting a further update which re-introduced backward compatibility with OS3.0 after the initial OS3.1 update was approved. In that case, if the backward compatibility issue still resulted in rejection at that point, the initial update would of course still remain available to 75% of OS3.1 users.
So I took the executive decision, for the sake of quickest possible approval, that the update would have a deployment target of OS3.1+.
The Release
Dec 10th, 10am – I was notified of the approval. The review and approval process had taken 12 days – par for the course at the moment. I was relieved that it hadn’t been rejected, and delighted it had happened well in advance of Christmas. For a while, I felt quite contented.
Dec 10th, 11:30am – Received an email from a user saying that although iTunes had notified them of the upgrade to v2.0, when they downloaded it it still appeared to be the old version (1.7.1), even after several retries. How odd, I thought! I was puzzled, but reassured the user that it was probably a server caching issue, and it would probably resolve itself later.
Dec 10th, 1:30pm – By now I had received 10 similar emails. I created a standard reply, suggesting that they contact Apple to report the problem. I was quite concerned because it would be unusual for more than 1 in 100 users to contact me directly, so there could be thousands of users with this problem. And of course they would all be assuming it is an Oz Weather fault.
Dec 10th, 7:20pm – By now had received about 40 similar emails, but suddenly my server started showing that many users were starting their trial – at the rate of about 10 per minute, indicating that of course the correct new app update version was now being served to them. Phew, I thought! Glad to see the back of that problem!
Dec 11th – One of the users who had contacted me yesterday was reporting that he now had the new version, but that it was crashing on startup. I had several email exchanges with him to try to get to the root of the problem, and by incredible good fortune, he was expert enough to be able to follow Apple’s instructions on where to find a crash log, and sent it to me. Within seconds of opening the log file I saw the problem. His device was running OS3.0, and a 3.1 deployment targeted app can simply do nothing but crash on an OS3.0 device. But how on earth did he manage to get an OS3.1 app onto his OS3.0 device?! iTunes controls that! Uh oh! My heart really started sinking at that point.
No doubt, I realised, this must be linked to the fact that iTunes was initially serving all these users with Oz Weather v1.7.1 – which was an OS2.2 targeted version. What if iTunes had based its list of customers to serve the update to was based on v1.7.1, and NOT on v2.0? That would explain this user’s problem. And it would also suggest that the problem might be much more widespread. Uh oh! My heart sank even further. And yes, iTunes was already showing a series of very recently posted one star reviews, complaining that the app crashed on startup.
My life flashed before my eyes. One star reviews?! Every developer gets them of course, at least once in a while. You can never please all users. But suddenly I was getting a constant stream of one star reviews from irate users – and it was all totally beyond my control, due to some weird Apple server glitch. And this kind of avalanche of negativity could probably kill an app quicker than the blink of an eye! And this app is my lifeblood. Of the various apps I have created and worked on, this is the one that truly supports me, and makes it possible for me to continue my “career” as an independent developer.
I fired off an email to Apple dev support, requesting assistance and pleading the urgency of the situation. Later I sent another email to a local Apple marketing rep, pleading urgency and lack of response from dev support. To his credit he responded promptly that he had forwarded the issue to the USA. However, even now, three days later, I have still not had any further response from them.
Dec 12th – Now, to make matters worse, a number of users (who obviously had OS3.1 on their devices, and therefore had no problem running the update) were leaving very negative reviews about the in-app purchase.
I was aghast. I had really gone out of my way to find the fairest possible system, which disadvantaged not one user, which offered a free 7-day trial of all additional features, and which even offered enhancements over v1.7.1 to those users who did not want pay for the new features. And yet I was being blasted for it. Many users latched onto the term “double dipping”. But how can it be double dipping when the user doesn’t have to pay for anything they don’t want to, and even if they have just got more functionality than they paid for in their original version anyway? The only explanation I can find for this is that these people had either not read, mis-read or just misunderstood my explanatory words about the trial and in-app purchase system. Perhaps I just hadn’t made it clear enough?
After a while I began to notice a bit of a pattern. There seemed to be a link between the app crash complaints and the in-app purchase complaints. It was as if they were fueling one another – scathing one star reviews cross pollenising one another in a kind of feeding frenzy – along the lines of “wow this guy wants to double charge us for an app that crashes on startup!”.
This really looked terminal. How can an app recover from this? What was the chance of Apple actually removing any of the reviews that erroneously blamed Oz Weather for the download and app crash schemozzle? And even if they were to do that (which would be completely unprecedented as far as I know, and hence extremely unlikely), would they also remove reviews referring to the in-app purchase which had been influenced by the crash reviews? And for that matter, why is it that users who haven’t and obviously won’t use the in-app-purchase be allowed to leave a bad review for the app supposedly based on their disapproval of the in-app purchase being offered, at all?
So what could I do now? The first step was to quickly submit a backward compatible 3.0 version, and submit a request for expedited review. Fortunately this was a very quick step, because I had already expected to be submitting such an update very soon, as explained earlier. And if approved, and providing Apple’s servers didn’t mess things up again, that would at least allow 3.0 based users (currently with what appears to them to be a dud app) to actually get it working. But it wouldn’t help anyone using OS2.2. And perversely, it would also no doubt appear to users as if I had submitted a new app update because the previous version was itself faulty – rather than the reality that the update was being submitted principally to help work around Apple’s server mess-up. And hence it would implicitly appear as an admission of fault/guilt on my part, thus appearing to vindicate all the reviews that misguidedly chastised me for having submitted a “faulty” update in the first place.
Another step was to recommend that users upgrade to OS3.1, as presumably this would also remedy the app crash. But of course only a small proportion of victims of the crash would actually contact me about it, so this wouldn’t reach many. So I decided to edit the app’s description in the hope that at least new users would be able to see my explanation of the reason for the slew of terrible reviews. And then in yet another quirky twist of fate, very shortly after I had done this, Apple suddenly started deploying a new app store layout, and all of a sudden most of the app description was hidden by default.
Sometimes you have to wonder, when so many coincidences stack up against you, whether you are simply fighting a losing battle. So that was when I started thinking about pulling Oz Weather from the app store altogether, at least until some real solutions emerge. Wow, that seems so drastic. A bit like cutting off your own leg to save your body after you get crumpled by an avalanche. But what exactly would the consequences of this be? Would any existing users be disadvantaged eg. would the in-app purchase become unavailable, for those existing users who did actually want to have the new features? If so, that would be making things even worse.
Despite all this, and helping me to keep things in perspective, there were still people leaving five star reviews, who obviously did really appreciate the new version and features, had not experienced Apple’s server glitch (or at least had understood that it was a temporary Apple issue), and had not misunderstood the in-app purchase system. And in fact some reviewers were obviously going out of their way to counter the negative reviews. So my faith in human kindness has not been lost. (If any of you are reading this – thank-you!)
What Next?
At time of writing (Sunday 13th Dec 2009), I have heard from Apple via the automated iTunes Connect system that they will provide an expedited review of an app update (v2.0.1) I submitted two days ago, which provides backward compatibility for OS3.0 users. This is something that they occasionally do when a developer requires an urgent update – usually of course due to bugs in their own app rather than due to an Apple problem. However, this will not help any pre OS3.0 users who were served the app update by Apple. According to my own analytics, this means that up to about 1000 users could still have an app that will not run, even after (and if) Apple approves this new update.
How on earth can that problem be fixed? Does Apple even have the ability to restore the older version to those users? But apart from that automated reply from iTunes Connect, I have not had any response at all to my pleas for help to Apple’s advertised email addresses for developers and iTunes Connect users, and I am not even sure whether or not Apple is *still* serving the existing Oz Weather update to users running older OS versions. (I am assuming that they are not, because the flood of emails has decreased, but I cannot be certain.)
And even if and when Apple manage to remedy that problem, what about all the negative reviews that Oz Weather has acquired due to Apple’s glitch, and the consequent loss of sales and rankings – possibly permanent? Having a new app upgrade will of course relegate those reviews to the pool of reviews for non-current versions, but my observation is that irate users can and will update their bad reviews whenever new updates come out, anyway.
The iPhone device, its SDK and the app store environment have provided me with a fantastic creative outlet, and overall it has been very gratifying and rewarding. But right now its is all feeling a bit tenuous. There seems to be little I can do now but wait for Apple to do something. Or nothing. Judging by past experience I should certainly not be holding my breath.
Ajnaware's Weblog 