- Apps were getting rated on deletes from devices. No way to log a rating without deleting.
- App rating distribution, meaning how many times your app had been rated a particular star level, weren't visible
- Reviews and ratings had no visible date associated with them
- Reviews and ratings had no visible app version associated with them
- The relationship between reviews and ratings was unclear
Apple slowly trickled out an overhaul of the review system for the App Store. It has been very strange how this has been done. Bits of functionality here, more there. It seems by March 13th, 2009 (Friday the 13th, must be an omen) Apple started breaking out the distribution of rating scores by stars. In the screenshot on the left, current version means iTimeZone 1.2.1
Here's why based on my analysis of the data for iTimeZone:
- Average Customer Ratings do not include Customer Reviews. The only way the math works, that Average rating for all versions is 3 stars, is if you exclude customer reviews. A 3 star rating has to at least fall between 3.0 and 3.25. Since we don't know Apple's rounding policy, and we know Apple does half-stars, it is entirely possible anything between 3.0 and 3.49 is a 3 star rating. Luckily for this exercise, the average of iTimeZone's customer reviews is 4.26. That happens to be just enough to make the combing ratings and review score 3.5, yet iTunes only shows a 3 start average rating for all versions. Thus, the aggregate rating can't include customer reviews, only ratings. So Customer Ratings are averaged for the current version, and across all versions, but customer reviews are not, they are only listed. If a customer took the time to write a review, surely that star score should be included in the ratings? That is just not intuitive at all. I would have to recommend at this point that if you write a review, you also leave a rating.
- Tracking Reviews/Ratings by Version is too literal. The accepted convention for version numbers, and the one I am using follows Apple's generic versioning scheme. Major.Minor.Revision, X.Y.Z, 10.5.6, 1.2.1. Major means a platform release, minor means a feature release, revision means changes/fixes to existing features. Mac and iPhone apps are thought of feature stable when X.Y remains the same. This is not controversial, except the App Store is now boxing reviews by X.Y.Z. So reviews for version 1.2 of iTimeZone mean the current version doesn't have any reviews because the current version is 1.2.1, and there are no written reviews yet for that version. Current version literally means the one on sale, there is no intelligence behind it. This is going to result in developers being reluctant to release fixes quickly, and instead only take the version review reset hit when they are going to release features. The other possibility is gaming the version system so that the app gets updated, but the version number doesn't change so reviews don't get reset. I haven't investigated this thoroughly yet, but I am sure it has already crossed more than my mind. Sure, developers could try and game the review by version system the way I propose, but if Apple had clear rules posted on versioning and then bounced apps during review for adding features without upping the X.Y version, developers would get the hint.
I can only hope Apple further tweaks the system and fixes the problems I have highlighted. Given that it took 8 months from App Store launch to get the current changes pushed through and the amount of work they surely have ahead of them with iPhone OS 3.0, I am not very hopeful.
Since I wrote the bulk of this, iTimeZone has received 2 more customer ratings both 5 stars. So the current version has a 3.5, yeah for me, but it doesn't change the unintuitive disconnection between Ratings and Reviews.