For those who came in late…
About seven months ago I came up with this idea of running a little experimentwith Amazon’s review policy. A new wave of complaints had cropped up about reviews being deleted or blocked, and—as they tended to-there wasn’t a lot to them past “Amazon took down my review of XxX!” Supposedly, the new rumor at the time was that everyone’s favorite online megastore was surfing social media sites, looking for potential connections between reviewers and reviewees—and using them as a reason to delete reviews. It struck me that I’d been hearing about review policies for years, but never seen any hard data on them. It always came back to he said this, she said that, lots of people had it happen to them. There were never any hard facts.
So… I decided to find some.
I reviewed thirty books I’d really loved. One every day for the month of August. I listed out all the social media connections between me and each author. I even did a handful of control reviews—ones that should get pulled regardless of social media connections. And I listed all of it out for everyone to see. And tweeted about it. And talked with folks on Twitter about it.
A month after my little experiment ended, nothing had happened. No warnings, no deletions, no reprisals… absolutely nothing. Even on the control reviews, which really should’ve been removed under every possible version of the review guidelines. I left it at that and decided to check in six months later.
Which is… right about now.
How many control reviews finally got spotted by Amazon’s algorithms? How many warnings were issued? Did my account get frozen?
Pretty much across the board… nothing’s changed.
All thirty reviews are still up, including all the blatant control reviews. Heck, two of the control reviews even have “people found this review helpful” checks. I never heard a peep from Amazon. Even with the tweets.
Seven months since the first review, Scott Sigler’s Alight (great series, check it out). Scott and I have known each other for almost four years, if memory serves. We follow each other on Twitter, we’re both with Random House subsidiaries, we’ve done panels together, he even interviewed me last year at WonderCon in front of an audience of about three or four hundred people. There is absolutely, no question a connection between us. That review is still up.
I feel pretty comfortable saying the social-media scanning algorithm is either a myth or reaaaaaaally poorly written. If it can’ t find a connection between me and Scott, it’s pretty inept. Same holds for me and the next two authors on the list—Chuck Wendig and Eloise Knapp. There’s social media connections and shared blurbs galore. Heck, with both of them I think there are pictures floating around. Incriminating pictures, for these purposes.
And yet… the reviews are still there.
So, yeah, the social media bot probably isn’t real. I wouldn’t bet anybody’s life on it, but the evidence sure seems to point that way.
I think there’s another possible conclusion we can draw here, too. I might be stretching here, so bear with me. Feel free to point out flawed logic.
The control reviews have nothing to do with the social media bot. As I mentioned above, just as they are they violate the basic rules for reviews. And all six of them are still there. Yeah, six examples isn’t a great number for a data pool, but considering the 100% survival rate… I think getting reviews pulled doesn’t have anything to do with the reviews themselves. I think it has to do with me. Or at least, my account. Last time, one of the spitball-hypotheses I tossed out was that Amazon only applied its all-seeing eye to accounts based on suspicious activity or complaints about said account. I’m more inclined to lean that way after six months of no activity.
So if my review of Yakko’s Yappy Dog Omnibus gets quickly pulled, I think it’s more likely because of something else I did in the past than anything about this particular review.
But, again, other ideas are always welcome.
If I happen to notice anything happen with these reviews, I may revisit this again. Barring that, though, I’m probably done with it. Feel free to share the data with anyone next time you hear about reviews being pulled.
Or, in the spirit of science, repeat the experiment and share your results.