The founders of the lyric annotation site Rap Genius received a Christmas present from Google. After the “Rap Genius blog affiliate program” was discussed by title="matt cutts, personal site">Matt Cutts, the head of Google’s Webspam team, on HackerNews, Mahbod Moghadam, Tom Lehman, and Ilan Zechory write that they “woke up on Christmas morning to a ‘manual action’ from Google, which bumped Rap Genius to the sixth page of search results for even queries like [rap genius].”
The rationale behind Google’s manual action was “Unnatural links to your site.” In Google’s words this amounts to “a pattern of unnatural artificial, deceptive, or manipulative links pointing to your site.” In other words, a big no-no in Mountain View. In a post this morning, Rap Genius’ founders, and their sleep-deprived engineering team, describe how they got back on Google in record time.
The short answer is that Rap Genius was repentant, proactive and thorough. When you are dealing with Google you have to remember that it knows everything! So the best way to address the 4-step approach it recommends to repairing your search status is to geek out big time. And that would seem to be how the team at Rap Genius celebrated Christmas and rung in the New Year.
What Google requires if you are caught link-spamming is that you scrutinize ALL of your inbound links, identify links that violate the rules, contact the appropriate webmasters and have them remove or deactivate the links from searches and, for the sites you can’t make contact with, use Google’s disavow links tool. OK, sounds simple enough. But Rap Genius compiled a list of 177,781 URLs! (They not only used Google’s own Webmaster Tools, but also Moz Open Site Explorer to get a their master list.)
This is where Rap Genius got an opportunity to live up to its name. The “technical digression” in the post describes how they used sophisticated web scrapping and concurrency techniques to download and evaluate all of those URLs in less than 15 minutes. Between obvious links they identified immediately along with the results of their web scrape, Rap Genius was able to get 286 links either taken down on disavowed, and that was evidently enough for Google. Along the way they also identified some “spammy aggregator / scraping sites” that picked up spammy links from third party sites that Rap Genius was technically not responsible for but cleaned up anyway.
Rap Genius explains in their post how the eventual, irrelevant and manipulative Justin Bieber links grew out of a process of facilitating legitimate bloggers to include relevant links to annotated lyrics on its site. They crossed a line by trying to juice up their SEO efforts by extending them to bloggers they did not know whose content was of dubious value and relevancy. But, in fact, Google (thanks to Matt Cutts) caught them quickly and it is remarkable that out of 178K of inbound links, less than 300 were actually problematic (that’s 0.16% for the statisticians out there!)
By being so fast and thorough, Rap Genius has demonstrated that the vast bulk of their activities are legit and Google-worthy. They have also used the publicity (albeit negative) to draw attention to their iOS app due to be released next week, which should give them a whole new avenue for traffic. To stay on Google’s good side, I would imagine the Android version is in the works as well.
– – – – – – – – – – – – – – – – – – – –