If you do a Google search for "Travel Blog" right now, you'll find Blogabond sitting at result number 361. That's not very good, especially since we're at #3 on Yahoo! and in the top 10 on most other search engines. What's the story?
I banged my head against the wall for months trying to figure out what the deal was here. We had climbed up to #60 for a while, but then got bumped down 300 points in a day. It looked to be a penalty of some sort that Google has been known to hand out for naughty web sites. But what was Blogabond doing that could be considered shady SEO?
For the longest time, the only thing I could come up with was that it didn't like our "Find Places" section that has links to every city in the world. So if your company was planning to send you to Mbayabougou, Mali, you could at least find it on the map. I figured that this page belonged in Google's index, but maybe they didn't really want it. Maybe they were actually penalizing me for putting a link to this page. Could be, since the crawler wouldn't see the Google Map up top. It would only see a blank page with the same words from the URL.
Anyway, today I made the painful decision to put rel="nofollow" links to all of these pages. I figure I'll wait for Blogabond users to put real content into these places before I expose them to the Spiders. Fair enough.
But then I came across this post on the Usenet. This guy is complaining about a minus-30 penalty for his site that has a bunch of Hotel Affiliate links on it. Hmm. I've read elsewhere that Google HATES hotel affiliate links. Wait a minute... I still have those crappy HostelBooker links all over the place in the Map pages. I guess they could be considered Hotel Affiliates. And hey, I bet those links are showing up on pages without any real content at all! Aha! I think I've finally found it!
Ok, change of plan. Hostelbookers, you're getting the boot. I never got any bookings from you anyway and now it looks like you're dragging down my Google rankings. While I'm at it, Amazon, you're gone too. You guys never made any money either and you take up a lot of space. We're back to a slim, nearly ad-free Blogabond. Cool.
I think I'll still leave the rel="nofollow"'s on those small town links for the time being. Give the spiders a chance to crawl all the location pages again and see that the affiliate links are gone. I'll follow this up in a month to see whether it worked.
Friday, March 16, 2007
Friday, March 02, 2007
Saturday, January 13, 2007
Hotel Katarino Bansko Bulgaria
Had a bad stay at Hotel Katarino Bansko Bulgaria, so I've built them a little website. If you ever book a trip to Bansko, be sure to avoid them!
Thursday, November 23, 2006
Defeating the Astroturfing Robot Spiders
Woke up to 30 variations on this today:
"As though I wanted will meet personally the author of clauses articles on your site, and personally to it him will get acquainted. But unfortunately I live in other country and I have no an opportunity to go on the world. Success to you the dear expensive friend. "
It looks like the SpamBots have finally found the comment links on Blog entries.
This one was especially fun, as it started out by simply commenting on a dozen different blogs, praising the author's "clauses" and offering ESL advice on how to better the world.
Then it started including helpful links to Porn.
What to do about it? I didn't want to force my users to create accounts before they could post messages. Most of the comments come from friends and family of our bloggers, and they have no reason to get real accounts for themselves. I also didn't want to put in an annoying CAPTCHA test, since again, our commenters are often just somebody's mom who might just get confused and give up.
So here it is, spammers of the world, the secret to Blogabond's anti-comment spamming engine. When a user pulls up the comment screen, we'll pick a random number and stick it into the ASPX ViewState on the outgoing page. We'll also dump out some javascript that counts to 5 Mississippi before sticking that same number into a hidden variable. Before we'll commit a new comment to the database, we check to make sure that the number in the hidden field matches the number in the ViewState. Done, and Done.
And yeah, it should be easy enough to get around. All a spammer needs to do is pull up the page in a real browser, wait 5 seconds and hit the submit button. He can even write code to do this for him, or go so far as to dissect the base64encoded viewstate in the html and construct his own custom HttpRequest. My bet though, is that he'll simply move on to somebody else's blog and do his spamming there. Maybe when we get BIG, this might turn into a problem. Who knows. But for now, the quick and dirty fix seems to be working pretty well.
"As though I wanted will meet personally the author of clauses articles on your site, and personally to it him will get acquainted. But unfortunately I live in other country and I have no an opportunity to go on the world. Success to you the dear expensive friend. "
It looks like the SpamBots have finally found the comment links on Blog entries.
This one was especially fun, as it started out by simply commenting on a dozen different blogs, praising the author's "clauses" and offering ESL advice on how to better the world.
Then it started including helpful links to Porn.
What to do about it? I didn't want to force my users to create accounts before they could post messages. Most of the comments come from friends and family of our bloggers, and they have no reason to get real accounts for themselves. I also didn't want to put in an annoying CAPTCHA test, since again, our commenters are often just somebody's mom who might just get confused and give up.
So here it is, spammers of the world, the secret to Blogabond's anti-comment spamming engine. When a user pulls up the comment screen, we'll pick a random number and stick it into the ASPX ViewState on the outgoing page. We'll also dump out some javascript that counts to 5 Mississippi before sticking that same number into a hidden variable. Before we'll commit a new comment to the database, we check to make sure that the number in the hidden field matches the number in the ViewState. Done, and Done.
And yeah, it should be easy enough to get around. All a spammer needs to do is pull up the page in a real browser, wait 5 seconds and hit the submit button. He can even write code to do this for him, or go so far as to dissect the base64encoded viewstate in the html and construct his own custom HttpRequest. My bet though, is that he'll simply move on to somebody else's blog and do his spamming there. Maybe when we get BIG, this might turn into a problem. Who knows. But for now, the quick and dirty fix seems to be working pretty well.
Tuesday, October 17, 2006
Street Cred
New idea for Blogabond here, though not really related to the blogger import. I want to do an 8a.nu style scoring system for backpackers. Give points for days on the road, countries & continents visited, etc. Let users vote to give eachother extra points for tales of hardship and coolness.
Publish the rankings by year, trip, and lifetime. Declare champions, award prizes. Bring in the MySpace kids that want the world to recognize how cool they are. This idea worked wonders at 8a.nu, with climbers posting every detail about the climbs they've done and even fudging the details a bit to get extra points.
Should be easy enough to do. Most of the stored procderes are written already. Just gotta implement voting.
1 - day on the road
10 - country visited
50 - continent visited
5 - peer vote
100 point antarctica bonus?
Publish the rankings by year, trip, and lifetime. Declare champions, award prizes. Bring in the MySpace kids that want the world to recognize how cool they are. This idea worked wonders at 8a.nu, with climbers posting every detail about the climbs they've done and even fudging the details a bit to get extra points.
Should be easy enough to do. Most of the stored procderes are written already. Just gotta implement voting.
1 - day on the road
10 - country visited
50 - continent visited
5 - peer vote
100 point antarctica bonus?
Saturday, October 07, 2006
API Woes - Intermittent Outages
Been having some trouble with the API lately. It seems that there is some sort of traffic limiter in place that will shut you out for making repeated API requests. For me, it translates into a major headache for development.
Here's what I see:
Assume an Http Request to the API, correctly formatted to request a list of blogs for a given username and password. Now, issue that request about 3 times a minute, as you would during a routine code/build/test/debug session. Notice that the request works as advertised for about 5 minutes of this sort of activity. Notice that after those 5 minutes, the request starts returning Blogger error screens instead of XML data. Notice that after about 10 more minutes of inactivity, the request starts working again.
To me, that looks like something put in place by the Blogger team to discourage Spammers from abusing the API. Unfortunately, it also discourages application developers from using the API, since it is basically unusable.
I've posted my experiences over at the Blogger API discussion group. Let's hope that somebody knows a workaround!
Here's what I see:
Assume an Http Request to the API, correctly formatted to request a list of blogs for a given username and password. Now, issue that request about 3 times a minute, as you would during a routine code/build/test/debug session. Notice that the request works as advertised for about 5 minutes of this sort of activity. Notice that after those 5 minutes, the request starts returning Blogger error screens instead of XML data. Notice that after about 10 more minutes of inactivity, the request starts working again.
To me, that looks like something put in place by the Blogger team to discourage Spammers from abusing the API. Unfortunately, it also discourages application developers from using the API, since it is basically unusable.
I've posted my experiences over at the Blogger API discussion group. Let's hope that somebody knows a workaround!
Tuesday, August 08, 2006
Snag Two
This one might be bigger. Locations. City Names. Places. How to keep them alive when we send stuff to Blogger???
With luck, we'll be able to stuff a little extra information into the Atom XML when we reverse-publish into Blogger. And with even more luck, it will be available to us when we ask Blogger to publish back to us.
For the purposes of our test, let's see if we can convince this entry that it was written in Arusha, Tanzania.
Looks like we can solve it with a simple XML tag:
<blogabond:location locationID="2172"/>
With luck, we'll be able to stuff a little extra information into the Atom XML when we reverse-publish into Blogger. And with even more luck, it will be available to us when we ask Blogger to publish back to us.
For the purposes of our test, let's see if we can convince this entry that it was written in Arusha, Tanzania.
Looks like we can solve it with a simple XML tag:
<blogabond:location locationID="2172"/>
The first big snag
Images. D'oh! Completely forgot about them.
At Blogabond, we store images locally and use little placeholders in the blog entry text to remind us where to show them on the page. I'm not sure quite what Blogger does, but we'll need to make sure we can switch back and forth without breaking anything or mangling anybody's blog. So here's a test:
Cool. Here's an image of Jackie eating tasty curry, and Blogger seems to have simply dumped it in as an HTML Anchor tag. That will be easy to play along with.
And here is an image being pulled straight out of the Blogabond Archives (of Yours Truly running with the bulls). Also done as simple HTML. Looks good so far!
One more photo, centered this time:
And finally, with no formatting at all:
Now to try sending it through the round trip...
At Blogabond, we store images locally and use little placeholders in the blog entry text to remind us where to show them on the page. I'm not sure quite what Blogger does, but we'll need to make sure we can switch back and forth without breaking anything or mangling anybody's blog. So here's a test:
Cool. Here's an image of Jackie eating tasty curry, and Blogger seems to have simply dumped it in as an HTML Anchor tag. That will be easy to play along with. And here is an image being pulled straight out of the Blogabond Archives (of Yours Truly running with the bulls). Also done as simple HTML. Looks good so far!One more photo, centered this time:
And finally, with no formatting at all:
Now to try sending it through the round trip...
Thursday, July 13, 2006
The idea
Blogabond is cool and all that. Blogger users that want to write travel blogs would be smart to move all their stuff across today.
But how do you go about moving, say, 5 years worth of stories spanning three separate blogs to this other site? More important, what if this Blogabond place really sucks and you want to move back? And what If you don't realize that this Blogabond place sucks until you've written a whole bunch of new stuff there? You'd be stuck, right???
Well, hopefully, no.
You can get your stuff back. Or at least you will be able to soon. I'm building a little importer/exporter that you'll be able to use to copy your stuff back and forth between Blogger and Blogabond without any hassles.
Follow the progress of this new feature here (and hopefully on Blogabond too!)
But how do you go about moving, say, 5 years worth of stories spanning three separate blogs to this other site? More important, what if this Blogabond place really sucks and you want to move back? And what If you don't realize that this Blogabond place sucks until you've written a whole bunch of new stuff there? You'd be stuck, right???
Well, hopefully, no.
You can get your stuff back. Or at least you will be able to soon. I'm building a little importer/exporter that you'll be able to use to copy your stuff back and forth between Blogger and Blogabond without any hassles.
Follow the progress of this new feature here (and hopefully on Blogabond too!)
Subscribe to:
Posts (Atom)