Solar OP Technology Blog

I just checked my blogs’ stats ( I use the Wordpress Firestat plugin ), and found a couple of very interesting ‘visits’ - visitors apparently coming from searches of  ’Twtitter’ and ‘Technology’ on search.live.com, which is Bing, Microsoft’s new search engine. That was somewhat suspicious - my blog is only about 2 weeks old. The chances of my blog appearing in top results for these two search terms is close to zero.

Here’s the screenshot of the two visits, who’s source are the keywords : ‘Twitter’ and ‘Technology’, from Bing search engine:

I tried to find out who are the visitors who were referred to my blog from Bing. I was surprised to find that they came from within Microsoft Corp network!! ( I used MaxMind IP geocoding services to find details of IP address 65.55.165.102, which is one of the visitors’ IP).

See screenshot below:

As far as I’m concerned, there are two possibilities here:

1.A real person working for Microsoft searched Bing for ‘Twitter’ and ‘Technology’, and my blog appeared high enough in the search results, and that visitor click on my link from Bing.

2. A robot identified my site as a blog (perhaps as a technology blog), and generated a spoof visit, knowing I will look through my stats (what bloggers do several times a day), and then follow the link that apparently brought a visitor, thus generating a Bing hit. Spoofing this kind of visit is pretty simple (you can pragmatically generate an HTTP request and set HTTP_Referrer to whatever you want).  As a matter of fact, I get several such spoofed visit to all of my blogs, but I never got one from a major website.

Keep in mind that there are millions of bloggers out there checking their stats to see who visited their website and where did they come from. If you can get only half of them to click on a spoof visit to check the visits’ referrer, you can generate a considerable amount of traffic.

So which is it, a real visit or a spoof? You decide for yourselves, but just try to search on Bing for Twitter or technology and see if you get a result from this blog (Solar OverPower). It might be that Microsoft is loosing it.

About 7 years ago I decided to build a search engine for video clips (yes, including adult clips). Youtube was still an egg, and low-quality video clips started popping like mushrooms after rain. I spent about 3 weeks on the project, until I realized that’s a lot harder than I though. The amount of data you need t crawl and index is huge, you never know where an innocent link will lead you.

Fast forward to 2009. I decided I’d build a search engine, inspired by the launch of Bing. I though I’d take it straight at good old B. gates (is he still with the company??). I  didn’t have much time to spend on the project, and I’m broke. So I decided I’ll build a search engine in 1 hour, spending no more than 10$.

The 1 hour, 10$ Search Engine: PinGoog

Here’ how:

1. Google Custom Search - why crawl, index and search when you can outsource all these activities? So I outsourced it to Google. They offer Google Custom Search - a Google search box anyone can embed inside any webpage, directing the search query to Google own search engine. Results are displayed on a Google’s page, or on your own page. Revenue generated from click on ads in the results page is shared.

2. Google App Engine - you need a webpage (a simple html page would do the trick) to accommodate the Google search box. There are many places yo can host the page (free or paid hosting etc.). I decided I’d use Google App Engine. First, It’s free. Second, I have experience using it (solarplanez.com, for example). And third, and most important , it’s infinitely scalable. So if I get a sudden spike of traffic when a top tech blogger links to PinGoog Google App Engine handles it easiliy - I don’t have to worry about the service crashing due to lack of resources or a poor hosting plan.

3. Register domain name - That’s where the 10US$ went (actually,the project went over-budget: 10.19$ for the domain name). I mistakenly used GoDaddy to register the domain name. What I should have done was use Google Apps own domain name registration services - it would make things more simple (read below why).

4. Register with Google Apps - since my application runs on Goolge App Engine, by default, its URL is something like: http://pingoog.appspot.com. I wanted to change that to a my own domain name,  pingoog.com. To do that, I had to sign up for Google Apps. Only through Google Apps was I able to change my application’s URL to http://www.pingoog.com. When you register with Google Apps, you can buy the domain name through them- it’s a good idea because you don’t have to configure DNS on GoDaddy (or any other domain register service you used).

Steps 1-4 took me about an hour of work. It cost me 10.19US$ for the domain name registration - everything else was free! Since then, I continued working on PinGoog, trying to spice things up.

5. Miscellaneous APIs - Once the search engine was up and running, I decide to add some real time data, collected from various web sources, to make the search experience a bit more fun, interesting and unexpected. I connected to API services from Digg, Yelp and Google News. From Digg - I pull the top rated stories in different topics and from Yelp, I provide localized review services, currently for the US only. So if you visit PinGoog from New York, it will show you reviews of bars, restaurants, movie theaters and other interesting information, all from New York, all randomly.

No two visits to PinGoog are the same.

And yes, if you keep visiting PinGoog to do your Google searches, you’ll get to play the games.

Bing is Microsoft’s new search engine. We all know that, part due to the fact that Microsoft will spend about 100MUS$ in advertisement. But someone didn’t quite agree with the definition of Bing as a search engine - arch-rival Google, the undisputed king of search. It’s gonna be a war, and I found earlier today that Bing drives higher quality traffic to my blog.

Google Analytics is a service Google provides to webmasters and bloggers who are interested in knowing how many people visit their blog and most importantly, where they come from.

But when Goolge Analytics display statistics for ‘Traffic Sources’ ‘, it divides them into categories - direct traffic, referring sites , search engines etc. I found it very surprising to see that traffic I get from Bing is listed under the ‘Referring Sites‘ category, and not the ‘Search Engine‘ category.

Here’s the screenshot:

I don’t believe this is a mistake, as someone had to classify Bing to a Traffic Source type. This being such a delicate issue, I’m sure it’s been discussed thoroughly inside Google.

To get the latest, hottest technology news, register to our RSS feed.

There’s  been a lot of bang around Bing, Microsoft’s new search engine over the last few days. According to StatCounter, Bing already bypassed Yahoo in search volumes. Personally, I don’t trust any Microsoft service since they deleted my entire Hotmail mailbox, which included some very important emails. I’ll never trust them again.

Looking through one of my blogs (Open Source ERP) Google analytics figures, I noticed something very surprising - although obviously,  most of the traffic to that blog comes from Google searches, Bing drives much higher quality traffic. Higher quality traffic means visitors who view more pages, stay longer on my blog (twice as long as Google-driven users!!), and most significantly, do not leave my blog without reading any post (’Bounce Rate’, the lower-the better). In fact, bounce rate of Google visitors is more than twice that of Bing users.

Here are my Google Analytics referrers stats(click to enlarge):

My  open source ERP blog doesn’t get much traffic (about 500 daily  visitors), so there might be a skew in my stats. It would be interesting to hear from other tech bloggers if they experience the same behaviour.