BeeShops.com was developed as a result of our futile attempts at building a search engine. Creating technology to easily manage millions of files and search millions of records we set up BeeShops with shopping data provided by merchants.

The concept was to allow anyone to search all of our affiliate stores from one location. We pushed the database to almost 4 million items and created a user friendly interface.

The real draw back was time and size. Although we created a system that would allow relatively quick searches we also created a site that occupied nearly 100gig of space.

That may not seem like a huge amount, at least if you have never built a site that big.

One big issue was downloading 4 million images from merchant websites, resizing them to fit and creating 4 million thumbnails. Since images were accessed one at a time and merchant site servers are not all the fastest machines online, the time to download the images was almost 4 days of continuous downloading over several channels eventially from nearly 400 websites.

Another issue is bandwith, uploading and downloading 100gig makes remote backups to other datacenters costly and time consuming. And although the companies using low end servers with IDE and SATA drives enjoy a cheap abundance of drive space, the rest of us that need serious performace and still foot the bill for SCSI Raid systems treat drive space like gold. Mostly because it costs more than gold.

Without extensive advertising, we can only sit back and wait for the search engines to bring us traffic. So the site will not even likely pay its own bandwith bill in the first year just feeding the bots. But with 4 million pages and some great content, the site will grow and has. Now producing a fair amount of sales and activity the site is begining to get some attention.