Getting jobs from an employer and onto a job board has been a challenge from the earliest days of the industry. Initially, the only way to do so was manually – the employer filled out a web form and submitted the job ad to the job board. Needless to say, this method (which, of course, is still widely used) was time-consuming and provided many opportunities for typos, dropped information, and other inaccuracies.
Needless to say, job boards didn’t like this arrangement – the harder it was to post jobs, the less likely their customers were to buy more job ads than absolutely needed. Employers didn’t like it, for it was tedious, slow, and yet another task for their (under-staffed) HR departments.
As more employers created their own websites, the first section they added – after the obligatory picture of their dynamic management team, of course – was a careers area. In addition to tidbits about benefits and working conditions, the careers area had job postings. These were created in exactly the same manner as those posted to a job board – via a simple web form. Almost immediately, HR managers everywhere began to wonder: “Why should I enter this information twice? Why not figure out some way to have it automatically transferred to the job board of my choice?”
Solutions quickly emerged. With a bit of programming, jobs could be ‘fed’ to job boards vis a RSS or XML file. For many companies, however, the ‘bit of programming’ was a stumbling block – they lacked programmers, time, money, or all of the above. Occasionally job boards would jump in and create the feeds for their clients. But by and large the responsibility fell on the employer to either create a feed or enter jobs the old-fashioned way.
Then along came a spider (sorry, couldn’t resist).
Spidering jobs is the ultimate ‘hands-off’ solution as far as employers are concerned. The software visits the employer site on a periodic basis (usually every 1-3 days), gathers the job data, then posts it directly to the job board. From the job board’s perspective, spidering involves either working with a 3rd party vendor or utilizing internal tech time – each time an employer’s site is added, the spider must be set up to read the correct areas of the site, then match the data fields to corresponding fields in the job board’s job posting database. In a perfect world, the spider is set up and then it just runs. In reality, spiders often require ‘fine tuning’ before they work – and if the employer changes their careers area, the spider must be changed as well.
So – is spidering jobs (or site scraping, or whatever you prefer to call it) a ‘must’ for job boards? A luxury? Or neither?
Well…it depends. Are you losing clients because they find it difficult or tedious to post jobs on your site? Are you losing revenue by making the posting of multiple jobs more difficult? Are your competitors offering spidering? (Don’t forget – some aggregators actually spider employer sites prior to making a sales call). The answers to these questions will determine whether spidering jobs is a nice option – or a necessity.[Want to get Job Board Doctor posts via email? Subscribe here.]. [Check out the JobBoardGeek podcast archive!]