Crawl Domain Find Expred Domains -- 2

We are looking for a crawler to crawl every page of a website looking for external links pointing to expired domains.

User should definde a list of sites to crawl via text file. Crawler should work logically crawling all pages of a site and not be sitemap dependent. Only unique external domains should be logged to prevent duplicate domain availability lookups.

User should also be able to define a list of urls to ignore checking for availability; eg. [login to view URL] etc. these domains should be user defined in a blacklist text file.

Results should be given in a csv file listing linking domain and available domain.

Квалификация: Python, Веб-скрейпинг

Показать больше find buyer domains, enom api find available domains, need script find expired domains name, expired domain marketplace, expired domain crawler, expired domain finder, expired domain finder software, domain hunter gatherer pro cracked, free expired domains, how to find expired domain names with traffic, expired domains with traffic, domain extractor extracts domains search engine, domain spy closed domains, crawl domain excel, crawl web find urls, crawl website find emails, perl script crawl links find text, crawl site find specific links, domain find, crawl website find files

О работодателе:
( 5 отзыв(-а, -ов) ) 保定市, China

ID проекта: #19765484

6 фрилансеров(-а) в среднем готовы выполнить эту работу за $28


Hi. I did read the project description and have a few questions. 1. Do you need the script as well or data only? 2. What is the format of the output data? CSV is OK? We can do other formats as well. 3. Which fields do Больше

$100 USD за 5 дней(-я)
(110 отзывов(-а))

‌Hi, I have gone through your requirement to scrape lots of websites. I am EXPERT in building scraping tools /scripts. Hence, I can SURELY work on your project. I am having 4 YEARS of EXPERIENCE in developing PHP-PYTHO Больше

$15 USD за 3 дней(-я)
(51 отзывов(-а))

Dear Prospect Hiring Manager. Thank you for giving me a chance to bid on your project. i am a serious bidder here and i have already worked on a similar project before and can deliver as u have mentioned "I can do th Больше

$13 USD за 7 дней(-я)
(1 отзыв)

*************************************************************************************************************************** Currently I am scrapping news sites for a US client and also woocommerce sites for a Netherlan Больше

$10 USD за 7 дней(-я)
(0 отзывов(-а))

I have experiment in crawling data using be4, scrapy,... with python, extract data to xml, json,... Contact me!

$20 USD за 2 дней(-я)
(0 отзывов(-а))

Hi there JUNA here. I understand that you need a crawler or A SPIDER for scraping expired domain but my question is will you provide the list of domains that you need to check for the availability otherwise this sho Больше

$12 USD за 3 дней(-я)
(0 отзывов(-а))