Закрыт

Create a Fast Web Crawler to Extract Google SERP Data

Hi,

I need someone to develop me a fast and powerful web crawler to extract data from Google SERP.

Important key features;

1. Must have a user interface protected by user and password where I can manage my campaigns.

2. Must have rotating proxies support

3. Multi threading support

I want a dashboard, where I can create/edit/delete my campaigns. For every campaign, I will set a google query (ie: intitle:"fashion" "blog") and decide how many results I want to extract.

When I will click on the "Start Crawler" button, I want to see the results in real time.. so If I decided to crawl the first 100 000 results for this query, if the crawler have extracted 1000 results, it should show 1000 results extracted out of 100 000.

Once the extraction is completed, I want to receive an email that notify me that the campaign has finished.

Once the process is completed, I want to have 3 options;

1. Remove duplicated domains (Remove all entries that is for the same domain) ie:

[login to view URL]

[login to view URL]

[login to view URL]

Should keep just first entry so "[login to view URL]" and delete all the other entries.

2. Update metrics (This will use the SEMrush API to update the keywords / organic traffic for every extracted domains)

3. Export the data in Excel format. (Columns; Full URL / SEMrush Keywords / SEMrush Organic Traffic)

Квалификация: Веб-скрейпинг, PHP, Python, Javascript, Архитектура ПО

Показать больше extract google analytics data limit, php extract google serp, extract google analytics data, web crawler extract description php mysql, extract google analytics data xml, fast web crawler mysql, web crawler extract data, java web crawler extract web page title, extract google analytics data daily, web crawler extract data website, web crawler extract phone number, web crawler extract data excel, extract google play data, web crawler script google, code web crawler extract data website, web crawler extract information java, program web crawler extract information website, create simple web-crawler and email sender, how to create a web crawler

О работодателе:
( 35 отзыв(-а, -ов) ) Sabrevois, Canada

ID проекта: #23957254

19 фрилансеров(-а) в среднем готовы выполнить эту работу за $589

zekovicm

Hi there,I am Web Scraping expert from Bosnia & Herzegovina,Europe. I have carefully gone through with your requirements and I would like to help you with this project ! I can start immediately and finish it within the Больше

$500 USD за 7 дней(-я)
(140 отзывов(-а))
7.6
AwaisChaudhry

Hey there, I have 4 years of working experience in web technologies and I have a strong working knowledge of every possible framework which is in the market. In terms of client-end I know: Javascript, PHP, Python, Sof Больше

$1250 USD за 29 дней(-я)
(84 отзывов(-а))
7.8
chirgeo

Hi. I did read the project description and have a few questions. 1. Do you need the script as well or data only? 2. What is the format of the output data? CSV is OK? We can do other formats as well. 3. Which fields Больше

$500 USD за 4 дней(-я)
(135 отзывов(-а))
7.5
umg536

Hi there, Being an expert in Python and matlab programming I can do this project for you. please leave a message on my chat so we can discuss the budget and deadline of the project. Also, have a look at my profile for Больше

$750 USD за 3 дней(-я)
(22 отзывов(-а))
6.2
songhku925

Hi. I will develop the web crawler that has GUI and key features using python. As you want, i will create 3 options I have over 5 years of experience with python. I have already completed several scraping projects on Больше

$750 USD за 5 дней(-я)
(34 отзывов(-а))
5.7
d1scoveritech

Good day! I'll develop you a fast and powerful web crawler to extract data from Google SERP. Important key features; 1. With a user interface protected by user and password where you can manage your campai Больше

$500 USD за 7 дней(-я)
(13 отзывов(-а))
5.7
stepinnsolution

Hi there, Greeting!! I have more than 10 years of experience in Web & App development and multiple technologies. I am committed to providing you high quality services. My goal is to satisfy clients by providing high q Больше

$750 USD за 7 дней(-я)
(36 отзывов(-а))
5.5
leonworldwide

Hi there, After reading your offer, this looks like a perfect fit for my skill sets so May I discuss with you for further details about this project? I'm a web developer with over 7 years experience and very strong i Больше

$250 USD за 4 дней(-я)
(8 отзывов(-а))
4.5
Martin718

Hello I'm familiar Page Scrapping with Phantom JS and php script. I have some examples. I read your description and i can start right now. Please discuss details on chat, Thanks

$700 USD за 7 дней(-я)
(3 отзывов(-а))
4.0
Jennycreation

hello sir , I am a full time freelancer and have 10 years of experience in web and window development in asp.net c#, javascript,sql server . and Android mobile development i can do work as per your requirement . Больше

$500 USD за 7 дней(-я)
(2 отзывов(-а))
3.3
hameedakbar688

Hello, I checked out your requirements.I have 5+ years of working experience and Expertise in IOS , Android , Web Development, PHP, JavaScript, NodeJS, HTML,C++ ,Wordpress, CSS, RestAPI, Third-Party integration. I a Больше

$500 USD за 7 дней(-я)
(2 отзывов(-а))
2.9
bangashdev5

Hello, I have done Scrapping from Google using PhantomJS and also with Python but PhantomJS is best tool and google can't detect it so message me to discuss some question which I have related to your project. Thanks

$500 USD за 12 дней(-я)
(10 отзывов(-а))
3.5
DevelopVladimir

Hi there. I have carefully read your job description with great interests. I have experience with python, django,selenium and bs4(beautifulSoup) for about 5 years. Please visit my past references here. -http://cars Больше

$500 USD за 7 дней(-я)
(3 отзывов(-а))
2.3
cofeeman15

Hi there! I have read your description carefully and I am confident I can full fill your requirement. I provide my skills in hopes of working with you. I am a web scraper and have 7+years of ample experienced in web s Больше

$500 USD за 7 дней(-я)
(2 отзывов(-а))
2.0
Servicehour24

Hello Dear, Yes, I can work for you as per your project description with amazing exploration and experimental research abilities of mine organization is perfect for that you may require. My work fulfills as well as ke Больше

$600 USD за 10 дней(-я)
(0 отзывов(-а))
0.0
worldmouse

Hi I have read your post content carefully and can develop your python project as my experience. That's why I have rich experience with Python. I am a new freelancer on freelancer.com, but I can provide you robust and Больше

$500 USD за 7 дней(-я)
(0 отзывов(-а))
0.0
dehafex758

I READ YOUR POST I CAN DO IT FOR YOU Hello I am a senior developer. I want to do it for you if you want. we can discuss more about this project. if you can send a reply. I waiting for you best response. let me discuss Больше

$500 USD за 7 дней(-я)
(0 отзывов(-а))
0.0
honeyballz

Dear Sir. Glad to meet you. I'm Web developer specializing in web scraping crawling and indexing web pages. Skills: python, scrapy, selenium, requests, beautifulsoup, mechanize, lxml, urllib2, automation, bots, spide Больше

$500 USD за 7 дней(-я)
(0 отзывов(-а))
0.0
networkdesign17

Nice to meet you Hi I am automation expert If there is any website you need to scrape data from (no matter the format, what blocking tools there are). I can deliver the data. If you do the same thing on the internet Больше

$637 USD за 9 дней(-я)
(0 отзывов(-а))
0.0