Закрыт

Fix Existing Scrapers

We have two scrapers that do not work consistently. We need them fixed.

Both scrapers are written in Python 2.7 using Scrapy. Both use Crawlera as the proxy to avoid getting banned.

One is for [login to view URL] and the other is for [login to view URL]

If possible, we would like the scrapers to retrieve the data directly from the HTTP response rather than rendering the page and then scraping it.

Both spiders are always running on a Google Cloud Platform Debian Virtual Machine (Google Compute Engine) and write all results to a MySQL 5.7 2nd generation database also on Google Cloud Platform.

When you submit your proposal, tell me how much experience you have with scrapy so that I know you read this.

Sometimes they work correctly, and retrieve data and write it to the database. Other times, they appear to get stuck on something and do not retrieve any data and write 0 records to the database. This project is to fix this inconsistency and to make the scrapers reliable, robust, and running continuously without problems, errors, or issues.

Квалификация: Веб-скрейпинг, Архитектура ПО, Scrapy, Python, Data Scraping

Показать больше fix browser alignment issues, fix browser compatibility issues, fix server configuration issues, fix css mouse issues, fix java scripting issues, fix stickam reconnecting issues, fix existing code, fix stickam connecting issues, fix browser incompatibility issues dreamweaver, fix browser compatibility issues dreamweaver file, dreamweaver fix browser compatibility issues, fix browser compatibility issues firefox, fix existing, fix existing auction site, fix ie6 css issues service, java script fix ie6 css issues, fix expanding box issues, fix website compatibility issues dreamweaver, fix cross browser issues, fix existing css

О работодателе:
( 3 отзыв(-а, -ов) ) Riverton, United States

ID проекта: #20556520

33 фрилансеров(-а) в среднем готовы выполнить эту работу за $134

zekovicm

Hi there, As an web scraping expert i can get it done for you. Please feel free to contact me to discuss further details. Best regards Miljan

$166 USD за 4 дней(-я)
(103 отзывов(-а))
7.3
liveexperts123

Hi there, I have read your project description and i'm confident i can do this project for you perfectly.I still have a few questions. please leave a message on my chat so we can discuss the budget and deadline of the Больше

$155 USD за 3 дней(-я)
(54 отзывов(-а))
7.0
DevStar925

How are you? As you can see from my profile, I am Python expert, And I have a good experience in Trading Bot project. I would like to discuss in detail via chat. Thanks

$140 USD за 7 дней(-я)
(55 отзывов(-а))
7.1
C3guru

Hello How are you? I'm very interested in your project I have built many python scripts So I'm sure I can complete your script Please share me more details over chat If you assign the job to me, I can start right now Больше

$140 USD за 7 дней(-я)
(41 отзывов(-а))
6.8
sky930320

Dear Sir. Glad to meet you. I'm Web developer specializing in web scraping crawling and indexing web pages. Skills: python, scrapy, selenium, requests, beautifulsoup, mechanize, lxml, urllib2, automation, bots, spide Больше

$140 USD за 7 дней(-я)
(24 отзывов(-а))
6.3
umg536

Hi there, please leave a message on my chat so we can discuss the budget and deadline of the project. I have read your project description and i'm confident i can do this project for you perfectly. Thanks . .

$155 USD за 3 дней(-я)
(18 отзывов(-а))
6.1
brianconey

Hello How are you I have checked your description I am a python expert and I am sure i can fix the scraper it is possible python request and beautifulsoup I can execute your script on google cloud platform It is th Больше

$140 USD за 1 день
(40 отзывов(-а))
6.2
Mickelson

Hi Nice to meet you. I have enough experience in python script. Below the libraries are I used in past project. selenium, pandas, matplotlib, lxml, beautifulsoup, scipy, and other useful libraries. I have written some Больше

$140 USD за 7 дней(-я)
(80 отзывов(-а))
6.5
kkc264043kkc

have an 3+ years experience in the web scraping and crawling with python Scrapy Can fix the scraper. want to know more about it. want to look at the script. These are my skills related to web scraping and crawling Больше

$200 USD за 2 дней(-я)
(45 отзывов(-а))
5.9
dungforever

Hi ⭐⭐⭐ I can complete your project in 1 day ⭐⭐⭐ I've read your requirements carefully. I understood your requirement and I have full experience in web scraping. I am a certified freelancer with great ideas & methodolog Больше

$140 USD за 2 дней(-я)
(77 отзывов(-а))
5.8
kunitsynartem

Hello! I have 2 years of experience in web scraping using Python and Scrapy and I'm interested in your project for your scrapers fixing. Is it possible to see the code of your scrapers? Thanks in advance for your answe Больше

$140 USD за 7 дней(-я)
(30 отзывов(-а))
5.3
NIKE9

Thanks for your job posting. I am a full-stack web developer with 5+ years of experience and I am very interested in your project. My main skill is Python, PHP, Ruby and I have the relevant javascript skills such Больше

$155 USD за 3 дней(-я)
(10 отзывов(-а))
5.3
fivestar10

Hello.. I have read your post very carefully and I have understood what you want. I am a Five Star Rated Developer. I have experience of development like this. As an experienced Scrapying Expert, I am sure I can handle Больше

$120 USD за 5 дней(-я)
(9 отзывов(-а))
4.5
phantom1222

Hi, I have read all your requirement I have rich experience with Python/Web Scraping and I can help you perfectly. I'm very interested in your job post.I believe my skills would be ideal for your project. And then I Больше

$100 USD за 7 дней(-я)
(12 отзывов(-а))
4.6
AlexanderPGR

⭐⭐⭐ Hi, Dear client. ⭐⭐⭐ I'm very interested your project. I read your description carefully. I'm very talented OPEN CART developer. if you assign to me this project, you can get a cool result. please contact me. thank Больше

$140 USD за 7 дней(-я)
(14 отзывов(-а))
4.6
Itblast

I am familiar with multiple programming languages, though most of my experience lies in scripting languages like python and object-oriented languages such as C# and Java. I have also worked a fair amount with various t Больше

$155 USD за 3 дней(-я)
(8 отзывов(-а))
4.4
seocloaker81

Hi sir, I have 4 years of experience in Scrapy & 7 years of experience in data extraction, & I am proficient in using tools such as Selenium, as well as an expert in creating Python/.Net scrapers. Regards Navpreet Sing Больше

$45 USD за 3 дней(-я)
(4 отзывов(-а))
3.9
bamtanh123

I can complete your task in 2 days. Hi, Nice to meet you. I am a rich experienced web/software developer. I have read your task description carefully and understood clearly. Scraping is my specialty. I can show you sev Больше

$110 USD за 7 дней(-я)
(5 отзывов(-а))
4.0
oswaldoodavidal

Hello, I have experience in web scraping with Python. My experience with Scrapy goes from extracting data from simple websites that doesn't require JavaScript rendering to extracting data and automating processes with Больше

$250 USD за 10 дней(-я)
(3 отзывов(-а))
4.1
shivansh121

Hi Employer, I can  extract any data from any [login to view URL] I have scraped the data from these websites & stored scraped data into CSVs and excels as per my clients requirement & I have made desktop applications u Больше

$88 USD за 3 дней(-я)
(9 отзывов(-а))
3.9