Закрыт

web crawler, C#

Hi,

I need a crawler (written in C#) for a specific real estate website.

It should be a console application which can crawl data (every 12 hours) and store it in MS SQL database (image files can be stored locally in a folder).

It should check all available listings (~100 000 listings).

To get listings you need to apply filters

- offer type (buy or rent)

- offer category (house, apartment, business, parking space, land/plot, apartment building

- postal code (list with postal codes will be provided)

Spec:

1. Data to store (if available) for a single offer:

- whole html

- offer type (buy or rent)

- offer category (house, apartment, business, parking space, land/plot, apartment building)

- title

- price

- construction year

- rooms

- living area

- city

- street

- postal code

- property type

- floor

- Floors in the house

- cubic area

- plot area

- offer Id (Immocode)

- Floors in the house

- Availability

- Renovated in

- Condition

- Features

- Description

- pictures (store image urls and images in the db or locally)

- Advertiser details: Name, address, phone number

2. The crawler should not make any damage on the website (prevent from DoS)

3. The crawler should not be detectable

4. The crawler should crawl the website every 12 hours without being blocked.

5. All data should be updated in case of any changes.

6. Price change for listings should be stored separately to have visibility if it grows or drops.

7. The crawler should send alerts (email or slack notification) if it's blocked.

Long-term cooperation possible.

Квалификация: .NET, Программирование на C#, Веб-скрейпинг

Показать больше crawling websites c#, web crawler in c# codeproject, web scraping c# code project, selenium web crawler c#, web crawler sample code c#, c# web crawler htmlagilitypack, multithreaded web crawler c#, c# pull data from website, java web crawler image capture, web crawler cpu usage, write web crawler, web crawler names, web crawler system requirement, web crawler demo gathering information function, java spring web crawler, web crawler job description, web crawler description title, web crawler extract description php mysql, web crawler extract phone number, web crawler fax number

О работодателе:
( 1 отзыв ) Lugano, Switzerland

ID проекта: #18542040

50 фрилансеров(-а) в среднем готовы выполнить эту работу за $736

mhmhz

Hi I did all my projects in C# and using mult-threading and proxies to enhance the performance. Thanks

$400 USD за 3 дней(-я)
(150 отзывов(-а))
7.6
seaanddream

Hi, my name is Selim. I read your `web crawler, C#` project descriptions carefully before bidding. I checked your requirements as well... I got what you need and ready to go ahead as soon as we can clarify further pr Больше

$900 USD за 10 дней(-я)
(260 отзывов(-а))
7.7
promactinfo

Hello, We have a very good experience in Dot Net Development and can offer you experienced developer(s) for your project. We have been very active using DotNet core as preferred platform. It allows application t Больше

$3000 USD за 30 дней(-я)
(51 отзывов(-а))
7.7
asin1987

Dear Employer, Please message me for your job. I have scraped more than a dozen Real Estate sites and I can share samples of those as well. I understand your job quite well and will adhere to eaxh and point. Do t Больше

$1000 USD за 15 дней(-я)
(162 отзывов(-а))
7.4
zekovicm

Hi there,I am Web Scraping expert from Bosnia & Herzegovina,Europe. I have carefully gone through with your requirements and I would like to help you with this project ! I can start immediately and finish it within th Больше

$611 USD за 5 дней(-я)
(85 отзывов(-а))
7.0
sergiuloghin

Hello! I can create the crowler using c#. the task is pretty much clear. Is 4-5 days delivery time ok for you? Can you send me the website? You will not be dissapointed, I have a big 33%rehire rate. thank you!

$777 USD за 5 дней(-я)
(143 отзывов(-а))
7.0
kyrbatow

Hi. I should test site and define, if it blocks crawlers. Can you send site url? Then I can prepare for 1-2 days sample script to be sure, that this site can be daily scraped. Thanks anyway.

$800 USD за 15 дней(-я)
(34 отзывов(-а))
6.6
DavidLindon

what is the real estate site? a proxy subscription will be required to rotate IPs to prevent it being blocked.

$250 USD за 7 дней(-я)
(54 отзывов(-а))
6.3
seefattechnologi

Hello!! As per the description I am highly interested to get started on this project. I am Asp.net Full-stack developer. I have 5+ Years experience in analysis and development of client/server, windows and web ap Больше

$800 USD за 20 дней(-я)
(45 отзывов(-а))
6.3
knikolovdev

Greetings, My name is Constantine and I specialize in C# and web scraping and data processing applications. Would be happy to do this project for you and future ones as well. I have good portfolio of many similar tas Больше

$600 USD за 3 дней(-я)
(51 отзывов(-а))
6.1
bianyunzhi95

Hello. I am really grateful to face your project. As I see, you 're looking for a senior c# developer and here is him. Confidentally I can say, I could be your wonderful choice on this project. Really look forward Больше

$800 USD за 10 дней(-я)
(34 отзывов(-а))
5.8
schoudhary1553

Hello, I have gone through your job posting and become very much interested to work with you. I am an expert in this field. I have already completed several projects like this. For evidence you can see my profile. Больше

$900 USD за 10 дней(-я)
(20 отзывов(-а))
5.9
DebasisB

Hi there, I am interested in the project. Can you please provide the link to the web site from where,data to be crawled? Thank you.

$611 USD за 10 дней(-я)
(63 отзывов(-а))
6.2
mmadi

Hello, Hope you are doing well! We at Flojics are most preferred agency for our client due to 100% recommendation and “5 STAR” rating from our clients which we achieve through an excellent client support as well as t Больше

$500 USD за 12 дней(-я)
(8 отзывов(-а))
6.0
narsim3128

With respect to this project I would like to present myself as a candidate for your consideration. I have more than 8 years of IT experience. I have successfully completed few projects which involved Programming, scrip Больше

$555 USD за 5 дней(-я)
(34 отзывов(-а))
5.7
Nidhijain123

Hi, I have my own script for scrapping. I can modify the existing script to suit your need and give you the require data in excel sheet. My expertise are in the area of scrapping using C#, My SQL, PHP. Lookin Больше

$611 USD за 10 дней(-я)
(19 отзывов(-а))
5.4
winlancerinc

Hello There, I'm available to assist you to work on web crawling task and I will start immediately. I am 6+ years experienced Web scraping specialist holding excellent background with creating scraping tools, a Больше

$611 USD за 10 дней(-я)
(15 отзывов(-а))
5.6
ig149

I have read your job description and I have done something like this in past it's a custom scrapper: LinkdIn: [login to view URL] I can certainly help you get this scra Больше

$800 USD за 15 дней(-я)
(42 отзывов(-а))
5.4
sagarvb2005

Hello Sir, I have read your whole project description and i believe that i will do it better. Could you please discuss more about it and provide some sample data if possible so i can give you proper Delivery time a Больше

$930 USD за 10 дней(-я)
(23 отзывов(-а))
5.2
shobhitpundir

Hi There, We can create a console app for scrapping. I can show you a sample of a scrapping tool I implemented. If you want not being detected the we can hop IP address for every request. Does the website you want Больше

$300 USD за 5 дней(-я)
(13 отзывов(-а))
4.9