web crawler, C#


I need a crawler (written in C#) for a specific real estate website.

It should be a console application which can crawl data (every 12 hours) and store it in MS SQL database (image files can be stored locally in a folder).

It should check all available listings (~100 000 listings).

To get listings you need to apply filters

- offer type (buy or rent)

- offer category (house, apartment, business, parking space, land/plot, apartment building

- postal code (list with postal codes will be provided)


1. Data to store (if available) for a single offer:

- whole html

- offer type (buy or rent)

- offer category (house, apartment, business, parking space, land/plot, apartment building)

- title

- price

- construction year

- rooms

- living area

- city

- street

- postal code

- property type

- floor

- Floors in the house

- cubic area

- plot area

- offer Id (Immocode)

- Floors in the house

- Availability

- Renovated in

- Condition

- Features

- Description

- pictures (store image urls and images in the db or locally)

- Advertiser details: Name, address, phone number

2. The crawler should not make any damage on the website (prevent from DoS)

3. The crawler should not be detectable

4. The crawler should crawl the website every 12 hours without being blocked.

5. All data should be updated in case of any changes.

6. Price change for listings should be stored separately to have visibility if it grows or drops.

7. The crawler should send alerts (email or slack notification) if it's blocked.

Long-term cooperation possible.

Квалификация: .NET, Программирование на C#, Веб-скрейпинг

Показать больше crawling websites c#, web crawler in c# codeproject, web scraping c# code project, selenium web crawler c#, web crawler sample code c#, c# web crawler htmlagilitypack, multithreaded web crawler c#, c# pull data from website, java web crawler image capture, web crawler cpu usage, write web crawler, web crawler names, web crawler system requirement, web crawler demo gathering information function, java spring web crawler, web crawler job description, web crawler description title, web crawler extract description php mysql, web crawler extract phone number, web crawler fax number

О работодателе:
( 1 отзыв ) Lugano, Switzerland

ID проекта: #18542040

48 фрилансеров(-а) в среднем готовы выполнить эту работу за $740


Hello, We have a very good experience in Dot Net Development and can offer you experienced developer(s) for your project. We have been very active using DotNet core as preferred platform. It allows application t Больше

$3000 USD за 30 дней(-я)
(51 отзывов(-а))

Hi, my name is Selim. I read your `web crawler, C#` project descriptions carefully before bidding. I checked your requirements as well... I got what you need and ready to go ahead as soon as we can clarify further pr Больше

$900 USD за 10 дней(-я)
(272 отзывов(-а))

Hi I did all my projects in C# and using mult-threading and proxies to enhance the performance. Thanks

$400 USD за 3 дней(-я)
(153 отзывов(-а))

Dear Employer, Please message me for your job. I have scraped more than a dozen Real Estate sites and I can share samples of those as well. I understand your job quite well and will adhere to eaxh and point. Do t Больше

$1000 USD за 15 дней(-я)
(175 отзывов(-а))

Hi. I should test site and define, if it blocks crawlers. Can you send site url? Then I can prepare for 1-2 days sample script to be sure, that this site can be daily scraped. Thanks anyway.

$800 USD за 15 дней(-я)
(38 отзывов(-а))

Hi there,I am Web Scraping expert from Bosnia & Herzegovina,Europe. I have carefully gone through with your requirements and I would like to help you with this project ! I can start immediately and finish it within th Больше

$611 USD за 5 дней(-я)
(98 отзывов(-а))

Hello! I can create the crowler using c#. the task is pretty much clear. Is 4-5 days delivery time ok for you? Can you send me the website? You will not be dissapointed, I have a big 33%rehire rate. thank you!

$777 USD за 5 дней(-я)
(159 отзывов(-а))

what is the real estate site? a proxy subscription will be required to rotate IPs to prevent it being blocked.

$250 USD за 7 дней(-я)
(59 отзывов(-а))

Hi there, I have checked the details I have rich experienced with .NET, C# Programming, Web Scraping. Please initiate chat so we can discuss this job.

$1000 USD за 10 дней(-я)
(17 отзывов(-а))

Hello!! As per the description I am highly interested to get started on this project. I am Asp.net Full-stack developer. I have 5+ Years experience in analysis and development of client/server, windows and web ap Больше

$800 USD за 20 дней(-я)
(47 отзывов(-а))

Hello, Hope you are doing well! We at Flojics are most preferred agency for our client due to 100% recommendation and “5 STAR” rating from our clients which we achieve through an excellent client support as well as t Больше

$500 USD за 12 дней(-я)
(8 отзывов(-а))

Hello, I have gone through your job posting and become very much interested to work with you. I am an expert in this field. I have already completed several projects like this. For evidence you can see my profile. Больше

$900 USD за 10 дней(-я)
(24 отзывов(-а))

Hi there, I am interested in the project. Can you please provide the link to the web site from where,data to be crawled? Thank you.

$611 USD за 10 дней(-я)
(65 отзывов(-а))

Greetings, My name is Constantine and I specialize in C# and web scraping and data processing applications. Would be happy to do this project for you and future ones as well. I have good portfolio of many similar tas Больше

$600 USD за 3 дней(-я)
(52 отзывов(-а))

Hello. I am really grateful to face your project. As I see, you 're looking for a senior c# developer and here is him. Confidentally I can say, I could be your wonderful choice on this project. Really look forward Больше

$800 USD за 10 дней(-я)
(38 отзывов(-а))

Hi, I have my own script for scrapping. I can modify the existing script to suit your need and give you the require data in excel sheet. My expertise are in the area of scrapping using C#, My SQL, PHP. Lookin Больше

$611 USD за 10 дней(-я)
(21 отзывов(-а))

Hello There, I'm available to assist you to work on web crawling task and I will start immediately. I am 6+ years experienced Web scraping specialist holding excellent background with creating scraping tools, a Больше

$611 USD за 10 дней(-я)
(16 отзывов(-а))

Long stroy short, I'm a full stack developer. You can see my profile. I'm 100% sure with C/C++/C#, Java, PHP, JSP, MySQL, Javascript, HTML, CSS, Android, iOS, etc. Anything you want, I'll satisfy you. Talking about Больше

$500 USD за 7 дней(-я)
(15 отзывов(-а))

With respect to this project I would like to present myself as a candidate for your consideration. I have more than 8 years of IT experience. I have successfully completed few projects which involved Programming, scrip Больше

$555 USD за 5 дней(-я)
(34 отзывов(-а))

hi,how are you?. i checked your bid in detail. i am C#, WebScrapping/WebCrawler expert and have full experience for 7 years. C#, WebScrapping/WebCrawler is my top skill and i can complete your project by using C#, .N Больше

$600 USD за 10 дней(-я)
(13 отзывов(-а))