use proxy server for VB.NET data scraping program

Job Description:

I have a program written in VB.NET that scrapes data from a public site. The program submits a case number to the site, the site looks up the case, and then my program stores the html returned by the site into an Access database.

My problem is that after looking up a few hundred cases, the site realizes my program is a bot, and it stops serving the pages to me.

My proposed solution is to use a proxy server ([login to view URL]) to use different IP addresses when scraping the site. Also, I think I should randomize the order of cases that I process. For example instead of scraping the data for ID 101V, then 102V, then 103V, 104V, 105V,106V, 107V, 108V, 109V, 110V, I think I should scrape the data in a random order like: 108V, 102V, 105V,110V, 101V, 107V, 109Vv, 104V, 103V, 106V.

I think this is an easy project. I just don’t have time to do it.

I can pay you $100 to do this project. And I will pay for the cost of proxy server.

Please let me know when you could have this done by. If your proposal looks ok, I will send you the VB.NET code and the Access database.


Навыки: VB.NET, Интеллектуальный анализ данных

О клиенте:
( 17 отзыв(-а, -ов) ) gaithersburg, United States

ID проекта: #35320503

7 фрилансеров(-а) готовы выполнить эту работу в среднем за $114


Hi there. I know which skill are necessary for you and I am expert in this fields. You can confirm about it via my previous client's reviews. Hope to contact you.

$100 USD за 4 дней(-я)
(0 отзывов(-а))

Hi there, I am Rashid, I am a full stack developer with 4+ years of experience in both front-end and back-end development. I have read your use proxy server for VB.NET data scraping program description very carefully a Больше

$250 USD за 8 дней(-я)
(0 отзывов(-а))

✔ Expert Here || Available Now.! Hello.! Will C. I’ve already understood the project because it requires the expertise in VB.NET and Data Mining. I've expertise in 400+ Technologies & awarded multiple times. You wil Больше

$30 USD за 1 день
(0 отзывов(-а))

Hello, I can complete the requested change to coded and implement the proxy in VB.NET no problem, Contact me so we can get started.

$120 USD за 1 день
(0 отзывов(-а))

hello. i am data scraping expert having rich experience for 4+ years. please contact me. i promise the best service. thanks.

$100 USD за 7 дней(-я)
(0 отзывов(-а))

HI What would you expect from the best candidate? I have scraper thousand of cases from different counties of different states. What approach do you use for scraping? With Chrome? Within VB.net? If different way? W Больше

$100 USD за 1 день
(0 отзывов(-а))