Закрыт

Build a Mobile E-commerce Application that pulls product info from other sites [with ability to Add ML and chats]

The goal of the project is to build a scalable e-commerce app with an inter grated blog section.

Most items will be created by web scraper which should scrape data from more a dozen different websites at first. Later on, it should be possible to upscale the scraper to a few thousand websites.

Those websites are known and should be added iteratively to the scraper. The websites have a different structure each which is why the development and maintenance costs per site need to stay as small as possible. The aim is to scrape the websites on a weekly basis at first. Later on, the scraping intervals should be reduced to a daily basis or even shorter. The scraped data needs to be stored in an useful and efficient way in a database in the cloud as it will be used to be products and personal shopper reccomendations.

Furthermore, the scraping must be intolerant to changes in the designs of the websites and it must prevent being blocked.

Currently, a simple scraper in Python exists which can scrape a few websites by using the Selenium library. However, this does not need to be continued at all cost.

The following tasks are part of your engagement for the project:

o Developing a modular and scalable software architecture for the web scraping project (preferably with Python)

o Containerizing the program in Docker

o Deploying and managing the containers in the cloud, probably with AWS and Kafka

o Implementing different measures to prevent blacklisting and being blocked

o Setting up a SQL database, probably PostgreSQL with AWS

The following tasks might be part of a further engagement:

o Implementing the web scrapers for a large number of different websites

o Maintaining and monitoring the scrapers for the websites

o Adding a web crawler to find additional websites

o Parsing the stored data and processing them into a more useful format

Your qualifications:

o Web Scraping (Importance: 9/10)

o Python (Importance: 7/10)

o Docker (Importance: 8/10)

o AWS (Importance: 5/10)

o Kafka or other Pipelining/Queuing Tools (Importance: 8/10)

o Cloud Databases (Importance: 6/10)

o PostgreSQL (Importance: 10/10)

Навыки: Веб-скрейпинг, PHP, MySQL, Machine Learning (ML), Mobile App Development

Показать больше: build an e-commerce website/marketplace - future integration with android/iphone, build my e-commerce website, mobile e commerce app, mobile e-commerce, e commerce application developer government, http www freelancer com build website e commerce hourly price 55gbp, pet e commerce application design, build web e-commerce, E-commerce application, Build a E-commerce website, social networking, e-commerce application, opencart, hotel and tourism booking system, opencart CMS, Build an e-commerce website, Build an e-commerce webite, e-commerce application testing interview questions, e commerce application testing process, e commerce application development, how e commerce application works, e-commerce application development, e-commerce application development pdf, build an e-commerce application using laravel and vue

О работодателе:
( 0 отзыв(-а, -ов) ) South Africa

ID проекта: #29013412

29 фрилансеров(-а) готовы выполнить эту работу в среднем за $2266

(109 отзывов(-а))
9.6
seekdeveloper

Hello, I have read your post and understood your requirement. Looking for the freelancer to work on your next project? Or just need some issues/bugs/fixes ASAP? I have 11+ years of software experience in fortune 50 Больше

$1500 USD за 15 дней(-я)
(125 отзывов(-а))
8.9
AwaisChaudhry

Hey there, I have 4 years of working experience in web technologies and I have a strong working knowledge of every possible framework which is in the market. In terms of client-end I know: MySQL, Web Scraping, Mobile Больше

$3000 USD за 14 дней(-я)
(78 отзывов(-а))
7.9
(54 отзывов(-а))
8.1
AppDuniya

Hi there Greetings! I am building E-commerce multivendor marketplace app. I can show you complete architecture of the mobile application. I hold 7+ years experience in building web and mobile applications. Please r Больше

$2250 USD за 7 дней(-я)
(72 отзывов(-а))
7.9
(206 отзывов(-а))
7.4
Julian29

Hi There,,, I saw your project description, you need to build a Mobile E-commerce Application(Android & iOS) that pulls product info from other sites, I have seen the project details carefully open chart window for mor Больше

$2250 USD за 7 дней(-я)
(99 отзывов(-а))
7.7
TheScorpion93

We can do scraping part proficiently using python also we can build app if u increase budget Please, contact me and send me the link to the site so I could make a FREE SAMPLE Please, contact me and send me the link t Больше

$1500 USD за 2 дней(-я)
(85 отзывов(-а))
6.8
(49 отзывов(-а))
6.7
AlphawizzTech

Hello, I have gone through your post for building an E-commerce mobile application that pulls product info from other sites ,please message me to discuss further . I am a full stack developer, I have worked much Appl Больше

$3000 USD за 25 дней(-я)
(15 отзывов(-а))
5.8
FiveExceptions

Hello, We've previous experience in developing a scraping solution for one of our eCommerce clients which scrape product's sales & stock information from different retail/eCommerce stores & save it into easy to unders Больше

$3000 USD за 45 дней(-я)
(19 отзывов(-а))
6.0
kdsighappsol

Hi, I saw your job post and you need app for " Scalable E-commerce App with an inter grated blog section App "and i have skills in coding similar app and i believe that i can do it for you perfectly. I would appro Больше

$1500 USD за 20 дней(-я)
(11 отзывов(-а))
5.9
rstomar

Hi i will build your e-commerce store /online store I have sound experience in it ,I have some query .please come for chat <<< MY Portfolio:- <<< Android:- =======a [login to view URL] Больше

$1500 USD за 7 дней(-я)
(57 отзывов(-а))
6.1
ProjectHunter87

I have reviewed project description and have some questions which i would like to discuss further. Please message me so we can discuss further. Thank you.

$2250 USD за 7 дней(-я)
(16 отзывов(-а))
5.6
spdsolusion

Hi There, I hope that you are doing well. We have gone through your project details deeply and we can meet your needs very well. I want to mention some of your needs here in brief. The following tasks might Больше

$2250 USD за 7 дней(-я)
(18 отзывов(-а))
6.4
fullstackappcode

Hello Dear!!!! I have check your project post as you need ++++"Mobile E-commerce Application that pulls product info from other sites [with ability to Add ML and chats]"++++ .The goal of the project is to build a scal Больше

$1500 USD за 7 дней(-я)
(18 отзывов(-а))
5.9
theredesign

Hi there in Kenya!! I'm a developer based in Cameroon with hands-on experience in providing scalable solution that scape various data over the Internet. My recent accomplishment is a price comparison solution which get Больше

$2500 USD за 20 дней(-я)
(8 отзывов(-а))
4.9
hiteshshar761

Hi there, Are you available for chat/calling to discuss the project? please let me know your availability in message box. I have 5 years of experience in iOS/Android mobile app development field. Looking forward to yo Больше

$2250 USD за 7 дней(-я)
(4 отзывов(-а))
4.6
programmernabeel

Hi there, I have more than 5+ years of experience in mobile application so I would like to start this project. Provided excellent solutions for mobile app development in the past 3 years consecutively. Both IOS and A Больше

$2900 USD за 7 дней(-я)
(13 отзывов(-а))
4.7
pinfytech

Hello There, I have seen your project post carefully and understand all you need E-COMMERCE APP. According to your required features i will make your app with complete features and functionality. I have already done s Больше

$1500 USD за 14 дней(-я)
(21 отзывов(-а))
5.7