Crawling, Web Scraping and API
I need to hire an experienced software engineer, who can design a script or make an API for price comparison website and collect data for all of the mobile phones including; description, memory size, images, make and model. These can be scraped from the Internet.
I’ll provide a list of websites where you have to extract the prices and tidy it on Excel for price analysing and adding your own prices after seeing the Minimum to Maximum price range. The script or the API will be integrated into a website and prices will be upgraded on a daily basis.
Further, all the data must be categorised, for example; Excel, etc.
Company Name Description Make Model Image Min Price Max Price Memory
Buying and selling categories’ list must be separate.
Note: If you also know how to design an e-commerce website, please let me know. Because I am finding it difficult to find someone how can merge these two different tasks. (web scraping and making an API and designing a website).
1) Crawling, Scraping and APIs; for price comparison from various competitors’ websites (for price analysis, marketing and pricing strategy).
• Tidy data
• Excel formatted
• Bar charts
• Available sources; [login to view URL], Scraperwiki, Tabula, Google Sheet (IMPORTHTML), etc.
• Additional Sources like, Kimono, etc.
• You could design in; R, Python, Bash, Java or PHP but it is better if designed in R or Python.
2) A full list of mobile phone data including; pictures, descriptions, make and models from all the Brands like Apple, Samsung, HTC, Nokia, Lenovo, Blackberry, Huawei, Sony, etc. This data can be scraped and then categorised.
• Brands logo or name will be shown on the landing page, after clicking on a brand logo all of the available model will appear.
• A simple search bar on the landing page. When a potential client types a name, model or IMEI number, his particular phone will appear. (An API will be provided for this).
3) Data removing feature for removing data from all models and makes.
• An email will be sent to the potential client in which explaining how to remove data from the mobile phone.
This section is for the website design:
Preferred and Important Features:
1) HTML, CSS, JS and Jquery. Additional features and languages are welcome
2) Complete CMS and CRM (Delivery, Postage, Accounts, Inventory management, SEO and Social Media integration, etc.)
3) SEO-friendly and W3C validated
4) Multichannel integration, like EBay, Amazon, Facebook and own website
5) Responsive and Dynamic
6) Complete sitemap, HTML or XML
7) Integration with Social Media like; Facebook, G+, Twitter, LinkedIn, Pinterest, YouTube, etc.
8) Integration with PayPal, and a few other companies
9) Adding Google Analytics and Webmaster on pages
• I personally like the disable/enable command with many features when coding/programming. These features are used for some strategic purposes.
• I will also be providing the sample websites and my own preferred design for better UI and UX Customer Interface, experience (Customer friendly website).
• There are certainly many other features which need to be incorporated into this project. However, those mentioned above are the main features which are the major contributors to this particular project.
Note: please only bid if you think you can complete this project. I had some bad experience in the past. I really do not want to waste my time.
I hope you can perform and cater what you just have read above and give your 100% to complete this project with sheer dedication and professionalism. Good Luck.
Thanks in advance
18 фрилансеров(-а) в среднем готовы выполнить эту работу за $543
Hi sir, i have read your description and i am interested in your project. Spare me some time that we can speak about project. Waiting for your positive response