Project Summary: Cloud Computing – Web Scrapping – Sync Google Drive – Merge Excel files - Reports on Excel file – Create alerts of Trading View.
1. Cloud Computing: Set up a cloud of 10 machines (preferably Amazon LightSail) for web scrapping. Install Microsoft Excel, Python, and Selenium.
2. **Web scraping: Extract statistical data from TradingView.com. Excel input files are running with Python, and at the end of the process, we generate one output file.
3. Sync Google Drive: Each virtual machine syncs with one Google Drive folder, where all generated output excel files are stored.
4. Merge Excell files: All output files (from all 10 synced Google Drive folders) merged into a “Master File.”
5. Reports on Excel : Reports are generated after a period of time (7 days) where a loop of web scrapping period finishes.
Every 7 days, we generate approximately 215 output files. Each file contains 11500 lines, which equals a total of 2.5 Million lines.
I understand that creating a report on excel with such a large amount of data is not efficient; I choose this method for the first version of this project for two reasons: 1. I need an interface to cross-check manually if the results of the report are correct, 2. The speed of generating the reports is not important. You can propose any other more efficient methods for this version of later versions of the project.
6. Create alerts of Trading View: Develop further an existing python script that can create alerts of [login to view URL] using data from an excel file.
What is ready:
Python script for web scraping on [login to view URL] runs smoothly and efficiently.
Python script for creating alerts on [login to view URL], needs further adjustments.
Input files (see attachment)
Output files (see attachment)
What is not ready:
Sync Google Drive folders
Master File –Reports (see attachments)
24 фрилансеров(-а) готовы выполнить эту работу в среднем за €527
Hi I have tons of experience in Web Scrapping and have created many crawlers using different technologies I can surely assist you in this task. Please feel free to connect with me on chat Thanks Regards
Hello I'm expert in Microsoft excel, cloud computing, data scrapping. I'll do this job perfectly as per your requirement and instructions. You Will get the best services within the deadline
Hey, I have checked your requirement and understand that as well. I have done SIMILAR work in past. Do you want to see the DEMO WORK??? Will show you Thanks.