Scrape data from website to MS Power BI using PBI Python

Job Description:

To execute this project, the freelancer will need the following skill and experience set:

Proficiency in Python programming language

Experience with web scraping using Python libraries such as BeautifulSoup or Selenium

Knowledge of Power BI and its integration with Python

Experience with incremental data refresh in Power BI

Understanding of website structure and HTML elements

The project will involve the following steps:

Setting up the Python environment and importing the necessary libraries:

Installing necessary Python libraries such as BeautifulSoup, Selenium, and Pandas

Importing the libraries into the script

Setting up the web driver for Selenium

Creating a script to navigate to the website and select the start and end dates:

Using Selenium to open the website "[login to view URL]"

Locating the start and end date pickers on the website using Selenium's find_element_by_* methods

Entering the start and end dates into the date pickers

Ticking the 3 boxes located below the date pickers to select all options of their related drop down lists:

Locate the 3 boxes using Selenium's find_element_by_* methods

Use Selenium's click() method to tick the boxes

Clicking the retrieve button to retrieve the data and storing it in a variable:

Locate the retrieve button using Selenium's find_element_by_* method

Use Selenium's click() method to click the button

Use BeautifulSoup to parse the HTML of the page and extract the table data

Store the table data in a variable for further processing

Looping through the pagination links and retrieving the data from each page:

Locate the pagination links using Selenium's find_elements_by_* method

Iterate through the pagination links and click on each one

Retrieve the table data on each page using BeautifulSoup and store it in the variable

Continue looping until the end of the pages is reached

Cleaning and formatting the data for better understanding:

Use Pandas to convert the data into a dataframe

Perform any necessary cleaning and formatting on the data, such as removing null values or renaming columns

Integrating the data with Power BI for visualization and analysis:

Use Power BI's Python integration to import the data into a report

Create visualizations and perform analysis on the data

Setting up incremental refresh in Power BI to update the data daily:

Use Power BI's incremental refresh feature to schedule the data to be updated daily

Configure the refresh settings and test the refresh process to ensure it is working properly.

The estimated hours required for this project is around 40-50 hours, assuming 2 revision rounds with the project owner. This includes time for debugging and testing the script, as well as integration with Power BI and setting up incremental refresh.

Навыки: Python, Веб-скрейпинг, BeautifulSoup, Selenium, Power BI

О клиенте:
( 6 отзыв(-а, -ов) ) Kuwait, Kuwait

ID проекта: #35823355

73 фрилансеров(-а) готовы выполнить эту работу в среднем за $497


Hello There, Experienced in web scraping / bot / software using Python, Java, PHP. I am experienced with scrap data from the given website, Also I am an expert to build a web scraper, desktop software, or script to pul Больше

$250 USD за 7 дней(-я)
(220 отзывов(-а))

Hello, my name is George and I'm an experienced web crawler expert with proven success in scraping websites and optimizing dataflows with web crawling software. With my extensive experience and knowledge in the Python Больше

$750 USD за 3 дней(-я)
(141 отзывов(-а))

Hi There, I am Python script developer with 5 years of experience. I can develop web scraper to scrape your required website by Python script/Bot with your instructions . Can we discuss please? Thanks & Regards Pooj Больше

$250 USD за 2 дней(-я)
(103 отзывов(-а))

Hello Good evening , I hope you are doing great. Just finished reading the brief details of your job and currently going through attached files . I see you have been looking for a freelancer who has experience with P Больше

$750 USD за 17 дней(-я)
(9 отзывов(-а))

Hello, Warm Greetings! I am pretty sure I can produce high-quality and perfect results for your project. By using Python, I used to make AI engines, BOT, Web Scraping Tools, and so on. PHP and Python are my majors, so Больше

$750 USD за 7 дней(-я)
(9 отзывов(-а))

Hi. I am a freelancer working in 24/7 service. I have much experience of +6 years in Scraping field, I can do your job with Python. I will produce the output in any format you desire. I think this project is for me and Больше

$250 USD за 2 дней(-я)
(51 отзывов(-а))

⭐⭐⭐⭐⭐ Hi there, I have strong expertise in web crawler, web scraper, web monitor, web automation and b-o-t-s using selenium webdriver framework. I have already made lots of scrapers and b-o-t-s for sites such as Googl Больше

$650 USD за 3 дней(-я)
(26 отзывов(-а))

Hello. I read your description carefully. I am python expert you want and am familiar with Power BI so I can make perfect result in time. Let's discuss about your project.

$500 USD за 7 дней(-я)
(26 отзывов(-а))

I can do it

$500 USD за 7 дней(-я)
(23 отзывов(-а))

Happy new year, Nice to meet you. I am very interested in your project. I am a scraping expert that have developed many scraping bot, web scraping, data mining, Data processing. I am very familiar with python, c#, and Больше

$500 USD за 7 дней(-я)
(4 отзывов(-а))

Hi There, I am web scraping expert. I have 10 years of working exposure on web scraping using python+selenium, BeautifySoap, UiPath etc. I have delivered many scraping projects to different clients. I can start work Больше

$250 USD за 3 дней(-я)
(16 отзывов(-а))

Scrape data from website to MS Power BI using PBI Python Good morning Fahad A. , Hi I am a very experienced statistician, data scientist and academic writer. I have completed several PhD level thesis projects involvin Больше

$750 USD за 3 дней(-я)
(30 отзывов(-а))

Hi! I have 8 years of experience in ⭐ python with power BI ⭐. I've read the project description carefully and can start work immediately. Let`s discuss about more detail in chat box. Best regards!

$350 USD за 3 дней(-я)
(20 отзывов(-а))

hii i went with the attachment kindly see my profile already done with the scraping part with help of python i know this how to do kindly tell me the details and text me once if interested and i'll show you my previous Больше

$500 USD за 4 дней(-я)
(22 отзывов(-а))

Hey! I am python and web scrapping expert. I did many scrapping tasks using python libraries. Please initiate a message to discuss further.

$350 USD за 8 дней(-я)
(6 отзывов(-а))

I will create scraping tool with PHP & CURL that extracts all requested data, And provides data in the required format (JSON, CSV, EXCEL, SQL...). Can we discuss more details about this project? Then I can provide sam Больше

$250 USD за 7 дней(-я)
(23 отзывов(-а))

Hi I have read your description carefully. I have rich experiences in ⭐ Python, web scraping, PowerBI, Excel, formula, vba, macro, data entry. ⭐ I worked on many similar projects. I can guarantee the quality of the job Больше

$250 USD за 5 дней(-я)
(17 отзывов(-а))

Hello! How are you? I'm glad to bid on your project. I'm a python expert with 8+ years experience and I'm very familiar with various libraries and packages including matplotlib,pyplot,selenium ,pandas,NLTK,tensorflow Больше

$500 USD за 7 дней(-я)
(10 отзывов(-а))

Hi, I just read your job posting and can see what you want exactly. As a senior developer, I have done many scarping projects using both python and extensions, fully mastered python as well as selenium, puppeteer, am Больше

$700 USD за 10 дней(-я)
(2 отзывов(-а))

Hi, there. I read your project description carefully and I am interested in your proposal. I'm a Python and Web Scraping developer and I have a lot of experience with web scraping. I usually use Beautiful Soup, Seleni Больше

$250 USD за 2 дней(-я)
(15 отзывов(-а))