web scraping for analysis using the Microsoft Power BI tool
source link: https://www.tuicool.com/articles/hit/ruQNbym
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Whenever we talk about web scraping or web data extraction we always think it would be a difficult part and trust me to exact HTML code and fetch what is needed is a bit complicated not an easy task.
But Power BI made it very easy to extract data on which we can do the analysis.
Let me take an example here, suppose we want to create a tool which extracts data from different shopping site and compare them to get the best deal of particular product.
This extraction of the data and comparison can be easily done in a few steps in Microsoft Power BI.
Let’s understand it more using hands on using few steps as mentioned below in this post.
In this example, we are going to extract data from Amazon.in web site
Step 1:- Open web data source and provide the URL of the web page from where you want the data. As shown in the below figure we are going to extract data from below URL
https://www.amazon.in/gp/goldbox?ref=nav_topnav_deals
Step 2: When you click on OK button you will get a new popup windows in which power BI tried to extract the data and provided you tables as you can see in below image there are two tables which Power BI provided Document & Table 0.
but when you view those table you didn’t get any fruitful information. So , Power BI has a feature to extract the data as per your requirement for that you have to click on the Button “Add table using Example”
Step 3:- After clicking the button you will get a new popup screen as shown in below image.
You can see there are 2 parts in the popup the first part is the web part view and you can see the page from where we want to extract data is visible here.
The second part is table structure which we are going to create by giving a few example to Power BI.
Step 4:- Now, Here we have to extract the Product Name, Price and percentage offer. So we are going to train the Power BI with 1 or 2 example. So , Let suppose Column1 will hold the Product name . As we can see the product Name in the web page is “Kevin 80 Cm”, “WOW Raw apple” etc. So , in the first column, we will try to write Kelvin and what you will see a list of data which Power BI extracted. Just select exact name, just follow this step for 2 or 3 rows and you will be surprised to see that rest other row data will fill automatically.
Step 5:- Now add another column in the below table where we will keep the price of the items.
so as you can see for the Kevin 80 cm TV the price is 8,499. So, when you click and try to write the amount of 8499. Power BI is intelligent enough to provide the list of extracted data with similar value. Just try the same step for next 2 or 3 rows and you will find all the prices against the product name.
Step 6:- Now, we will fetch data of percentage off . As you can see for kevin TV price is 39% off. so , to achieve this just add another column in the grid and try to type 39% . As you can see in the below image. Just repeat the step for few other rows and Power BI will do for rest other rows.
Step 7 : Wow, you have extract the data which you want. so far as you can see in below image.
Step 8: Now, Just click on Load button to proceed further you will get the following screen. Proceed further by selecting Custom Tables .
Step 9: Now, great news you got the data whatever you want as you can see the grid
Step 10 : The only thing which might be bother you that in Percentage Off column you were seeing brackets and % Off extra text as well. Now, to avoid this just add another column by example as you can see below screen.
Step 11: That’s great now you have the data which you want.
Now, we have data so we can apply any analysis which we want.
Hope this feature might help you.
Happy learning !
Recommend
-
7
Machine Learning requires a lot of data and not always it is easy to get the data you want. Have you ever wondered how Kaggle and other such websites provide us with huge datasets? The answer is web scraping. So, let us see how we can extract...
-
5
When web-scraping, CSS selectors are one of the best friends. This tutorial will tell you what they're, their pros and cons, and why they matter from a web scraping perspective with Python examples to get you going.
-
2
As a programmer we should think to automate anything related to our daily task every single time if possible. For instance when you gathering amount of data on a web page, rather than copying the text one-by-one you could do a simple web scra...
-
2
@codejediCode_JediPython, Machine Learning, Web-Scraping, Web-Automation and more...NEWABOUT PAG...
-
0
@darshan12SerpdogSerpdog is a Google Search API that allows you to access ...
-
6
Web scraping police data using selenium and python So I have a few posts in the past on scraping data. One shows downloading and parsing...
-
6
Web Scraping Web Scraping Google Search Results Using PHP ByDarshan
-
3
An AI Scraping Tool Is Overwhelmi...
-
3
Web scraping using a headless browser in NodeJSJuly 2nd 2023 New Story by
-
2
Page Replica "Page Replica" is a versatile web scraping and caching tool built with Node.js, Express, and Puppeteer. It helps prerender web app (React, Angular, Vue,...) pages, which can be served via Nginx for SEO or other purposes....
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK